Web Scraper avatar

Web Scraper

Try for free

No credit card required

Go to Store
Web Scraper

Web Scraper

apify/web-scraper
Try for free

No credit card required

Crawls arbitrary websites using the Chrome browser and extracts data from pages using JavaScript code. The Actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

Do you want to learn more about this Actor?

Get a demo
BS

Am I doing something wrong?

Closed

BascharSeven opened this issue
2 months ago

I have been trying your solution for a long time. I try it again and again. I want to scrape a web page. But I only get 2 entries as output. Why, what do I have to do? Please help me.

jindrich.bar avatar

Hello, and thank you for your interest in this Actor!

This behavior is caused by using the Glob Patterns input option - by setting it to https://www.management-qualifizierung.de/, you're telling the Actor only to crawl this one specific URL (the example.com is added in the default Page function as an example.)

You can set the Glob Patterns option to https://www.management-qualifizierung.de/**/* to tell the Actor to crawl all the pages under this domain. See my "fixed" run here. Note that in this run, I set the maxResultsPerCrawl option to 100, to crawl more pages (in your run, this was set to 10). Adjust this value if you want to get more / less results.

I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!

Developer
Maintained by Apify

Actor Metrics

  • 2.4k monthly users

  • 278 stars

  • >99% runs succeeded

  • 38 days response time

  • Created in Mar 2019

  • Modified 4 months ago