Web Scraper avatar
Web Scraper
Try for free

No credit card required

View all Actors
Web Scraper

Web Scraper

apify/web-scraper
Try for free

No credit card required

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

User avatar

Crawling taking too long time and timed out

Closed

openmart opened this issue
4 months ago

Crawling taking too long time and timed out

User avatar

Hello and thank you for your interest in this Actor.

The timeout happened because you've set the Actor timeout to 140 seconds. From what I see, the Actor scrapes the first two web pages just fine - and only timeouts when loading the pdf file. It seems that changing the Performance and limits > Navigation waits until from networkidle2 to domcontentloaded helps with that - feel free to try it out!

I'll close this issue now, but feel free to ask any additional questions, in case your problem persists. Thanks!

Developer
Maintained by Apify
Actor metrics
  • 3.4k monthly users
  • 99.9% runs succeeded
  • 3.2 days response time
  • Created in Mar 2019
  • Modified about 2 months ago