Web Scraper avatar
Web Scraper

Pricing

Pay per usage

Go to Store
Web Scraper

Web Scraper

Developed by

Apify

Apify

Maintained by Apify

Crawls arbitrary websites using a web browser and extracts structured data from web pages using a provided JavaScript function. The Actor supports both recursive crawling and lists of URLs, and automatically manages concurrency for maximum performance.

4.5 (23)

Pricing

Pay per usage

813

Total users

86K

Monthly users

4.5K

Runs succeeded

>99%

Issues response

31 days

Last modified

16 days ago

OP

Crawling taking too long time and timed out

Closed

openmart opened this issue
a year ago

Crawling taking too long time and timed out

jindrich.bar avatar

Hello and thank you for your interest in this Actor.

The timeout happened because you've set the Actor timeout to 140 seconds. From what I see, the Actor scrapes the first two web pages just fine - and only timeouts when loading the pdf file. It seems that changing the Performance and limits > Navigation waits until from networkidle2 to domcontentloaded helps with that - feel free to try it out!

I'll close this issue now, but feel free to ask any additional questions, in case your problem persists. Thanks!