No credit card required
Web Scraper
No credit card required
Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.
fail to crawl ( worked before)
Closed
this url returns an error and i had to abort it run URL https://console.apify.com/organization/LKLE3onx9hqxuNRz9/actors/runs/sJ4eIO5jgO1PH75ir#log
oblanceolate_epoch
happy to get an update
Hello and thank you for your interest in this Actor (and sorry for the wait)!
I tried running the Actor with the same input under my account - just with Datacenter proxies instead of Residentials - and it worked just fine. While Residential proxies should be in general more reliable, they are also more costly and noticeably slower (this is balanced by their good score with antibot protection services). Switching to Datacenter proxies gives you a faster connection to the target server with still a reasonably good (i.e. low) bot score.
Check out my example run at https://console.apify.com/view/runs/hGShY3iKdDgLIsxvr and feel free to reuse the input from there (the only difference is the proxy setup, though).
I'll close this issue now, but feel free to ask more questions regarding this problem, if you have any. Thanks again!
Apify- 3.7k monthly users
- 98.8% runs succeeded
- 3.6 days response time
- Created in Mar 2019
- Modified about 1 month ago
