Web Scraper avatar
Web Scraper
Try for free

No credit card required

View all Actors
Web Scraper

Web Scraper

apify/web-scraper
Try for free

No credit card required

Crawls arbitrary websites using the Chrome browser and extracts data from pages using a provided JavaScript code. The actor supports both recursive crawling and lists of URLs and automatically manages concurrency for maximum performance. This is Apify's basic tool for web crawling and scraping.

User avatar

Crawl not working and eating up credits

Closed

chaxg opened this issue
4 months ago

When i was using the CRawl API, the crawl didnt work on my site kipper.ai and continued to retry without giving any results

User avatar

Hi, thanks for reaching out, we'll look into it with the team and let you know.

User avatar

Hello @chaxg and thank you for your interest in this Actor!

The problem you have encountered was most likely caused by some random proxy / network / server issue.

Rerunning the Actor with your input resulted in a 20-second run with one result, just like expected. The only difference is that I switched to PRODUCTION mode - the DEVELOPMENT mode requires much more setup and can slow down your Actor even more. See my Actor rerun here: https://console.apify.com/view/runs/Zw5sEx7kPmyPwhve7

I'll close this issue now, but feel free to reopen it with more questions if the problem still persists. Thanks!

Developer
Maintained by Apify
Actor metrics
  • 3.4k monthly users
  • 99.9% runs succeeded
  • 3.2 days response time
  • Created in Mar 2019
  • Modified about 2 months ago