No credit card required
Website Content Crawler
No credit card required
Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.
Always see the message: There was an uncaught exception during the run of the Actor and it was not handled.
Hello and thank you for your interest in this Actor!
I see that you are using an experimental JSDOM crawler type. Unfortunately, this crawler type can run into issues like the one you just encountered, as the JSDOM DOM API is still not 1:1 with the regular browsers.
I just tried running the crawl with the Cheerio Crawler - and it seems to work just fine. On top of that, the Cheerio Crawler is much faster (~300 results in around 3 minutes) and much more predictable. Check out my run example here - feel free to copy and reuse the input for your Actor run.
I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!
- 2k monthly users
- 99.9% runs succeeded
- 2.9 days response time
- Created in Mar 2023
- Modified 3 days ago