No credit card required
Website Content Crawler
No credit card required
Automatically crawl and extract text content from websites with documentation, knowledge bases, help centers, or blogs. This Actor is designed to provide data to feed, fine-tune, or train large language models such as ChatGPT or LLaMA.
Problem: Crawling requests for G2 never succeeds in runs after 10 session rotations. This is consistent with run over the last few days.
Not expecting this as there are G2 scraping actors and the site overall does not seem anti-scraping. Thank you!
Hello,
Thanks for the report. Actually, g2.com has one of the strongest Cloudflare protection we have seen (at least some types of pages). There is a customized approach to scrape it, the team will look into how to incorporate it to this actor.
Thank you! Looking forward to the update. G2 is a pretty key part of our project use case.
- 2k monthly users
- 99.9% runs succeeded
- 2.9 days response time
- Created in Mar 2023
- Modified 3 days ago