
Website Content Crawler
Pricing
Pay per usage
Go to Store

Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
4.0 (40)
Pricing
Pay per usage
1392
Total users
53K
Monthly users
7.9K
Runs succeeded
>99%
Issues response
6.8 days
Last modified
4 days ago
Didn't crawl the entire page and seemed to do it in no particular orer
Open
arsia opened this issue
6 days ago
I an trying to crawl https://vcnewsdaily.com/ and pull up all the reports on there, but when I ran it it didn't crawl all of them and seemed to do it out of order. Can you please assist?