
Website Content Crawler
Pricing
Pay per usage
Go to Store

Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
3.7 (41)
Pricing
Pay per usage
1501
Total users
58K
Monthly users
8.1K
Runs succeeded
>99%
Issues response
7.6 days
Last modified
an hour ago
Crawling with markdown give half of the data where on the other pages gives us complete data
Closed
formidable_quagmire opened this issue
a month ago
So for specifically one of the url the data is partially in the markdown. Not sure as we are not also removing any of the viewport from css selctor as well how this can be possible,
https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=OJ:L_202500914