
Website Content Crawler
Pricing
Pay per usage

Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
4.0 (40)
Pricing
Pay per usage
1392
Total users
53K
Monthly users
7.9K
Runs succeeded
>99%
Issues response
6.8 days
Last modified
4 days ago
No text parsed from from webpage.
Closed
Even though text is there on url for some of the url It's coming as a empty. even after removing css selector we are getting it.
https://diariodarepublica.pt/dr/detalhe/aviso-extrato/13315-919368519
https://diariodarepublica.pt/dr/detalhe/aviso-extrato/13328-919368536
Hello, and thank you for your interest in this Actor.
This seems to be caused by the Mozilla Readability HTML Extractor. I tried switching the HTML Extractor option to None
, which seems to have fixed the issue.
See my example run here.
I'll close this issue now, but feel free to ask additional questions if you have any. Cheers!