Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
Do you want to learn more about this Actor?
Get a demoHello,
The actor replace any URL including (%) with (%25) which broke the link and getting no data ( Page not found).
for example If i put this URL : example.com/%D8%A3%D8
it will be replace by the actor to be like this : example.com/%25D8%25A3%25D8
Please assist on this. Thanks
Hello, and thank you for bringing this issue to our attention.
We've just released a patch (in 0.3.42) which fixes this problem.
Rerunning the Actor with the latest
build tag should now work as expected.
I'll close this issue now, but let us know in case anything doesn't work for you. Cheers!
- 3.8k monthly users
- 544 stars
- 99.9% runs succeeded
- 3.4 days response time
- Created in Mar 2023
- Modified 1 day ago