Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
Unable to retrieve key-value stores via API
When trying to access the key-value store items, I continue to get { "error": { "type": "record-not-found", "message": "Record was not found" } }
However, I know there are key value store values - when I check the "key-value store" tab next to the dataset, I get the screenshot & html files I asked for.
Hi, thank you for using the Website Content Crawler.
I’m not sure if I completely understand your question.
If you’re in the key-value store and click on the "View" link for a particular file, you’ll be redirected to the file. The search bar in your browser will display the complete path to your file, such as:
https://api.apify.com/v2/key-value-stores/XpDuj*****/records/SCREENSHOT-www-li*****.jpg
If I misunderstood your question, could you please clarify? Thank you, Jiri
shawnveltman
Hi - thanks for responding! My question is this - why include this option in the API list if it never works?
Hi, oh, I understand now. This works only when an Actor stores its results in the OUTPUT file.
I agree that this is misleading, and not just in the Website Content Crawler.
Thank you for bringing this up—I’ll make convey this internally!
I'll go ahead and close this issue for now, but feel free to ask questions or raise a new issue. Jiri
ApifyActor Metrics
5.5k monthly users
-
999 bookmarks
>99% runs succeeded
1.1 days response time
Created in Mar 2023
Modified 14 days ago
