Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
Crawler timeout setting doesn't work
My problem is the following: Even though I have increased the timeout value of my Website COntent Crawler a lot by Actors -> Website Content Crawler -> Input -> Run Options -> Timeout My runs still regularly timeout after 30 seconds. I have tried now a lot of different things. What could I do?
Hi, thank you for using the Website Content Crawler.
Perhaps this issue will help. It provides details on how to call the Actor with a timeout. However, it sees there may be an issue with Zapier. I'll notify @drobnikj to check it as well.
Thank you, Jiri
Hey there,
if you run the Actor from Zapier, some settings override this timeout. You can find in the Zap setting at the very bottom.
Can you check what timeout is there in Zapier?
I’ll go ahead and close this issue due to inactivity. If you have any further questions or need assistance, feel free to ask or open a new issue. Best, Jiri
ApifyActor Metrics
5.4k monthly users
-
990 bookmarks
>99% runs succeeded
1 days response time
Created in Mar 2023
Modified 13 days ago
