Website Content Crawler
Pricing
Pay per usage
Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
3.6 (39)
Pricing
Pay per usage
1406
Total users
54K
Monthly users
8K
Runs succeeded
>99%
Issues response
7.6 days
Last modified
2 days ago
2025-05-13T10:00:05.221Z WARN PlaywrightCrawler: Reclaiming failed request back to the list or queue. page.evaluate: Execution context was destroyed, most likely because of a navigation.
Open
Recently some of the pages started throwing this errors.
Hi, thank you for using Website Content Crawler.
The issue you're encountering is related to the element click automation — specifically, the crawler is trying to click elements with aria-expanded="false", such as:
However, not all such elements are actually expandable or interactable. In this case, Playwright tries to evaluate the page, but navigation or a DOM change likely causes the “Execution context was destroyed” error, leading to a failed request.
To avoid clicking on such unstable elements, you can modify the crawler’s Click elements CSS selector (clickElementsCssSelector) to only target elements that are already expanded (and thus safely skipped):
This effectively disables the auto-click behavior while retaining the configuration. It’s a useful workaround for pages with non-functional or unstable expandable elements. Here is my run with a fix.
Best regards, Jiri
