Website Content Crawler
No credit card required
Website Content Crawler
No credit card required
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
Do you want to learn more about this Actor?
Get a demoHung crawl
Opened 14 hours ago by mcantrell-owner, last comment 10 hours ago by Jiří Spilka (jiri.spilka)
Poor results
Opened 4 days ago by Digital_Mole, last comment 9 hours ago by Jiří Spilka (jiri.spilka)
Integration with openai’s assistant through their functions calling tool
Opened 5 days ago by Cirula, last comment 5 days ago by Jiří Spilka (jiri.spilka)
"Run actor synchronously with input and get dataset items" supported?
Opened 6 days ago by Kilian, last comment 6 days ago by Dušan Vystrčil (dusan.vystrcil)
Blocked by cookies
Opened 7 days ago by andrewtshannon, last comment 6 days ago by andrewtshannon
Not getting PDF Files
Opened 15 days ago by laius, last comment 15 days ago by Jiří Spilka (jiri.spilka)
Website crawler less outputs than requests
Opened 16 days ago by codewithcheese.com, last comment 15 days ago by Jiří Spilka (jiri.spilka)
I can't "get output"
Opened 20 days ago by nicco.teodori, last comment 19 days ago by Jiří Spilka (jiri.spilka)
crawled a website and it just returns messages about cookies, not the text of the site
Opened 23 days ago by jhonovich, last comment 20 days ago by Jiří Spilka (jiri.spilka)
The actor is trying to proceed too many requests
Opened 25 days ago by capola, last comment 22 days ago by Jiří Spilka (jiri.spilka)
I am having trouble putting in Cookies as input.
Opened a month ago by chirag_xcvii, last comment 23 days ago by Jiří Spilka (jiri.spilka)
Input url's path is re-encoded when it contains specific characters, breaking the url
Opened a month ago by iadvize, last comment a month ago by iadvize
Configuring the RAM limits
Opened a month ago by Robmobius, last comment a month ago by Dušan Vystrčil (dusan.vystrcil)
Why does this job take so long?
Opened a month ago by grossjo, last comment a month ago by Dušan Vystrčil (dusan.vystrcil)
Strictly Necessary cookies are absolutely necessary for core functions such as navigating the page or accessing secure areas.
Opened a month ago by lei.niu.bigco, last comment a month ago by Dušan Vystrčil (dusan.vystrcil)
Can't get globs to work
Opened a month ago by matt333, last comment a month ago by matt333
set the depth to 0
Opened a month ago by environmental_mammal, last comment a month ago by Jiří Spilka (jiri.spilka)
Scrapping only few elements on the page and save them in the separate fields
Opened a month ago by igocza, last comment a month ago by igocza
not the whole content is scraped
Opened a month ago by igocza, last comment a month ago by Jiří Spilka (jiri.spilka)
can't crawl the whole website
Opened a month ago by visable, last comment a month ago by Jiří Spilka (jiri.spilka)
Actor Metrics
4k monthly users
-
839 stars
>99% runs succeeded
1 days response time
Created in Mar 2023
Modified 17 hours ago