
Website Content Crawler
Pricing
Pay per usage

Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
4.6 (38)
Pricing
Pay per usage
1222
Monthly users
6.4k
Runs succeeded
>99%
Response time
4.6 days
Last modified
a day ago
Is it possible to speed up the processing time?
Opened 8 hours ago by sheldon-supreme, last comment 8 hours ago by sheldon-supreme
simple page is throwing an error
Opened a day ago by burgundy_zebra, last comment a day ago by burgundy_zebra
Crawling did not support cookie rotating
Opened 2 days ago by meddlesome, last comment 2 days ago by meddlesome
Execution context was destroyed
Opened 6 days ago by benjaminprevot, last comment 6 days ago by benjaminprevot
can we get the images on the pages too?
Opened 6 days ago by disarming_rutabaga, last comment 6 days ago by luca.bartolini88
Exclude Start URL and Disallowed Paths from Output + Return Clean JSON Structure
Opened 7 days ago by rudy-seo, last comment 2 days ago by rudy-seo
Error on Zapier Actor
Opened 7 days ago by insiderperks-owner, last comment 7 days ago by insiderperks-owner
Issue Crawling Content from Paid Websites Like New York Times
Opened 8 days ago by onlinereach, last comment 8 days ago by Jakub Kopecký (jakub.kopecky)
Date Format
Opened 10 days ago by rizlene, last comment 10 days ago by Jiří Spilka (jiri.spilka)
crawler wont click on a specific button
Opened 11 days ago by shikh.sn2021, last comment 8 days ago by Jakub Kopecký (jakub.kopecky)
Adsterra .com
Opened 11 days ago by Tijjeboy, last comment 10 days ago by Jiří Spilka (jiri.spilka)
number of saved lines
Opened 15 days ago by kocsi, last comment 7 days ago by kocsi
Large number of requests fail
Opened 16 days ago by cirez_d, last comment 16 days ago by cirez_d
How to only have the home page or about us page?
Opened 19 days ago by xemivo2655, last comment 10 days ago by Jiří Spilka (jiri.spilka)
Add Full File Name to the Key-Value-Stores
Opened a month ago by CtrlAltElite, last comment 8 days ago by CtrlAltElite
Ability to Group Crawled Page with Followed Link and Its Content in a Single Row
Opened a month ago by randomname1234, last comment a month ago by randomname1234
Navigating frame was detached
Opened 2 months ago by stephen.kim, last comment 2 months ago by Jiří Spilka (jiri.spilka)
scraper don't scrape all the website content like product description
Opened 3 months ago by maabada.shivok, last comment 3 months ago by maabada.shivok
Crawl hung at finished
Opened 3 months ago by mcantrell, last comment 3 months ago by mykola_scrapes
Decode non-UTF-8 text in crawlerType cheerio
Opened a year ago by consoling_knock, last comment a year ago by Jindřich Bär (jindrich.bar)
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.