
Website Content Crawler
Pricing
Pay per usage

Website Content Crawler
Crawl websites and extract text content to feed AI models, LLM applications, vector databases, or RAG pipelines. The Actor supports rich formatting using Markdown, cleans the HTML, downloads files, and integrates well with 🦜🔗 LangChain, LlamaIndex, and the wider LLM ecosystem.
4.6 (38)
Pricing
Pay per usage
1223
Monthly users
6.4k
Runs succeeded
>99%
Response time
4.6 days
Last modified
a day ago
Is it possible to speed up the processing time?
Opened 12 hours ago by sheldon-supreme, last comment 12 hours ago by sheldon-supreme
simple page is throwing an error
Opened a day ago by burgundy_zebra, last comment a day ago by burgundy_zebra
Crawling did not support cookie rotating
Opened 2 days ago by meddlesome, last comment 2 days ago by meddlesome
Execution context was destroyed
Opened 6 days ago by benjaminprevot, last comment 6 days ago by benjaminprevot
can we get the images on the pages too?
Opened 7 days ago by disarming_rutabaga, last comment 6 days ago by luca.bartolini88
Exclude Start URL and Disallowed Paths from Output + Return Clean JSON Structure
Opened 7 days ago by rudy-seo, last comment 2 days ago by rudy-seo
Error on Zapier Actor
Opened 7 days ago by insiderperks-owner, last comment 7 days ago by insiderperks-owner
Issue Crawling Content from Paid Websites Like New York Times
Opened 9 days ago by onlinereach, last comment 8 days ago by Jakub Kopecký (jakub.kopecky)
Date Format
Opened 10 days ago by rizlene, last comment 10 days ago by Jiří Spilka (jiri.spilka)
crawler wont click on a specific button
Opened 11 days ago by shikh.sn2021, last comment 8 days ago by Jakub Kopecký (jakub.kopecky)
Adsterra .com
Opened 11 days ago by Tijjeboy, last comment 10 days ago by Jiří Spilka (jiri.spilka)
Received blocked status code: 429
Opened 14 days ago by josephfalla, last comment 10 days ago by Jiří Spilka (jiri.spilka)
number of saved lines
Opened 15 days ago by kocsi, last comment 7 days ago by kocsi
Large number of requests fail
Opened 16 days ago by cirez_d, last comment 16 days ago by cirez_d
Increased usage limit not continuing run
Opened 18 days ago by anlaics2, last comment 16 days ago by anlaics2
How to only have the home page or about us page?
Opened 19 days ago by xemivo2655, last comment 10 days ago by Jiří Spilka (jiri.spilka)
the crawler stops half way through the crawling process
Opened 19 days ago by avkarma, last comment 16 days ago by Jiří Spilka (jiri.spilka)
how to extract "date" meta data?
Opened 21 days ago by avkarma, last comment 10 days ago by Jiří Spilka (jiri.spilka)
get this error
Opened 22 days ago by esc4dinh4, last comment 16 days ago by Jakub Kopecký (jakub.kopecky)
Crawling not extracting all text on page
Opened a month ago by agungbmtra, last comment 23 days ago by Jakub Kopecký (jakub.kopecky)
Pricing
Pricing model
Pay per usageThis Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage.