Pricing

from $2.10 / 1,000 pages

Try for free

Go to Apify Store

Website Content Crawler

Try for free

Website content crawler for RAG and LLMs: each page as clean Markdown, menus and footers removed, optional chunks. $3 per 1,000 saved pages ($2.10 on Business); refused and failed pages are free. HTML only: no JavaScript rendering, proxy or PDF. Max 5,000 pages per website.

Pricing

from $2.10 / 1,000 pages

Rating

0.0

(0)

Developer

Pumpkin Farm

Actor stats

Bookmarked

Total users

Monthly active users

6 hours ago

Last modified

Configure MCP server with Website Content Crawler for RAG: Markdown & Chunks

Add the configuration above to your project's .mcp.json file for Claude Code, then run /mcp in Claude Code to sign in to Apify. It connects to the hosted Apify MCP server with the Website Content Crawler for RAG: Markdown & Chunks Actor already loaded. Clients without OAuth support can authenticate with an Authorization: Bearer header carrying an API token from API & Integrations in Apify Console.

For other MCP clients, get a client-specific configuration with the Website Content Crawler for RAG: Markdown & Chunks Actor preconfigured at https://mcp.apify.com/?tools=fetch-actor-details,pumpkinapi/website-content-crawler.

You can connect to the Apify MCP server using an MCP client that supports Streamable HTTP.

If you want to learn more about our Apify MCP implementation, check out our MCP documentation. To learn more about the Model Context Protocol in general, refer to the official MCP documentation or read our blog post.

Google Search Scraper ($0.15 per 1000 results)

igolaizola/google-search-scraper-ppe

$0.15 per 1000 results 💸 Fast & Low Cost Google Search scraper that returns clean, structured SERP data. Get position, title, URL and snippet. Control region, language and domain, with built-in Apify proxy support for reliable, geo-accurate results.

Iñigo Garcia Olaizola

3.2K

4.1

(6)

Youtube Transcript Scraper

pintostudio/youtube-transcript-scraper

Looking for a reliable way to extract transcripts from YouTube videos? 🎥✨ Look no further! The YouTube-Transcript-Scraper has you covered. 🚀 It effortlessly retrieves transcripts while offering additional valuable insights. Ready to start? Let’s scrape away! 🕵️‍♂️💻

Pinto Studio

28K

3.7

(49)

YouTube Comments Scraper

streamers/youtube-comments-scraper

This alternative YouTube Data API has no limits or quotas. Extract YouTube comments data from one or multiple YouTube videos: full comment text, posting date, author username, video title, videoId. Download YouTube comments in JSON, CSV, and Excel.

Streamers

30K

4.9

(42)

YouTube Thumbnail Downloader

moving_beacon-owner1/my-actor-73

Download high-quality thumbnails from YouTube videos. Bulk processing allows simultaneous downloads from multiple URLs. Choose from quality options: maxresdefault, sddefault, hqdefault, or mqdefault. Automatic fallback switches to a lower quality if necessary.

Jamshaid Arif

573

Website Content Crawler

mikolabs/website-content-crawler

Deep-crawl websites to extract clean text, Markdown, or HTML for AI/LLM apps, RAG pipelines, and vector databases. Supports adaptive crawling, HTML cleaning, file downloads, and structured dataset output. Easily integrates with LangChain, LlamaIndex, and other LLM tools.

Mikolabs

5.0

(1)

Fast Website Content Crawler

6sigmag/fast-website-content-crawler

A high-performance web scraper that rapidly extracts and analyzes content from multiple websites simultaneously. Perfect for competitive research, content aggregation, and website structure analysis.

David

4.8K

4.9

(7)

Website Content Crawler – Markdown for AI, Emails & Contacts

locomint/website-crawler-content-contacts

Website content crawler that turns a whole site into clean Markdown for LLM and RAG pipelines, with the role emails, social profiles and WhatsApp numbers found on each page. Seeded from the sitemap, honours robots.txt, plain HTTP with no browser.

Locomint

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.

Top Api

Facebook Groups Scraper

apify/facebook-groups-scraper

Extract data from one or multiple public Facebook groups. Get group and post URLs, post text, comments, timestamp, likes and comments count, and basic commentator info. Download the data in JSON, CSV, and Excel and use it in apps, spreadsheets, and reports.