Pricing

from $1.50 / 1,000 results

Website Content Crawler for AI & LLM Data

Crawl any website from a seed URL and extract clean Markdown content, ready for LLM training data, RAG pipelines, and vector databases. Set crawl depth, page limits, and domain scope.

Pricing

from $1.50 / 1,000 results

Rating

0.0

(0)

Developer

Code With Aqib

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Configure MCP server with Web Scraper to Markdown for AI, RAG & LLM

Get a ready-to-use configuration for your MCP client with the Web Scraper to Markdown for AI, RAG & LLM Actor preconfigured at mcp.apify.com?tools=your_scraper_guy/website-content-crawler-lite .

You can connect to the Apify MCP Server using clients like Tester MCP Client, or any other MCP client of your choice.

If you want to learn more about our Apify MCP implementation, check out our MCP documentation. To learn more about the Model Context Protocol in general, refer to the official MCP documentation or read our blog post.

Wikipedia Scraper - Articles, Summaries, Metadata

santamaria-automations/wikipedia-scraper

Extract Wikipedia articles including full content, summary, thumbnails, categories, external links, coordinates, and Wikidata IDs. Multi-language support for 12+ languages. Export data, run via API, schedule and monitor runs, or integrate with other tools.

NanoScrape

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Hitman studio

Website Main Content Extractor

sync-network/website-main-content-extractor

Alam

Website Content Scraper: Clean Markdown for AI and RAG

scrapemint/website-content-scraper

Crawl any website and get clean markdown, text, or HTML per page, ready for RAG pipelines, chatbots, and LLM fine tuning. Plain HTTP, no browser, no API key. Pay per page.

Ken M

Website Content Crawler for AI — Clean Markdown, 4x Cheaper

joyouscam35875/website-content-crawler

Crawl any website and extract clean text/markdown for LLMs, RAG pipelines, vector databases. BFS crawl with depth control, robots.txt support, boilerplate removal. Perfect for feeding AI models. $0.001/page — 4x cheaper than the official Apify crawler.

Ken Digital

Stealth Website Crawler

nocturne/stealth-website-crawler

Crawl websites protected by Cloudflare, DataDome, and other anti-bot systems. Extract clean text or markdown for AI/LLM pipelines. Like Website Content Crawler, but for sites that block you.

Nocturne

Youtube

canadesk/youtube

Collect channels, videos (with comments), shorts, streams, playlists, podcasts and posts from Youtube! Filters included. It's fast and costs little!

Canadesk Support

2.0

(1)

Youtube Bulk

canadesk/youtube-bulk

No limits and Bulk mode! Collect channels, videos (with comments), shorts, streams, playlists, podcasts and posts from Youtube! Supports a lot of filters. It's fast and costs little!

Canadesk Support

Website Content Crawler Pro

datascoutapi/website-content-crawler-pro

Crawl websites and extract clean, structured content in Markdown, JSON, or plain text for AI models, LLMs, vector DBs, or RAG pipelines. Fast, reliable, and stealthy, with bulk processing, advanced metadata extraction, and seamless integration with LangChain, LlamaIndex, and AI workflows.

halam

558

3.7

(3)

Quick Website Content Scraper ( Extract Text for RAG & LLMs )

automateitplease/ai-web-content-scraper-extract-text-for-rag-llms

Extract clean text from any website for AI/LLM applications. Supports both static and JavaScript-rendered sites (React, Vue, Angular). Perfect for RAG systems, chatbot training, and content analysis.