Pricing

$1.79 / 1,000 page converted to markdowns

Try for free

Go to Apify Store

Website to Markdown Crawler - LLM & RAG Ready Content

Try for free

Crawl documentation sites, blogs and knowledge bases into clean Markdown for LLMs and RAG: main content only, links and tables kept, chunking, sitemap seeding. HTTP-only, fast. $0.0018 per page with a hard page limit; failed pages free. Respects robots.txt.

Pricing

$1.79 / 1,000 page converted to markdowns

Rating

0.0

(0)

Developer

DigiVibeTech

Actor stats

Bookmarked

Total users

Monthly active users

2 days ago

Last modified

Quick start: example input, output and cost

Turn docs sites and blogs into LLM-ready Markdown with chunks. Hard page limit so the cost is known up front, robots.txt respected, failed pages free.

Example input

{
  "startUrls": [
    "https://www.rfc-editor.org/rfc/rfc9309.html"
  ]
}

Cost example: A 300-page docs site x $0.0018 = $0.54 (the page limit caps it).

Why this one

Predictable cost. Fixed price per page plus a hard limit, so a run cannot surprise you with a large bill.
Fast and light. Plain HTTP crawling (no browser), so it runs in 512 MB and does not run out of memory in n8n/Make flows. Sites that need JavaScript to show content are out of scope: use a browser-based crawler for those.
Deterministic. Breadth-first crawl, same input = same pages; stayWithinPath, maxDepth, include/exclude regex, optional sitemap seeding.
Robust. One slow or broken site never fails the others; every skipped URL is listed with a reason in ERRORS.
Privacy by default. Email addresses and phone numbers are replaced with [redacted] (switch off with redactContactDetails: false for your own sites). Social networks and profile sites are not crawled.
PDFs? Use our Document to Markdown OCR actor for linked PDF files.

Output (one item per page)

url, requestedUrl, title, markdown, characters, words, chunks[] (if chunkSize > 0), redactions, httpStatus, crawledAt. Totals in SUMMARY.

Responsible crawling

We follow each site's robots.txt for every page and sitemap (disallowed = skipped, free), send at most about one request per second per site (slower if the site asks for a crawl-delay), use an honest User-Agent (SiteToMarkdown/1.0), and never log in or bypass bot protection. Crawl sites you own or are allowed to reuse, and respect copyright when you use the content.

Pricing

Event	Price
Page converted to Markdown	$0.0018

A 500-page documentation site costs at most $0.90.

Maintained by DigiVibeTech. Questions or bugs: use the Issues tab; we reply within 2–5 business days.

Website Content Crawler - Clean Text, Markdown & RAG Chunks

topapi/website-content-crawler

Crawl any website or URL list for $1.20 per 1,000 pages: one row per page with clean text, Markdown, HTML, metadata, links, headings, tables and RAG chunks. Plain HTTP first, headless Chrome only for JavaScript pages. Sitemaps, llms.txt, PDF/DOCX text, Bing/DuckDuckGo search, change monitor.

Top Api

Universal Web to Markdown (Bulk & AI-Ready)

lentic_october/web-to-markdown-converter

Bulk convert any website URLs to clean Markdown for AI & LLMs. Universal scraper that removes ads, scripts, and clutter. Optimized for RAG, ChatGPT, Claude, and LangChain. Fast, async, and API-ready.

kalthireddy Abhishek

Website Content Extractor for RAG: Markdown, HTML, Text

nezha/website-content-crawler

Turn docs sites, help centers, blogs, and websites into clean markdown, text, or HTML for RAG, AI knowledge bases, and internal search. Crawl from start URLs or sitemaps and keep the crawl in scope.

nezha

Website Content Crawler — Markdown for LLM & RAG

brilliant_gum/web-content-crawler

Deep-crawl websites and extract LLM-ready Markdown with OG tags, JSON-LD, author, dates, token estimates, native RAG chunking, language filtering, content-hash dedup, and per-page error reporting. Enforced timeouts. Zero silent failures.

Yuliia Kulakova

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

Manas Mantri

Markdown Maker: HTML to Markdown 📝

shahidirfan/Markdown-Maker

Instantly convert complex HTML into clean, structured Markdown. This lightweight actor is optimized to render web content into a format that is easily readable for AI LLMs, reducing token usage and improving context. Perfect for RAG pipelines and preparing data for training.

Shahid Irfan

5.0

(1)

Website to Markdown for RAG: Crawl & Changed Pages

cliqtomedia/website-to-markdown

Turn a website or docs section into clean Markdown, one row per page, for RAG and LLM pipelines. Stays inside your start path, respects robots.txt and maxPages exactly. Repeat runs can return only new and changed pages. No login.

Cliqto Media

Llm Ready Documentation Scraper

direct_duty/llm-ready-documentation-scraper

Developers and AI agents need to read documentation (e.g. Stripe Docs, Next.js Docs), but standard scrapers return noisy HTML that includes: navigation bars headers / footers ads / cookie banners This Actor must return pure content-only Markdown, suitable for vectorization and semantic search.

Sean

Website to Markdown Crawler for AI, RAG & LLMs

precious_bathmat/website-to-markdown

Crawl any website or docs site and get every page as clean Markdown for ChatGPT, Claude, RAG and vector databases. Main content only, ready-made chunks, llms.txt and one combined file. Uses the sitemap, respects robots.txt. No browser, no API key.

Mariam Ahmed

Web to LLM Spanish — RAG-ready content extractor

fanciful_geode/web-to-llm-es

Crawls Spanish-language websites and extracts clean markdown content optimized for LLM RAG pipelines. Handles Spanish character encoding, removes boilerplate, and outputs structured chunks ready for embedding.