Pricing

Pay per usage

Website to Markdown for RAG & LLMs

Crawls a website and converts every page into clean Markdown, ready for RAG pipelines, vector databases, and LLM context. Pay per page converted.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Aaron Marxsen

Actor stats

Bookmarked

Total users

Monthly active users

15 minutes ago

Last modified

Why Markdown?

LLMs work dramatically better with clean Markdown than with raw HTML: fewer tokens, preserved structure (headings, lists, code blocks, links), no markup noise. This Actor does the cleanup for you at crawl time.

Features

🕷️ Crawls an entire site or just a section (URL glob filters, e.g. https://example.com/docs/**)
🧹 Removes navigation, headers, footers, sidebars, cookie banners, and scripts
🎯 Prefers <main> / <article> content when the page provides it
📝 GitHub-style Markdown: ATX headings, fenced code blocks
🚫 Skips near-empty pages so your dataset stays clean
💸 Pay per page converted — skipped pages cost nothing

Input

Field	Type	Description
`startUrls`	array	Where the crawl starts (stays on the same domain)
`maxPages`	integer	Max pages to crawl (default 50)
`includeUrlGlobs`	array	Only crawl URLs matching these globs (optional)
`minTextLength`	integer	Skip pages with less extracted text than this (default 100)
`proxyConfiguration`	object	Optional proxy settings

Output

One dataset item per page:

{
    "url": "https://example.com/docs/getting-started",
    "title": "Getting Started — Example",
    "description": "Learn how to ...",
    "markdown": "# Getting Started\n\nWelcome to ...",
    "charCount": 5321,
    "crawledAt": "2026-07-05T12:00:00.000Z"
}

Export as JSON/CSV from the Storage tab, or stream straight into your pipeline via the Apify API — perfect for feeding LangChain, LlamaIndex, or a vector DB loader.

Typical uses

Build a RAG knowledge base from your product docs
Keep an LLM assistant in sync with a changing website (run on a schedule)
Create clean text corpora for evaluation and fine-tuning
Migrate CMS content to Markdown

Website to Markdown for LLM & RAG — Crawl4AI URL to Clean

bikram07/web-to-markdown-crawl4ai

Convert any URL, sitemap, or whole website into clean, LLM-ready Markdown for RAG, vector databases, and AI agents. Hosted Crawl4AI in a real Chromium browser — renders JavaScript and SPAs, strips boilerplate, and exports JSON/CSV. Callable over MCP from Claude and Cursor.

Bikram

Website to Markdown – Clean LLM & RAG Content Extractor

dataquarry/website-to-markdown

Convert any public web page to clean, LLM-ready Markdown with metadata — by URL, a list of URLs, or a whole-site crawl. Strips nav/ads/boilerplate, keeps headings/lists/tables/code. Respects robots.txt. No API key.

Daniel Brenner

Website to Markdown Crawler for LLM & RAG

logiover/website-text-markdown-crawler

Crawl any website to clean Markdown and plain text for LLM training and RAG. HTML to Markdown, no API or login. Export website text to CSV or JSON.

Logiover

Website To Markdown

smart_api/website-to-markdown

Convert any webpage into clean, LLM-ready Markdown in seconds — perfect for AI training data, RAG pipelines, and content archiving.

SmartApi

5.0

(1)

Website to Text & Markdown — AI / RAG Content Crawler

inexhaustible_glass/rag-website-crawler

Scrape any website into clean text & Markdown with RAG-ready chunks and token counts for LLMs, vector databases (Pinecone, Qdrant) and AI chatbots. Also extracts linked PDF/Word/Excel. Anti-block, robots.txt-aware. Website-to-text for beginners, full RAG pipeline for pros. CPU only, no API key.

Hitman studio

Website & Docs to Markdown + RAG Chunks

awesome_highboy/aeo-rag-ready-content-structurer

Turn websites & docs into clean Markdown plus token-bounded, embeddings-ready RAG chunks (heading lineage + sha256) ready for Pinecone, Weaviate, Qdrant or pgvector. Optional no-hallucination field extraction and AEO mode (FAQ, answer-first, llms.txt, citations). Robots honored; ownership required.

Adam

Website to Markdown Scraper - HTML to MD for RAG API

pink_comic/website-content-to-markdown

Convert web pages and HTML to clean Markdown for RAG, LLM training, AI knowledge bases, and content migration. Strips nav, ads, scripts, and styling while preserving structure. Bulk output includes word/link/image counts.

Ava Torres

Website Content Pipeline for AI: Markdown, Tokens, RAG Chunks

scrapemint/website-content-crawler

Crawl any website and ship clean Markdown, plain text, and HTML for AI, LLM, and RAG pipelines. Each row carries token estimates, JSON LD metadata, link graph, and optional auto chunk splitting for vector databases. Pay per page.

Ken M

Website to Markdown for LLMs and RAG

rodrgds/website-to-markdown

Convert webpages into clean markdown for LLMs, RAG pipelines, AI datasets, archives, and content extraction. Simple pay-per-page pricing.

Rodrigo Dias

Website to Markdown for LLM and RAG

jeweled_jockstrap/my-actor-3

Convert any URL to clean Markdown text for AI applications. Strips HTML extracts content. For LLM training RAG pipelines and vector databases. Free Firecrawl alternative.

Juan Triviño

Website to Markdown Crawler - Full-Site Text for LLMs & RAG

entranced_gelato/website-to-markdown-crawler

Crawl any website from a start URL and get every page as clean text + Markdown for LLMs, RAG, and AI agents. Follows internal links with depth and page limits, strips nav and ads, and returns one structured record per page. A fast, no-config site-to-Markdown crawler.