Pricing

$4.00 / 1,000 page converteds

Website to Markdown for RAG & LLMs

Crawls a website and converts every page into clean, LLM-ready Markdown for RAG pipelines, vector databases, and AI agents. Removes nav, ads, and boilerplate. Predictable pricing: $0.004 per page converted.

Pricing

$4.00 / 1,000 page converteds

Rating

0.0

(0)

Developer

Aaron Marxsen

Actor stats

Bookmarked

Total users

Monthly active users

19 days ago

Last modified

Website to Markdown Converter for RAG and LLMs

Convert any website into clean, structured Markdown ready for RAG pipelines, vector databases, LLM context, and AI agents. This actor crawls a site page by page, strips away navigation, footers, cookie banners, and scripts, and returns the real content of every page as Markdown. It is the fastest way to turn a website into LLM ready text without writing a scraper or cleaning HTML by hand.

Built for AI engineers, agent developers, and anyone feeding web content to a large language model.

Features

Convert a website to Markdown for RAG and vector database ingestion
Crawl an entire site or a single section with URL glob filters
Strip navigation, headers, footers, sidebars, and cookie banners automatically
Prefer main and article content when a page provides it
GitHub style Markdown with ATX headings and fenced code blocks
Skip near empty pages so your dataset stays clean
Pay per page converted, no subscription

Use cases

Build a RAG knowledge base from your product documentation
Keep an LLM assistant in sync with a changing website on a schedule
Create clean text corpora for evaluation and fine tuning
Give an AI agent a tool to read and summarize any website
Migrate CMS content to Markdown

Input

Field	Type	Description
`startUrls`	array (required)	Where the crawl starts (stays on the same domain)
`maxPages`	integer (default 50)	Maximum pages to crawl and convert
`includeUrlGlobs`	array (optional)	Only crawl URLs matching these globs
`minTextLength`	integer (default 100)	Skip pages with less extracted text than this

{ "startUrls": [{ "url": "https://docs.apify.com/platform" }], "maxPages": 100 }

Output

One dataset item per page:

{
  "url": "https://docs.apify.com/platform/actors",
  "title": "Actors | Platform | Apify Documentation",
  "description": "Learn how Apify Actors work ...",
  "markdown": "# Actors\n\nAn Actor is a serverless program ...",
  "charCount": 5321,
  "crawledAt": "2026-07-07T12:00:00.000Z"
}

Pricing: worked example

Pay per event. You are charged page-converted at $0.004 for each page that is converted and returned. Skipped near empty pages cost nothing.

Converting 100 pages triggers 100 page events at $0.004 each. Total: $0.40.

How to use

Open the actor and click Try for free.
Paste one or more start URLs (your docs site, blog, or knowledge base).
Set maxPages to bound cost, and optionally includeUrlGlobs to target a section.
Click Start.
Watch the run convert each page to Markdown in real time.
Download the results as JSON from the Storage tab, or stream them via the API.
Feed the Markdown into your RAG loader, vector database, or LLM context.

FAQ

How do I convert a website to Markdown for a RAG pipeline? Give this actor your start URLs and it crawls the site, cleans each page, and returns Markdown you can load straight into LangChain, LlamaIndex, or a vector database.

Does it remove navigation and ads? Yes. It removes nav, headers, footers, sidebars, cookie banners, and scripts, and prefers main or article content.

Can it convert a whole documentation site? Yes. Raise maxPages. Cost scales linearly and predictably with pages converted.

What format is the output? GitHub style Markdown (ATX headings, fenced code blocks) plus the page title, description, and character count.

Can an AI agent call this automatically? Yes. It supports agentic payments, so an agent can discover, run, and pay for it without a human in the loop.

API and agents

Run it from the Apify API, or let an AI agent call it. This actor supports agentic payments via x402 and Skyfire (pay per event, limited permissions, no Standby), so agents can buy exactly the pages they need.

curl -X POST "https://api.apify.com/v2/acts/hereditary_model~website-to-markdown/runs?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{ "startUrls": [{ "url": "https://example.com" }], "maxPages": 50 }'

Part of a connected toolkit. Chain these in your agent workflows:

Website Content Crawler - Markdown & Text for LLM / RAG

pear_fight/website-content-crawler-markdown-text-for-llm-rag

Crawl any website and extract clean article text and Markdown, ready to feed into LLMs, ChatGPT, vector databases and RAG pipelines. Removes navigation, ads and boilerplate. Configurable crawl depth and page limits. Export to JSON, CSV, Excel.

Harald

Website to Markdown

cool_ya/website-to-markdown

Convert any web page into clean, LLM-ready Markdown. Strips nav, ads and boilerplate and returns the main article text plus title, description and word count. Perfect for RAG and AI pipelines.

Y A

RAG Pipeline Scraper — Website to Markdown & JSONL

yuchiaoniu/rag-pipeline-scraper

Transform any website into clean Markdown and JSONL ready for RAG pipelines, vector databases (Pinecone, Weaviate, Chroma), and LLM training. Removes ads, navigation, and boilerplate automatically.

Niu Yuchiao

Website to Markdown Crawler - Full-Site Text for LLMs & RAG

entranced_gelato/website-to-markdown-crawler

Crawl any website from a start URL and get every page as clean text + Markdown for LLMs, RAG, and AI agents. Follows internal links with depth and page limits, strips nav and ads, and returns one structured record per page. A fast, no-config site-to-Markdown crawler.

AIDevs

Website to Markdown for LLMs and RAG

rodrgds/website-to-markdown

Convert webpages into clean markdown for LLMs, RAG pipelines, AI datasets, archives, and content extraction. Simple pay-per-page pricing.

Rodrigo Dias

Web Content Extractor - Clean Markdown for AI

geekguymj/web-content-extractor

Extract clean, readable markdown content from any web page. Removes navigation, ads, footers, and boilerplate — outputs structured markdown optimized for LLM training, RAG pipelines, and AI agents. Pay-per-event pricing. $0.002/page.

Matthew Jenkins

Webpage to Markdown

technicaldost/webpage-to-clean-markdown

Convert any web page into clean, LLM-ready Markdown. Strips ads, nav and boilerplate, keeping headings, links, tables and code. Perfect for RAG pipelines and AI agents.

Technical Dost Solutions

AI Web Content Crawler - Markdown for LLMs

intelscrape/ai-web-content-crawler

Crawl any website and extract clean Markdown optimized for LLM training, RAG pipelines, and AI knowledge bases - removes boilerplate and outputs structured JSON with URL, title, markdown, and metadata.

IntelScrape

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

Manas Mantri

Website Content Scraper: Clean Markdown for AI and RAG

scrapemint/website-content-scraper

Crawl any website and get clean markdown, text, or HTML per page, ready for RAG pipelines, chatbots, and LLM fine tuning. Plain HTTP, no browser, no API key. Pay per page.