URL to Markdown — clean, LLM-ready Markdown from any web page

URL to Markdown turns web pages into clean Markdown for LLMs, RAG pipelines, AI agents and note-taking. It strips navigation, cookie banners, ads and footers, keeps the real content (headings, lists, tables, code blocks, links), and returns page metadata plus a token estimate. Optional chunking gives you RAG-ready pieces with their section headings.

It is HTTP-first: most pages are fetched and converted without a browser, which is fast and cheap. Only pages that need JavaScript (single-page apps) are rendered in headless Chrome, and you are only billed the browser price when rendering actually produced more content.

What it does

• Main-content extraction (Mozilla Readability, the engine behind Firefox Reader View) with a smart fallback for pages Readability mangles (e.g. big Wikipedia data tables are kept).
• Proper GFM tables: real data tables become Markdown tables; layout tables (old-school HTML, Hacker News) are flattened so you don't get junk pipes.
• Absolute links and image URLs, so the Markdown works outside the page. Link titles, tracking attributes and javascript: links are removed to save tokens.
• JavaScript pages: auto-detects client-rendered apps and renders them in Chrome (or force it with renderMode: "browser").
• PDF links are converted too (text-based PDFs, page markers included), as are plain-text and native-Markdown URLs (.md, llms.txt, sites that serve text/markdown).
• Crawl mode: follow links up to N levels deep, staying under the start URL's path — convert a whole docs section in one run. Respects robots.txt.
• Chunking: split each page into chunks of ≤ N characters on heading/paragraph boundaries, each tagged with its nearest heading.
• Metadata: title, description, author, published date, site name, language, canonical URL, word count, character count, token estimate.

Why use it

• RAG and vector databases — feed clean chunks to Pinecone, Qdrant, Weaviate, pgvector…
• AI agents and LLM tools — give your agent a "read this URL" tool that returns compact Markdown instead of 200 KB of HTML (typically 5–20× fewer tokens).
• Docs ingestion — crawl docs.example.com/guide/ into a folder of Markdown files.
• Content archiving and research — save articles to Obsidian, Notion or a Git repo.

How to use

1. Paste one or more URLs.
2. Leave Rendering on auto (recommended).
3. Optional: set Crawl depth to 1–2 to convert linked pages, and Chunk size (e.g. 2000) for RAG.
4. Click Start, then export as JSON/CSV/Excel or call the Apify API. Each page's Markdown is in the markdown field.

Input

Field	Default	What it does
urls	Wikipedia "Markdown" article	Pages to convert.
renderMode	auto	auto = HTTP first, Chrome only for JavaScript apps or blocked pages. http = never use a browser (cheapest). browser = always render in Chrome.
mainContentOnly	true	Remove navigation, headers, footers, sidebars and cookie banners. Turn off to convert the whole page.
includeLinks	true	Keep hyperlinks as [text](url). Off = plain text only.
includeImages	false	Keep images as .
chunkSize	0	If > 0, also return chunks of at most this many characters (split on headings and paragraphs).
crawlDepth	0	0 = only the given URLs. 1 = also pages they link to, etc. (same site, max 10).
maxPages	50	Hard cap on pages converted per run (also caps cost).
sameUrlPrefixOnly	true	When crawling, stay under the start URL's folder (e.g. /docs/).
respectRobotsTxt	true	Skip crawled (linked) pages disallowed by robots.txt. Start URLs you list yourself are always converted. If a site's robots.txt answers with a server error or cannot be fetched, linked pages on that site are skipped (RFC 9309).
maxPdfPages	50	Pages to extract from each linked PDF.
maxConcurrency	5	Pages processed in parallel.
timeoutSecs	30	Per-page timeout.

Example — convert a docs section for RAG:

{
  "urls": [{ "url": "https://crawlee.dev/js/docs/quick-start" }],
  "crawlDepth": 1,
  "maxPages": 25,
  "chunkSize": 2000
}

Output

One dataset item per page. Real example (Oct 22 2026, Markdown shortened):

{
  "url": "https://en.wikipedia.org/wiki/Markdown",
  "finalUrl": "https://en.wikipedia.org/wiki/Markdown",
  "statusCode": 200,
  "contentType": "text/html",
  "title": "Markdown - Wikipedia",
  "description": null,
  "author": "Contributors to Wikimedia projects",
  "publishedAt": "2005-08-09T19:56:00Z",
  "siteName": "Wikimedia Foundation, Inc.",
  "language": "en",
  "canonicalUrl": "https://en.wikipedia.org/wiki/Markdown",
  "markdown": "# Markdown - Wikipedia\n\n…\n\n## History\n\nMarkdown was inspired by pre-existing [conventions](https://en.wikipedia.org/wiki/Convention_\\(norm\\)) for…",
  "wordCount": 5297,
  "charCount": 36635,
  "tokensEstimate": 9159,
  "renderedWithBrowser": false,
  "error": null,
  "crawlDepth": 0,
  "convertedAt": "2026-10-22T23:55:31.859Z"
}

With chunkSize set, each item also has chunks: [{ "index": 0, "heading": "History", "text": "…", "charCount": 1987 }, …]. PDF items add pdfPages and pdfPagesExtracted.

Pages that fail (404, timeout, DNS error, unsupported file type) are listed with error and errorCode (HTTP_404, TIMEOUT, NETWORK_ERROR, UNSUPPORTED_TYPE, TOO_LARGE, BLOCKED_URL, …) and are not charged. Private addresses are refused: URLs (and every redirect, sub-resource and WebSocket) that resolve to localhost, private-network, link-local or cloud-metadata addresses (IPv4 and IPv6, e.g. 169.254.169.254) are not fetched and fail with BLOCKED_URL. A run summary is saved to the key-value store as OUTPUT.

Pricing

Pay per event — failed pages are free:

Event	Price
Page converted over HTTP (most pages)	$0.001 ($1 per 1,000 pages)
Page rendered in headless Chrome (JavaScript apps, renderMode: browser)	$0.003
PDF page extracted (from a linked PDF; blank/scanned pages free)	$0.0005
Actor start	Apify default ($0.00005)

Example: converting 1,000 normal articles costs about $1. In auto mode you are only charged the Chrome price when rendering actually returned more content than plain HTTP. Set Max pages or a max cost per run to cap spending — the Actor stops cleanly and keeps everything converted so far.

FAQ

How is this different from Apify's free Website Content Crawler? Website Content Crawler is a full crawler with many knobs and platform-usage pricing. URL to Markdown is a simple, predictable per-page price, HTTP-first (so most runs finish in seconds on 1 GB of memory), with token estimates and RAG chunks built in.

Does it bypass logins, paywalls or CAPTCHAs? No. It converts publicly accessible pages only. Only convert content you are allowed to use.

Why is my page empty or short? Some sites block automated requests or need interaction (cookie walls, infinite scroll). Try renderMode: "browser" and mainContentOnly: false.

Scanned PDFs? Text is extracted from text-based PDFs; scanned (image-only) pages are returned empty and not charged. For dedicated PDF jobs see our PDF to Markdown Actor.

Limitations

• No proxy rotation; heavily bot-protected sites may return errors.
• Images are referenced by URL, not downloaded.
• tokensEstimate is characters ÷ 4 (a good approximation for English with GPT/Claude-style tokenizers).