URL to Markdown API — Clean Page Text for LLMs, RAG & Agents

Give it web page URLs and get each page back as clean Markdown and plain text — main content only, with title, description, language, canonical URL, author, publish date, headings, link count and word count. Built for LLM pipelines, RAG ingestion and AI agents that need to read a page, not scrape it.

Try it in 1 minute:

1. Press Start — the prefilled input reads a Wikipedia article and the Apify blog.
2. Wait a few seconds.
3. Open the Output tab: one record per URL with markdown, text, title, wordCount and the metadata, as JSON, CSV or Excel.

Price: $0.50 per 1,000 pages. Pages that cannot be read (404, not HTML, blocked by robots.txt, no text) are free.

Use it to:

• Feed web pages to an LLM as Markdown instead of raw HTML — 5–20× fewer tokens, structure kept.
• Build a RAG corpus: one clean document per URL with headings, word count and a stable canonical URL for deduplication.
• Give an AI agent a "read this page" tool through the Apify MCP server or the API.
• Archive articles and docs as Markdown with their title, author and publish date.
• Pull the text of hundreds of pages in one batch from a sheet of URLs.

How it works

Each URL is fetched with browser-like headers and a 30-second timeout (one retry on a network error through the Apify proxy). Mozilla Readability — the engine behind Firefox Reader View — finds the main content and drops navigation, footer, sidebars and cookie notices; Turndown turns it into Markdown with headings, lists, links, images and GFM tables. Relative links and images are made absolute, so the Markdown works anywhere. Metadata comes from the page head: <title>, Open Graph, <link rel=canonical>, lang, JSON-LD.

Pages that build their text with JavaScript come back as NO_CONTENT from a plain fetch — turn on Use a browser to render them in headless Chromium first (slower, and worth more compute per page).

robots.txt is respected by default: a page the site disallows for bots is skipped with a free ROBOTS_DISALLOWED record.

Input

Field	Default	What it does
urls	—	Web pages to read, one per line (required)
useBrowser	false	Render in headless Chromium first — for JavaScript pages
mainContentOnly	true	Readability main content; off = whole page body
includeLinks	false	Add links (href + anchor text, up to 500)
respectRobotsTxt	true	Skip pages the site's robots.txt disallows
waitUntil, extraWaitMs	load, 0	Browser mode: when the page counts as loaded, and an extra wait
timeoutSecs	30	Per-page timeout
outputFormat	both	markdown, text or both
includeHtml	false	Also deliver the extracted content as cleaned HTML
maxTextLength	0	Cut markdown/text after N characters
maxConcurrency, proxyConfiguration	5, Apify proxy	Speed and retry proxy

Output

One record per URL:

Field	What it is
url, canonicalUrl, httpStatus, contentType	the page after redirects, its canonical link, HTTP status and content type
title, description, siteName, language	from the page head (Open Graph preferred)
author, publishedAt, image	byline, ISO 8601 publish time and lead image when the page declares them
markdown, text	the content as Markdown (starts with # Title) and as plain text
wordCount, headings, linksCount, links	size, structure and links
mainContentDetected, renderedWithBrowser, truncated	how the record was made
input, status, error, scrapedAt	what you gave, OK or the reason, and when

Status codes: OK, NOT_FOUND, NOT_HTML, NO_CONTENT, ROBOTS_DISALLOWED, TOO_LARGE, INVALID_INPUT, FAILED. Only OK records are charged.

Example

{
  "input": "https://en.wikipedia.org/wiki/Web_scraping",
  "url": "https://en.wikipedia.org/wiki/Web_scraping",
  "httpStatus": 200,
  "title": "Web scraping",
  "description": null,
  "language": "en",
  "canonicalUrl": "https://en.wikipedia.org/wiki/Web_scraping",
  "siteName": "Wikipedia",
  "markdown": "# Web scraping\n\nWeb scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...\n\n## History\n\n...",
  "text": "Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites...",
  "wordCount": 3120,
  "headings": [{ "level": 2, "text": "History" }, { "level": 2, "text": "Techniques" }],
  "linksCount": 412,
  "mainContentDetected": true,
  "renderedWithBrowser": false,
  "status": "OK",
  "scrapedAt": "2026-10-19T12:00:00.000Z"
}

FAQ

Is this an official API? No, it's an unofficial API built on public pages.

Why is a page NO_CONTENT? Its HTML has no text — the content is rendered by JavaScript. Run it again with Use a browser on.

Can it read pages behind a login or a paywall? No. It reads public pages only and does not log in, solve captchas or bypass bot walls.

How is this different from a plain fetch? You get the content, not the page: Readability removes menus, footers and cookie notices, and the Markdown keeps headings, lists, tables and links — far fewer tokens for an LLM.

Does it respect robots.txt? Yes, by default. A disallowed page is a free ROBOTS_DISALLOWED record; you can turn the check off for pages you have the right to read.

Like it? A short review on the Store page helps other people find this Actor. Something missing or broken? Tell us on the Issues tab — we read every one.

Related Actors

• Sitemap Extractor & Scraper — every URL, date and image from a sitemap
• PDF Text Extractor & Scraper — page-by-page PDF text and metadata
• YouTube Transcript & Subtitles Scraper — captions as text, SRT or VTT
• Web Fetch API — Fetch any URL as HTML, text, Markdown or JSON
• Markdown Crawler API — crawl a whole website to Markdown for RAG