URL to Markdown API - Web Pages to Clean Text for LLMs

URL to Markdown API is a REST API on the Apify platform: one request returns the main content of any public web page as clean Markdown, with its metadata, as JSON, CSV or Excel. You can also run it from a form in the Apify Console, without code.

What does URL to Markdown API do?

It turns any public web page into clean Markdown for LLMs, RAG pipelines and AI agents, and adds the page's metadata. Give it a list of URLs, or a start page plus crawling, and get one record per page: the main content as Markdown and plain text (navigation, cookie banners and footers removed), the title, description, language, canonical URL, author, publish and update dates, site name, main image, the heading outline, every link, the content images and the JSON-LD the page publishes.

• Main content only: navigation, cookie banners, sidebars and footers are removed, so prompts and embeddings stay clean
• Clean Markdown: headings, lists, GitHub-style tables, code blocks with their language and absolute links
• Rich metadata on every page: title, description, language, canonical URL, author, dates, site name, main image and the JSON-LD the page publishes
• Page outline, every link (internal or external) and the content images as separate fields
• Every URL that fails returns a free error item saying why: robots.txt, not found, bot protection, no content
• Pay only per page delivered: failed URLs cost nothing, no monthly rental
• Fast HTTP fetching without a browser; robots.txt respected; no login and no bot-protection bypass

Use cases

• Feed web pages into LLM prompts, RAG pipelines and vector databases as clean Markdown
• Give AI agents a reliable "read this URL" tool through MCP
• Build a knowledge base from documentation sites, help centers and blogs
• Monitor pages for content changes with the content hash
• Collect page metadata (titles, descriptions, canonical URLs, dates, structured data) for SEO audits

Why this web page to Markdown converter?

• Several competitors have unreliable runs (<90% success in 30 days)
• Cheaper than typical alternatives ($0.75 vs $2.69 per 1,000 results)

How it compares

Web page to Markdown scraper	Users (last 30 days)	Successful runs (last 30 days)	Price per 1,000 results
RAG Web Browser	32912	98.8%	$2.28
Web Fetch	27790	91.0%	$1.50
Fast Website Content Crawler	140	99.7%	$3.00
This Actor	new	every release is tested on live pages first	$0.75

Apify Store data from 2026-11-06, free-plan prices. Prices and stats change; check each listing before you compare.

URL to Markdown API

Every Apify Actor is also a REST endpoint, so URL to Markdown API works as a URL to Markdown API, a web to Markdown API or a HTML to Markdown API, without a server of your own. One POST starts a run, waits for it and returns the results:

POST https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items

Send your Apify API token in the Authorization header and the input as the JSON body. Add ?format=csv or ?format=xlsx to get CSV or Excel instead of JSON. The request waits up to 300 seconds; for a longer run, start it with POST /v2/acts/rocketyard~web-page-to-markdown/runs and read the dataset when it finishes. It takes page URLs and reads their HTML; it does not accept pasted HTML.

How to use it

1. Click Try for free.
2. Add page URLs to Start URLs (for example https://en.wikipedia.org/wiki/Web_scraping).
3. Optionally choose Output formats (Markdown, text, HTML), or turn on Crawl the website to convert a whole docs site or blog, narrowed with Link pattern.
4. Click Start, then download results as JSON, CSV, Excel or via the API.

Input example

{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "outputFormats": [
    "markdown",
    "text"
  ],
  "onlyMainContent": true,
  "maxItems": 100
}

Output example

A real record from the latest test run (nasa.gov) on 2026-11-07. Fields the page didn't publish are left out here; the Actor returns them as null.

Long values are shortened here: markdown and text hold the whole main content, and the lists hold every heading, link and image.

{
  "url": "https://www.nasa.gov/humans-in-space/",
  "finalUrl": "https://www.nasa.gov/humans-in-space/",
  "statusCode": 200,
  "contentType": "text/html",
  "title": "Humans In Space - NASA",
  "description": "In this original series, NASA astronauts take viewers on a journey to the International Space Station, exploring the life-changing experience of an orbital perspective.",
  "language": "en-US",
  "canonical": "https://www.nasa.gov/humans-in-space/",
  "author": "Dacia Massengill",
  "publishedAt": "2022-12-02T21:06:45.000Z",
  "modifiedAt": "2026-10-05T15:20:09.000Z",
  "siteName": "NASA",
  "mainImage": "https://www.nasa.gov/wp-content/uploads/2022/12/iss072e595426orig.jpg",
  "wordCount": 302,
  "tokenEstimate": 893,
  "markdown": "# Humans In Space - NASA\n\nBuilding on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\n### NASA+ Series\n\n## Down to Earth\n\nIn this original series, N…",
  "text": "Building on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\nNASA+ Series\n\nDown to Earth\n\nIn this original series, NASA astronauts take viewers on a…",
  "headings": [
    {
      "level": 1,
      "text": "Humans In Space - NASA"
    },
    {
      "level": 3,
      "text": "NASA+ Series"
    },
    {
      "level": 2,
      "text": "Down to Earth"
    },
    {
      "level": 2,
      "text": "Artemis II Lunar Flyby"
    },
    {
      "level": 3,
      "text": "Humans in Space"
    },
    {
      "level": 2,
      "text": "Human Spaceflight"
    }
  ],
  "links": [
    {
      "url": "https://www.nasa.gov/",
      "text": "",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/news/",
      "text": "News & Events",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/2026-news-releases/",
      "text": "News Releases",
      "internal": true
    }
  ],
  "images": [
    {
      "src": "https://images-assets.nasa.gov/image/iss075e0070083/iss075e0070083~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "NASA astronaut and Expedition 75 flight engineer Anil Menon is attached to the Canadarm2 robotic arm’s latching end effector during a spacewalk to install a high-speed space-to-ground antenna on the International Space Station’s Z1 truss segment above the Unity module. Menon and ESA (European Space Agency) astronaut and Expedition 75 flight engineer Sophie Adenot (out of frame) spent six hours and 30 minutes replacing a failed antenna with a spare and making the precise connections needed to activate it for high-speed transmissions with mission control."
    },
    {
      "src": "https://images-assets.nasa.gov/image/jsc2026e391565/jsc2026e391565~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "Four astronauts are pictured in orange Orion Crew Survival System suits. The astronauts on the far left and far right of the image are standing, holding their helmets. In between them, are the other two astronauts, who are seated and holding their gloves."
    }
  ],
  "contentHash": "sha256:b084d35db91f86024ac4f634e5a7f917c794a7d59ba24d873f6b69d6692dd797",
  "scrapedAt": "2026-11-07T07:01:38.852Z"
}

Output fields

Field	Description
url	The URL you gave (or found by crawling)
finalUrl	URL after redirects
statusCode	HTTP status of the page
contentType	MIME type (text/html)
title	Page title as published (og:title or <title>)
description	Meta description, or the first paragraph when the page has none
language	Language code, e.g. en-US
canonical	Canonical URL the page declares
author	Author or byline, when published
publishedAt	Publication date (ISO 8601), when published
modifiedAt	Last update (ISO 8601), when published
siteName	Name of the website
mainImage	Social preview image, or the first content image
wordCount	Words in the main content
tokenEstimate	Rough LLM token count of the Markdown (characters / 4)
markdown	Main content as clean Markdown: headings, lists, tables, code blocks with their language, absolute links
text	Main content as plain text
html	Main content as cleaned HTML (only when requested in Output formats)
headings	Outline of the main content: { level, text } for every heading
links	Every link on the page: { url, text, internal }, absolute and de-duplicated
images	Images of the main content: { src, alt }
jsonLd	Every JSON-LD block the page publishes (schema.org structured data), parsed
contentHash	sha256: hash of the main text: an unchanged hash means unchanged content
scrapedAt	ISO timestamp

Error items

When an input produces no data, the dataset gets one error item for it, so nothing is skipped silently. Error items are free: only result records are charged.

{
  "url": "https://example.com/a-page-that-was-removed",
  "input": "https://example.com/a-page-that-was-removed",
  "error": {
    "type": "https://rocketyard.dev/errors#not-found",
    "title": "Not found",
    "status": 404,
    "detail": "HTTP 404",
    "instance": "https://example.com/a-page-that-was-removed",
    "code": "NOT_FOUND"
  }
}

The error object follows RFC 9457 (Problem Details). Its type links to the explanation of the code, and status is the site's HTTP status when there was one. Filter on error.code to separate failures from data; it has its own column in CSV and Excel exports.

Code	Meaning	Retry?
ROBOTS_DISALLOWED	The site's robots.txt does not allow automated access to this URL (or to the page it redirects to), so it was not fetched.	no
ROBOTS_UNREACHABLE	The site's robots.txt could not be loaded because of a server or network error. Under RFC 9309, the page is then not fetched.	yes
PRIVATE_ADDRESS	The URL, or a redirect from it, points to a private or local network address. Only public websites are fetched.	no
NOT_FOUND	The page or source does not exist (HTTP 404 or 410).	no
BLOCKED	The site refused access (HTTP 401, 403 or 429) or answered with a bot-check or consent page instead of the content.	sometimes
HTTP_ERROR	The site answered with another HTTP error: a server error (5xx) that persisted after retries, or an unexpected 4xx status.	sometimes
TIMEOUT	The site did not answer in time, even after retries.	yes
NETWORK_ERROR	The connection failed (DNS, TLS or a reset connection), even after retries.	yes
WRONG_CONTENT_TYPE	The URL returned something the Actor doesn't read, such as a PDF or image instead of an HTML page, or a web page where an API response was expected.	no
TOO_LARGE	The response exceeded the size limit (10 MiB for web pages, 48 MiB for API responses).	no
PARSE_ERROR	The response could not be read (malformed HTML or JSON).	sometimes
EMPTY	The source has nothing to return right now: a list with no open items, a store or collection with no products, a search with no results, or a web page whose HTML has no readable text (often a page built only by JavaScript in the browser).	no

Pricing

Pay per page delivered: $0.75 per 1,000 pages, plus a tiny per-run start fee. Error items are free. There is no monthly rental, and you can cap spend per run with Max results or Maximum cost per run.

Integrations and API

Call it from the Apify API (the run-sync-get-dataset-items endpoint returns the records in one request), the JavaScript/Python clients, LangChain, LlamaIndex, Make, Zapier or n8n. AI agents can use it as a "read this URL" tool through Apify's MCP server: https://mcp.apify.com?tools=rocketyard/web-page-to-markdown.

FAQ

How much does it cost? You pay per result: $0.75 per 1,000 pages, plus a start fee of $0.00005 per run, with no subscription. Cap any run with Maximum cost per run.

How do I call it from the API? One request starts a run and returns its results, using your Apify API token:

$curl -X POST "https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10}'

How do I use it with AI agents? Connect Apify's MCP server at https://mcp.apify.com?tools=rocketyard/web-page-to-markdown, and the agent can call this Actor as a tool.

Which pages does it work on? Any public HTML page: articles, documentation, blogs, help centers, product and landing pages. It reads the HTML the server sends, without a browser, so it is fast and cheap.

What about pages that need JavaScript? If a page builds its content only with JavaScript in the browser, the server sends an empty shell. You get a free EMPTY error item for it instead of an empty record.

Does it get around bot protection or logins? No. It respects robots.txt and crawl-delay, never logs in and never solves CAPTCHAs. Pages behind bot protection return a free BLOCKED error item.

How do I keep the whole page instead of the main content? Turn off Only main content: the Markdown then covers the whole visible page.

What is tokenEstimate? A rough token count of the Markdown (characters / 4), to plan LLM context and cost before you send the text to a model.

Is it legal? The Actor reads public pages only, obeys robots.txt and doesn't bypass protections. You are responsible for how you use the content under each site's terms and the law (for example copyright and GDPR).

What happens when a URL fails? If a page can't be read, is blocked or has no content, the dataset gets one error item for it with a code and the reason, and you are not charged for it. The other URLs in the run carry on.

I need a feature. Open an issue on the Actor's Issues tab. Requests are reviewed regularly and drive new features.

You might also like

• Article Scraper - Any Website (Schema.org): article data from any website with schema.org markup
• Events Scraper - Any Event Site to Structured Data: event data from any website with schema.org markup
• Company Info Scraper - Name, Logo & Social Profiles: organization data from any website with schema.org markup
• Product Data API & Scraper - Prices & Stock: product data from any website with schema.org markup
• Shopify Products API & Scraper: exports the whole public catalog of a Shopify store

Made by Rocketyard

Rocketyard builds pay-per-result scrapers for public web data, and every release is tested on live pages before it ships. Guides for this Actor: overview, REST API, MCP for AI agents and n8n, Make and Zapier.