# URL to Markdown API - Web Pages to Clean Text for LLMs (`rocketyard/web-page-to-markdown`) Actor

URL to Markdown API: turns the HTML of any public web page into clean Markdown for LLMs and RAG, with title, description, author, dates, language, canonical URL, headings, links, images and JSON-LD, over REST. Failed URLs are free. JSON, CSV or Excel. $0.75 per 1,000 pages, no subscription.

- **URL**: https://apify.com/rocketyard/web-page-to-markdown.md
- **Developed by:** [Rocketyard](https://apify.com/rocketyard) (community)
- **Categories:** AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.49 / 1,000 results

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## URL to Markdown API - Web Pages to Clean Text for LLMs

**URL to Markdown API** is a REST API on the Apify platform: one request returns the main content of any public web page as clean Markdown, with its metadata, as JSON, CSV or Excel. You can also run it from a form in the Apify Console, without code.

### What does URL to Markdown API do?

It turns **any public web page into clean Markdown** for LLMs, RAG pipelines and AI agents, and adds the page's metadata. Give it a list of URLs, or a start page plus crawling, and get one record per page: the main content as Markdown and plain text (navigation, cookie banners and footers removed), the title, description, language, canonical URL, author, publish and update dates, site name, main image, the heading outline, every link, the content images and the JSON-LD the page publishes.

- Main content only: navigation, cookie banners, sidebars and footers are removed, so prompts and embeddings stay clean
- Clean Markdown: headings, lists, GitHub-style tables, code blocks with their language and absolute links
- Rich metadata on every page: title, description, language, canonical URL, author, dates, site name, main image and the JSON-LD the page publishes
- Page outline, every link (internal or external) and the content images as separate fields
- Every URL that fails returns a free error item saying why: robots.txt, not found, bot protection, no content
- Pay only per page delivered: failed URLs cost nothing, no monthly rental
- Fast HTTP fetching without a browser; robots.txt respected; no login and no bot-protection bypass

### Use cases

- **Feed web pages into LLM prompts, RAG pipelines and vector databases as clean Markdown**
- **Give AI agents a reliable "read this URL" tool through MCP**
- **Build a knowledge base from documentation sites, help centers and blogs**
- **Monitor pages for content changes with the content hash**
- **Collect page metadata (titles, descriptions, canonical URLs, dates, structured data) for SEO audits**

### Why this web page to Markdown converter?

- Several competitors have unreliable runs (<90% success in 30 days)
- Cheaper than typical alternatives ($0.75 vs $2.69 per 1,000 results)

### How it compares

| Web page to Markdown scraper | Users (last 30 days) | Successful runs (last 30 days) | Price per 1,000 results |
| --- | --- | --- | --- |
| RAG Web Browser | 32912 | 98.8% | $2.28 |
| Web Fetch | 27790 | 91.0% | $1.50 |
| Fast Website Content Crawler | 140 | 99.7% | $3.00 |
| **This Actor** | new | every release is tested on live pages first | **$0.75** |

Apify Store data from 2026-11-06, free-plan prices. Prices and stats change; check each listing before you compare.

### URL to Markdown API

Every Apify Actor is also a REST endpoint, so URL to Markdown API works as a URL to Markdown API, a web to Markdown API or a HTML to Markdown API, without a server of your own. One `POST` starts a run, waits for it and returns the results:

```text
POST https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items
```

Send your [Apify API token](https://console.apify.com/settings/integrations) in the `Authorization` header and the [input](#input-example) as the JSON body. Add `?format=csv` or `?format=xlsx` to get CSV or Excel instead of JSON. The request waits up to 300 seconds; for a longer run, start it with `POST /v2/acts/rocketyard~web-page-to-markdown/runs` and read the dataset when it finishes. It takes page URLs and reads their HTML; it does not accept pasted HTML.

### How to use it

1. Click **Try for free**.
2. Add page URLs to **Start URLs** (for example `https://en.wikipedia.org/wiki/Web_scraping`).
3. Optionally choose **Output formats** (Markdown, text, HTML), or turn on **Crawl the website** to convert a whole docs site or blog, narrowed with **Link pattern**.
4. Click **Start**, then download results as JSON, CSV, Excel or via the API.

### Input example

```json
{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "outputFormats": [
    "markdown",
    "text"
  ],
  "onlyMainContent": true,
  "maxItems": 100
}
```

### Output example

A real record from the latest test run (nasa.gov) on 2026-11-07. Fields the page didn't publish are left out here; the Actor returns them as `null`.

Long values are shortened here: `markdown` and `text` hold the whole main content, and the lists hold every heading, link and image.

```json
{
  "url": "https://www.nasa.gov/humans-in-space/",
  "finalUrl": "https://www.nasa.gov/humans-in-space/",
  "statusCode": 200,
  "contentType": "text/html",
  "title": "Humans In Space - NASA",
  "description": "In this original series, NASA astronauts take viewers on a journey to the International Space Station, exploring the life-changing experience of an orbital perspective.",
  "language": "en-US",
  "canonical": "https://www.nasa.gov/humans-in-space/",
  "author": "Dacia Massengill",
  "publishedAt": "2022-12-02T21:06:45.000Z",
  "modifiedAt": "2026-10-05T15:20:09.000Z",
  "siteName": "NASA",
  "mainImage": "https://www.nasa.gov/wp-content/uploads/2022/12/iss072e595426orig.jpg",
  "wordCount": 302,
  "tokenEstimate": 893,
  "markdown": "# Humans In Space - NASA\n\nBuilding on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\n### NASA+ Series\n\n## Down to Earth\n\nIn this original series, N…",
  "text": "Building on 60 years of exploration experience and more than two decades of continuous human presence on the International Space Station, NASA will expand humanity’s reach deeper into space than ever before.\n\nNASA+ Series\n\nDown to Earth\n\nIn this original series, NASA astronauts take viewers on a…",
  "headings": [
    {
      "level": 1,
      "text": "Humans In Space - NASA"
    },
    {
      "level": 3,
      "text": "NASA+ Series"
    },
    {
      "level": 2,
      "text": "Down to Earth"
    },
    {
      "level": 2,
      "text": "Artemis II Lunar Flyby"
    },
    {
      "level": 3,
      "text": "Humans in Space"
    },
    {
      "level": 2,
      "text": "Human Spaceflight"
    }
  ],
  "links": [
    {
      "url": "https://www.nasa.gov/",
      "text": "",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/news/",
      "text": "News & Events",
      "internal": true
    },
    {
      "url": "https://www.nasa.gov/2026-news-releases/",
      "text": "News Releases",
      "internal": true
    }
  ],
  "images": [
    {
      "src": "https://images-assets.nasa.gov/image/iss075e0070083/iss075e0070083~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "NASA astronaut and Expedition 75 flight engineer Anil Menon is attached to the Canadarm2 robotic arm’s latching end effector during a spacewalk to install a high-speed space-to-ground antenna on the International Space Station’s Z1 truss segment above the Unity module. Menon and ESA (European Space Agency) astronaut and Expedition 75 flight engineer Sophie Adenot (out of frame) spent six hours and 30 minutes replacing a failed antenna with a spare and making the precise connections needed to activate it for high-speed transmissions with mission control."
    },
    {
      "src": "https://images-assets.nasa.gov/image/jsc2026e391565/jsc2026e391565~large.jpg?w=1920&h=1280&fit=clip&crop=faces%2Cfocalpoint",
      "alt": "Four astronauts are pictured in orange Orion Crew Survival System suits. The astronauts on the far left and far right of the image are standing, holding their helmets. In between them, are the other two astronauts, who are seated and holding their gloves."
    }
  ],
  "contentHash": "sha256:b084d35db91f86024ac4f634e5a7f917c794a7d59ba24d873f6b69d6692dd797",
  "scrapedAt": "2026-11-07T07:01:38.852Z"
}
```

#### Output fields

| Field | Description |
|---|---|
| `url` | The URL you gave (or found by crawling) |
| `finalUrl` | URL after redirects |
| `statusCode` | HTTP status of the page |
| `contentType` | MIME type (`text/html`) |
| `title` | Page title as published (og:title or `<title>`) |
| `description` | Meta description, or the first paragraph when the page has none |
| `language` | Language code, e.g. `en-US` |
| `canonical` | Canonical URL the page declares |
| `author` | Author or byline, when published |
| `publishedAt` | Publication date (ISO 8601), when published |
| `modifiedAt` | Last update (ISO 8601), when published |
| `siteName` | Name of the website |
| `mainImage` | Social preview image, or the first content image |
| `wordCount` | Words in the main content |
| `tokenEstimate` | Rough LLM token count of the Markdown (characters / 4) |
| `markdown` | Main content as clean Markdown: headings, lists, tables, code blocks with their language, absolute links |
| `text` | Main content as plain text |
| `html` | Main content as cleaned HTML (only when requested in **Output formats**) |
| `headings` | Outline of the main content: `{ level, text }` for every heading |
| `links` | Every link on the page: `{ url, text, internal }`, absolute and de-duplicated |
| `images` | Images of the main content: `{ src, alt }` |
| `jsonLd` | Every JSON-LD block the page publishes (schema.org structured data), parsed |
| `contentHash` | `sha256:` hash of the main text: an unchanged hash means unchanged content |
| `scrapedAt` | ISO timestamp |

### Error items

When an input produces no data, the dataset gets one **error item** for it, so nothing is skipped silently. Error items are **free**: only result records are charged.

```json
{
  "url": "https://example.com/a-page-that-was-removed",
  "input": "https://example.com/a-page-that-was-removed",
  "error": {
    "type": "https://rocketyard.dev/errors#not-found",
    "title": "Not found",
    "status": 404,
    "detail": "HTTP 404",
    "instance": "https://example.com/a-page-that-was-removed",
    "code": "NOT_FOUND"
  }
}
```

The `error` object follows [RFC 9457](https://www.rfc-editor.org/rfc/rfc9457) (Problem Details). Its `type` links to the [explanation of the code](https://rocketyard.dev/errors), and `status` is the site's HTTP status when there was one. Filter on `error.code` to separate failures from data; it has its own column in CSV and Excel exports.

| Code | Meaning | Retry? |
|---|---|---|
| [`ROBOTS_DISALLOWED`](https://rocketyard.dev/errors#robots-disallowed) | The site's robots.txt does not allow automated access to this URL (or to the page it redirects to), so it was not fetched. | no |
| [`ROBOTS_UNREACHABLE`](https://rocketyard.dev/errors#robots-unreachable) | The site's robots.txt could not be loaded because of a server or network error. Under RFC 9309, the page is then not fetched. | yes |
| [`PRIVATE_ADDRESS`](https://rocketyard.dev/errors#private-address) | The URL, or a redirect from it, points to a private or local network address. Only public websites are fetched. | no |
| [`NOT_FOUND`](https://rocketyard.dev/errors#not-found) | The page or source does not exist (HTTP 404 or 410). | no |
| [`BLOCKED`](https://rocketyard.dev/errors#blocked) | The site refused access (HTTP 401, 403 or 429) or answered with a bot-check or consent page instead of the content. | sometimes |
| [`HTTP_ERROR`](https://rocketyard.dev/errors#http-error) | The site answered with another HTTP error: a server error (5xx) that persisted after retries, or an unexpected 4xx status. | sometimes |
| [`TIMEOUT`](https://rocketyard.dev/errors#timeout) | The site did not answer in time, even after retries. | yes |
| [`NETWORK_ERROR`](https://rocketyard.dev/errors#network-error) | The connection failed (DNS, TLS or a reset connection), even after retries. | yes |
| [`WRONG_CONTENT_TYPE`](https://rocketyard.dev/errors#wrong-content-type) | The URL returned something the Actor doesn't read, such as a PDF or image instead of an HTML page, or a web page where an API response was expected. | no |
| [`TOO_LARGE`](https://rocketyard.dev/errors#too-large) | The response exceeded the size limit (10 MiB for web pages, 48 MiB for API responses). | no |
| [`PARSE_ERROR`](https://rocketyard.dev/errors#parse-error) | The response could not be read (malformed HTML or JSON). | sometimes |
| [`EMPTY`](https://rocketyard.dev/errors#empty) | The source has nothing to return right now: a list with no open items, a store or collection with no products, a search with no results, or a web page whose HTML has no readable text (often a page built only by JavaScript in the browser). | no |

### Pricing

Pay per page delivered: **$0.75 per 1,000 pages**, plus a tiny per-run start fee. Error items are free. There is no monthly rental, and you can cap spend per run with **Max results** or **Maximum cost per run**.

### Integrations and API

Call it from the [Apify API](https://docs.apify.com/api/v2) (the `run-sync-get-dataset-items` endpoint returns the records in one request), the JavaScript/Python clients, LangChain, LlamaIndex, Make, Zapier or n8n. AI agents can use it as a "read this URL" tool through Apify's MCP server: `https://mcp.apify.com?tools=rocketyard/web-page-to-markdown`.

### FAQ

**How much does it cost?** You pay per result: $0.75 per 1,000 pages, plus a start fee of $0.00005 per run, with no subscription. Cap any run with **Maximum cost per run**.

**How do I call it from the API?** One request starts a run and returns its results, using your [Apify API token](https://console.apify.com/settings/integrations):

```bash
curl -X POST "https://api.apify.com/v2/acts/rocketyard~web-page-to-markdown/run-sync-get-dataset-items" -H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" -d '{"startUrls":[{"url":"https://en.wikipedia.org/wiki/Web_scraping"}],"outputFormats":["markdown"],"maxItems":10}'
```

**How do I use it with AI agents?** Connect Apify's MCP server at `https://mcp.apify.com?tools=rocketyard/web-page-to-markdown`, and the agent can call this Actor as a tool.

**Which pages does it work on?** Any public HTML page: articles, documentation, blogs, help centers, product and landing pages. It reads the HTML the server sends, without a browser, so it is fast and cheap.

**What about pages that need JavaScript?** If a page builds its content only with JavaScript in the browser, the server sends an empty shell. You get a free `EMPTY` error item for it instead of an empty record.

**Does it get around bot protection or logins?** No. It respects robots.txt and crawl-delay, never logs in and never solves CAPTCHAs. Pages behind bot protection return a free `BLOCKED` error item.

**How do I keep the whole page instead of the main content?** Turn off **Only main content**: the Markdown then covers the whole visible page.

**What is `tokenEstimate`?** A rough token count of the Markdown (characters / 4), to plan LLM context and cost before you send the text to a model.

**Is it legal?** The Actor reads public pages only, obeys robots.txt and doesn't bypass protections. You are responsible for how you use the content under each site's terms and the law (for example copyright and GDPR).

**What happens when a URL fails?** If a page can't be read, is blocked or has no content, the dataset gets one [error item](#error-items) for it with a code and the reason, and you are not charged for it. The other URLs in the run carry on.

**I need a feature.** Open an issue on the Actor's Issues tab. Requests are reviewed regularly and drive new features.

### You might also like

- [Article Scraper - Any Website (Schema.org)](https://apify.com/rocketyard/article-schema-scraper): article data from any website with schema.org markup
- [Events Scraper - Any Event Site to Structured Data](https://apify.com/rocketyard/event-schema-scraper): event data from any website with schema.org markup
- [Company Info Scraper - Name, Logo & Social Profiles](https://apify.com/rocketyard/organization-schema-scraper): organization data from any website with schema.org markup
- [Product Data API & Scraper - Prices & Stock](https://apify.com/rocketyard/product-schema-scraper): product data from any website with schema.org markup
- [Shopify Products API & Scraper](https://apify.com/rocketyard/shopify-products-scraper): exports the whole public catalog of a Shopify store

### Made by Rocketyard

[Rocketyard](https://rocketyard.dev) builds pay-per-result scrapers for public web data, and every release is tested on live pages before it ships. Guides for this Actor: [overview](https://rocketyard.dev/web-page-to-markdown), [REST API](https://rocketyard.dev/web-page-to-markdown#api), [MCP for AI agents](https://rocketyard.dev/web-page-to-markdown#mcp) and [n8n, Make and Zapier](https://rocketyard.dev/web-page-to-markdown#automation).

# Actor input Schema

## `startUrls` (type: `array`):

Public web pages to convert, e.g. a docs page, an article or a blog post.

## `outputFormats` (type: `array`):

Formats of the main content in each record. Formats you leave out are null.

## `onlyMainContent` (type: `boolean`):

Keep the article or docs body only, without navigation, sidebars, cookie banners and footers. Turn off to convert the whole visible page.

## `excludeSelectors` (type: `string`):

Optional: elements to drop before conversion, e.g. .newsletter-signup, #comments (max 500 characters).

## `maxItems` (type: `integer`):

Stop after this many records in total. This also caps what the run can charge for results.

## `crawl` (type: `boolean`):

Follow links on the same site from the start URLs to find more pages. When off, only the start URLs are read.

## `maxPagesPerCrawl` (type: `integer`):

Only used when crawling: the most pages to fetch in one run.

## `linkPattern` (type: `string`):

Only used when crawling: follow only links matching this regular expression, e.g. /products/ (max 200 characters, no nested quantifiers).

## `maxConcurrency` (type: `integer`):

Pages fetched in parallel. A polite delay between requests to the same host always applies.

## `proxyConfiguration` (type: `object`):

Optional. Leave empty unless a site blocks direct requests.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ],
  "outputFormats": [
    "markdown",
    "text"
  ],
  "onlyMainContent": true,
  "maxItems": 100,
  "crawl": false,
  "maxPagesPerCrawl": 50,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `results` (type: `string`):

One record per page (Markdown, text and metadata), plus a free error item for each URL that produced none.

## `runSummary` (type: `string`):

Pages visited, items extracted, pages skipped by robots.txt and errors.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://en.wikipedia.org/wiki/Web_scraping"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rocketyard/web-page-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://en.wikipedia.org/wiki/Web_scraping" }] }

# Run the Actor and wait for it to finish
run = client.actor("rocketyard/web-page-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://en.wikipedia.org/wiki/Web_scraping"
    }
  ]
}' |
apify call rocketyard/web-page-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rocketyard/web-page-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Fbm6ZS1gTYdl2Wdtm/builds/h4TiuFAkgBR05XTLh/openapi.json