# Web Page to Markdown for LLMs & AI Agents (`fleet261114/page-to-markdown`) Actor

Turn any list of web page URLs into clean Markdown for LLMs, RAG and AI agents: main article content (or the whole page) with headings, lists, tables, code and absolute links, plus title, description and word count. Fast HTTP, no browser.

- **URL**: https://apify.com/fleet261114/page-to-markdown.md
- **Developed by:** [Fleet](https://apify.com/fleet261114) (community)
- **Categories:** AI, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.70 / 1,000 page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Page to Markdown for LLMs & AI Agents

Give it page URLs; get back **clean Markdown** ready for an LLM prompt, a RAG index or an AI agent. By default it keeps only the **main content** (found with Mozilla Readability, the engine behind Firefox Reader View) and drops navigation, ads, cookie banners and footers. Headings, lists, tables, code blocks and links survive, and every link is made absolute so it still works out of context.

It uses a plain HTTP request per page (no browser), so it is fast and cheap: **$0.001 per page**.

### Use it for

- **RAG and knowledge bases:** convert documentation, help centres or blog posts into Markdown chunks.
- **AI agents:** a cheap "read this URL" tool through Apify's MCP server.
- **LLM prompts:** paste the Markdown of a page into a prompt, cut to your context size with "Max characters per page".
- **Content archiving and diffing:** store pages as readable text and compare versions.

### When to use a browser-based tool instead

Pages that build their text with JavaScript after loading (some single-page apps) return little or nothing here. The run summary says "no readable text" for them and you are **not charged**. For those, use a browser-based crawler such as Apify's Website Content Crawler. For server-rendered pages (most docs, blogs, news, Wikipedia, government sites), this Actor gives the same text faster and cheaper.

### Input

| Field | What it does |
|---|---|
| **Page URLs** (required) | One per line, up to 20,000. Links are not followed. |
| What to convert | Main content (default; falls back to whole page if none is found) or whole page minus navigation/header/footer/sidebars. |
| Keep links | Default on, absolute URLs. Off = link text only. |
| Keep images | Default off (saves tokens). |
| Max characters per page | Cut long pages; `truncated` and `fullMarkdownLength` tell you how much was cut. |
| Respect robots.txt | Default on: disallowed pages are skipped and not charged. If a site's robots.txt cannot be fetched (server or network error, timeout), its pages are not requested either (RFC 9309); `RUN_SUMMARY` gives the reason. |
| Timeout / parallel requests | Defaults 30 s, 10 in total, 2 per website. |

### Output

One item per page converted. A real example from a test run:

```json
{
  "url": "https://example.com/",
  "finalUrl": "https://example.com/",
  "statusCode": 200,
  "contentType": "text/html; charset=utf-8",
  "title": "Example Domain",
  "description": null,
  "siteName": null,
  "lang": "en",
  "publishedTime": null,
  "canonicalUrl": null,
  "mode": "page",
  "markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
  "markdownLength": 156,
  "fullMarkdownLength": 156,
  "truncated": false,
  "wordCount": 25,
  "fetchedAt": "2026-11-15T07:03:31.531Z"
}
```

`mode` says what was actually converted: `article` (main content), `page` (whole page, also used when no main content was found) or `text` (a plain-text file). Tables with merged cells are written as `cell | cell` lines instead of raw HTML.

### Pricing

Pay per result: **$0.001 per page converted** ($1 per 1,000 pages), with lower prices on Apify's Bronze, Silver and Gold plans, plus Apify's small run-start fee. Pages that cannot be read (network or HTTP errors, non-text files such as images and PDFs, robots.txt disallowed, no readable text), invalid lines and duplicates are **not charged**; `RUN_SUMMARY` lists them with the reason. You can set a maximum cost per run; the Actor stops cleanly when it is reached.

### Limits and honest notes

- No JavaScript rendering (see above). PDFs and images are not converted.
- Some sites block requests from data centers; those pages are reported as not read and not charged.
- Author bylines are removed where the page marks them (`.byline`, `rel=author`, `itemprop=author`). Other text is returned as the page shows it; you are responsible for using it in line with the site's terms and copyright.
- Up to 8 MB of HTML is read per page.

### Use it from the API, a schedule or an AI agent

**API.** Convert pages and get the Markdown in one call (replace `YOUR_APIFY_TOKEN`):

```bash
curl -X POST "https://api.apify.com/v2/acts/fleet261114~page-to-markdown/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://docs.python.org/3/tutorial/introduction.html"], "maxCharsPerPage": 20000}'
```

**AI agents (MCP).** Add the Actor as a tool in any MCP client (Claude, Cursor and others) through Apify's hosted MCP server:

```
https://mcp.apify.com?tools=fleet261114/page-to-markdown
```

The agent sees the input schema and field descriptions and can call it with a list of URLs. Set a maximum cost per run in the call options to cap spending.

**Whole documentation site.** Get the URL list from the site's sitemap (for example with [Sitemap URL Extractor](https://apify.com/fleet261114/sitemap-url-extractor)), then pass it here.

**Schedules and integrations.** Save the input as a task and schedule it; send results to a vector database, Google Drive, Make, Zapier or n8n with Apify's integrations.

### FAQ

**Why is a page's Markdown empty or short?** It is probably rendered by JavaScript. You are not charged for pages with no readable text; use a browser-based crawler for those.

**Can it follow links and crawl a site?** No, it converts exactly the URLs you give. Use a sitemap for whole sites.

**How many tokens is a page?** Roughly `markdownLength / 4` tokens for English text. Use "Max characters per page" to cap it.

**What does a typical run cost?** At $0.001 per page, 1,000 documentation pages cost about $1.

### Questions and problems

Open an issue on the Actor's **Issues** tab with the run link.

# Actor input Schema

## `urls` (type: `array`):

One page URL per line (up to 20,000 per run). Each page is fetched once with a plain HTTP request; links are not followed. Plain-text pages (text/plain, text/markdown) are returned as they are.

## `mode` (type: `string`):

Main content: the article or documentation body, found with Mozilla Readability (navigation, ads, footers removed); falls back to the whole page when no main content is found. Whole page: the full <body> minus scripts, navigation, header, footer and sidebars.

## `includeLinks` (type: `boolean`):

Keep hyperlinks as Markdown links with absolute URLs. Off = link text only (fewer tokens).

## `includeImages` (type: `boolean`):

Keep images as Markdown image links (absolute URLs, alt text). Off by default to save tokens.

## `maxCharsPerPage` (type: `integer`):

Cut each page's Markdown at about this many characters (at a line break when possible); truncated pages are marked. 0 = no limit. Useful to fit an LLM context window.

## `respectRobotsTxt` (type: `boolean`):

Skip pages the site's robots.txt disallows for this Actor's user agent (listed in RUN_SUMMARY, not charged). Turn off only for your own sites.

## `timeoutSecs` (type: `integer`):

How long to wait for each page before giving up.

## `concurrency` (type: `integer`):

Pages fetched at the same time across all websites.

## `perHostConcurrency` (type: `integer`):

Keeps the load on any one website low.

## Actor input object example

```json
{
  "urls": [
    "https://docs.python.org/3/tutorial/introduction.html",
    "https://example.com"
  ],
  "mode": "article",
  "includeLinks": true,
  "includeImages": false,
  "maxCharsPerPage": 0,
  "respectRobotsTxt": true,
  "timeoutSecs": 30,
  "concurrency": 10,
  "perHostConcurrency": 2
}
```

# Actor output Schema

## `pages` (type: `string`):

No description

## `summary` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://docs.python.org/3/tutorial/introduction.html",
        "https://example.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fleet261114/page-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "https://docs.python.org/3/tutorial/introduction.html",
        "https://example.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("fleet261114/page-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://docs.python.org/3/tutorial/introduction.html",
    "https://example.com"
  ]
}' |
apify call fleet261114/page-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fleet261114/page-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/gHWb7Zqfw49d6NrXF/builds/ms4YXmDQ42vkLDUvO/openapi.json
