Web Page to Markdown for LLMs & AI Agents avatar

Web Page to Markdown for LLMs & AI Agents

Pricing

from $0.70 / 1,000 page converteds

Go to Apify Store
Web Page to Markdown for LLMs & AI Agents

Web Page to Markdown for LLMs & AI Agents

Turn any list of web page URLs into clean Markdown for LLMs, RAG and AI agents: main article content (or the whole page) with headings, lists, tables, code and absolute links, plus title, description and word count. Fast HTTP, no browser.

Pricing

from $0.70 / 1,000 page converteds

Rating

0.0

(0)

Developer

Fleet

Fleet

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

14 hours ago

Last modified

Features

Limited permissions

Share

Give it page URLs; get back clean Markdown ready for an LLM prompt, a RAG index or an AI agent. By default it keeps only the main content (found with Mozilla Readability, the engine behind Firefox Reader View) and drops navigation, ads, cookie banners and footers. Headings, lists, tables, code blocks and links survive, and every link is made absolute so it still works out of context.

It uses a plain HTTP request per page (no browser), so it is fast and cheap: $0.001 per page.

Use it for

  • RAG and knowledge bases: convert documentation, help centres or blog posts into Markdown chunks.
  • AI agents: a cheap "read this URL" tool through Apify's MCP server.
  • LLM prompts: paste the Markdown of a page into a prompt, cut to your context size with "Max characters per page".
  • Content archiving and diffing: store pages as readable text and compare versions.

When to use a browser-based tool instead

Pages that build their text with JavaScript after loading (some single-page apps) return little or nothing here. The run summary says "no readable text" for them and you are not charged. For those, use a browser-based crawler such as Apify's Website Content Crawler. For server-rendered pages (most docs, blogs, news, Wikipedia, government sites), this Actor gives the same text faster and cheaper.

Input

FieldWhat it does
Page URLs (required)One per line, up to 20,000. Links are not followed.
What to convertMain content (default; falls back to whole page if none is found) or whole page minus navigation/header/footer/sidebars.
Keep linksDefault on, absolute URLs. Off = link text only.
Keep imagesDefault off (saves tokens).
Max characters per pageCut long pages; truncated and fullMarkdownLength tell you how much was cut.
Respect robots.txtDefault on: disallowed pages are skipped and not charged. If a site's robots.txt cannot be fetched (server or network error, timeout), its pages are not requested either (RFC 9309); RUN_SUMMARY gives the reason.
Timeout / parallel requestsDefaults 30 s, 10 in total, 2 per website.

Output

One item per page converted. A real example from a test run:

{
"url": "https://example.com/",
"finalUrl": "https://example.com/",
"statusCode": 200,
"contentType": "text/html; charset=utf-8",
"title": "Example Domain",
"description": null,
"siteName": null,
"lang": "en",
"publishedTime": null,
"canonicalUrl": null,
"mode": "page",
"markdown": "This domain is for use in documentation examples without needing permission. This is not a service; avoid relying on it for testing and monitoring purposes.",
"markdownLength": 156,
"fullMarkdownLength": 156,
"truncated": false,
"wordCount": 25,
"fetchedAt": "2026-11-15T07:03:31.531Z"
}

mode says what was actually converted: article (main content), page (whole page, also used when no main content was found) or text (a plain-text file). Tables with merged cells are written as cell | cell lines instead of raw HTML.

Pricing

Pay per result: $0.001 per page converted ($1 per 1,000 pages), with lower prices on Apify's Bronze, Silver and Gold plans, plus Apify's small run-start fee. Pages that cannot be read (network or HTTP errors, non-text files such as images and PDFs, robots.txt disallowed, no readable text), invalid lines and duplicates are not charged; RUN_SUMMARY lists them with the reason. You can set a maximum cost per run; the Actor stops cleanly when it is reached.

Limits and honest notes

  • No JavaScript rendering (see above). PDFs and images are not converted.
  • Some sites block requests from data centers; those pages are reported as not read and not charged.
  • Author bylines are removed where the page marks them (.byline, rel=author, itemprop=author). Other text is returned as the page shows it; you are responsible for using it in line with the site's terms and copyright.
  • Up to 8 MB of HTML is read per page.

Use it from the API, a schedule or an AI agent

API. Convert pages and get the Markdown in one call (replace YOUR_APIFY_TOKEN):

curl -X POST "https://api.apify.com/v2/acts/fleet261114~page-to-markdown/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" \
-H "Content-Type: application/json" \
-d '{"urls": ["https://docs.python.org/3/tutorial/introduction.html"], "maxCharsPerPage": 20000}'

AI agents (MCP). Add the Actor as a tool in any MCP client (Claude, Cursor and others) through Apify's hosted MCP server:

https://mcp.apify.com?tools=fleet261114/page-to-markdown

The agent sees the input schema and field descriptions and can call it with a list of URLs. Set a maximum cost per run in the call options to cap spending.

Whole documentation site. Get the URL list from the site's sitemap (for example with Sitemap URL Extractor), then pass it here.

Schedules and integrations. Save the input as a task and schedule it; send results to a vector database, Google Drive, Make, Zapier or n8n with Apify's integrations.

FAQ

Why is a page's Markdown empty or short? It is probably rendered by JavaScript. You are not charged for pages with no readable text; use a browser-based crawler for those.

Can it follow links and crawl a site? No, it converts exactly the URLs you give. Use a sitemap for whole sites.

How many tokens is a page? Roughly markdownLength / 4 tokens for English text. Use "Max characters per page" to cap it.

What does a typical run cost? At $0.001 per page, 1,000 documentation pages cost about $1.

Questions and problems

Open an issue on the Actor's Issues tab with the run link.