Web to Markdown: URL to Markdown for LLMs, RAG and AI agents

A URL to Markdown converter and RAG web browser alternative for LLMs, RAG and AI agents, at $2 per 1,000 pages. Give it a list of URLs, or a search query, and it returns the main content of each page as Markdown with headings, lists, tables, code blocks and links kept, and the navigation, footer and cookie banners removed. It fetches over plain HTTP first and starts a headless browser only for pages that come back empty or need JavaScript, so most pages return in a few seconds.

It costs $0.002 per page that returns content. Failed, blocked and error pages return an error item and are free. A blocked page's error carries a hint to retry with proxyConfiguration set to residential proxies.

Built for AI agents

Agents call Apify Actors as tools through Apify's MCP server, and page fetchers are among the tools they use most. This one is priced and shaped for that job: one cheap call, a predictable output, and a failure that costs nothing.

• Fast. Plain HTTP first and 10 pages in parallel by default; the run's status message reports the median time per page.
• Reliable. Retries with backoff, and a browser fallback instead of an empty answer.
• Low flat price. $0.002 per page, a flat price with no start fee.
• Also a Standby API. A low-latency HTTP endpoint for agents, same price (see Standby mode below).
• Safe in a loop. A bad URL never fails the run, a maxCharsPerPage cap protects your context window, and the run stops at your maximum charge.

Tip for agents: set maxCharsPerPage to the budget you can afford in context, and leave outputFormats on ["markdown"] so the response stays small.

Input

Nothing is required. With empty input the Actor fetches three well-known pages (Wikipedia on Markdown, the Python tutorial, a Paul Graham essay), which is also what Apify's daily auto-test runs. You pay the normal $0.002 per page for them.

Empty input example: {}

Field	Type	What it does
urls	list of text	Pages to fetch. One item per URL. A URL without https:// gets it added.
query	text	Optional search phrase. The Actor searches DuckDuckGo (Google if that fails), takes the top results and fetches them. No separate search proxy or API key needed. A query that is a full URL (starts with http:// or https://) is fetched directly, not searched.
maxResults	number	Search results to fetch for the query. Default 3, maximum 20.
outputFormats	list	markdown (default), text, html.
includeLinks	true / false	Keep links as [text](url). Default on.
maxCharsPerPage	number	Cut each content field at this length. 0 means unlimited (default). Cut items get truncated: true and originalLength.
timeoutSecs	number	Per page, for HTTP and browser each. Default 30.
concurrency	number	Pages in parallel. Default 10.
renderJs	select	auto (default): plain HTTP first, a headless browser only when the page comes back empty or looks like a JavaScript app. http: never use a browser; cheapest and fastest, but a JavaScript-only page returns a "no readable content" error. browser: load every page in a browser first, for single-page apps that auto-detection misses; slower than HTTP. The price per page is the same in all three modes.
removeElementsCssSelector	text	CSS selectors of elements to delete before conversion, comma separated, e.g. nav, footer, .sidebar. Leave empty to use the built-in list (nav, footer, aside and common cookie banners). Your list replaces the built-in one; enter a single space to remove nothing. An invalid selector is skipped with a warning in the log.
proxyConfiguration	object	Optional Apify Proxy. Default is no proxy. Residential proxies help on sites that block datacenter IPs.

{
  "urls": ["https://docs.python.org/3/tutorial/introduction.html", "https://en.wikipedia.org/wiki/Web_scraping"],
  "query": "model context protocol servers",
  "maxResults": 3,
  "outputFormats": ["markdown"],
  "maxCharsPerPage": 20000
}

Output

One item per URL, in the order they finish. Export as JSON, CSV, Excel or via the API. Every item has the same 22 keys; a key that does not apply is null, so you can rely on one fixed shape.

Field	Meaning
url, finalUrl	The URL you gave and where it ended up after redirects.
statusCode	HTTP status of the page.
title, description, language	From the page metadata.
author, publishedAt, siteName	author and publishedAt are what the page declares in its meta tags or JSON-LD, as written (a PDF's author comes from its metadata). siteName is the site name from the page metadata. null when absent.
markdown	Main content as Markdown.
text, html	Plain text and cleaned HTML, only if requested in outputFormats.
wordCount	Words in the extracted content.
truncated, originalLength	truncated is true when maxCharsPerPage cut a field; originalLength is the uncut length in characters.
fetchedVia	http or browser.
searchRank, searchQuery, searchTitle, searchSnippet	Only for pages found through query.
scrapedAt	ISO 8601 UTC time.
error	Set only on failed URLs, whose content fields are null and which are not charged.

{
  "url": "https://docs.python.org/3/tutorial/introduction.html",
  "finalUrl": "https://docs.python.org/3/tutorial/introduction.html",
  "statusCode": 200,
  "title": "3. An Informal Introduction to Python",
  "description": "In the following examples, input and output are distinguished by the presence or absence of prompts...",
  "language": "en",
  "author": null,
  "publishedAt": null,
  "siteName": null,
  "markdown": "# 3. An Informal Introduction to Python\n\nIn the following examples, input and output are distinguished by ...",
  "text": null,
  "html": null,
  "wordCount": 3050,
  "truncated": false,
  "originalLength": 17342,
  "fetchedVia": "http",
  "searchQuery": null,
  "searchRank": null,
  "searchTitle": null,
  "searchSnippet": null,
  "error": null,
  "scrapedAt": "2026-10-09T09:12:44+00:00"
}

Standby mode (HTTP API for agents)

Enable Standby for the Actor in the Apify Console, then call it as a plain HTTP endpoint with no run to start or poll:

curl -H "Authorization: Bearer <APIFY_TOKEN>" \
  "https://datagleaner--web-to-markdown.apify.actor/?url=https://example.com&format=markdown"

You can send the token as ?token= instead of the header. Parameters:

• url: page to fetch; repeat it for up to 10 pages.
• query (or q): search phrase, as in the input. Use maxResults (default 3, max 20).
• format: json (default), markdown or text. markdown and text return the raw body; Accept: text/markdown does the same.
• outputFormats (markdown,text,html), maxChars, includeLinks (default true), timeoutSecs (default 30).
• renderJs (auto, http or browser, default auto) and removeElementsCssSelector, as in the input.

?url=... returns one JSON item (a list for several URLs or a query). Standby uses no proxy, because the request cannot set one yet. A blocked page comes back as an item with an error that carries the residential-proxy hint, and is free; when every page in the request fails, the response is HTTP 502. Use the normal run mode with proxyConfiguration for those sites. Same $2 per 1,000 pages, same page event, and no start fee. It is a low-latency alternative to apify/rag-web-browser Standby, at a flat price.

Use with Python

# pip install apify-client
import os
from apify_client import ApifyClient

client = ApifyClient(os.environ["APIFY_TOKEN"])
run = client.actor("datagleaner/web-to-markdown").call(run_input={
    "urls": ["https://en.wikipedia.org/wiki/Markdown", "https://docs.python.org/3/tutorial/introduction.html"],
    "maxCharsPerPage": 5000,
})
if run is None:
    raise SystemExit("The Actor run did not start.")

for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item["url"], item.get("title"), item.get("wordCount"), item.get("error"))

Two pages cost at most $0.004.

Use with JavaScript / Node.js

// npm install apify-client
import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('datagleaner/web-to-markdown').call({
    urls: ['https://en.wikipedia.org/wiki/Markdown', 'https://docs.python.org/3/tutorial/introduction.html'],
    maxCharsPerPage: 5000,
});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
    console.log(item.url, item.title, item.wordCount, item.error);
}

Or call the Apify API directly and get the items back in the same request:

curl -X POST "https://api.apify.com/v2/acts/datagleaner~web-to-markdown/run-sync-get-dataset-items?token=<APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://en.wikipedia.org/wiki/Markdown"], "maxCharsPerPage": 20000}'

Use it from n8n, Make, Zapier or an AI agent

Actor ID: datagleaner/web-to-markdown

Minimal input:

{"urls": ["https://en.wikipedia.org/wiki/Markdown"]}

Each tool below runs this Actor with your own Apify API token.

• n8n: add the Apify node (@apify/n8n-nodes-apify). On n8n Cloud you install it from the community node registry. Choose Run an Actor and get dataset, set Actor to datagleaner/web-to-markdown and paste the input above.

• Make: use the Apify app's Run an Actor module, then Get Dataset Items to read the results. Watch Actor Runs can trigger a scenario when a run finishes.

• Zapier: use the Apify action Run Actor, then the search Fetch dataset items. The trigger Finished Actor run starts a Zap when a run ends.

• AI agents (MCP): connect to https://mcp.apify.com?tools=datagleaner/web-to-markdown. In Claude Code:

  claude mcp add --transport http apify "https://mcp.apify.com?tools=datagleaner/web-to-markdown"
  

  Then run /mcp to sign in to Apify in your browser. Clients that take a header instead can send Authorization: Bearer <APIFY_TOKEN>. Claude Desktop and other clients that run local servers can use Apify's stdio server (npx -y @apify/actors-mcp-server with APIFY_TOKEN set). It does not preload this Actor, so the agent finds and runs datagleaner/web-to-markdown by name with its search-actors and call-actor tools. Then ask your agent in plain words, for example: "Use datagleaner/web-to-markdown to read https://docs.apify.com/platform/actors and https://modelcontextprotocol.io/introduction as Markdown, capped at 15,000 characters each, and summarise how they differ." The tool input is the same JSON as in the Input section.

• LangChain (Python):

# pip install langchain-apify, then set APIFY_TOKEN in your environment
import json
from langchain_apify import ApifyActorsTool
tool = ApifyActorsTool("datagleaner/web-to-markdown")
result = tool.invoke({"run_input": json.loads('{"urls": ["https://en.wikipedia.org/wiki/Markdown"]}')})

Pricing

Pay per event: $0.002 per page that returns content ($2 per 1,000), no start fee. This price applies from 21 October 2026. Failed URLs, 404s and blocked pages are free. Platform usage is included, and the run stops by itself when your maximum total charge is reached.

Pages	Cost
1,000	$2.00
10,000	$20.00
100,000	$200.00

Worked example: a research agent reads 5 pages to answer each question and answers 2,000 questions a month, which is 10,000 pages, or $20.00 here. A failed fetch costs nothing.

Limits

• Bot protection. Some sites (large news publishers, Cloudflare or DataDome challenge pages, Reddit) refuse automated readers. The Actor then returns an error item such as "blocked by the site's bot protection", free of charge. It does not solve CAPTCHAs.
• Login and paywalls. Only what a logged-out visitor sees. No cookies, no sign-in.
• PDFs. Text-based PDFs return their text (first 100 pages) in markdown. Scanned PDFs without a text layer return an error.
• Search. Uses DuckDuckGo's HTML results, then Google, with no SERP proxy or key. Search engines can refuse datacenter IPs; if a query returns no results, pass the URLs directly.
• Main content only. The extractor keeps the article or documentation body. A page that is mostly navigation or a dashboard returns little.
• Speed. Plain pages come back over HTTP in a few seconds. The browser fallback is slower and is used only for pages that come back empty or need JavaScript.

FAQ

How do I convert a URL to Markdown? Put the URL in urls and run the Actor. Each page comes back as one item whose markdown field holds the main content. Add text or html to outputFormats if you want those too.

Is there an API for web to Markdown? Yes. Every Apify Actor is an API: use the Python or JavaScript client above, or the run-sync-get-dataset-items endpoint, which returns the Markdown in the same HTTP response.

Can I use it as an MCP tool? Yes. Add https://mcp.apify.com?tools=datagleaner/web-to-markdown to Claude, Cursor or any MCP client, as shown above.

Can it convert a whole website to Markdown? Not by crawling. It fetches exactly the URLs you give and does not follow links. To get every page of a site, list its URLs first with the Sitemap Extractor Actor, then pass them in urls.

When does it use a browser? By default only when the plain HTTP answer has almost no readable text or looks like an empty JavaScript app. fetchedVia tells you which path each page took. Set renderJs to browser to always render first (for single-page apps that slip through) or to http to never use a browser.

Responsible use

This Actor reads only publicly visible pages. Respect each site's terms and robots rules, keep your request volume reasonable, and remember that page content is often copyrighted: the output is for analysis, retrieval and research, not for republishing. Pages may contain personal data: you are responsible for having a lawful basis and meeting your obligations under GDPR, CCPA and any other law that applies to your use. This is not legal advice.