PageMark: Website to LLM-Ready Markdown & RAG Chunks

PageMark crawls a website and turns every page into clean Markdown for LLMs, RAG and AI agents. You get the main content only, with no menus, cookie banners or footers. Code blocks keep their language and indentation, tables stay tables, and headings stay headings. Choose whole pages or token-counted chunks that you can load straight into a vector database.

$0.45 per 1,000 pages. Pages that need a real browser cost $2.25 per 1,000. You pay only for pages that are saved.

Why PageMark

• Correct content. Code blocks become fenced code with the language tag (```python), with no backslash escapes that break code for an LLM. Doc pages keep their explanations, not only the code samples.
• Fast and cheap. PageMark fetches pages over plain HTTP first and only starts a browser for pages that come back empty without JavaScript. A 30-page docs crawl takes about 10 seconds.
• RAG-ready chunks. Chunks follow the page headings, every chunk carries its heading path (Guide > Install > Linux) and token count (OpenAI cl100k), and a code block is never cut in half. Optional overlap.
• Only new or changed pages. On a schedule, PageMark skips pages whose content has not changed since the last run, and you are not charged for skipped pages. Re-embed only what changed.
• Predictable price. You pay a fixed price per page, so a heavy site doesn't change your bill.
• Plays by the rules. PageMark respects robots.txt (including crawl-delay), identifies itself, and retries politely on 429/5xx. It does not bypass CAPTCHAs or bot protection.

Benchmark

We ran the same three crawls (30 pages each, 26 October 2026) with PageMark and with Apify's Website Content Crawler on its default settings.

Site	Tool	Cost per 1,000 pages	Time	Fenced code blocks	Backslash escapes	Image links
docs.python.org tutorial	Website Content Crawler	$1.45 (compute)	105 s	0	2,358	0
	PageMark	$0.45	7 s	344	0	0
react.dev/learn	Website Content Crawler	$5.36 (compute)	753 s	226	1,918	70
	PageMark	$0.45	11 s	315	0	0
bbc.com/news	Website Content Crawler	$3.50 (compute)	434 s	0	4	551
	PageMark	$0.45	14 s	0	0	0

Website Content Crawler has many settings, and other settings will give different numbers. These are the defaults most people run.

What you get per page

{
  "url": "https://react.dev/learn/state-a-components-memory",
  "title": "State: A Component's Memory – React",
  "description": "The library for web and native user interfaces",
  "language": "en",
  "markdown": "# State: A Component's Memory\n\nComponents often need to change what’s on the screen as a result of an interaction. ...",
  "stats": { "words": 2802, "tokens": 5038, "codeBlocks": 12, "tables": 0 },
  "crawl": { "requestedUrl": "https://react.dev/learn/state-a-components-memory", "depth": 1, "referrerUrl": "https://react.dev/learn",
             "httpStatus": 200, "renderedWithBrowser": false, "fetchedAt": "2026-10-26T15:16:36+00:00" },
  "contentHash": "cea8cbc84bd2ddf956ec2c5634319c5c"
}

In RAG chunks mode you get one row per chunk instead:

{ "url": "https://fastapi.tiangolo.com/tutorial/path-params/", "title": "Path Parameters - FastAPI",
  "chunkIndex": 9, "chunkCount": 20,
  "headingPath": ["Path Parameters", "Predefined values", "Create an `Enum` class"],
  "text": "### Create an `Enum` class\n\nImport `Enum` and create a sub-class that inherits from `str` and from `Enum`. ...",
  "tokens": 257 }

The run's key-value store has a SUMMARY record with counts and the exact pages that failed or were blocked by robots.txt. Nothing is skipped silently.

How to use it

1. Add one or more Start URLs. PageMark follows links that stay under each URL: https://docs.example.com/guide/ crawls only the guide.
2. Set Max pages. You pay only for pages that are saved.
3. Pick Output: pages, pages with chunks, or one row per chunk.
4. Optionally turn on Also use sitemaps to find pages that no menu links to, or Only new or changed pages for scheduled runs.

To fetch just a list of URLs without crawling, set Max crawl depth to 0.

Main settings

Setting	What it does
Max crawl depth	How many links deep to go. 0 = only the start URLs.
Include / Exclude URLs	Glob patterns, e.g. https://example.com/blog/** or **/tag/**.
JavaScript rendering	Auto (browser only when needed), Never (cheapest), Always.
Page formats	Markdown, plain text and/or clean HTML of the main content.
Chunk size / overlap	Tokens per chunk (default 512) and optional overlap.
Keep links / Include images	Keep [text](url) links (on by default); add  images (off by default).
Remove elements	Extra CSS selector to remove. Menus, footers, cookie bars and sidebars are already removed.

Pricing

Event	Price
Page saved (HTTP)	$0.00045 ($0.45 per 1,000)
Page saved with browser (JavaScript sites)	$0.00225 ($2.25 per 1,000)

Pages that fail, are blocked by robots.txt, are duplicates, are not HTML, or are unchanged in "only new or changed" mode are not charged. You can set a maximum cost per run in the run options, and PageMark stops cleanly when it reaches it.

Use it with your AI stack

• LangChain / LlamaIndex: load the dataset with the Apify dataset loader and use markdown (pages) or text (chunks) as the document content, with url and headingPath as metadata.
• Vector databases (Pinecone, Qdrant, Weaviate…): run in RAG chunks mode and embed text. contentHash tells you when a page changed.
• AI agents (MCP): PageMark works as a tool through the Apify MCP server.
• API: start a run with POST https://api.apify.com/v2/acts/1701_engage~pagemark-website-to-markdown/run-sync-get-dataset-items.

FAQ

Does it work on JavaScript sites? Yes. In Auto mode a page that is empty without JavaScript is loaded in a headless browser. Those pages are marked renderedWithBrowser: true and billed at the browser price.

Why did a page fail? See the SUMMARY record. Common reasons are a 403/429 from bot protection, a login wall, or robots.txt. PageMark does not get around bot protection or logins.

PDFs and other files? PageMark only converts HTML pages. Links to PDFs, images and archives are skipped and not charged.

Personal data? PageMark saves page content as published. Comply with the website's terms and with privacy law (such as the GDPR) for whatever you crawl.

Missing a feature or found a page that converts badly? Open an issue with the URL. Conversion bugs are fixed first.