# Website & Page to Markdown for RAG and LLMs (`humble-echidna/page-to-markdown`) Actor Turn web pages or whole sites into clean Markdown for RAG and LLMs: main content only, with title, links and word count. Whole-site mode reads the sitemap first, can return only changed pages, and splits pages into heading-based chunks. Respects robots.txt and AI opt-outs. No browser. - **URL**: https://apify.com/humble-echidna/page-to-markdown.md - **Developed by:** [Michael Costa](https://apify.com/humble-echidna) (community) - **Categories:** AI, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $1.00 / 1,000 pages This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ### What does Website & Page to Markdown for RAG and LLMs do? **Page to Markdown** converts **web pages to clean Markdown** for RAG, LLM prompts and AI agents. Give it page URLs or a whole docs site; you get each page's **main content as Markdown**, without menus, footers or cookie banners, plus its title, language and links. Headings, lists, tables, code blocks (with their language) and links are kept. It can **follow same-site links** (up to 2 clicks deep), convert a **whole site or section, sitemap first**, return **only the pages that changed** since your last run, and **split each page into chunks by heading** for your vector store. **Try it in one click:** the input comes pre-filled with two pages of Apify's own documentation (the Storage overview and the Key-value store page). That's 2 pages: $0.002, plus the $0.00005 start fee, so about $0.0021 in all. **Then replace them with the pages you actually want.** ### Monitor a docs site: get only new and changed pages, in Slack, email or a webhook With **Only changed pages** on, each run returns (and charges for) only the pages that are new or changed since the last run of the same input, so your RAG index stays current without re-embedding everything. A run where nothing changed returns 0 pages and costs only the $0.00005 start fee. 1. Put the docs root in **Web page URLs** (for example `https://docs.example.com`), turn on **Whole site (sitemap first)** (`"crawlWholeSite": true`) and **Only changed pages** (`"onlyChangedPages": true`), set **Max pages per site** (`maxPagesPerSite`) to cover the site, and click **Start**. This first run returns every page and is the baseline. 2. Click **Save as a new task** (top right of the actor page). The memory is kept separately for each URL and set of options, so the task compares with its own last run; changing the options starts a new baseline. 3. In Apify Console, open **Schedules**, click **Create new**, set how often in **Schedule setup** (for example weekly, Monday 06:00), then **Add** your task. 4. On the task, open the **Integrations** tab and pick where the changes go: - **HTTP webhook** (for a vector store): event `ACTOR.RUN.SUCCEEDED`, your URL. Apify POSTs `{"eventType": ..., "resource": {...}}`; `resource.defaultDatasetId` is the run's dataset, and `GET https://api.apify.com/v2/datasets//items?format=json` (with your API token) returns the changed pages. Upsert them by `id`, which stays the same for a page from run to run. - **Slack**: click **Configure**, sign in, pick the workspace and channel, and the "run succeeded" event. A useful message: `{{resource.statusMessage}}` (the example below's repeat run said "0 pages converted from 2/2 URLs; 2 unchanged since the last run") and a link, ``. - **Gmail**: click **Connect with Google**, set the subject and body, and attach the dataset. It sends after each successful run. Apify's integrations fire after every successful run, including quiet ones: a quiet run's dataset is empty, and its status message says how many pages were unchanged. ### What data does Page to Markdown return? | Field | Example | Notes | |---|---|---| | `id` | `96dc7de0c0bcc49ea35fac19` | From the page URL; the same on every run. Use it as your vector-store key. | | `url` | `https://docs.apify.com/storage/key-value-store` | After redirects. | | `title` | `json — JSON encoder and decoder` | | | `description` | `Source code: Lib/json/__init__.py ...` | Meta description; `null` if none. | | `language` | `en` | From the page; `null` if not declared. | | `markdown` | `# Key-value store\n\nThe key-value store is ...` | The main content, up to 500,000 characters. | | `wordCount` | `1899` | | | `links` | `[{"url": "https://json.org", "text": "JSON ..."}]` | Links inside the content, absolute, each once. | | `canonicalUrl` | `https://docs.python.org/3/library/json.html` | `null` if none. | | `discoveredVia`, `lastmod` | `input`, `null` | How the page was found (`input`, `link` or `sitemap`); the sitemap's date, if any. | | `contentHash` | `7a6b543f5ef8fa31` | Same text, same hash on every run. | | `chunks`, `chunkCount` | `[{"headingPath": ["Storage", "Datasets"], ...}]` | When chunking is on. | One row per converted page. The full list is under [Output](#output). ### How much does it cost to convert web pages to Markdown? You pay per page converted: **$1.00 per 1,000 pages**, plus $0.00005 each time a run starts. - **The example below:** 2 pages × $0.001 = $0.002, plus the $0.00005 start fee; the repeat run $0.00005. - **A month, for example:** a 500-page docs site in whole-site mode (**Max pages per site** 500), weekly, only changed pages: the first run is 500 × $0.001 = $0.50; if 25 pages change a week, the next 3 runs are 75 × $0.001 = $0.075; **about $0.58** in all. - **Caps:** **Max pages per run** in the input, and **Maximum cost per run** in the run options. The run stops cleanly at whichever comes first. Each page is counted against the limit before it's fetched (and the count given back if the page fails), so a capped run never fetches pages it can't return. **Never charged:** pages that fail, need JavaScript, are disallowed by `robots.txt` or opted out of AI use, a page returned twice (each page is converted once per run, even when several of your URLs link to it), and, with "Only changed pages", pages that haven't changed since your last run. **Chunks don't cost extra.** Splitting into chunks is included in the page price, whichever chunk output you pick: with one row per chunk, the chunk rows go to a separate dataset of the run (`chunks`), which isn't charged; the default dataset still has one (charged) row per page. ### How to convert a web page or a whole site to Markdown 1. Open Page to Markdown and click **Try for free** (or **Start** if you're signed in). 2. Put your pages in **Web page URLs**, one per line (a missing `https://` is added for you). 3. Optional: set **Follow links (crawl depth)**, or turn on **Whole site (sitemap first)** for a site or section; add **Only pages matching** / **Skip pages matching** patterns; turn on **Split into chunks for RAG**. 4. Click **Start**, then open the **Output** tab and export as JSON, CSV or Excel. ### Example: two documentation pages, then a re-check The pre-filled input, with **Only changed pages** turned on: ```json {"urls": ["https://docs.apify.com/storage", "https://docs.apify.com/storage/key-value-store"], "onlyChangedPages": true} ``` One of the two pages (real output from a local run on 2026-09-25; `markdown` shortened, `links` left out): ```json { "id": "96dc7de0c0bcc49ea35fac19", "url": "https://docs.apify.com/storage/key-value-store", "title": "Key-value store | Platform | Apify Documentation", "language": "en", "markdown": "# Key-value store\n\nThe key-value store is simple storage that can be used for storing any kind of data. ...", "wordCount": 1899, "discoveredVia": "input", "contentHash": "7a6b543f5ef8fa31" } ``` Run again straight after, the same input returned **0 pages**: neither page had changed, so only the start fee was charged. For a whole site, use **Whole site (sitemap first)** with the site's docs root instead of single pages. ### Input | Field | What it does | |---|---| | **Web page URLs** (`urls`) | The pages to convert, one per line. A missing `https://` is added for you. | | Follow links (crawl depth) (`crawlDepth`) | `0` (default): only the pages you list. `1`: also the pages they link to on the same site. `2`: one more click further. Other sites and file links (PDF, images, ZIP, ...) are never followed. `www.` counts as the same site. | | Max pages per URL when following links (`maxPagesPerUrl`) | At most this many pages are tried for each URL you list, including itself (default 10). | | Whole site (sitemap first) (`crawlWholeSite`) | Convert every page of each site you list (see below). Crawl depth is ignored then. | | Max pages per site (`maxPagesPerSite`) | In whole-site mode, at most this many pages are returned per URL you list (default 100, up to 10,000). | | Only pages matching / Skip pages matching (`includeUrlPatterns` / `excludeUrlPatterns`) | URL patterns for pages found by crawling (either mode). A pattern without `*` matches anywhere in the URL (`/docs/`); with `*` it must match the whole URL (`https://example.com/blog/*`). Case-insensitive. Pages that don't pass are never fetched; the URLs you list are always converted. | | Only changed pages (`onlyChangedPages`) | Return only pages that are new or changed since the last run of the same input (see below). | | Include images (`includeImages`) | Keep images as `![alt text](image URL)`. Off by default. | | Split into chunks for RAG (`chunkMarkdown`) | Also split each page's Markdown into chunks by heading (see below). | | Max chunk size (characters) (`maxChunkChars`) | No chunk is longer than this (default 2,000). | | Chunk output (`chunkOutput`) | One row per page with a `chunks` array (default), or one row per chunk in the run's `chunks` dataset. | | Max pages per run (`maxResults`) | Cap the total number of pages across all URLs. | ```json { "urls": ["https://docs.python.org/3/tutorial/index.html"], "crawlDepth": 1, "maxPagesPerUrl": 25 } ``` #### Whole-site mode Each line is a site (`https://docs.example.com`), a section of one (`https://docs.example.com/guide`: only pages under `/guide`), or a sitemap URL (read as given). Pages are found **sitemap first**: the `Sitemap:` lines of the site's `robots.txt`, else `/sitemap.xml` or `/sitemap_index.xml`, with sitemap indexes followed. Pages on other sites and links to files are skipped. If the site has no usable sitemap, or it lists nothing in your section, the actor follows the site's own links instead, starting from your URL, with no depth limit (if your URL redirects, the section is taken from where it lands, e.g. `/platform/storage` → `/storage`). `RUN_STATS` says which way each site was read (`discoveredVia`) and why. The page limit counts pages returned. To keep a run bounded when most pages turn out unchanged or broken, at most 5 pages are fetched per page returned (plus 100). ```json { "urls": ["https://docs.apify.com/platform/storage"], "crawlWholeSite": true, "maxPagesPerSite": 500, "excludeUrlPatterns": ["/changelog"], "onlyChangedPages": true } ``` #### Only changed pages The first run returns every page and remembers each one's sitemap `lastmod` date and a hash of its Markdown. Later runs of the **same input** skip a page when its sitemap `lastmod` is unchanged (without fetching it), or, for pages without a date, when its Markdown is identical to last time (it's fetched to compare, but not returned or charged). The memory is kept in a key-value store named `page-to-markdown-changes` in your own Apify account, separately for each URL and set of options, so two different searches never hide each other's pages. Only pages that were actually delivered are remembered: a page cut by your max cost per run comes back next time. Delete that store to start over. #### Chunks for RAG Each page's Markdown is split at its headings: one chunk per section, and a section longer than the max chunk size is split at paragraph breaks, then line breaks (code blocks keep their lines). A heading with no text of its own (a page title right before its first subsection) doesn't become a chunk; it appears in the heading path of the sections under it. Each chunk has: - `headingPath`: the headings it sits under, outermost first, e.g. `["Storage", "Datasets", "Retention"]` - `chunkIndex` (from 0) and `charCount`, plus, in one-row-per-chunk output, `id`, `pageId`, `url`, `startUrl`, `title`, `language`, `chunkCount` and `scrapedAt`. With **one row per page** (the default), the chunks are a `chunks` array on the page's row. With **one row per chunk**, the page rows stay in the default dataset and each chunk is its own row in the run's second dataset, `chunks` (the run's status message and `RUN_STATS.chunksDataset` give its id). A chunk's `id` is its page's `id` plus its index, so it's stable while the page doesn't change. ### Output One item per converted page. Fields a page doesn't have are `null`. ```json { "id": "c9c50a504fb81ab27af5a4d0", "startUrl": "https://docs.python.org/3/library/json.html", "url": "https://docs.python.org/3/library/json.html", "canonicalUrl": "https://docs.python.org/3/library/json.html", "depth": 0, "title": "json — JSON encoder and decoder", "description": "Source code: Lib/json/__init__.py JSON (JavaScript Object Notation), specified by RFC 7159 ...", "language": "en", "markdown": "# `json` — JSON encoder and decoder\n\n---\n\n[JSON (JavaScript Object Notation)](https://json.org), specified by ...", "markdownTruncated": false, "wordCount": 3757, "links": [{"url": "https://json.org", "text": "JSON (JavaScript Object Notation)"}], "discoveredVia": "input", "lastmod": null, "contentHash": "5e0f3c1d2a9b8e7f", "chunkCount": null, "chunks": null, "scrapedAt": "2026-09-24T12:00:00Z" } ``` - `id` is derived from the page URL and is the same on every run: use it to deduplicate or as your vector-store key. - `url` is the address after redirects; `startUrl` is the URL you listed that led to it; `depth` is how many links away from it the page is (`null` for pages found in a sitemap); `discoveredVia` is `input`, `link` or `sitemap`, and `lastmod` the date the sitemap gives, if any. - `contentHash` is a hash of the Markdown: the same text gives the same hash on every run. - `chunkCount` and `chunks` are filled when chunking is on (see above). - `links` lists the links inside the extracted content, made absolute, each once. - `markdown` is capped at 500,000 characters (`markdownTruncated` says when that happened). Pages larger than 5 MB of HTML are skipped. - When the content has no heading of its own, the page title is added as the first line (`# Title`), so every chunk knows what document it came from. ### Run it on a schedule, or from your own code 1. Save your input as a **task** and add it to a **schedule** (Console → Schedules), as in *Monitor a docs site* above. 2. Collect results: download the dataset as JSON, CSV or Excel; fetch the latest run's results from the API (`GET https://api.apify.com/v2/actor-tasks//runs/last/dataset/items?status=SUCCEEDED&format=csv`, with your API token); let a webhook tell your system when a run succeeds; or connect it to Make, Zapier or n8n through Apify's integrations. Each page's `id` stays the same from run to run: use it as the key in your vector store, so a changed page replaces its old version instead of being added twice. #### Can I use Page to Markdown from an AI agent (MCP)? Yes, through Apify's MCP server: add `https://mcp.apify.com?tools=humble-echidna/page-to-markdown` to your MCP client (or let the agent find it with the server's actor search). The agent passes the pages it wants to read, e.g. `{"urls": ["https://docs.example.com/guide/install"]}`, and gets each page's Markdown back; set `maxResults` to keep the answer small. ### Who it's for Teams that keep an LLM, RAG index or AI assistant supplied with documentation they're allowed to use (their own product docs, a vendor's public docs, an internal knowledge base on a public site). The recurring job: convert the docs site once, then re-run on a schedule and get only the pages that changed. ### Why this one? - **Clean output, not the whole page.** A readability-style extractor keeps the article and drops the site chrome around it, so you don't pay tokens (or embedding space) for menus and footers. - **Honest about what it can't do.** Pages that only render with JavaScript (single-page apps) are **reported as "needs JavaScript" and not charged**, instead of being returned as an empty or boilerplate-only result. The same goes for PDFs and other files, missing pages, and sites that refuse the request. - **Polite by design.** It checks each site's `robots.txt` before every request and honours its `Crawl-delay` (see below). - **Fast and cheap.** Plain HTTP, no browser, so there's no page rendering to wait for and no browser to pay for. Most of a run's time is the sites' own response time and the politeness limits below. - **Whole sites, sitemap first.** In whole-site mode it reads the site's sitemap (the same way search engines find it), so it gets every listed page without guessing, and falls back to following links when there is no sitemap. - **Only what changed.** Re-run it on a schedule with "Only changed pages" and you get (and pay for) only new and changed pages: sitemap `lastmod` dates skip unchanged pages without even fetching them, and a content hash catches the rest. - **Chunks ready for your vector store**, split by heading with a size cap, each with its heading path. Chunks are free: you pay per page. - **Reliable.** One failing URL never affects the others in your run. The run log and the `RUN_STATS` record say exactly which URL had a problem and why. ### robots.txt and politeness - Before fetching any page, including every redirect hop and every page found by following links, it checks that site's `robots.txt` and skips pages it disallows. `robots.txt` is read once per origin (`https://` + host) per run, not once per page. Skipped pages are listed in the log and in `RUN_STATS` (`blockedByRobots`), and are never charged. - It identifies itself honestly, with the user agent `HumbleEchidnaApify/1.0 (+https://apify.com/humble-echidna)`. Site owners can allow or block it by name: `robots.txt` rules written for `User-agent: HumbleEchidnaApify` apply to it (otherwise the `*` rules do). - **It also honours opt-outs from AI use.** Its output is meant for LLMs, so when a site's `robots.txt` disallows a page for any of the well-known AI crawlers (GPTBot, ChatGPT-User, OAI-SearchBot, CCBot, ClaudeBot, Claude-User, anthropic-ai, Claude-Web, Google-Extended, PerplexityBot, Applebot-Extended, Bytespider, cohere-ai, Meta-ExternalAgent, Amazonbot, Diffbot, Omgilibot), the page isn't fetched. It's reported as "site opts out of AI use" (`optedOutOfAI` in `RUN_STATS`) and not charged. - If a site's `robots.txt` can't be read because the server errors, the site is treated as disallowing everything, as the robots.txt standard (RFC 9309) requires. A site without a `robots.txt` allows everything. - It honours `Crawl-delay` (retries included) and `Retry-After`, and it never makes more than 2 requests at a time to one site. A site asking for more than 30 seconds between requests, or before a retry, is reported as such rather than fetched slowly. - It doesn't log in, and doesn't try to get around paywalls, logins or bot protection: a site that answers "403 Forbidden" is reported as refusing the request. ### Limits - No browser: pages whose content is built by JavaScript are reported as "needs JavaScript" and not charged. - HTML pages only: PDFs and other files are reported as "not an HTML page" and not charged. - `markdown` is capped at 500,000 characters per page; pages larger than 5 MB of HTML are skipped. - Only public web pages on the standard ports (80 and 443); no logins, no proxies. - Main-content detection is heuristic: it works best on articles, documentation and blog posts. ### FAQ #### Why did a page come back as "needs JavaScript"? Its content is built in the browser by JavaScript, and this version doesn't run a browser. The page is not charged. Many such sites also publish the same content somewhere static (a docs site, a blog, an RSS feed) that works. #### Why did a page come back as "blocked by robots.txt"? The site's owner has asked crawlers not to fetch it. This actor respects that, and the page is not charged. #### Why did a page come back as "site opts out of AI use"? The site's `robots.txt` disallows that page for AI crawlers (see the list above). Converting it for an LLM is exactly that use, so this actor skips it, and it's not charged. #### Does whole-site mode respect AI opt-outs too? Yes. The sitemaps are fetched with the same checks as the pages: `robots.txt`, and its opt-outs for AI crawlers. A page the site opts out of AI use for is never fetched, even when its sitemap lists it. #### Can I get only new and changed pages since my last run? Yes: turn on **Only changed pages**. See *Monitor a docs site* and *Only changed pages* above. #### Why does it refuse `localhost`, `10.x.x.x`, an internal hostname or a URL with a port? It only fetches public web pages, on the standard web ports (80 for `http://`, 443 for `https://`); a URL with any other port, such as `:8080`, is refused. Every hostname (and every redirect) is resolved first, and a page is refused if any address it resolves to is private, loopback, link-local (cloud metadata) or otherwise not on the public internet; the connection then goes to the address that was checked. These refusals are reported as "not a public web address", not as robots.txt blocks, and aren't charged. A domain that doesn't exist is reported as such. #### Can it convert PDFs to Markdown? Not in this version: PDFs and other files are reported as "not an HTML page" and not charged. Use [PDF, DOCX & XLSX to Text](https://apify.com/humble-echidna/document-to-text) for those. #### The output contains a bit of menu or misses a section. What now? Main-content detection is heuristic and works best on articles, documentation and blog posts. Please open an issue with the URL. #### Something that used to work now fails. Why? Sites change their markup without notice. The run log names the URL and what went wrong, and every other URL in the run is unaffected. Please open an issue with the input you used. #### Is it legal to convert web pages to Markdown? It fetches only the pages you give it (and, if you ask, same-site pages they link to or the site's sitemap lists), as a normal logged-out visitor, and follows each site's `robots.txt`, including its opt-outs for AI crawlers. It doesn't log in or get around any protection. What you do with the content (copyright, the site's terms) is your responsibility, as with any web page you save. ### Related actors | Actor | Use it when | |---|---| | [Article Extractor: Text, Author, Date](https://apify.com/humble-echidna/article-extractor) | You want **news and blog articles**: it finds the newest posts through a site's feed or sitemap and returns author and publication date. Pick Page to Markdown for documentation and other pages, whole sites, and heading-based chunks. | | [PDF, DOCX & XLSX to Text](https://apify.com/humble-echidna/document-to-text) | The docs site links to PDFs, Word or Excel files (this actor reports those as "not an HTML page" and doesn't charge for them). | | [Sitemap URL Extractor](https://apify.com/humble-echidna/sitemap-urls) | You want to see a site's full URL list, with `lastmod` dates, before choosing URL patterns. | ### Feedback and support Found a bug, or need a field or option that isn't here? Open an issue on the **Issues** tab with the input you used. ### Versions Current version: **1.1**. See the Changelog tab for what changed in each version. # Changelog This Actor's version history is a separate document: https://apify.com/humble-echidna/page-to-markdown/changelog.md # Actor input Schema ## `urls` (type: `array`): The pages to convert, one per line, e.g. https://docs.apify.com/storage; a missing https:// is added. With crawlWholeSite, each line is a site (https://docs.example.com), a section of one (https://docs.example.com/guide: only pages under /guide) or a sitemap URL. Pages the site's robots.txt disallows, or whose robots.txt opts out of AI crawlers, are skipped and listed in the log. ## `crawlWholeSite` (type: `boolean`): Default false. When true, convert every page of each site in urls instead of only the page itself: the pages its sitemap lists (found from robots.txt, /sitemap.xml or /sitemap\_index.xml), or, with no usable sitemap, the pages reached by following its same-site links. A URL with a path limits it to pages under that path. Capped by maxPagesPerSite; crawlDepth and maxPagesPerUrl are ignored in this mode. ## `maxPagesPerSite` (type: `integer`): Whole-site mode only (crawlWholeSite true): return at most this many pages for each URL in urls. 1 to 10,000, default 100. Unchanged pages skipped by onlyChangedPages don't count. At most 5 pages are fetched per page returned (plus 100), so a site where almost nothing changed can't run for ever. ## `crawlDepth` (type: `integer`): How far to follow links from each URL: 0 to 2, default 0. 0 converts only the pages you list. 1 also converts the same-site pages they link to, 2 goes one more link further, up to maxPagesPerUrl pages. Links to other sites and to files (PDF, images, ...) are never followed. Ignored when crawlWholeSite is true. ## `maxPagesPerUrl` (type: `integer`): When following links (crawlDepth 1 or 2): fetch at most this many pages for each URL in urls, the URL itself included. 1 to 1,000, default 10. No effect at crawlDepth 0 or with crawlWholeSite. ## `includeUrlPatterns` (type: `array`): When crawling (crawlWholeSite, or crawlDepth 1 or 2), convert only pages whose URL matches one of these patterns, one per line. Without \* a pattern matches anywhere in the URL (e.g. /docs/); with *, it must match the whole URL (e.g. https://example.com/blog/*). Case-insensitive. Pages that don't match are never fetched. The URLs you list are always converted. Default: empty (every page). ## `excludeUrlPatterns` (type: `array`): When crawling (crawlWholeSite, or crawlDepth 1 or 2), never fetch pages whose URL matches one of these patterns, one per line, same syntax as includeUrlPatterns, e.g. /tag/ or ?page=. Default: empty. ## `onlyChangedPages` (type: `boolean`): Default false. When true, return only pages that are new or changed since the last run of this same input. A sitemap page whose lastmod date hasn't changed is skipped without being fetched; any other page is fetched and skipped (not charged) when its Markdown is identical to last time. The memory is kept in a key-value store named page-to-markdown-changes in your account, separately for each URL and set of options. ## `includeImages` (type: `boolean`): Default false. When true, keep images in the Markdown as !\[alt text]\(image URL). Off by default, since images are usually noise for text models. ## `chunkMarkdown` (type: `boolean`): Default false. When true, also split each page's Markdown into chunks: one per heading section, with sections larger than maxChunkChars split at paragraph breaks. Each chunk carries its heading path (the headings it sits under), the page URL, title and its index. Where chunks go: chunkOutput. Chunks are free: you still pay per page. ## `maxChunkChars` (type: `integer`): Only with chunkMarkdown: no chunk is longer than this many characters. 200 to 50,000, default 2000. A longer section is split at paragraph breaks, then at line breaks. ## `chunkOutput` (type: `string`): Only with chunkMarkdown. `page` (the default): each page's chunks go in a chunks array on its row. `chunk`: every chunk is written as its own row to this run's second dataset, named chunks (the default dataset keeps one row per page; chunk rows are not charged). ## `maxResults` (type: `integer`): Stop after this many converted pages in total, e.g. 20. Minimum 1; leave empty (the default) for no limit. The run also stops cleanly at the maximum cost per run you set in the run options, whichever comes first. Set it to keep a run small and quick. ## Actor input object example ```json { "urls": [ "https://docs.apify.com/storage", "https://docs.apify.com/storage/key-value-store" ], "crawlWholeSite": false, "maxPagesPerSite": 100, "crawlDepth": 0, "maxPagesPerUrl": 10, "includeUrlPatterns": [], "excludeUrlPatterns": [], "onlyChangedPages": false, "includeImages": false, "chunkMarkdown": false, "maxChunkChars": 2000, "chunkOutput": "page" } ``` # Actor output Schema ## `results` (type: `string`): No description ## `runStats` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [ "https://docs.apify.com/storage", "https://docs.apify.com/storage/key-value-store" ] }; // Run the Actor and wait for it to finish const run = await client.actor("humble-echidna/page-to-markdown").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [ "https://docs.apify.com/storage", "https://docs.apify.com/storage/key-value-store", ] } # Run the Actor and wait for it to finish run = client.actor("humble-echidna/page-to-markdown").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [ "https://docs.apify.com/storage", "https://docs.apify.com/storage/key-value-store" ] }' | apify call humble-echidna/page-to-markdown --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,humble-echidna/page-to-markdown" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/3Ubpa67JBiKbD3drs/builds/Puo79d3GAXweM6FeR/openapi.json