Documentation to Markdown & Text

Turn public documentation and release-note pages into clean article text and Markdown for search indexes, RAG preparation, and content review. Each saved page includes its source URL, same-host canonical URL, title, heading outline, content hash, links, and fetch time.

Quick start

{
  "startUrls": [{"url": "https://requests.readthedocs.io/en/latest/user/quickstart/"}],
  "maxPages": 3,
  "maxDepth": 1,
  "includePrefixes": ["https://requests.readthedocs.io/en/latest/user/"],
  "outputFormat": "markdown"
}

The dataset contains one row per saved article. A real extraction of the Requests Quickstart page begins:

{
  "url": "https://requests.readthedocs.io/en/latest/user/quickstart/",
  "canonical_url": "https://requests.readthedocs.io/en/latest/user/quickstart/",
  "title": "Quickstart",
  "headings": [
    {"level": 1, "text": "Quickstart", "id": "quickstart"},
    {"level": 2, "text": "Make a Request", "id": "make-a-request"}
  ],
  "text": "Quickstart Eager to get started? This page gives a good introduction in how to get started with Requests. …",
  "markdown": "# Quickstart\n\nEager to get started? This page gives a good introduction in how to get started with Requests. …"
}

The complete row also includes a SHA-256 content_hash, article links, and UTC fetched_at. The examples/requests-quickstart-preview.json shows representative field values; its content strings and arrays are shortened for readability.

Inputs and scope

startUrls accepts 1–10 public HTTPS documentation pages. maxPages defaults to 10 and allows 1–1000 pages across all starts; it counts pages considered, including errors and skips. maxDepth defaults to 2, with start pages at depth 0. includePrefixes and excludePrefixes are optional lists of HTTPS URL prefixes. Crawling stays on each start page's host. Query strings and fragments are removed before page deduplication, so pages whose content depends on a query are outside this product's scope.

The Actor reads server-rendered HTML. It selects the main article body, drops navigation, sidebars, footers, and scripts, and keeps headings, article links, and code blocks. It can process a supplied release-note page as an article. outputFormat: "text" saves text while leaving markdown empty; the default markdown saves both. This is a snapshot of the pages fetched during one run.

Pages that require login, browser rendering, OCR, or a private network are unsupported. Markdown is a readable conversion of common article elements, not a byte-for-byte copy of the site's source. Complex tables and interactive examples may lose layout. Sites may block anonymous requests or change their markup.

Results, limits, and billing

The dataset contains only successfully extracted pages. OUTPUT in the default key-value store reports fetched, saved, skipped, errors, status, max_pages, max_depth, queued_remaining, and concise per-page errors. A run with page errors may leave a partial dataset. A page cap or spending limit can stop a run before every discovered link is visited. Inspect OUTPUT to judge coverage.

The billing unit is one saved page. A page that fails extraction or is skipped is not a result row. The Actor stops adding pages when the configured charge limit is reached. The Apify pricing panel shows the current price before a paid run.