# Convert documentation pages to Markdown for RAG

**Use case:** 

Prepare documentation pages for a vector database. The task reads each page and returns its main content as Markdown with headings, tables and code blocks, plus the heading outline for chunking, a content hash that shows when a page changed, and an estimated token count. The example input uses MDN and the Python tutorial.

## Input

```json
{
  "startUrls": [
    {
      "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Overview"
    },
    {
      "url": "https://docs.python.org/3/tutorial/introduction.html"
    }
  ],
  "outputFormats": [
    "markdown"
  ],
  "onlyMainContent": true,
  "maxItems": 10,
  "crawl": false,
  "maxPagesPerCrawl": 50,
  "maxConcurrency": 5
}
```

## Output

```json
{
  "url": {
    "label": "URL",
    "format": "link"
  },
  "title": {
    "label": "Title",
    "format": "text"
  },
  "siteName": {
    "label": "Site name",
    "format": "text"
  },
  "language": {
    "label": "Language",
    "format": "text"
  },
  "wordCount": {
    "label": "Word count",
    "format": "number"
  },
  "tokenEstimate": {
    "label": "Token estimate",
    "format": "number"
  },
  "publishedAt": {
    "label": "Published at",
    "format": "date"
  },
  "description": {
    "label": "Description",
    "format": "text"
  }
}
```

## How it works

### 1. Sign up on Apify or log in

The URL to Markdown API - Web Pages to Clean Text for LLMs Actor is ready to run with the inputs already set for "Convert documentation pages to Markdown for RAG". [Sign up or log in](https://console.apify.com/create-task-from-example/IQU9KbC6bZGybYzaZ) and you'll land in Apify Console with a copy of this exact task.

### 2. Start the "Convert documentation pages to Markdown for RAG" task

The URL to Markdown API - Web Pages to Clean Text for LLMs Actor will start running based on the input automatically. You can also change all the input fields, like start urls, output formats, and more, to match your specific use case.

### 3. Receive the "Convert documentation pages to Markdown for RAG" output

Monitor the progress in real time. You will be notified as soon as your dataset is complete and ready for review. Each result includes data such as URL, title, site name, and more.

### 4. Integrate into your workflow

The final output is delivered in JSON, CSV, or Excel format, ready to be plugged into your workflow. You can also [run the "Convert documentation pages to Markdown for RAG" task via API](https://apify.com/rocketyard/web-page-to-markdown.md).

## FAQ

### How do I detect that a page changed?

Compare the contentHash field between runs of the documentation to Markdown for RAG task. It is a SHA-256 hash of the main text, so an unchanged hash means unchanged content.

### Does it split pages into chunks?

No. The documentation to Markdown for RAG task returns one record per page, with a headings field listing each heading and its level, so you can chunk by section in your own pipeline.

### Why is the text field empty?

The documentation to Markdown for RAG task asks for Markdown only through outputFormats, and formats left out are null. Add "text" or "html" to outputFormats to get them too.

## About this Actor

This example demonstrates how to use [URL to Markdown API - Web Pages to Clean Text for LLMs](https://apify.com/rocketyard/web-page-to-markdown.md) with a specific input configuration. Visit the [Actor detail page](https://apify.com/rocketyard/web-page-to-markdown.md) to learn more, explore other use cases, and run it yourself.


## How to integrate an Actor?

This Task's input is already configured above. Use it as-is rather than inventing a new one.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For full API examples (JavaScript, Python, CLI, MCP, OpenAPI), see this Task's Actor page: https://apify.com/rocketyard/web-page-to-markdown.md

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).