# PDF to Markdown & RAG Chunks — OCR, Tables, Page Citations (`opendatafrontierlab/pdf-to-markdown-rag`) Actor

Convert PDF URLs into clean LLM-ready Markdown (headings, tables) or heading-aware RAG chunks with page numbers and citations. Automatic OCR for scanned pages. No external AI APIs or token costs. Great for RFPs, regulations, reports and manuals. $1.50 per 1,000 pages.

- **URL**: https://apify.com/opendatafrontierlab/pdf-to-markdown-rag.md
- **Developed by:** [OpenDataFrontierLabs](https://apify.com/opendatafrontierlab) (community)
- **Categories:** AI, Developer tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## PDF to Markdown & RAG Chunks: PDF Parser with OCR, Tables and Page Citations

**Paste PDF links and get back clean, LLM-ready Markdown**, with headings, lists and tables preserved, or **heading-aware RAG chunks with page numbers and ready-made citations** that go straight into your vector database. Scanned pages are OCR'd automatically. Everything runs inside the Actor: no OpenAI or other external AI APIs, no per-token costs, and your documents never leave Apify.

**Price:** $1.50 per 1,000 pages + $0.003 per document. Scanned pages that need OCR cost $0.004 extra. Failed or non-PDF URLs are free.

**Open source:** the source code is on GitHub at [opendatafrontierlab/pdf-to-markdown-rag](https://github.com/opendatafrontierlab/pdf-to-markdown-rag) (AGPL-3.0).

**Tutorials & examples:** [Turn government PDFs into RAG-ready chunks with page citations](https://github.com/opendatafrontierlab/awesome-public-data-actors/blob/main/tutorials/pdf-rag-chunks.md), with a runnable Python example, on GitHub.

### Three output modes

| `outputMode` | You get | Best for |
|---|---|---|
| `chunks` (default) | One item per chunk with `headingPath`, `pageStart`/`pageEnd`, `approxTokens` and `citation` | RAG, vector DBs (Pinecone, Qdrant, pgvector, Weaviate), AI agents |
| `pages` | One item per page | Page-accurate citations, page-level search, QA |
| `document` | One item per PDF with full Markdown, table of contents and metadata | Summarization, archiving, fine-tuning datasets |

The full Markdown file of every document is always saved to the run's key-value store as well (`markdownUrl`).

### Why this PDF parser

- **Built for long, structured documents:** government reports, RFP/RFQ packages, Federal Register rules, standards (NIST, ISO-style), court and agency filings, manuals, annual reports and academic papers.
- **Layout-aware extraction:** it understands headings and multi-column reading order and renders tables as Markdown or HTML (`tableFormat`). Repeating page headers, footers and page numbers are removed (`removeHeadersFooters`).
- **Real citations:** every chunk carries its section path (e.g. `["Rules and Regulations", "SUPPLEMENTARY INFORMATION:"]`), its page range and a human-readable `citation` such as *"The NIST Cybersecurity Framework (CSF) 2.0, p. 8"*.
- **Built-in OCR:** `ocrMode: "auto"` OCRs only the pages that have no text layer, so you pay for OCR only when it's needed.
- **Change detection:** each document gets a SHA-256 fingerprint, so you can re-run on a schedule and re-index only the documents that changed.
- **Private and predictable:** no LLM calls, no token bills and no data sent to third parties.

### Who uses it

- **AI and RAG developers** building chatbots or agents over policies, regulations, codes, manuals or research papers.
- **Bid and proposal teams** turning 200-page solicitation PDFs into searchable sections (pairs with our [Government Bids & RFP Scraper](https://apify.com/opendatafrontierlab/us-public-bids)).
- **Compliance, policy and legal researchers** extracting rules, standards and guidance with exact page references.
- **Data teams** converting public PDF archives into Markdown for search, analytics or LLM training data.

### How to use it

1. Click **Try for free** and paste one or more public PDF URLs (`startUrls`).
2. Choose `outputMode`: `chunks` for RAG, `pages` or `document`.
3. Click **Start**. A 30-page digital PDF typically converts in seconds. Download the results as JSON, CSV or Excel, or fetch them through the API.

### Input example

```json
{ "startUrls": [{ "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf" }], "outputMode": "chunks", "chunkSizeChars": 2000, "chunkOverlapChars": 200, "ocrMode": "auto" }
```

| Field | What it does |
|---|---|
| `startUrls` | Public http(s) PDF links |
| `outputMode` | `chunks`, `pages` or `document` |
| `chunkSizeChars` / `chunkOverlapChars` | Chunk size target (~4 characters ≈ 1 token) and overlap for long sections |
| `ocrMode` | `auto` (OCR only scanned pages), `off` or `force` |
| `tableFormat` | `markdown`, or `html` for tables with merged cells |
| `maxPagesPerDocument`, `maxFileSizeMb` | Limits; skipped pages are not charged |
| `removeHeadersFooters` | Strips running headers, footers and page numbers (default on) |

### Output example (real chunk from NIST CSF 2.0, text shortened)

```json
{
  "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf",
  "docId": "8eeaabe261e4",
  "processedAt": "2026-10-29T18:39:50+00:00",
  "finalUrl": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf",
  "fileName": "NIST.CSWP.29.pdf",
  "title": "The NIST Cybersecurity Framework (CSF) 2.0",
  "author": "National Institute of Standards and Technology",
  "pageCount": 32,
  "pagesProcessed": 32,
  "ocrPages": 0,
  "sha256": "3c31f46fee98cac0c4323453e5109291a213b4de7fef8c058af9bf67f717433c",
  "markdownUrl": "https://api.apify.com/v2/key-value-stores/<storeId>/records/doc-8eeaabe261e4.md",
  "text": "# **2. Introduction to the CSF Core**\n\nAppendix A is the CSF Core \u2014 a set of cybersecurity outcomes arranged by Function, then Category, and finally Subcategory, as depicted in Fig. 1. These outcomes are not a checklist of actions to perform; specific actions taken to achieve an outcome will vary by organization and use case, as will the individual responsible for those actions. Additionally, the order and size of Functions, Categories, and Subcategories in the Core does not imply the sequence or importance of achieving them. The structure of the Core is intended to resonate most with those ch \u2026",
  "headingPath": [
    "2. Introduction to the CSF Core"
  ],
  "pageStart": 8,
  "pageEnd": 8,
  "chunkIndex": 13,
  "charCount": 1780,
  "approxTokens": 445,
  "chunkCount": 54,
  "citation": "The NIST Cybersecurity Framework (CSF) 2.0, p. 8"
}
```

### Pricing and cost examples (pay per event)

| Event | Price |
|---|---|
| Document converted | $0.003 |
| Page converted | $0.0015 |
| Scanned page OCR (in addition to the page price) | $0.004 |
| Actor start | $0.00005 per GB of memory |

| Scenario | Cost |
|---|---|
| 30-page digital report | ~$0.05 |
| 200-page RFP package | ~$0.30 |
| 1,000 ten-page PDFs (10,000 pages) | ~$18 |
| 10-page scanned document (all OCR) | ~$0.06 |

Failed downloads, password-protected files and non-PDF URLs are not charged. Platform compute is included in the price.

### Integrations

- **API and SDKs:** use the [Apify API](https://docs.apify.com/api/v2) or `apify-client` for Python/JavaScript. `run-sync-get-dataset-items` returns chunks in a single HTTP call.
- **LangChain and LlamaIndex:** load the results with the Apify dataset loaders and embed them directly. Each chunk's `citation` and `pageStart` fit into metadata.
- **AI agents (MCP):** connect Claude, Cursor or any MCP client through the [Apify MCP server](https://mcp.apify.com) (`opendatafrontierlab/pdf-to-markdown-rag`), so your agent can read any PDF link it finds.
- **Zapier, Make, n8n:** e.g. "new PDF link in a Google Sheet → convert → push chunks to your vector DB".

Python example:

```python
from apify_client import ApifyClient
client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("opendatafrontierlab/pdf-to-markdown-rag").call(run_input={
    "startUrls": [{"url": "https://www.govinfo.gov/content/pkg/FR-2026-10-29/pdf/2026-22093.pdf"}],
    "outputMode": "chunks"})
for chunk in client.dataset(run.default_dataset_id).iterate_items():
    print(chunk["citation"], chunk["approxTokens"], chunk["text"][:80])
```

### FAQ

**Does it work on scanned PDFs?** Yes. With `ocrMode: "auto"`, pages without a text layer are OCR'd with RapidOCR. OCR is optimized for English and Latin-script text, and quality depends on scan quality.

**Does it use GPT or another LLM?** No. Extraction and layout analysis run locally inside the Actor, so there are no token costs and nothing is sent to an AI provider.

**Can I convert private or password-protected PDFs?** Only public http(s) URLs are supported. Password-protected PDFs are reported as errors and not charged.

**How big can a PDF be?** By default up to 300 pages per document (adjustable up to 5,000) and 100 MB per file.

**Which chunk size should I use?** 1,000–2,000 characters (~250–500 tokens) works well for most embedding models. Chunks never cross a heading boundary unless a section is very short.

**Is the source code available?** Yes. It's open source under AGPL-3.0 (built on PyMuPDF / PyMuPDF4LLM); see the Source tab.

### Related Actors

- [Government Bids & RFP Scraper](https://apify.com/opendatafrontierlab/us-public-bids): find the RFPs, then convert their PDFs here.
- [Building Permits Scraper](https://apify.com/opendatafrontierlab/us-building-permits): construction leads from official permit data.

# Changelog

This Actor's version history is a separate document: https://apify.com/opendatafrontierlab/pdf-to-markdown-rag/changelog.md

# Actor input Schema

## `startUrls` (type: `array`):

Public http(s) links to PDF files, e.g. https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf. Non-PDF and failed URLs return an error item and are not charged.

## `outputMode` (type: `string`):

'chunks' = one dataset item per RAG chunk with headingPath, page range and citation (best for vector DBs). 'pages' = one item per page. 'document' = one item per PDF with full Markdown. The full Markdown is always also saved to the key-value store.

## `chunkSizeChars` (type: `integer`):

Target maximum characters per chunk in 'chunks' mode (~4 characters ≈ 1 token; 2000 ≈ 500 tokens).

## `chunkOverlapChars` (type: `integer`):

Characters repeated between consecutive chunks when a long section must be split.

## `ocrMode` (type: `string`):

'auto' OCRs only pages without a text layer (recommended). 'off' never OCRs. 'force' OCRs every page. Each OCR'd page costs $0.004 extra.

## `tableFormat` (type: `string`):

How detected tables are rendered.

## `maxPagesPerDocument` (type: `integer`):

Only the first N pages of each PDF are converted. Pages beyond this are skipped and not charged.

## `maxFileSizeMb` (type: `integer`):

Larger files are skipped.

## `removeHeadersFooters` (type: `boolean`):

Drop running headers, footers and page numbers so chunks contain only body text.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf"
    }
  ],
  "outputMode": "chunks",
  "chunkSizeChars": 2000,
  "chunkOverlapChars": 200,
  "ocrMode": "auto",
  "tableFormat": "markdown",
  "maxPagesPerDocument": 300,
  "maxFileSizeMb": 100,
  "removeHeadersFooters": true
}
```

# Actor output Schema

## `results` (type: `string`):

Markdown documents, pages or RAG chunks, one item per output unit.

## `files` (type: `string`):

Full Markdown file per document (linked from each item's markdownUrl).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("opendatafrontierlab/pdf-to-markdown-rag").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf" }] }

# Run the Actor and wait for it to finish
run = client.actor("opendatafrontierlab/pdf-to-markdown-rag").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://nvlpubs.nist.gov/nistpubs/CSWP/NIST.CSWP.29.pdf"
    }
  ]
}' |
apify call opendatafrontierlab/pdf-to-markdown-rag --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,opendatafrontierlab/pdf-to-markdown-rag"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Mm7iB5lYnIUR3IPhG/builds/iueulsAejwayuu1Ui/openapi.json