# PDF to Text & HTML to Markdown for LLMs and RAG (`textokit/document-to-llm-markdown`) Actor

Convert public HTTPS HTML pages and text-based PDFs to Markdown or plain text for LLM and RAG workflows. Get source URLs, PDF page metadata, character counts and truncation warnings. No OCR or JavaScript rendering.

- **URL**: https://apify.com/textokit/document-to-llm-markdown.md
- **Developed by:** [Textokit](https://apify.com/textokit) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 document processeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Document to LLM Markdown

Run it on Apify Store: https://apify.com/textokit/document-to-llm-markdown

Convert public HTTPS articles, documentation pages and text-based PDFs into Markdown or plain text for LLM workflows. Get extracted content, source URLs, character counts and explicit truncation warnings in the default Apify dataset.

Useful for developers building retrieval pipelines, research assistants, document indexing and summarization workflows. No external LLM API or API key is required.

### What it extracts

- HTML: page title and main article content, with navigation, scripts and common page furniture removed. Markdown preserves headings, links, lists and code blocks where extraction supports them.
- PDF: embedded text in page order, PDF title when available, page numbers and per-page character counts. Empty pages remain represented.
- Both: original extracted-content length, returned-content length, truncation status and warnings. Counts use Unicode code points.

PDF output contains page text in both formats; it does not reconstruct rich Markdown layout.

### Input

For one document:

```json
{
  "url": "https://example.com/article",
  "format": "markdown",
  "maxChars": 20000
}
```

For a batch:

```json
{
  "urls": [
    "https://example.com/article",
    "https://example.com/report.pdf"
  ],
  "format": "text",
  "maxChars": 50000
}
```

These URLs illustrate the input shape; replace them with your documents.

| Field | Meaning |
| --- | --- |
| `url` | One public HTTPS document URL. |
| `urls` | Between 1 and 50 public HTTPS document URLs. |
| `format` | `markdown` (default) or `text`. |
| `maxChars` | Between 1 and 200000; default 20000. Applies separately to each document. |

Provide exactly one of `url` or `urls`. Runtime validation enforces this rule for API and Console inputs. URL fragments are removed and identical normalized input URLs are processed once per run. Different URLs that redirect to the same document can still be processed separately.

The character budget covers returned content, including Markdown syntax, and excludes titles and metadata. A PDF shares one budget across all pages. Pages remain in the output after the budget is exhausted, with empty content and truncation metadata when applicable. Truncating output does not reduce PDF processing or page charges.

### Output

Each successfully extracted document produces one dataset item. This illustrative HTML result shows the output shape:

```json
{
  "url": "https://example.com/article",
  "finalUrl": "https://example.com/article",
  "status": "success",
  "documentType": "html",
  "format": "text",
  "title": "Example document",
  "fullTextCharacters": 12,
  "outputCharacters": 12,
  "textTruncated": false,
  "warnings": [],
  "content": "Hello world."
}
```

PDF items replace the document-level `content` field with `pageCount` and a `pages` array. Each page contains `pageNumber`, `content`, `fullTextCharacters`, `outputCharacters` and `textTruncated`. Document-level counts sum page content without adding separators.

Download or extraction failures produce an error item and no custom processing charges. The run continues with the next URL. Storage or charging failures stop the run; previously saved results remain subject to platform retention. Dataset items do not certify that their corresponding charges completed.

### Pricing

Pay per event: **USD 0.005 per successfully extracted document, plus USD 0.001 per processed PDF page.** You pay only for documents that were extracted.

| Document | Total |
| --- | --- |
| One HTML page | USD 0.005 |
| Three-page PDF | USD 0.008 |
| Ten-page PDF | USD 0.015 |

PDF page charges include empty pages and pages whose returned text is truncated. A successful extraction with no embedded text is still billable and includes a warning. Failed downloads and failed extractions are not charged. The pricing shown on the Apify Store page is the one that applies; event names and the charging sequence are described in [docs/PRICING.md](docs/PRICING.md).

### Supported documents and limits

- Public HTTPS destinations on port 443, without URL credentials. Private and special-use IP destinations are rejected. Every redirect is independently checked and its resolved address pinned for connection.
- Static HTML and text-based PDFs; no JavaScript rendering, authentication, cookies, proxy configuration or OCR.
- Maximum download size: 10 MiB per document. Compressed HTTP responses are rejected.
- Up to five redirects. Each HTTP request has a 20-second deadline; DNS resolution is outside that deadline.
- Maximum PDF length: 200 pages; maximum accumulated extracted PDF text: 2000000 Unicode code points. Exceeding either limit fails that document.
- HTML is decoded as UTF-8. Other character encodings can produce incorrect text.
- Main-content detection is heuristic. Short or unusual pages can use cleaned-page fallback content; extraction quality varies with the source.
- PDF reading order depends on embedded text. Complex columns, tables, scanned pages and font encoding can reduce quality.

### Development

Source, tests and the validation record are in [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md). Run `npm ci` and `npm test`; tests use local fixtures only.

### Implementation and official references

`src/main.js` uses the official Apify SDK lifecycle and connects validation, download, extraction, output storage and explicit custom-event charging, with a document-wide Unicode budget and page metadata.

Official Apify documentation used:

- [Actor SDK reference](https://docs.apify.com/sdk/js/reference/class/Actor): `Actor.init`, `Actor.getInput`, `Actor.pushData`, `Actor.charge` and `Actor.exit`.
- [ChargeOptions](https://docs.apify.com/sdk/js/reference/interface/ChargeOptions.md): event names and counts.
- [ChargeResult](https://docs.apify.com/sdk/js/reference/interface/ChargeResult.md): actual charged counts and event-specific spending limits.
- [Pay-per-event pricing](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event): saving results before charging, custom and synthetic events, spending limits and developer revenue.
- [Actor definition](https://docs.apify.com/platform/actors/development/actor-definition/actor-json): Actor metadata and paths relative to `.actor/`.
- [Input schema](https://docs.apify.com/platform/actors/development/actor-definition/input-schema): Console input configuration.

### License

MIT. See [LICENSE](LICENSE).

# Actor input Schema

## `url` (type: `string`):

A public HTTPS HTML or PDF URL. Use this field or the batch field.

## `urls` (type: `array`):

Up to 50 public HTTPS URLs. Duplicate URLs are processed once per run.

## `format` (type: `string`):

HTML supports Markdown structure or plain text. PDF output preserves extracted text by page without reconstructing document layout.

## `maxChars` (type: `integer`):

Unicode code-point budget shared across all PDF pages; excludes title and metadata. Truncating output does not reduce the number of PDF pages processed or billed.

## Actor input object example

```json
{
  "url": "https://example.com/",
  "format": "markdown",
  "maxChars": 20000
}
```

# Actor output Schema

## `documents` (type: `string`):

Extraction results in the default dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://example.com/"
};

// Run the Actor and wait for it to finish
const run = await client.actor("textokit/document-to-llm-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "url": "https://example.com/" }

# Run the Actor and wait for it to finish
run = client.actor("textokit/document-to-llm-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://example.com/"
}' |
apify call textokit/document-to-llm-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,textokit/document-to-llm-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Ikpdg7Edgw0hOzCyQ/builds/LsFVIMAKt1Meqqipn/openapi.json