# URL to Markdown for LLM & RAG: Pages, PDFs, Podcasts (`4b47/llm-ready-markdown`) Actor

Convert web pages, PDFs, Word/PowerPoint/Excel files, RSS feeds and podcast transcripts into clean Markdown for LLMs and RAG: main content, metadata, exact GPT-4o token counts and heading-aware chunks. HTTP-only, respects robots.txt. $1 per 1,000 pages; failed URLs free.

- **URL**: https://apify.com/4b47/llm-ready-markdown.md
- **Developed by:** [Kacper Gumieniuk](https://apify.com/4b47) (community)
- **Categories:** AI, Developer tools, Agents
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 page converteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**URL to Markdown for LLM & RAG converts web pages, PDF files, Word, PowerPoint and Excel files (DOCX, PPTX, XLSX),
RSS/Atom feeds and podcast transcripts into clean, LLM-ready Markdown.** Each result has the main content only, metadata, an outline, **exact GPT-4o token counts** and
**heading-aware RAG chunks**, ready for a vector database, a LangChain/LlamaIndex loader, a custom GPT knowledge base
or an AI agent's context window.

**Try it now:** press **Start**. The prefilled input converts a Wikipedia article, the 15-page "Attention Is All You
Need" PDF and 3 podcast episodes in under a minute for about one cent. Price: **$1 per 1,000 pages**. **Failed URLs
are never charged.** Apify's free plan ($5 of monthly credit) covers about **4,900 pages a month**.

It is fast and cheap because it uses plain HTTP (no browser) and respects robots.txt.

### Use cases

- Build a **RAG knowledge base** from documentation sites, help centers, research PDFs and blog archives.
- Feed an **AI agent** clean context from any URL (works through the Apify API and the Apify MCP server).
- Turn **podcast episodes into searchable, quotable text** for show notes, summaries, newsletters or fine-tuning data,
  without paying for speech-to-text when the publisher already has a transcript.
- Keep a **competitor or news monitor** that pulls full articles from RSS feeds every day.
- Convert **reports, papers and whitepapers (PDF)** into Markdown for summarization or extraction.
- Read **Word, PowerPoint and Excel files** in n8n Cloud, Make or Zapier, where you cannot install Python converters:
  one HTTP call with the file's URL returns Markdown your AI step can use.

### What it converts

- **Web pages → Markdown:** keeps the main article content and strips navigation, headers, footers, cookie banners,
  sidebars and link farms. Headings, lists, tables (GFM), code blocks with language, quotes and links are preserved.
  Metadata comes from Open Graph, JSON-LD and meta tags: title, description, author, published date, language, site
  name, canonical URL and image.
- **PDF → Markdown:** reads the PDF text layer, turns larger fonts into headings, joins wrapped lines into paragraphs
  (with de-hyphenation), drops running headers, footers and page numbers, and marks pages with `<!-- page N -->` for
  citations.
- **Word, PowerPoint, Excel → Markdown:** DOCX headings, bold/italic, links, bulleted and numbered lists and tables;
  PPTX one `## Slide N: title` section per slide with bullets and speaker notes; XLSX one section per visible sheet as
  a Markdown table (first row = header, up to 5,000 rows x 50 columns per sheet). Old binary .doc/.ppt/.xls are not
  supported (save them as the newer format).
- **Podcast transcripts → Markdown:** give a podcast RSS feed. Episodes that publish a transcript
  (`<podcast:transcript>`, Podcasting 2.0: Buzzsprout, Fireside, Podnews, many others) become clean Markdown with
  **speaker names and \[hh:mm:ss] timestamps**.
- **RSS/Atom feeds → full articles:** a blog or news feed expands into the full Markdown of each linked article, not
  the truncated feed summary.
- **Caption files:** direct `.vtt` / `.srt` / Podcasting 2.0 JSON links → timestamped Markdown.
- **Also:** Markdown, plain text and JSON URLs. Sites that serve Markdown natively are used as-is.
- **For LLMs and RAG:** exact token counts (o200k_base, the GPT-4o / GPT-4.1 / o-series tokenizer), chunks of a
  configurable token size cut at heading and paragraph boundaries, each with its heading path (`Guide > Setup >
  Install`) and token count, and optional truncation to a token budget.

### How to use URL to Markdown

1. Paste URLs into **URLs**, one per line: web pages, PDFs, DOCX/PPTX/XLSX files, RSS/Atom feeds or transcript files.
2. Pick a **chunk size** for RAG (default 1,000 tokens; 0 = no chunks).
3. For feeds, set **Max items per feed** (prefilled with 3 for a quick test).
4. Press **Start**. Each document becomes one dataset item; export it as JSON, JSONL, CSV or Excel, or read it with the
   API.

### Input

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
    "https://arxiv.org/pdf/1706.03762",
    "https://feeds.buzzsprout.com/1004689.rss",
    "https://podnews.net/audio/podnews261102.mp3.vtt"
  ],
  "chunkTokens": 800,
  "maxFeedItems": 5
}
```

| Field | Default | What it does |
| --- | --- | --- |
| `urls` | required | Web pages, PDFs, Word/PowerPoint/Excel files, RSS/Atom feeds, transcript/caption files. Up to 5,000 per run. |
| `chunkTokens` | 1000 | Chunk size in tokens; 0 turns chunking off. |
| `contentMode` | `main` | `main` = article content, `full` = whole page body. |
| `feedItems` | `auto` | `auto` = podcast transcripts if the feed has them, otherwise linked articles; `transcripts`, `articles`, or `feed` (the feed itself as one document). |
| `maxFeedItems` | 10 | Items taken from each feed (newest first). |
| `transcriptLanguage` | (none) | Preferred transcript language, e.g. `en`. |
| `timestamps` | true | `[hh:mm:ss]` at the start of transcript paragraphs. |
| `maxTokens` | 0 | Truncate each document to about this many tokens (0 = no limit). |
| `keepLinks` / `keepImages` / `includeLinkList` | true / false / false | Link and image handling. |
| `maxPdfPages` | 200 | Pages converted per PDF (you pay per page). |
| `maxConcurrency` | 5 | Parallel downloads (one site is always hit at most every 250 ms). |

### Output

One dataset item per document. Example (podcast episode, shortened):

```json
{
  "url": "https://podnews.net/audio/podnews261030.mp3.vtt",
  "type": "podcast-transcript",
  "title": "Who’s buying PodcastOne?",
  "author": "Podnews LLC",
  "publishedAt": "2026-10-30T09:15:44.000Z",
  "language": "en-gb",
  "markdown": "# Who’s buying PodcastOne?\n\n- Show: Podnews Daily\n- Duration: 00:03:56\n...\n\n## Transcript\n\n**[00:00:00]** From our free daily newsletter, the latest from podnews.net ...",
  "wordCount": 642,
  "tokens": 878,
  "truncated": false,
  "outline": [{ "level": 1, "text": "Who’s buying PodcastOne?" }, { "level": 2, "text": "Transcript" }],
  "chunks": [{ "index": 0, "heading": "Who’s buying PodcastOne? > Transcript", "tokens": 790, "markdown": "..." }],
  "podcast": { "show": "Podnews Daily", "episode": "Who’s buying PodcastOne?", "durationSeconds": 236, "transcriptFormat": "vtt" },
  "httpStatus": 200,
  "fetchedAt": "2026-11-02T20:01:43.878Z"
}
```

`type` is one of `web-page`, `pdf`, `docx`, `pptx`, `xlsx`, `podcast-transcript`, `captions`, `feed`, `markdown`,
`text`, `json`. PDFs add `pdf: { pages, pagesConverted, producer }`; presentations and workbooks add
`office: { slides }` or `office: { sheets }`. A URL that cannot be converted produces an **uncharged** error item that
says what to do:

```json
{ "url": "https://httpbin.org/status/404", "type": "error", "error": { "code": "target_http_404", "message": "httpbin.org answered HTTP 404; not charged" } }
```

The dataset has three views: **Overview** (type, title, tokens, words), **Markdown**, and **Podcast episodes**.

### How much does it cost to convert URLs to Markdown?

| Event | Price | Example |
| --- | --- | --- |
| Page converted (web page, feed, caption, text file; a whole DOCX, PPTX or XLSX file) | **$0.001** | 1,000 pages = **$1.00** |
| PDF page converted | $0.0003 | a 15-page paper = $0.0045 |
| Podcast transcript converted (one episode) | $0.002 | 100 episodes = $0.20 |
| Actor start (per run) | $0.001 | |

**Failed URLs cost nothing:** 404s, robots.txt blocks, timeouts, unsupported files and empty pages. Platform usage is
included. Set a **maximum cost per run** in the run options and the actor stops when it is reached.

### FAQ

#### How is it different from RAG Web Browser or Website Content Crawler?

RAG Web Browser searches Google and renders pages in a browser; Website Content Crawler crawls whole sites. This actor
does no search, no crawling and no JavaScript rendering: it converts the URLs and feeds you already have, so it is
faster and cheaper per document, and it adds PDFs, podcast transcripts, token counts and RAG chunks.

#### Does it run JavaScript?

No. Pages that render their text in the browser (some single-page apps) return an uncharged `empty_content` error;
use Website Content Crawler for those.

#### Does it OCR scanned PDFs?

No, only PDFs with a text layer. Scanned PDFs return an uncharged `empty_content` error; use
[Image to Text OCR](https://apify.com/4b47/image-pdf-ocr) for them.

#### Which tokenizer is used?

`o200k_base` (GPT-4o, GPT-4.1, o1/o3/o4). Counts for Claude or Llama models differ by a few percent.

#### How big can documents be?

Downloads up to 25 MB. Markdown is capped at 400,000 characters per document (use `maxTokens` for a smaller budget).

#### Is YouTube supported?

No. YouTube's Terms of Service forbid automated access and its robots.txt disallows the caption endpoints.

#### Can I use it from an AI agent?

Yes: through the Apify MCP server, the REST API (`run-sync-get-dataset-items`), or the LangChain / LlamaIndex Apify
integrations reading the dataset.

### Responsible use

robots.txt is respected on every request and redirect (RFC 9309). Sites can opt out with
`User-agent: LLMReadyMarkdown`. The actor identifies itself in its user agent and spaces requests to the same site. No
logins, no paywall bypass, no CAPTCHA solving, no proxy rotation to evade blocks. You are responsible for how you use
converted content (copyright, terms of the source site).

### Related actors

- [Image to Text OCR](https://apify.com/4b47/image-pdf-ocr): scanned PDFs and images to text, 80+ languages.
- [Career Site Jobs Scraper](https://apify.com/4b47/career-site-jobs): job descriptions as Markdown from
  Greenhouse, Lever, Workday and more.
- [Email Domain Validator](https://apify.com/4b47/domain-trust-checker): MX, disposable, SPF/DMARC and
  domain age checks.

# Actor input Schema

## `urls` (type: `array`):

Web pages, PDF files, Word/PowerPoint/Excel files (DOCX, PPTX, XLSX), RSS/Atom feeds (blogs or podcasts), or direct transcript/caption files (.vtt, .srt, Podcasting 2.0 JSON). One per line, up to 5,000. A feed expands into its items (see Feed items).

## `chunkTokens` (type: `integer`):

Split each document into chunks of about this many tokens, cut at heading and paragraph boundaries, each with its heading path. 0 = no chunks.

## `contentMode` (type: `string`):

Main content strips navigation, headers, footers, cookie banners and sidebars. Whole page keeps everything in the body.

## `feedItems` (type: `string`):

What to do with an RSS/Atom feed. Auto: podcast episode transcripts if the feed publishes them (<podcast:transcript>), otherwise the full article behind each item link. Feed only: one Markdown document of the feed itself.

## `maxFeedItems` (type: `integer`):

Newest items first, as listed in the feed.

## `transcriptLanguage` (type: `string`):

Language code such as en or de, used when an episode offers transcripts in several languages.

## `timestamps` (type: `boolean`):

Start each transcript paragraph with \[hh:mm:ss].

## `maxTokens` (type: `integer`):

Truncate long documents at a paragraph boundary to about this many tokens. 0 = no limit (documents are capped at 400,000 characters).

## `keepLinks` (type: `boolean`):

Keep [text](url) links in the Markdown. Turn off for the most compact text.

## `keepImages` (type: `boolean`):

Keep images as ![alt](src).

## `includeLinkList` (type: `boolean`):

Add a links array (url, text) of the links found in the content of web pages.

## `maxPdfPages` (type: `integer`):

Convert at most this many pages of each PDF (you pay per converted page).

## `maxConcurrency` (type: `integer`):

Documents fetched in parallel. Requests to one site are always spaced at least 250 ms apart.

## Actor input object example

```json
{
  "urls": [
    "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
    "https://arxiv.org/pdf/1706.03762",
    "https://feeds.buzzsprout.com/1004689.rss"
  ],
  "chunkTokens": 1000,
  "contentMode": "main",
  "feedItems": "auto",
  "maxFeedItems": 3,
  "timestamps": true,
  "maxTokens": 0,
  "keepLinks": true,
  "keepImages": false,
  "includeLinkList": false,
  "maxPdfPages": 200,
  "maxConcurrency": 5
}
```

# Actor output Schema

## `results` (type: `string`):

One item per converted document (page, PDF, feed item or podcast episode). Failed URLs have an error field and are not charged.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
        "https://arxiv.org/pdf/1706.03762",
        "https://feeds.buzzsprout.com/1004689.rss"
    ],
    "maxFeedItems": 3
};

// Run the Actor and wait for it to finish
const run = await client.actor("4b47/llm-ready-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": [
        "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
        "https://arxiv.org/pdf/1706.03762",
        "https://feeds.buzzsprout.com/1004689.rss",
    ],
    "maxFeedItems": 3,
}

# Run the Actor and wait for it to finish
run = client.actor("4b47/llm-ready-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://en.wikipedia.org/wiki/Retrieval-augmented_generation",
    "https://arxiv.org/pdf/1706.03762",
    "https://feeds.buzzsprout.com/1004689.rss"
  ],
  "maxFeedItems": 3
}' |
apify call 4b47/llm-ready-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,4b47/llm-ready-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/PqmIgOKpw0mEfebAW/builds/eMnqmj5s1m2iJ1AuV/openapi.json