# Web Page to Markdown & Article Extractor for LLMs and RAG (`manageable_gully/page-to-markdown`) Actor

Convert a list of URLs into clean Markdown for LLMs and RAG: main article text only (no nav, ads or cookie banners), title, author, date, word and token counts, and optional chunks. HTML to Markdown and article extraction at $1 per 1,000 pages.

- **URL**: https://apify.com/manageable_gully/page-to-markdown.md
- **Developed by:** [WebIntel Tools](https://apify.com/manageable_gully) (community)
- **Categories:** AI, Developer tools, Agents
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 converted pages

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Web Page to Markdown & Article Extractor: Clean Text and RAG Chunks for LLMs

Turn any list of URLs into **clean Markdown** that is ready for ChatGPT, Claude, embeddings or a RAG index. Only the main content is kept. Navigation, footers, cookie banners, scripts and ads are removed, while headings, lists, tables and links are preserved.

### What you can use it for

- **RAG and vector databases:** turn documentation, help-center and blog URLs into clean Markdown chunks for embeddings.
- **AI agents:** give ChatGPT, Claude or your own agent the readable text of a page instead of raw HTML (fewer tokens, better answers).
- **Article extraction:** collect title, author, date and body text from news and blog articles.
- **Content migration:** convert old HTML pages to Markdown for a static site or a new CMS.
- **Summaries and monitoring:** feed the text to an LLM for summaries; pair it with the [RSS Feed Reader](https://apify.com/manageable_gully/rss-feed-reader) for new articles.

### How to use it

1. Add URLs under **Start URLs**.
2. Choose `article` mode (main content) or `full` mode (whole page body).
3. Optionally set **chunk size** and **overlap** for RAG.
4. Click **Start** and use the `markdown` or `chunks` fields.

### Features

- **Main-content extraction** (Trafilatura). It falls back automatically to the full page body on landing pages that have no clear article.
- **Metadata:** title, description, author, published date, site name, language, word count, estimated tokens.
- **RAG chunks:** optional heading- and paragraph-aware splitting with overlap.
- **Any language,** including right-to-left Hebrew and Arabic pages.
- **Polite:** respects robots.txt, fetches each URL once (no crawling), never logs in, and refuses private or internal addresses.

### Input

```json
{ "startUrls": [{"url": "https://en.wikipedia.org/wiki/Markdown"}], "extractionMode": "article", "chunkSize": 1500, "chunkOverlap": 200 }
```

### Output (one item per URL)

```json
{
  "url": "https://en.wikipedia.org/wiki/Markdown", "statusCode": 200, "title": "Markdown",
  "author": null, "publishedDate": "2026-10-01", "language": "en", "extractionMode": "article",
  "wordCount": 6134, "estimatedTokens": 11000,
  "markdown": "# Markdown\n\n**Markdown** is a lightweight markup language ...",
  "chunks": ["# Markdown\n\n...", "## History\n\n..."]
}
```

### Limits

- Pages are fetched without running JavaScript, which is fast and cheap. Content that only renders client-side (some single-page apps) may come back thin. Check `wordCount`.
- PDFs and other non-HTML files are rejected with an error and not charged. Use the [PDF & Image OCR Extractor](https://apify.com/manageable_gully/pdf-ocr-extractor) Actor for documents.
- Use it only on content you are allowed to process. The Actor doesn't get around paywalls or logins.

### Pricing

Pay per event, no monthly rent: **$0.001 per page converted** (**$1.00 per 1,000**).

| Example | Cost |
|---|---|
| 1,000 pages | $1.00 |
| 10,000 pages | $10.00 |
| 100,000 pages | $100.00 |

Pages that fail, return an error or are disallowed by robots.txt are free.

You can set a **maximum cost per run** in the run options; the Actor stops cleanly when it is reached. New to Apify? The free plan includes $5 of platform credit every month, so you can try this Actor at no cost.

### API, integrations and AI agents (MCP)

Run it from your own code with the [Apify API](https://docs.apify.com/api/v2) and get the results in one call:

```bash
curl -X POST "https://api.apify.com/v2/acts/manageable_gully~page-to-markdown/run-sync-get-dataset-items?token=<YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" -d @input.json
```

```python
from apify_client import ApifyClient
run = ApifyClient("<YOUR_APIFY_TOKEN>").actor("manageable_gully/page-to-markdown").call(run_input={...})
```

- **No-code:** use the Apify app in **n8n, Make or Zapier** ("Run Actor", then "Get dataset items"), or send results to Google Sheets, Slack or a webhook.
- **AI agents (MCP):** add `https://mcp.apify.com?tools=manageable_gully/page-to-markdown` to Claude, Cursor, VS Code or any MCP client and the agent can call this Actor as a tool.
- **Schedules:** run it hourly, daily or weekly from Apify Console.

### FAQ

**Does it crawl a whole website?** No. It converts exactly the URLs you give it, which keeps cost predictable. Get the URL list from a sitemap first.

**How is it different from Website Content Crawler?** It is a lightweight, no-browser converter for known URLs: cheaper and faster, but it doesn't crawl or render JavaScript.

**Can AI agents use it?** Yes. It is available as a tool through the Apify MCP server (`https://mcp.apify.com?tools=manageable_gully/page-to-markdown`) and the Apify API, and it returns structured JSON that agents can read directly.

### More tools from WebIntel Tools

- [PDF & Image OCR Extractor](https://apify.com/manageable_gully/pdf-ocr-extractor): PDFs and images to text, Markdown and tables (incl. Hebrew/Arabic)
- [RSS Feed Reader & Monitor](https://apify.com/manageable_gully/rss-feed-reader): read and monitor RSS, Atom and JSON feeds, only new items
- [Website Screenshot & URL to PDF](https://apify.com/manageable_gully/url-to-pdf-screenshot): full-page screenshots and PDFs of URLs or your own HTML
- [Broken Link Checker](https://apify.com/manageable_gully/broken-link-checker): crawl a site and find every 404, broken image and bad redirect

See [all WebIntel Tools Actors](https://apify.com/manageable_gully).

*Made by WebIntel Tools. This Actor was built and is maintained with AI assistance. Report issues on the Issues tab; they are monitored daily.*

# Actor input Schema

## `startUrls` (type: `array`):

Pages to convert (each URL is fetched once; no crawling).

## `extractionMode` (type: `string`):

'Main content' falls back to full page automatically when no article is found.

## `includeLinks` (type: `boolean`):

Keep hyperlinks as [text](url).

## `includeImages` (type: `boolean`):

Keep images as ![alt](src).

## `includeTables` (type: `boolean`):

Convert HTML tables to Markdown tables. Turn off to drop tables from the output.

## `chunkSize` (type: `integer`):

Also return the Markdown split into heading/paragraph-aware chunks for embeddings/RAG.

## `chunkOverlap` (type: `integer`):

Number of characters repeated between consecutive chunks, so context is not lost at chunk boundaries.

## `includeRawHtml` (type: `boolean`):

Also store the original HTML of each page in the dataset. Makes items much larger.

## `maxConcurrency` (type: `integer`):

Maximum number of pages fetched in parallel.

## `timeoutSecs` (type: `integer`):

How long to wait for each page to download.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://crawlee.dev/python/docs/quick-start"
    }
  ],
  "extractionMode": "article",
  "includeLinks": true,
  "includeImages": false,
  "includeTables": true,
  "chunkSize": 0,
  "chunkOverlap": 200,
  "includeRawHtml": false,
  "maxConcurrency": 10,
  "timeoutSecs": 30
}
```

# Actor output Schema

## `results` (type: `string`):

One item per URL: clean Markdown, title, metadata, word/token counts and optional RAG chunks.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://crawlee.dev/python/docs/quick-start"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("manageable_gully/page-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [{ "url": "https://crawlee.dev/python/docs/quick-start" }] }

# Run the Actor and wait for it to finish
run = client.actor("manageable_gully/page-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://crawlee.dev/python/docs/quick-start"
    }
  ]
}' |
apify call manageable_gully/page-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,manageable_gully/page-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/25X6JuVQ4hxHggppE/builds/DI4zKyCaRXaDeiZZP/openapi.json