# Webpage to Markdown for LLMs & RAG (`digital-tools/webpage-to-markdown`) Actor

Convert any web page URL into clean, LLM-ready Markdown. Strips nav, ads and boilerplate, keeps headings, tables and links. Fast raw-HTTP, handles Chinese/Japanese/Korean pages, cheap per page. Ideal for AI agents, RAG and MCP.

- **URL**: https://apify.com/digital-tools/webpage-to-markdown.md
- **Developed by:** [凡 方](https://apify.com/digital-tools) (community)
- **Categories:** AI, Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Webpage to Markdown do?

**Webpage to Markdown** turns any public web page into **clean, LLM-ready Markdown**. Give it a list of URLs and get back the main content with headings, lists, tables and links preserved, and menus, cookie banners, ads, scripts and footers stripped out.

Use it as a **website to Markdown converter**, **HTML to Markdown API** or **URL to text** tool. It is built for **AI agents, RAG pipelines, MCP tools and LLM prompts** where you need the text of a page, not its HTML, and you need it **fast and cheap**.

### Why use it?

- ⚡ **Fast** – plain HTTP fetching (no browser), most pages convert in under a second.
- 🧹 **Clean output** – main-content extraction (Trafilatura) with automatic fallback to full-page conversion for homepages and landing pages.
- 🌏 **CJK & legacy encodings** – correctly decodes GBK, GB2312, Big5 and Shift-JIS pages that break many other converters.
- 📏 **Fits your context window** – optional `maxCharacters` truncation, plus `wordCount` and `characterCount` for every page.
- 💸 **Pay per page** – you only pay for pages that were converted successfully. Failed URLs are free.
- 🤖 **Agent-friendly** – simple input, flat JSON output, works through the Apify API, MCP server, Make, Zapier, n8n and LangChain.

### How to use it

1. Paste one or more URLs into **URLs**.
2. (Optional) switch off **Main content only** to convert the whole page, or turn on **Keep images**.
3. Click **Start**. Results appear in the **Output** tab and can be downloaded as JSON, CSV, Excel or Markdown text.

#### Input example

```json
{
  "startUrls": [{ "url": "https://docs.apify.com/platform/actors" }],
  "mainContentOnly": true,
  "includeLinks": true,
  "maxCharacters": 20000
}
```

#### Output example

```json
{
  "url": "https://docs.apify.com/platform/actors",
  "loadedUrl": "https://docs.apify.com/platform/actors",
  "statusCode": 200,
  "title": "Actors | Platform | Apify Documentation",
  "language": "en",
  "markdown": "# Actors\n\nActors are serverless cloud programs that ...",
  "wordCount": 260,
  "characterCount": 1754,
  "truncated": false,
  "error": null
}
```

### How much does it cost?

This Actor uses **pay-per-event (PPE)** pricing: you pay only for results, and the Apify platform compute it uses is already included.

| Event | Price | Per 1,000 |
|---|---|---|
| Page converted | **$0.001** per page | **$1.00** per 1,000 pages |
| Actor start | $0.0005 per run | – |

URLs that fail to load are **not charged**. For example, 1,000 pages cost about **$1.00**. The Actor start fee is charged once per run (one event per GB of memory; the default memory setting counts as one). You can set a **maximum cost per run** in the run options and the Actor stops cleanly when it is reached. Apify's free plan includes $5 of monthly credit, enough to try it at no cost.

### Use cases

- **RAG and vector databases** – convert documentation, help centers and blogs into Markdown chunks for embeddings.
- **LLM prompts and AI agents** – let ChatGPT, Claude or your own agent read a web page as clean text.
- **Content archiving and migration** – move web pages into Notion, Obsidian, GitHub or a CMS as Markdown.
- **Whole-site conversion** – list all pages with [Sitemap URL Extractor](https://apify.com/Digital-Tools/sitemap-url-extractor), then convert them here.

### Limitations

- Pages are fetched with plain HTTP, so content that is rendered only by JavaScript (some single-page apps) may come back empty. Static sites, docs, blogs, news and Wikipedia work well.
- Pages behind logins or anti-bot walls are not supported. Please only convert pages you are allowed to access.
- Non-HTML text (plain text, Markdown, JSON) is returned as-is. PDFs are not converted – use [Document to Text Extractor](https://apify.com/Digital-Tools/document-to-text) for PDF, Word and PowerPoint files.

### FAQ

**Can I call it from my AI agent?** Yes. Use the Apify MCP server or call `run-sync-get-dataset-items` on the API and read the `markdown` field.

**Is it legal?** The Actor only fetches public pages you point it to and identifies itself with a clear user agent. You are responsible for respecting each website's terms.

### Other tools by Digital-Tools

- [URL Metadata & Open Graph Link Preview Extractor](https://apify.com/Digital-Tools/url-metadata-extractor) – Get title, description, preview image, favicon, canonical URL, Open Graph, Twitter Card, RSS feeds and JSON-LD schema from any list of URLs.
- [Broken Link Checker - Find 404s & Dead Links](https://apify.com/Digital-Tools/broken-link-checker) – Crawl a website and check every internal and external link for 404s, 5xx errors, timeouts and redirects.
- [Chinese Simplified/Traditional Converter & Pinyin Toolkit](https://apify.com/Digital-Tools/chinese-text-toolkit) – Bulk convert Simplified ↔ Traditional Chinese (Taiwan, Hong Kong variants with phrase conversion), generate pinyin with tones or initials, segment Chinese words and extract keywords.
- [Sitemap URL Extractor - Get All URLs from sitemap.xml](https://apify.com/Digital-Tools/sitemap-url-extractor) – Extract every URL from a website's XML sitemaps.
- [RSS & Atom Feed Reader - Parse Any Feed to JSON](https://apify.com/Digital-Tools/rss-feed-reader) – Read RSS, Atom and RDF feeds in bulk and get clean JSON: title, link, date, author, categories, summary, full content, images and podcast enclosures.
- [SSL Certificate & DNS Checker - Bulk Domain Expiry Monitor](https://apify.com/Digital-Tools/ssl-dns-checker) – Check SSL/TLS certificates and DNS records for many domains at once: expiry date, days left, issuer, SANs, TLS version, validity, plus A, AAAA, CNAME, MX, NS, TXT, CAA, SOA, SPF and DMARC.
- [PDF, Word, PowerPoint & Excel to Text Extractor](https://apify.com/Digital-Tools/document-to-text) – Extract text from PDF, DOCX, PPTX and XLSX files by URL.

**Found a bug or need a feature?** Open an issue in the **Issues** tab and I will fix it quickly.

# Actor input Schema

## `startUrls` (type: `array`):

Web page URLs to process. Paste one per line or upload a file.

## `mainContentOnly` (type: `boolean`):

Extract only the main article/body and drop navigation, footers, cookie banners and sidebars. Turn off to convert the whole page.

## `includeLinks` (type: `boolean`):

Keep hyperlinks as Markdown links.

## `includeImages` (type: `boolean`):

Keep images as Markdown image tags.

## `maxCharacters` (type: `integer`):

Truncate Markdown to this many characters (0 = no limit). Useful to fit LLM context windows.

## `maxConcurrency` (type: `integer`):

How many URLs to fetch in parallel.

## `timeoutSecs` (type: `integer`):

Timeout for each HTTP request.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://docs.apify.com/platform/actors"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Markdown"
    }
  ],
  "mainContentOnly": true,
  "includeLinks": true,
  "includeImages": false,
  "maxCharacters": 0,
  "maxConcurrency": 10,
  "timeoutSecs": 20
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset with one item per processed input.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://docs.apify.com/platform/actors"
        },
        {
            "url": "https://en.wikipedia.org/wiki/Markdown"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("digital-tools/webpage-to-markdown").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://docs.apify.com/platform/actors" },
        { "url": "https://en.wikipedia.org/wiki/Markdown" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("digital-tools/webpage-to-markdown").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://docs.apify.com/platform/actors"
    },
    {
      "url": "https://en.wikipedia.org/wiki/Markdown"
    }
  ]
}' |
apify call digital-tools/webpage-to-markdown --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,digital-tools/webpage-to-markdown"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/jGDDp6eNHlRIS6v0z/builds/3WloBVNT75fEY4sKm/openapi.json