Website to Markdown Converter for LLM Prompts

📝 What is Website to Markdown?

Reading public HTML, Website to Markdown returns the readable body as Markdown, ready for any LLM, retrieval setup or agent. Tables, code, quotes and images keep their structure, with absolute addresses. Name, summary, language and timestamp come along for each result.

• RAG and vector databases: load documentation, help centers or blogs into Pinecone, Qdrant, pgvector or LangChain as ready-made chunks
• Prompting and AI agents: give Claude, ChatGPT or Cursor the readable text of any page instead of raw HTML
• Fine-tuning and datasets: collect clean, structured text of whole sites with word counts for filtering
• Content and SEO analysis: compare headings, word counts and links across your own and competitors' pages
• Archiving and migration: move pages of an old CMS to Markdown for a static site or a wiki

✨ Why use this Actor?

• ✅ Only the main content: picks <main>, role=main or the longest <article>, and removes nav, header, footer, aside, cookie and newsletter boxes, share bars and hidden elements
• ✅ RAG chunks built in: switch it on and every page is also cut at sections, then paragraphs, into pieces of up to chunkSize characters, each with its heading path and word count
• ✅ Crawl a whole section: follow links on the same host to the depth you set, with include and exclude globs such as https://example.com/docs/**
• ✅ Polite crawling: robots.txt honoured (can be switched off for your own sites), noindex pages skipped and nofollow links not followed in crawls, retries on 429 and 5xx
• ✅ Light on resources: plain HTTP requests and a built-in HTML parser, no browser to start and no proxy costs
• ✅ Pay as you go: 1 USD per 1,000 pages; failed and skipped ones cost nothing

📦 What data can you get with Website to Markdown?

📝 Markdown of the main content	🏷️ Page title	💬 Meta description
🌐 Language	🧭 Headings with level	🔗 Links in the content, with link text
🔢 Word count	🧩 RAG chunks with heading path	↪️ Final URL after redirects
✅ HTTP status code	🕸️ Crawl depth	🕒 Crawl time

🚀 How to use Website to Markdown

Website to Markdown works in a few clicks, no coding needed:

1. Create a free Apify account (the free plan includes 5 USD of monthly credit).
2. Open Website to Markdown and click Try for free.
3. Paste the URLs of the pages you want. To convert a whole section, set a crawl depth of 1 or more and an include glob. Switch on RAG chunks if you feed a vector database.
4. Click Start and wait for the run to finish (most runs take seconds).
5. Download your data as JSON, CSV, Excel, XML or HTML, or send it straight to Google Sheets, Make, Zapier or your AI agent.

⬇️ Input

Add one or more start URLs. With the defaults the Actor converts exactly those pages; everything else is optional.

Single pages or a crawl?

At maxCrawlDepth 0 (the default) only the start URLs are handled. At 1, the pages they link to on the same host follow, and so on, up to maxPages (10 unless changed).

Stay in one section

includeGlobs and excludeGlobs decide which links are followed: ** matches any depth, * a single level. Example: https://docs.example.com/guide/**.

Chunks for RAG

Switch on chunks and set chunkSize (200 to 20,000 characters, default 2,000). Each chunk keeps the headings above it, so your retriever knows where the text came from.

Field	Type	What it does	Default
Start URLs (startUrls)	array	Pages to convert.	[{"url":"https://docs.apify.com/academy/
Crawl depth (maxCrawlDepth)	integer	0 converts only the start URLs; 1 also the pages they link to on the same host, and so on.	0
Maximum pages (maxPages)	integer	Stop after this many pages (each one is a charged event).	10
Include globs (includeGlobs)	array	Follow only links matching one of these, e.g.	[]
Exclude globs (excludeGlobs)	array	Never follow links matching these, e.g.	[]
RAG chunks (chunks)	boolean	Also split each page into chunks with their heading path.	false
Chunk size (characters) (chunkSize)	integer	Maximum characters per chunk.	2000
Honour robots.txt (respectRobotsTxt)	boolean	Skip pages robots.txt disallows.	true
Concurrency (maxConcurrency)	integer	How many pages are fetched in parallel (1 to 20).	5
Maximum retries (maxRetries)	integer	Retries per page on network errors, 429 and 5xx.	2

Input example

{
    "startUrls": [
        {
            "url": "https://docs.apify.com/academy/web-scraping-for-beginners"
        }
    ],
    "maxCrawlDepth": 1,
    "maxPages": 5,
    "includeGlobs": [
        "https://docs.apify.com/academy/**"
    ]
}

⬆️ Output

Each result is one row: Markdown plus metadata. The run summary in the key-value store (OUTPUT) holds the totals, every failed URL with its error in plain language, and the skipped pages (robots.txt, not HTML, noindex, duplicate after a redirect).

Output example

One row of the dataset (local crawl of the Apify Academy, 2026-10-16; Markdown and lists shortened):

{
    "url": "https://docs.apify.com/academy/scraping-basics-javascript/devtools-inspecting",
    "requestedUrl": "https://docs.apify.com/academy/scraping-basics-javascript/devtools-inspecting",
    "statusCode": 200,
    "depth": 1,
    "title": "Inspecting web pages with browser DevTools | Academy | Apify Documentation",
    "description": "Lesson about using the browser tools for developers to inspect and manipulate the structure of a website.",
    "language": "en",
    "markdown": "# Inspecting web pages with browser DevTools\n\n**In this lesson we'll use the browser tools for developers to inspect and manipulate the structure of a website.**\n\n---\n\nA browser is the most complete tool for navigating websites. Scrapers are like automated browsers - and sometimes, they actually are automated browsers. ...",
    "headings": [
        {
            "level": 1,
            "text": "Inspecting web pages with browser DevTools"
        },
        {
            "level": 2,
            "text": "Opening DevTools"
        },
        {
            "level": 2,
            "text": "Selecting an element"
        }
    ],
    "links": [
        {
            "url": "https://developer.chrome.com/docs/devtools",
            "text": "Chrome DevTools"
        },
        {
            "url": "https://developer.apple.com/documentation/safari-developer-tools/web-inspector",
            "text": "Web Inspector"
        }
    ],
    "wordCount": 1240,
    "crawledAt": "2026-10-16T06:16:20.296Z"
}

RAG chunks

With chunks on, each row also has a list of pieces. Two of them for the Crawlee Quick Start page (local run with chunkSize 1,500, 2026-10-16; 16 in total):

{
    "chunks": [
        {
            "index": 1,
            "headings": [
                "Quick Start"
            ],
            "text": "# Quick Start\n\nWith this short tutorial you can start scraping with Crawlee in a minute or two. To learn in-depth how Crawlee works, read the [Introduction](https://crawlee.dev/js/docs/introduction), which is a comprehensive step-by-step guide for creating your first scraper.",
            "wordCount": 40
        },
        {
            "index": 2,
            "headings": [
                "Quick Start",
                "Choose your crawler​"
            ],
            "text": "## Choose your crawler​\n\nCrawlee comes with three main crawler classes: [`CheerioCrawler`](https://crawlee.dev/js/api/cheerio-crawler/class/CheerioCrawler), [`PuppeteerCrawler`](https://crawlee.dev/js/api/puppeteer-crawler/class/PuppeteerCrawler) and [`PlaywrightCrawler`](https://crawlee.dev/js/api/playwright-crawler/class/PlaywrightCrawler). All classes share the same interface for maximum flexibility when switching between them.",
            "wordCount": 27
        }
    ]
}

View the results as a table or JSON in the Output tab and download them as JSON, CSV, Excel, XML, HTML or RSS, all fields or only those you pick.

💰 How much does Website to Markdown cost?

Website to Markdown is priced per event: 1 USD per 1,000 documents (0.001 USD per page), with no monthly rental. Failed or skipped inputs (robots.txt, not HTML, noindex, duplicates) cost nothing.

You get	You pay
100 documents	0.1 USD
1,000 documents	1 USD
10,000 documents	10 USD
100,000 documents	100 USD

💡 Apify's free plan gives you 5 USD of credit every month, enough for about 5,000 documents at no cost. Set a spending limit per run and the Actor stops there.

⚖️ Website to Markdown vs. the alternatives

	Website to Markdown	Browser-based crawlers	Copy and paste
Menus, footers and cookie banners removed	✅	✅ usually	❌ by hand
RAG chunks with heading path	✅	some	❌
Headings, links and word count per page	✅	some	❌
Speed per page	✅ plain HTTP, no browser	slower, a browser per page	minutes
Pages that need JavaScript to show text	❌ not supported	✅	✅
API, schedules and MCP for AI agents	✅	✅	❌

🔌 Integrations, API and MCP

Connect Website to Markdown to your AI agent (MCP)

Give Claude, ChatGPT, Cursor or any MCP client direct access to Website to Markdown through the Apify MCP server. Add this URL to its settings:

https://mcp.apify.com/?actors=bepolabs/web-to-markdown

For Claude Desktop, add it under mcpServers in claude_desktop_config.json:

{
    "mcpServers": {
        "web-to-markdown": {
            "command": "npx",
            "args": [
                "-y",
                "@apify/actors-mcp-server",
                "--actors",
                "bepolabs/web-to-markdown"
            ],
            "env": {
                "APIFY_TOKEN": "<your Apify API token>"
            }
        }
    }
}

Then ask: "Convert the Getting Started pages of docs.example.com to Markdown and summarise how to install the product."

Run it from your code

Python (pip install apify-client):

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("bepolabs/web-to-markdown").call(run_input={"startUrls":[{"url":"https://docs.apify.com/academy/web-scraping-for-beginners"}],"maxCrawlDepth":1,"maxPages":5,"includeGlobs":["https://docs.apify.com/academy/**"]})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

JavaScript (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: '<YOUR_APIFY_TOKEN>' });
const run = await client.actor('bepolabs/web-to-markdown').call({"startUrls":[{"url":"https://docs.apify.com/academy/web-scraping-for-beginners"}],"maxCrawlDepth":1,"maxPages":5,"includeGlobs":["https://docs.apify.com/academy/**"]});
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

No-code integrations

Send results to Google Sheets, Slack, Airtable, HubSpot, Make, Zapier, n8n and more from the Integrations tab, start runs on a schedule, or trigger your own endpoint with a webhook when a run finishes. The Apify API gives you full control over runs and datasets.

❓ FAQ

Does it work on pages built with JavaScript?

It reads the HTML the server sends, which covers documentation sites, blogs, news, wikis and most marketing pages. Anything that shows its text only after JavaScript runs (some web apps) comes back nearly empty; the word count makes it easy to spot.

How do I convert a whole documentation site?

Give the docs home page as the start URL, set maxCrawlDepth to 2 or 3, raise maxPages, and add an include glob such as https://docs.example.com/** so the crawl stays inside that section.

Which chunk size should I use?

1,000 to 2,000 characters suits most embedding models. Chunks break at headings first, then at paragraphs, and never mix two sections.

What happens when a page fails?

The other pages still run. The failed URL and the reason (for example HTTP 404 or a timeout) are listed in the run summary, and it is not charged. The run fails only when no page could be converted.

Does it follow links to other websites?

No. The crawl stays on the host of each start URL. Links to other sites still appear in the links field.

Is it legal to use Website to Markdown?

Converting public web pages is generally legal, but the content belongs to its publishers. The Actor honours robots.txt by default, respects noindex and nofollow, and identifies itself with its own user agent. You are responsible for how you use and republish the text, for copyright and personal data, and for each site's terms; when in doubt, ask the owner.

What if something is missing or does not work?

Open an issue on the Issues tab; we read each one and reply. Feature requests are welcome too.

💬 Your feedback

Tell us what Website to Markdown should add on the Issues tab or leave a review; your notes decide what we work on next.

🔗 Related Actors

Actor	What it does
🕰️ Wayback Machine Scraper	The archive history of any URL or domain: first and last seen, every capture, and the page text of old versions.
🔗 Broken Link Checker	Every link and image on your pages checked: HTTP status, each redirect hop, final URL and response time.
🩺 SEO Audit	Any page to a 0 to 100 SEO score and grade, with the fixes that matter most first: on-page, links and Core Web Vitals.
🗺️ Sitemap URL Extractor	Every URL of a website from its sitemaps, in one deduplicated list with lastmod, changefreq and priority.
🔤 Image OCR	Extract text from images in 29 languages: plain text, clean Markdown and every word with its pixel box and confidence.
📄 PDF to Text and Markdown	PDFs to clean text and Markdown, per page, per document or as RAG chunks, with columns in reading order and headers and footers removed.
🌐 Domain RDAP and DNS	Domains to registrar, creation and expiry dates, age, status, nameservers, DNSSEC and DNS records from official RDAP, in bulk.
⚡ Bulk PageSpeed Insights	Lighthouse scores, lab metrics and real-user Core Web Vitals for any list of URLs, on mobile, desktop or both.