Website to Markdown for AI: Clean Content, RAG Chunks

Feed web pages to ChatGPT, Claude, your RAG pipeline or your AI agent without the junk. Give it a list of pages or a site to crawl, and get back:

• Clean Markdown of the main content: headings, lists, tables, code blocks and links kept; menus, footers, cookie banners, ads and share buttons removed
• Metadata: title, description, author, publish date, language, site name, image, canonical URL
• Word count and approximate token count per page, so you can plan context windows and costs
• Optional RAG chunks: pieces of about N tokens, split on paragraph boundaries with overlap
• Whole sites via the sitemap: optionally queue every page the site lists in its sitemap (robots.txt, sitemap indexes and .gz supported), filtered to the paths you want
• Works on JavaScript apps too: if a page comes back empty, it's opened in a real headless Chrome automatically (React, Vue, Next.js client pages)
• Instant API for AI agents (Standby mode): one HTTP request with a URL, Markdown back in seconds
• Fixed price per page. Same price with or without the browser.

Main content is found with Mozilla Readability, the engine behind Firefox Reader View. Links and images are made absolute, so the Markdown works anywhere.

Who it's for

• RAG and chatbots: index docs, help centers and blogs as clean chunks.
• AI agents: read any page as Markdown with one call.
• Content teams and researchers: archive or analyze articles in a readable format.
• Training and evaluation data: clean text with word counts and dates.

Input

Option	Default	What it does
URLs	–	Pages to convert, or start pages to crawl
Crawl the site	off	Follow links on the same site
Use the site's sitemap	off	Also queue every page listed in the sitemap
Max pages	50	Stop after this many converted pages
Max link depth	3	How far from the start pages to crawl
Only / skip URLs containing	–	Keep the crawl to e.g. /docs/
Use a real browser	Auto	Auto = only when the page needs JavaScript; or always / never
Content	Main content	Or the whole page minus menus and footer
Chunk size, overlap	0, 50	Tokens per RAG chunk (0 = off)
Include plain text, links	off	Extra fields
Respect robots.txt	on	Skip pages the site asks bots not to visit

Output

Real result, shortened:

{
  "url": "https://github.blog/engineering/the-cost-of-saying-yes-has-changed/",
  "status": "ok",
  "title": "The cost of saying yes has changed",
  "author": "Dalia Abuadas",
  "published": "2026-07-17T16:46:47+00:00",
  "language": "en-US",
  "markdown": "The cost of writing code dropped; the cost of owning it didn’t. …",
  "wordCount": 1174,
  "approxTokens": 1877,
  "chunks": [
    { "index": 0, "text": "…", "approxTokens": "…" }
  ]
}

status	meaning
ok	Converted.
unreachable	The page didn't load. Not charged.
no_content	The page had no readable text. Not charged.
not_html	Not a web page (PDF, image…). Not charged.
blocked_by_robots_txt	The site asks bots not to visit. Not charged.
blocked_by_bot_protection	The site showed a bot check. We don't try to get around it. Not charged.

Instant API (Standby mode)

For agents and apps that need one page at a time, skip the run and call the Standby URL shown in the API tab (Standby) with your Apify token:

$curl "https://swiftkit--web-to-markdown.apify.actor/?url=https://docs.apify.com&format=markdown"   -H "Authorization: Bearer YOUR_APIFY_TOKEN"

• format=markdown returns plain Markdown; leave it out for the full JSON row (same fields as above).
• Optional: contentMode=full, renderJs=always|never, chunkSize=500, includeLinks=true.
• POST / with a JSON body like {"url": "https://..."} works too.
• Each converted page is one "Page converted" event. Failed pages return an error and are free.

Use it from your AI agent (MCP)

Claude, Cursor and other MCP clients can call this tool directly through Apify's MCP server. Add it as an MCP server / connector:

{
  "mcpServers": {
    "apify": {
      "url": "https://mcp.apify.com?tools=swiftkit/web-to-markdown",
      "headers": { "Authorization": "Bearer <YOUR_APIFY_TOKEN>" }
    }
  }
}

Then just ask, for example: "Read https://docs.apify.com/platform/actors and summarize the key steps". Each call is billed like a normal run.

Pricing

You pay per page converted, the same for every page size. Failed and blocked pages are free. See the Pricing tab.

Limits, honestly

• Pages are downloaded directly first; the browser is used only when that comes back empty. A page that shows some text without JavaScript but loads more later may still come back partial; set "Use a real browser" to Always for such sites.
• Token counts are approximate (about 4 characters per token). Exact counts depend on your model.
• Pages behind a login, a paywall or a bot check aren't converted.
• You're responsible for how you use the content; respect each site's terms and copyright.

More tools from SwiftKit

• Sitemap Extractor & Broken Link Checker: every URL from a site’s sitemaps, plus 404 and redirect checks
• RSS Feed Reader: news, blogs and podcasts from any feed or website, only new items
• Website Screenshot Tool: bulk full-page and mobile screenshots, PNG/JPEG/PDF
• PDF to Text & Markdown: clean text from PDF links for AI, priced per page

Questions?

Open an issue on the Issues tab.