Website to Markdown: HTML to LLM-Ready Markdown avatar

Website to Markdown: HTML to LLM-Ready Markdown

Pricing

from $0.55 / 1,000 page converted to markdowns

Go to Apify Store
Website to Markdown: HTML to LLM-Ready Markdown

Website to Markdown: HTML to LLM-Ready Markdown

Convert any web page to clean Markdown for LLMs and RAG with title, headings, links, token count and chunks. Crawl a site or sitemap. No browser, fast.

Pricing

from $0.55 / 1,000 page converted to markdowns

Rating

0.0

(0)

Developer

Detroit Web Agent - Open Data

Detroit Web Agent - Open Data

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

3 days ago

Last modified

Features

Limited permissions

Share

Website to markdown, done over plain HTTP with no browser: give it page URLs, a sitemap or a whole site and get clean, LLM-ready Markdown back, one dataset row per page. Each row carries the title, description, author, published date, language, the Markdown itself, the heading outline, optional links and images, word count, approximate token count, a content hash for change detection and optional RAG chunks split by heading.

It is built for the jobs people actually search for when they type "html to markdown" or "website to markdown": feeding docs and blogs into ChatGPT, Claude or a vector database, archiving a site as plain text, migrating content to a static site generator, and cutting the token cost of pasting web pages into prompts.

Run in 30 seconds

  1. Open the Input tab. The prefilled input starts at the Python library docs, follows links within that section for 30 pages and also returns 500-token chunks.
  2. Click Start. Thirty pages finish in a few seconds.
  3. Open the Pages table in the Output tab, or export as JSON, CSV or Excel. The markdown field is the converted page.
  4. Replace the URLs with your own, or switch on Discover pages from sitemap.xml to convert a whole site.

What does this Actor do?

For every URL it downloads the HTML, finds the main content (article, documentation body or page text), removes menus, footers, cookie banners and scripts, and converts what is left to Markdown with headings, lists, bold and italic, links, code blocks with their language, quotes and GitHub-style tables. Relative links and image paths are turned into absolute ones, so the Markdown still works when you copy it somewhere else.

Because it never starts a browser, it is fast and cheap. A page converts in well under a second of CPU time and the Actor runs many pages at once (16 by default, up to 50). 100 pages normally finish in far less than a minute on a 1 GB run.

GroupFields
Requesturl, finalUrl, statusCode, status, error, fetchedAt
Page factstitle, description, author, publishedDate, language, siteName, canonicalUrl, ogImage
Contentmarkdown, mainContentOnly, headings
Optionallinks, images, chunks
SizewordCount, approxTokens, contentHash

status is ok for converted pages. Pages that cannot be converted still get a row with a clear reason (blocked, not-found, http-error, non-html, empty, failed) and are not charged.

Main content or the whole page

Main content only is on by default. The Actor reads the page twice: once with a reader-mode extractor and once by looking for the main or article element, and keeps whichever holds the real body, so long reference pages do not lose sections. mainContentOnly in each row tells you whether a main body was found. If a page has no clear body (a home page, a link list), the Actor falls back to the whole page with scripts and styles removed, and the flag is false. Turn the option off to always get the full page.

RAG chunks

Set RAG chunk size (tokens) to a number such as 500 and every row gets a chunks array. Chunks follow your headings first, small sections are merged up to the size, and long sections are split on paragraph breaks. Each chunk has index, heading, headingPath (for example ["Install", "Linux"]), text and tokens. Embed the chunk text and keep the heading path as metadata for much better retrieval.

Crawl a site

  • Follow links on the same site converts pages linked from your start pages, same domain only, down to Max crawl depth.
  • Discover pages from sitemap.xml reads robots.txt and sitemap.xml, including sitemap indexes, and adds those pages.
  • Only URLs matching and Skip URLs matching take glob patterns such as https://example.com/docs/** or **/tag/**.
  • Max pages is a hard stop for both the crawl and your bill.

Pages you list yourself are always fetched. The filters apply to pages found by crawling or sitemaps.

Limits

  • It reads the HTML the server sends. Pages that build their text with JavaScript after loading (some single-page apps) come back as empty. Use a browser-based tool for those.
  • No login, no cookies by default. You can pass your own headers, for example a session cookie, in Extra request headers for pages you are allowed to read.
  • Sites that actively block automated traffic return blocked rows. No proxy is used.
  • PDFs and other non-HTML files return a non-html row.
  • approxTokens is an estimate (characters divided by four), good for budgeting, not exact for any one model.
  • Pages are read up to 6 MB.

Input

FieldWhat it does
urlsPages to convert, one per line. Also the start pages for crawling.
mainContentOnlyKeep only the article or documentation body. Default on.
includeLinksAdd a links array with absolute URL, text and an internal flag.
includeImagesAdd an images array with absolute URL and alt text.
chunkTokens0 for none, or 50 to 8000 for RAG chunks.
frontMatterStart the Markdown with a YAML block (title, source, author, date).
removeSelectorsExtra CSS selectors to delete before converting.
crawlSameDomainFollow links on the same domain.
useSitemapAdd pages from sitemap.xml.
maxPagesStop after this many converted pages. Default 100.
maxCrawlDepthLink hops from the start pages. Default 2.
includeGlobs, excludeGlobsGlob filters for crawled and sitemap URLs.
concurrencyPages fetched in parallel. Default 16.
requestTimeoutSecsPer-page timeout. Default 20.
extraHeadersExtra headers as Name: value.

Example input

{
"urls": ["https://docs.python.org/3/library/json.html"],
"useSitemap": false,
"crawlSameDomain": true,
"maxPages": 50,
"maxCrawlDepth": 1,
"includeGlobs": ["https://docs.python.org/3/library/**"],
"chunkTokens": 500,
"includeLinks": true
}

Output sample

{
"url": "https://en.wikipedia.org/wiki/Markdown",
"finalUrl": "https://en.wikipedia.org/wiki/Markdown",
"statusCode": 200,
"status": "ok",
"error": null,
"fetchedAt": "2026-11-16T09:12:44.120Z",
"title": "Markdown - Wikipedia",
"description": null,
"author": "Contributors to Wikimedia projects",
"publishedDate": "2005-08-09T19:56:00Z",
"language": "en",
"mainContentOnly": true,
"markdown": "## History\n\nMarkdown was created by John Gruber ...",
"headings": [{ "level": 2, "text": "History" }],
"wordCount": 4659,
"approxTokens": 7200,
"contentHash": "9f2c...e1",
"chunks": [{ "index": 0, "heading": "History", "headingPath": ["History"], "text": "## History ...", "tokens": 480 }]
}

The values above are an illustration. Your own run returns the real pages.

Use cases

  • Feed documentation, help centres and blogs into a RAG pipeline or vector store with ready-made chunks.
  • Give an AI agent clean page text instead of raw HTML and cut token use by a large factor.
  • Archive or migrate a site to Markdown for Hugo, Jekyll, Astro, Obsidian or Notion.
  • Detect changed pages by comparing contentHash between scheduled runs.
  • Build a training or evaluation corpus from a list of URLs with title, author, date and language already attached.

Pricing and cost control

Pay per page converted: $0.0009 per page on the free plan, $0.000554 on the Gold plan (Bronze $0.000837, Silver $0.000715), plus a $0.00005 start fee per run. That is $0.90 per 1,000 pages. Blocked, missing, empty and non-HTML pages are not charged. Use Maximum cost per run or Max pages to cap spend; the run stops cleanly at the limit.

Integrations and scheduling

Run it on a schedule and compare contentHash to see what changed, then send results to Google Sheets, Slack, email, Make, Zapier or n8n with the Apify integrations, or call it through the REST API and the Python and JavaScript clients. Send the dataset straight into a vector database with the Apify integrations for Pinecone, Qdrant and similar stores.

Use with the API, n8n, Make, Zapier or AI agents

API (one call, returns the rows as JSON):

$curl -X POST "https://api.apify.com/v2/acts/transparent_meteorite~website-to-markdown/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" -H "Content-Type: application/json" -d '{"urls":["https://example.com"],"chunkTokens":500}'

Python

from apify_client import ApifyClient
client = ApifyClient("YOUR_APIFY_TOKEN")
run = client.actor("transparent_meteorite/website-to-markdown").call(run_input={"urls": ["https://example.com"], "chunkTokens": 500})
for row in client.dataset(run["defaultDatasetId"]).iterate_items():
print(row["url"], row["wordCount"], row["markdown"][:200])
  • n8n, Make, Zapier: use the Apify app's "Run Actor" step with actor transparent_meteorite/website-to-markdown, paste the input JSON, then map markdown to the next step.
  • AI agents (MCP): connect the Apify MCP server at https://mcp.apify.com?tools=transparent_meteorite/website-to-markdown to Claude, Cursor or any MCP client, and your agent can turn any URL into Markdown as a tool call.

The Actor requests public web pages the way a browser does and converts the text it receives. Check the site's terms and robots rules before bulk crawling, keep Parallel requests modest on small sites, and respect copyright and privacy rules that apply to the content you collect. This is an independent tool.

FAQ

Do I need an API key or proxy? No. It uses plain HTTP and no proxy.

Does it run JavaScript? No. That is why it is fast and cheap. Pages that render their text only in the browser return empty.

Why does a page have fewer words than I expected? With Main content only on, menus and sidebars are removed on purpose. Turn it off for the whole page.

What happens when a URL fails? You get a row with status and error, and you are not charged for it.

Can it convert a whole site? Yes: switch on the sitemap option or link following, set Max pages and optionally glob filters.

Which languages work? Any. The language comes from the page and the character set is decoded from the response headers or the page.

Is the Markdown GitHub compatible? Yes. Headings use #, code uses fenced blocks with a language when the page gives one, and tables use pipe syntax.

Support

Found a page that converts badly? Open an issue on the Issues tab with the run URL and the page link.

Changelog

  • 0.1: First private build. URL to Markdown, main-content extraction, headings, links, images, chunks, sitemap and same-domain crawl.