Website to Markdown: HTML to LLM-Ready Markdown
Pricing
from $0.55 / 1,000 page converted to markdowns
Website to Markdown: HTML to LLM-Ready Markdown
Convert any web page to clean Markdown for LLMs and RAG with title, headings, links, token count and chunks. Crawl a site or sitemap. No browser, fast.
Pricing
from $0.55 / 1,000 page converted to markdowns
Rating
0.0
(0)
Developer
Detroit Web Agent - Open Data
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Features
Share
Website to markdown, done over plain HTTP with no browser: give it page URLs, a sitemap or a whole site and get clean, LLM-ready Markdown back, one dataset row per page. Each row carries the title, description, author, published date, language, the Markdown itself, the heading outline, optional links and images, word count, approximate token count, a content hash for change detection and optional RAG chunks split by heading.
It is built for the jobs people actually search for when they type "html to markdown" or "website to markdown": feeding docs and blogs into ChatGPT, Claude or a vector database, archiving a site as plain text, migrating content to a static site generator, and cutting the token cost of pasting web pages into prompts.
Run in 30 seconds
- Open the Input tab. The prefilled input starts at the Python library docs, follows links within that section for 30 pages and also returns 500-token chunks.
- Click Start. Thirty pages finish in a few seconds.
- Open the Pages table in the Output tab, or export as JSON, CSV or Excel. The
markdownfield is the converted page. - Replace the URLs with your own, or switch on Discover pages from sitemap.xml to convert a whole site.
What does this Actor do?
For every URL it downloads the HTML, finds the main content (article, documentation body or page text), removes menus, footers, cookie banners and scripts, and converts what is left to Markdown with headings, lists, bold and italic, links, code blocks with their language, quotes and GitHub-style tables. Relative links and image paths are turned into absolute ones, so the Markdown still works when you copy it somewhere else.
Because it never starts a browser, it is fast and cheap. A page converts in well under a second of CPU time and the Actor runs many pages at once (16 by default, up to 50). 100 pages normally finish in far less than a minute on a 1 GB run.
| Group | Fields |
|---|---|
| Request | url, finalUrl, statusCode, status, error, fetchedAt |
| Page facts | title, description, author, publishedDate, language, siteName, canonicalUrl, ogImage |
| Content | markdown, mainContentOnly, headings |
| Optional | links, images, chunks |
| Size | wordCount, approxTokens, contentHash |
status is ok for converted pages. Pages that cannot be converted still get a row with a clear reason (blocked, not-found, http-error, non-html, empty, failed) and are not charged.
Main content or the whole page
Main content only is on by default. The Actor reads the page twice: once with a reader-mode extractor and once by looking for the main or article element, and keeps whichever holds the real body, so long reference pages do not lose sections. mainContentOnly in each row tells you whether a main body was found. If a page has no clear body (a home page, a link list), the Actor falls back to the whole page with scripts and styles removed, and the flag is false. Turn the option off to always get the full page.
RAG chunks
Set RAG chunk size (tokens) to a number such as 500 and every row gets a chunks array. Chunks follow your headings first, small sections are merged up to the size, and long sections are split on paragraph breaks. Each chunk has index, heading, headingPath (for example ["Install", "Linux"]), text and tokens. Embed the chunk text and keep the heading path as metadata for much better retrieval.
Crawl a site
- Follow links on the same site converts pages linked from your start pages, same domain only, down to Max crawl depth.
- Discover pages from sitemap.xml reads
robots.txtandsitemap.xml, including sitemap indexes, and adds those pages. - Only URLs matching and Skip URLs matching take glob patterns such as
https://example.com/docs/**or**/tag/**. - Max pages is a hard stop for both the crawl and your bill.
Pages you list yourself are always fetched. The filters apply to pages found by crawling or sitemaps.
Limits
- It reads the HTML the server sends. Pages that build their text with JavaScript after loading (some single-page apps) come back as
empty. Use a browser-based tool for those. - No login, no cookies by default. You can pass your own headers, for example a session cookie, in Extra request headers for pages you are allowed to read.
- Sites that actively block automated traffic return
blockedrows. No proxy is used. - PDFs and other non-HTML files return a
non-htmlrow. approxTokensis an estimate (characters divided by four), good for budgeting, not exact for any one model.- Pages are read up to 6 MB.
Input
| Field | What it does |
|---|---|
urls | Pages to convert, one per line. Also the start pages for crawling. |
mainContentOnly | Keep only the article or documentation body. Default on. |
includeLinks | Add a links array with absolute URL, text and an internal flag. |
includeImages | Add an images array with absolute URL and alt text. |
chunkTokens | 0 for none, or 50 to 8000 for RAG chunks. |
frontMatter | Start the Markdown with a YAML block (title, source, author, date). |
removeSelectors | Extra CSS selectors to delete before converting. |
crawlSameDomain | Follow links on the same domain. |
useSitemap | Add pages from sitemap.xml. |
maxPages | Stop after this many converted pages. Default 100. |
maxCrawlDepth | Link hops from the start pages. Default 2. |
includeGlobs, excludeGlobs | Glob filters for crawled and sitemap URLs. |
concurrency | Pages fetched in parallel. Default 16. |
requestTimeoutSecs | Per-page timeout. Default 20. |
extraHeaders | Extra headers as Name: value. |
Example input
{"urls": ["https://docs.python.org/3/library/json.html"],"useSitemap": false,"crawlSameDomain": true,"maxPages": 50,"maxCrawlDepth": 1,"includeGlobs": ["https://docs.python.org/3/library/**"],"chunkTokens": 500,"includeLinks": true}
Output sample
{"url": "https://en.wikipedia.org/wiki/Markdown","finalUrl": "https://en.wikipedia.org/wiki/Markdown","statusCode": 200,"status": "ok","error": null,"fetchedAt": "2026-11-16T09:12:44.120Z","title": "Markdown - Wikipedia","description": null,"author": "Contributors to Wikimedia projects","publishedDate": "2005-08-09T19:56:00Z","language": "en","mainContentOnly": true,"markdown": "## History\n\nMarkdown was created by John Gruber ...","headings": [{ "level": 2, "text": "History" }],"wordCount": 4659,"approxTokens": 7200,"contentHash": "9f2c...e1","chunks": [{ "index": 0, "heading": "History", "headingPath": ["History"], "text": "## History ...", "tokens": 480 }]}
The values above are an illustration. Your own run returns the real pages.
Use cases
- Feed documentation, help centres and blogs into a RAG pipeline or vector store with ready-made chunks.
- Give an AI agent clean page text instead of raw HTML and cut token use by a large factor.
- Archive or migrate a site to Markdown for Hugo, Jekyll, Astro, Obsidian or Notion.
- Detect changed pages by comparing
contentHashbetween scheduled runs. - Build a training or evaluation corpus from a list of URLs with title, author, date and language already attached.
Pricing and cost control
Pay per page converted: $0.0009 per page on the free plan, $0.000554 on the Gold plan (Bronze $0.000837, Silver $0.000715), plus a $0.00005 start fee per run. That is $0.90 per 1,000 pages. Blocked, missing, empty and non-HTML pages are not charged. Use Maximum cost per run or Max pages to cap spend; the run stops cleanly at the limit.
Integrations and scheduling
Run it on a schedule and compare contentHash to see what changed, then send results to Google Sheets, Slack, email, Make, Zapier or n8n with the Apify integrations, or call it through the REST API and the Python and JavaScript clients. Send the dataset straight into a vector database with the Apify integrations for Pinecone, Qdrant and similar stores.
Use with the API, n8n, Make, Zapier or AI agents
API (one call, returns the rows as JSON):
$curl -X POST "https://api.apify.com/v2/acts/transparent_meteorite~website-to-markdown/run-sync-get-dataset-items?token=YOUR_APIFY_TOKEN" -H "Content-Type: application/json" -d '{"urls":["https://example.com"],"chunkTokens":500}'
Python
from apify_client import ApifyClientclient = ApifyClient("YOUR_APIFY_TOKEN")run = client.actor("transparent_meteorite/website-to-markdown").call(run_input={"urls": ["https://example.com"], "chunkTokens": 500})for row in client.dataset(run["defaultDatasetId"]).iterate_items():print(row["url"], row["wordCount"], row["markdown"][:200])
- n8n, Make, Zapier: use the Apify app's "Run Actor" step with actor
transparent_meteorite/website-to-markdown, paste the input JSON, then mapmarkdownto the next step. - AI agents (MCP): connect the Apify MCP server at
https://mcp.apify.com?tools=transparent_meteorite/website-to-markdownto Claude, Cursor or any MCP client, and your agent can turn any URL into Markdown as a tool call.
Fair use and legal
The Actor requests public web pages the way a browser does and converts the text it receives. Check the site's terms and robots rules before bulk crawling, keep Parallel requests modest on small sites, and respect copyright and privacy rules that apply to the content you collect. This is an independent tool.
FAQ
Do I need an API key or proxy? No. It uses plain HTTP and no proxy.
Does it run JavaScript? No. That is why it is fast and cheap. Pages that render their text only in the browser return empty.
Why does a page have fewer words than I expected? With Main content only on, menus and sidebars are removed on purpose. Turn it off for the whole page.
What happens when a URL fails? You get a row with status and error, and you are not charged for it.
Can it convert a whole site? Yes: switch on the sitemap option or link following, set Max pages and optionally glob filters.
Which languages work? Any. The language comes from the page and the character set is decoded from the response headers or the page.
Is the Markdown GitHub compatible? Yes. Headings use #, code uses fenced blocks with a language when the page gives one, and tables use pipe syntax.
Support
Found a page that converts badly? Open an issue on the Issues tab with the run URL and the page link.
Related actors
- Sitemap Link Inventory to list every URL of a site before you convert it.
- Website Email Extractor for emails and contact details from the same pages.
- Technical SEO Page Audit to check titles, headings and meta tags on the pages you convert.
Changelog
- 0.1: First private build. URL to Markdown, main-content extraction, headings, links, images, chunks, sitemap and same-domain crawl.