Website to Markdown Converter: clean, LLM-ready content from any web page

Turn web pages into clean, token-efficient Markdown for LLMs, RAG and agents. Only the main content is kept: navigation, headers, footers, sidebars, ads, share bars and cookie banners are stripped. $1.50 per 1,000 pages, and you only pay for pages that succeed.

What you get

• Main content only: Readability-style extraction plus cleanup.
• Token-efficient Markdown: collapsed whitespace, no empty links, absolute URLs, tables kept as GFM tables, code blocks with their language.
• Optional JSON per page: url, title, description, markdown, links, word_count, token_estimate.
• Single pages or crawls: same-domain crawl with maxPages, maxDepth, and include/exclude globs.
• Polite by default: respects robots.txt, per-domain delay, and concurrency and requests-per-minute caps.
• Honest skips: login walls, paywalls and 401/403/402 pages are skipped with a reason, never charged.

Input

Field	Default	Notes
startUrls	required	The pages to convert, or the crawl roots
mode	single	single or crawl
maxPages / maxDepth	10 / 1	Crawl limits; maxPages counts successful pages
sameDomainOnly	true	Follow only the start hosts
includeGlobs / excludeGlobs	[]	e.g. https://example.com/docs/**
outputFormat	both	markdown, json or both
respectRobotsTxt	true
maxConcurrency, maxRequestsPerMinute, sameDomainDelaySecs	5, 60, 1	Politeness settings

Output (one dataset item per page)

{"url":"https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status/402","title":"402 Payment Required - HTTP | MDN",
 "description":"The HTTP 402 Payment Required client error response status code…","markdown":"# 402 Payment Required\n\nThe HTTP **`402 Payment Required`** …",
 "links":["https://developer.mozilla.org/…"],"word_count":343,"token_estimate":907}

A skipped page is recorded as {"url":…,"skipped":true,"reason":"robots_txt_disallow|login_wall|paywall|auth_required_http_403|http_404|fetch_failed: …"} and is not charged.

Pricing

Pay per event: page = $0.0015 for each successfully converted page. Skipped and failed pages cost nothing. Platform usage is included in the event price.

	Price per 1,000 pages
This Actor	$1.50, no subscription
Firecrawl Hobby ($19 for 5,000 credits; $5 per extra 1,000)	$3.80 to $5.00
Firecrawl Standard ($99 for 100,000 credits)	$0.99, but only with a $99/month commitment
Apify Website Content Crawler (platform usage)	about $0.20 (raw HTTP) to $0.50–$5 (headless browser)

Limits (v0.1)

• Raw HTTP fetch, no JavaScript rendering. Pages that render client-side only come back thin; they are flagged empty_content and not charged.
• PDFs and other non-HTML files are skipped.
• token_estimate is about characters ÷ 4, not an exact tokenizer count.

Use cases

• RAG and vector databases: crawl a docs site and feed clean Markdown chunks into your embeddings pipeline.
• LLM context and AI agents: convert a URL to Markdown before passing it to ChatGPT, Claude or a custom agent.
• HTML to Markdown for knowledge bases: archive articles and docs in a readable, portable format.

FAQ

How do I convert a web page to Markdown? Add the URL to startUrls, keep mode as single, and run. Each page becomes one dataset item with a markdown field.

Can it crawl a whole website or docs site? Yes. Set mode to crawl and use maxPages, maxDepth and include/exclude globs. Crawls stay on the same domain by default.

Does it render JavaScript? No, v0.1 uses raw HTTP fetch. Client-side-only pages are flagged empty_content and not charged.

What does it cost? $0.0015 per successfully converted page ($1.50 per 1,000). Skipped and failed pages are free.

Built by withgrokbot.