Website to Markdown Crawler — LLM & RAG, Firecrawl Alt avatar

Website to Markdown Crawler — LLM & RAG, Firecrawl Alt

Pricing

from $44.00 / 1,000 pages converted (per 5)s

Go to Apify Store
Website to Markdown Crawler — LLM & RAG, Firecrawl Alt

Website to Markdown Crawler — LLM & RAG, Firecrawl Alt

Turn any web page into clean, LLM/RAG-ready Markdown. Strips nav/ads/boilerplate, optional same-domain crawl and overlap-aware chunking for vector databases. Keyless and fast.

Pricing

from $44.00 / 1,000 pages converted (per 5)s

Rating

0.0

(0)

Developer

Fredrik Hansson

Fredrik Hansson

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

8 days ago

Last modified

Security

Limited permissions

Share

Website to Markdown — Clean Markdown for LLMs & RAG

Turn any web page into clean, LLM/RAG-ready Markdown. Point it at one URL or a whole docs site: it strips navigation, ads and boilerplate, keeps the main content, and returns tidy Markdown — optionally split into overlap-aware chunks ready to embed into a vector database.

Deterministic and keyless — no OpenAI key, no per-token cost. Fast, cheap, and built for people and AI agents (available over MCP).

A clean Firecrawl / Jina Reader alternative on Apify: pay-per-use, no subscription.

What it does

  • Boilerplate removal — drops nav, header, footer, aside, forms, scripts, ads and common menu/sidebar classes, then picks the richest <main>/<article> container.
  • Clean Markdown — turndown + GFM, so tables, code blocks and lists survive intact (ATX headings, fenced code).
  • Same-domain crawl — set a crawl depth to walk a whole docs site; filter with include/exclude URL globs.
  • RAG chunking — optional: split each page into ~N-token chunks with overlap so vector-DB retrieval keeps context across chunk edges.
  • Token + word counts — every record carries a rough token estimate for budgeting.

Input

FieldTypeDefaultNotes
startUrlsstring[]—Seed URLs (required).
maxPagesinteger50Hard cap 200. One charge per page converted.
maxDepthinteger0Same-domain crawl depth. 0 = only the seeds.
maxConcurrencyinteger5Pages fetched/converted in parallel.
chunkForRagbooleanfalseEmit chunk records instead of page records.
chunkSizeinteger512Target chunk size in tokens.
chunkOverlapinteger50Token overlap between chunks (< chunkSize).
includeUrlGlobsstring[]—Only follow matching links when crawling.
excludeUrlGlobsstring[]—Never follow matching links (wins over include).

Output

One dataset record per page (or per chunk when chunkForRag is on):

{
"url": "https://quotes.toscrape.com/",
"title": "Quotes to Scrape",
"markdown": "# Quotes to Scrape\n\n> \"The world as we have created it...\"\n...",
"wordCount": 412,
"tokenEstimate": 540
}

Chunk records add chunkIndex and drop wordCount.

Example — messy page in, clean Markdown out

A typical article arrives wrapped in nav bars, cookie banners, share widgets and footers. You get only the content:

In: one blog/docs URL (full HTML, ~180 KB of markup)

Out:

# RAG best practices
Retrieval-augmented generation works best when your chunks keep their context.
## 1. Clean your source
Strip boilerplate before you embed — nav links and cookie banners poison retrieval.

Flip on chunkForRag and the same page comes back as embed-ready chunks with overlap and a token estimate per chunk — drop them straight into a vector DB.

Pricing

Pay-per-event: $0.10 per 5 pages converted (≈ $0.02 / page). No subscription, no token fees — you only pay for pages actually converted.

Part of the No Niner no-code data toolkit

One of 90+ no-code data tools — many chain directly into each other. Browse them all