Website to Markdown for LLM & RAG (Fast, Lightweight)
Pricing
from $1.00 / 1,000 page converteds
Website to Markdown for LLM & RAG (Fast, Lightweight)
Convert web pages or whole websites into clean Markdown for ChatGPT, LLM fine-tuning and RAG. Main-content extraction, chunking with token estimates, respects robots.txt, no browser needed.
Pricing
from $1.00 / 1,000 page converteds
Rating
0.0
(0)
Developer
age break
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
an hour ago
Last modified
Categories
Features
Share
Website to Markdown for LLM & RAG
Convert any web page, or a whole website, into clean Markdown for ChatGPT, Claude, custom GPTs, fine-tuning and RAG vector databases. Navigation, headers, footers, cookie banners and ads are removed. Headings, lists, tables, code blocks and links are kept. Optional chunking with token estimates gives you ready-to-embed pieces.
Why use this actor
- Main-content extraction: no menus or footers in your embeddings.
- Built-in chunking on paragraph boundaries with overlap, plus a token estimate per page and per chunk.
- Lightweight HTTP crawler: much cheaper and faster than browser-based crawlers for static and server-rendered sites.
- Crawl mode with include/exclude regex (e.g. only
/docs/), depth and page limits. - Respects robots.txt and flags pages that look JavaScript-rendered instead of returning empty junk.
Use cases
- Build a RAG chatbot over product docs or a help center.
- Create LLM fine-tuning or evaluation datasets from websites.
- Give an AI agent clean page content (works well with Apify MCP).
- Archive articles as Markdown in Notion or Obsidian.
How to use
- Click Try for free and fill in the input (or paste the JSON below into the JSON tab).
- Click Start. Small runs finish in seconds.
- Download the results as JSON, CSV, Excel or HTML, or get them through the Apify API, webhooks, Make, Zapier, n8n or the Apify MCP server for AI agents.
- Optional: schedule the actor to run daily or weekly for monitoring.
Input example
{"startUrls": [{"url": "https://crawlee.dev/js/docs/quick-start"}],"crawlLinks": true,"maxPages": 50,"includeUrlPatterns": ["/js/docs/"],"chunkSizeTokens": 800}
Output example
{"url": "https://crawlee.dev/js/docs/quick-start","title": "Quick Start | Crawlee","wordCount": 800,"tokenEstimate": 1990,"markdown": "# Quick Start\n\nWith this short tutorial you can start scraping ...\n\n## Choose your crawler\n...","chunks": [{"index": 0,"text": "...","tokenEstimate": 790}]}
Limits
No browser: pages that render their content only with JavaScript come back with an error saying so (try Apify's Website Content Crawler for those). The token estimate is about 4 characters per token.
Pricing
Pay per event: $1.00 per 1,000 pages converted. Skipped or failed pages are free.
Apify's free plan includes monthly platform credit, which is enough to try this actor at no cost.
Is it legal?
This actor only reads publicly available data, without logging in. The crawler respects robots.txt and paces requests per host. It does not collect personal data by default. You are responsible for using the results in line with the source's terms and with laws such as the GDPR.
Built with AI assistance
This actor was built with AI assistance (Claude by Anthropic). A human operator reviews and publishes it. It was tested with real inputs before release. If something breaks or you need a feature, please open an issue on the Issues tab.
Related actors and keywords
website to markdown, html to markdown, url to markdown api, llm ready data, rag web scraper, firecrawl alternative, jina reader alternative, web page to text for chatgpt