Website to Markdown: Fast Crawler for LLM & RAG (No Browser)
Pricing
from $0.70 / 1,000 page converteds
Website to Markdown: Fast Crawler for LLM & RAG (No Browser)
Turn web pages into clean Markdown for LLMs, RAG and AI agents: main content only, menus and footers removed, can follow site links. $1 per 1,000 pages.
Pricing
from $0.70 / 1,000 page converteds
Rating
0.0
(0)
Developer
Open Data Tools
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
9 hours ago
Last modified
Categories
Features
Share
Give page links, get clean Markdown ready for an LLM, a RAG index or an AI agent: headings, paragraphs, lists, links, tables and code, without the menus, headers, footers, sidebars and cookie banners. One page, or a whole documentation: the Actor can follow the links of a site under the same path. It reads pages directly, without a browser, so it is fast and cheap: $1 per 1,000 pages.
What you get
One row per page:
- Markdown of the main content (or of the whole page if you prefer).
- Title, description, language and canonical link.
- Dates the page gives: published and modified, in ISO 8601.
- Counts: words, links and images in the Markdown.
- Depth of the page from the link you gave (0 for the page itself).
Use cases
- RAG and chatbots: feed a documentation, a help center or a blog into a vector database.
- AI agents: one call to read a page as text an LLM understands.
- Content audits: word counts and structure of many pages at once.
- Archives and monitoring: keep the text of pages that matter to you.
How to use
- Without changing anything: the Apify documentation page about Actors.
- Pages: paste links or bare domains, one per line, up to 10,000 per run.
- Pages per link: 1 converts the page alone. More follows its links on the same site, under the same path: from
https://example.com/docsthe Actor reads/docs/...pages only, up to the number you choose. - Main content only, Keep links, Keep images: switches for shorter or fuller text.
Input example
{"urls": ["https://docs.apify.com/platform/actors"],"maxPagesPerUrl": 5,"onlyMainContent": true,"includeLinks": true,"includeImages": false}
Output example
{"recordType": "page","id": "https://docs.apify.com/actors","url": "https://docs.apify.com/actors","startUrl": "https://docs.apify.com/platform/actors","depth": 0,"title": "Actors | Platform | Apify Documentation","description": "Learn how to develop, run and share serverless cloud programs.","lang": "en","canonicalUrl": "https://docs.apify.com/actors","publishedAt": null,"modifiedAt": null,"markdown": "# Actors\n\nActors are serverless cloud programs that take a structured JSON input, perform a task…","wordCount": 360,"linkCount": 16,"imageCount": 0,"status": "ok","sourceName": "Website docs.apify.com","sourceUrl": "https://docs.apify.com/actors","licence": "Text of a public page, as its owner publishes it; check the site terms before reuse","sourceUpdatedAt": null,"actorVersion": "0.1.1","retrievedAt": "2026-11-17T21:00:00.000Z"}
Pricing
Pay per event: $1.00 per 1,000 pages converted (page-converted, $0.001 each), plus Apify's start fee of $0.00005 per run. Pages that do not answer, are missing or are not web pages give a free error row.
| Pages | Price |
|---|---|
| 100 | $0.10 |
| 1,000 | $1 |
| 100,000 | $100 |
Data source
The pages you give, read directly as their server sends them. The Actor introduces itself as OpenDataToolsBot, follows robots.txt (RFC 9309), waits between two pages of the same site, and never logs in, solves no CAPTCHA and uses no proxy to get round a block.
Limit: pages that build their text with JavaScript after loading (some single-page apps) give little or no Markdown, since no browser runs here. Most sites, documentations, blogs and news pages send their text directly.
Is it legal?
The Actor reads public pages, as each site's robots.txt allows. The text stays its owner's: check the site's terms before you republish it or use it to train a model.
FAQ
- Why so few words on a page? The page may build its text with JavaScript, or the main part may be small. Try "Main content only" off.
- Why did the crawl stop early? It reads links under the path you gave, on the same site, and skips pages that robots.txt closes, files (PDF, images) and pages that fail.
- Can I get shorter text for an AI model? Turn "Keep links" off: links keep their text only.
Related Actors
- AI Model Prices: Compare 400+ LLM Prices and Features
- llms.txt Generator: AI-Ready Site Map for Any Website
- AI Readiness Checker: GPTBot, ClaudeBot, llms.txt & AI SEO
- Structured Data Extractor: Schema.org, JSON-LD & Open Graph
- Sitemap Extractor: All URLs of a Site, Lastmod & Priority
Every Actor here also works as a tool for AI agents (Claude, ChatGPT, Cursor…) through Apify’s MCP server.
Changelog
See CHANGELOG.md.
En français
Pages web en Markdown propre, pour les IA
Donnez des liens de pages, obtenez leur texte en Markdown (titres, listes, liens, tableaux, code) sans les menus, en-têtes, pieds de page et bandeaux cookies. Le robot peut suivre les liens d'un site sous le même chemin (toute une documentation, par exemple).
Prix
1 $ les 1 000 pages. Les lignes d'erreur sont gratuites.
Source et légalité
Les pages publiques elles-mêmes, lues dans le respect de leur robots.txt, sans contournement. Le texte reste à son auteur : vérifiez les conditions du site avant de le republier.