Pricing

from $2.00 / 1,000 page scrapeds

Try for free

Go to Apify Store

Docs Site to Markdown Corpus

Try for free

Turn any documentation site into a clean Markdown corpus: one file per page with front matter (title, URL, breadcrumbs), headings, code blocks with languages, tables and absolute links, plus an INDEX.json. Built for LLM wikis, RAG and offline docs. Pay per page.

Pricing

from $2.00 / 1,000 page scrapeds

Rating

0.0

(0)

Developer

Vellum Kasane

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Categories

You can access the Docs Site to Markdown Corpus programmatically from your own applications by using the Apify API. You can also choose the language preference from below. To use the Apify API, you’ll need an Apify account and your API token, found in API & Integrations in Apify Console.

Python

JavaScript

CLI

OpenAPI

HTTP

MCP

1import { ApifyClient } from 'apify-client';
2
3// Initialize the ApifyClient with your Apify API token
4// Replace the '<YOUR_API_TOKEN>' with your token
5const client = new ApifyClient({
6    token: '<YOUR_API_TOKEN>',
7});
8
9// Prepare Actor input
10const input = {
11    "startUrls": [
12        {
13            "url": "https://docs.apify.com/actors"
14        }
15    ],
16    "maxPages": 20
17};
18
19// Run the Actor and wait for it to finish
20const run = await client.actor("vellumlabs/docs-to-markdown-corpus").call(input);
21
22// Fetch and print Actor results from the run's dataset (if any)
23console.log('Results from dataset');
24console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
25const { items } = await client.dataset(run.defaultDatasetId).listItems();
26items.forEach((item) => {
27    console.dir(item);
28});
29
30// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

Docs Site to Markdown Corpus API in JavaScript

The Apify API client for JavaScript is the official library that allows you to use Docs Site to Markdown Corpus API in JavaScript or TypeScript, providing convenience functions and automatic retries on errors.

Install the apify-client

$npm install apify-client

Other API clients include:

Docs Site to Markdown Corpus API in Python

Docs Site to Markdown Corpus API through CLI

Docs Site to Markdown Corpus OpenAPI definition

Docs Site to Markdown Corpus API

Site Crawler: Website → Markdown Corpus for LLM/RAG

boxbox10/site-crawler

Crawl a whole website or docs site and get one clean, LLM-ready Markdown + JSON record per page (title, headings, content, links, token count). Built for RAG ingestion and AI knowledge bases.

Marvin Eguilos

Docs → RAG Corpus Builder (LLM-ready Markdown + llms.txt)

yasaslive/docs-rag-builder

Crawl any documentation site into an embeddings-ready corpus: clean markdown chunks with heading-path metadata, generated llms.txt, corpus.jsonl, and optional OpenAI embeddings.

Eonix Pvt Ltd

Docs To Markdown

excellent_mustang/docs-to-markdown

Gorav Agarwal

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

Mintlify Docs Scraper — Docs to Clean Markdown

straightforward_hydra/mintlify-docs-scraper

Extract any Mintlify / llms.txt documentation site as clean Markdown: one row per page with title, section, summary & content. For RAG & LLMs. No key.

Dev D

Knowledge Intelligence Engine — Website to Markdown for RAG

ryanclinton/website-content-to-markdown

Turn any website, documentation site or help centre into a retrieval-ready knowledge corpus for RAG and AI search. Clean Markdown plus chunks, change detection, deduplication, retrieval scoring, version awareness and a full corpus audit, in one run.

Ryan Clinton

Website to RAG Markdown Crawler

knotted_tussock/rag-markdown-crawler

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Ralph T

AI Web Extractor: URL → Clean Markdown + JSON for LLM/RAG

boxbox10/ai-web-extractor

Turn any URL into clean, LLM-ready Markdown + structured JSON (title, headings, main content, links, metadata, token count). Perfect for RAG pipelines, AI agents, and LLM context.

Marvin Eguilos

Website Pages to Clean Markdown for LLM and RAG

usta/url-to-markdown

Website to markdown for rag: turn public pages or a sitemap into llm ready text, one row per page. Headings, links and tables stay; menus and scripts go. robots.txt is always obeyed.

US Tech Automations

Website to Markdown – Clean LLM & RAG Content Extractor

dataquarry/website-to-markdown

Convert any public web page to clean, LLM-ready Markdown with metadata — by URL, a list of URLs, or a whole-site crawl. Strips nav/ads/boilerplate, keeps headings/lists/tables/code. Respects robots.txt. No API key.