Under maintenance

Pricing

Pay per usage

Try for free

Go to Apify Store

Crawl Documentation Site — Data, Details & Metadata

Under maintenance

Try for free

Crawl documentation site data at scale with this powerful Apify actor. Extracts data, details & metadata with automatic pagination and proxy rotation. Perfect for market research, competitive intelligence, and data-driven decision making.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Donny Nguyen

Actor stats

Bookmarked

Total users

Monthly active users

a day ago

Last modified

Configure MCP server with Documentation Site Crawler | Apify

Get a ready-to-use configuration for your MCP client with the Documentation Site Crawler | Apify Actor preconfigured at mcp.apify.com?tools=tropical_quince/documentation-site-crawler .

You can connect to the Apify MCP Server using clients like Tester MCP Client, or any other MCP client of your choice.

If you want to learn more about our Apify MCP implementation, check out our MCP documentation. To learn more about the Model Context Protocol in general, refer to the official MCP documentation or read our blog post.

Tech Docs to LLM-Ready Markdown

hedelka/tech-docs-scraper

Scrapes technical documentation sites (Docusaurus, GitBook, MkDocs, ReadTheDocs) and converts them to clean, structured Markdown for RAG pipelines, LLM training, and AI assistants. Automatically detects documentation framework and removes navigation elements.

Dmitry Goncharov

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

Website to Markdown

tropical_quince/website-to-markdown

Convert webpages to clean Markdown. Perfect for RAG pipelines, LLM training, and documentation.

Donny Nguyen

RAG Knowledge Loader

botflowtech/rag-knowledge-loader

Scrapes documentation sites (GitBook, ReadTheDocs, Notion public pages) and converts them into vector-ready JSON format for RAG applications.

BotFlowTech

GitHub Documentation Extractor (Agentic)

himanshi1rana/github-docs-intelligence

An agentic AI actor that automatically extracts and analyzes documentation from GitHub repositories to help developers understand projects faster.

Himanshi Rana

GithubScraper

fornace/githubscraper

Automatically scrapes and downloads Markdown documentation from GitHub repositories, for easy AI finetuning.

Fornace

Actor README Generator Agent

jkuzz/actor-readme-generator-agent

Generates a README for an Apify Actor using an AI Agent. You only need to provide the actor id to generate a readable and SEO optimized README for the Actor. The generation process utilizes Apify's Official README Guide. This Agent is open source (link in readme).

Jan Kuželík

Get Site to Markdown

jhaisley/get-site

Website to Markdown Crawler An asynchronous web crawler that mirrors websites into a single organized markdown file, with handling for images and directory structure preservation. Designed to operate with low cost. This works great to build context for AI agents.