Pricing

from $3.99 / 1,000 results

RAG-Ready Documentation Scraper

Scrape documentation to framework-optimized Markdown. Features semantic chunking for LLM, vector database, and RAG pipelines. Parse XML sitemaps easily.

Pricing

from $3.99 / 1,000 results

Rating

0.0

(0)

Developer

Alaricus

Actor stats

Bookmarked

Total users

Monthly active users

4 days ago

Last modified

Categories

Automation

Developer tools

Start URLs

start_urls

Required

List of URLs of the documentation pages or XML sitemaps to start crawling.

Type:array

Documentation Framework

framework

Optional

Select the documentation framework to scrape. 'auto' will attempt to auto-detect.

Type:string

Default:auto

Options:

autodocusaurusgitbooksphinxreadthedocs

Enable Semantic Chunking

enable_chunking

Optional

Split the generated Markdown pages into smaller text chunks suitable for vector databases.

Type:boolean

Default:true

Chunk Size (Characters)

chunk_size

Optional

Approximate character size for each semantic chunk.

Type:integer

Minimum:100

Default:1500

Chunk Overlap (Characters)

chunk_overlap

Optional

Overlap between consecutive chunks to maintain local context.

Type:integer

Minimum:0

Default:200

Maximum Pages to Scrape

max_pages

Optional

Limit the number of pages to scrape to prevent runaway runs and excessive compute usage.

Type:integer

Minimum:1

Default:50

Include Image Links

include_images

Optional

Include image markdown syntax (e.g. Alt text ) in the output.

Type:boolean

Default:true

Include Hyperlinks

include_links

Optional

Include markdown hyperlinks (e.g. Link Text) in the output.

Type:boolean

Default:true

Docs To Rag

gabrielaxy/docs-to-rag

Transform documentation websites into RAG-ready chunks with semantic understanding, quality scoring, and direct vector database integration.

Gabriel Antony Xaviour

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.

Manas Mantri

Tech Docs to LLM-Ready Markdown

hedelka/tech-docs-scraper

Scrapes technical documentation sites (Docusaurus, GitBook, MkDocs, ReadTheDocs) and converts them to clean, structured Markdown for RAG pipelines, LLM training, and AI assistants. Automatically detects documentation framework and removes navigation elements.

Dmitry Goncharov

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Dev with Bobby

AI Content Crawler

kai-agent/ai-content-crawler

Crawl any website and get clean, AI-ready content in markdown format. Perfect for RAG pipelines, LLM training data, and vector database ingestion. Features smart chunking, metadata extraction, and multiple output formats.

Kai Agent

rag-docs-scraper

marbled_jury/my-actor

Extract clean, RAG-optimized Markdown from any technical documentation. Built for LLMs and AI agents. No noise, just high-fidelity data.

Hastin S.

Website to Markdown for LLM and RAG

jeweled_jockstrap/my-actor-3

Convert any URL to clean Markdown text for AI applications. Strips HTML extracts content. For LLM training RAG pipelines and vector databases. Free Firecrawl alternative.

Juan Triviño

Documentation Crawler for RAG

liquid_bark/docs-crawler-for-rag

Specialized crawler for developer documentation sites. Detects frameworks (Docusaurus, GitBook, ReadTheDocs, MkDocs, Sphinx), extracts clean content, and outputs semantically chunked Markdown optimized for RAG pipelines.

Izz

Docs-to-RAG Optimizer

vamsi-krishna/docs-to-rag-optimizer

Convert public developer documentation into clean Markdown, semantic RAG chunks, token counts, duplicate hashes, JSONL exports, and quality warnings for AI assistants.