Docs-to-RAG Pipeline Builder

Docs Markdown Rag Ready Crawler

devwithbobby/docs-markdown-rag-ready-crawler

Turn any documentation site or website into clean, structured markdown—ready for RAG, embeddings, and AI agents.

Docs to Markdown + AI Embeddings → Vector DB Crawler

badruddeen/docs-to-markdown-ai-embeddings---vector-db-crawler

Turn any documentation site into clean Markdown, intelligently chunked content with embeddings (Azure/OpenAI), and directly upsert into MongoDB Atlas, Pinecone, Weaviate, Qdrant, or Milvus — ready for RAG, AI assistants, and semantic search in minutes.

Docs → RAG Corpus Builder (LLM-ready Markdown + llms.txt)

yasaslive/docs-rag-builder

Crawl any documentation site into an embeddings-ready corpus: clean markdown chunks with heading-path metadata, generated llms.txt, corpus.jsonl, and optional OpenAI embeddings.

Crawl any website or docs site and export clean Markdown plus JSONL-style chunks for RAG, LLM apps, and AI agents.

Fetch documentation from GitHub repositories — READMEs, docs folders, wikis — and convert to clean, chunked markdown optimized for RAG pipelines. Use via Apify Console/API or connect as an MCP server for Claude, Cursor, and other AI agents.

Rag Embedding Generator

labrat011/rag-embedding-generator

Generate vector embeddings from text or chunked datasets using OpenAI or Cohere. Chains with RAG Content Chunker for end-to-end RAG pipelines. Outputs raw vectors ready for any vector database.

Website to RAG Chunks

cynix_dev/web-to-rag-chunks

Crawl any website and turn its pages into clean, chunked, metadata-rich Markdown records ready for RAG pipelines, vector stores, and custom GPTs.

🧩 RAG Dataset Builder - Crawl to Embedding-Ready Chunks

that_red_bird/rag-dataset-builder

🧩 Crawl a site and get back a FINISHED, embedding-ready RAG dataset — not just clean text. ✅ Semantic chunking on heading boundaries, never mid-code-block or mid-table. ✅ Token estimator, configurable overlap, near-duplicate removal, full provenance per chunk.

Website to Markdown Crawler | Pre-Chunked for RAG

themineworks/rag-crawler

Crawl any website into clean, pre-chunked Markdown with per-chunk token counts for RAG pipelines, vector DBs (Pinecone, Qdrant) and LLM context. MCP native for Claude & ChatGPT. SPA support via Playwright. Pay only for pages that crawl. A Firecrawl alternative.

RAG Docs Extractor - Documentation to Chunks

ambitious_door/ragdocs-extractor

Turn any documentation site into clean, RAG-ready chunks in a single call. Semantic boundaries, preserved structure, per-chunk metadata.