🧩 RAG Dataset Builder - Crawl to Embedding-Ready Chunks

🧩 LLM Dataset Builder - Site to Embedding-Ready Chunks

that_red_bird/llm-dataset-builder

⚡ Turn any site into a RAG dataset in one run. ✅ Clean markdown (code blocks, tables, headings kept; nav and cookie banners stripped), split into semantic chunks on heading boundaries with token estimates, configurable overlap and a source URL on every chunk. ✅ Ready to embed, no post-processing.

Download any PDF and chunk into semantically coherent segments ready for embedding/RAG. Configurable chunk size + overlap. Returns one row per chunk with page, char count, token estimate. Feed directly into OpenAI text-embedding-3 / Voyage / Cohere. $0.005 per PDF + $0.0002 per chunk.

RAG Docs Extractor - Documentation to Chunks

ambitious_door/ragdocs-extractor

Turn any documentation site into clean, RAG-ready chunks in a single call. Semantic boundaries, preserved structure, per-chunk metadata.

Crawl a site and export clean Markdown with token counts and chunks, ready for RAG.

PDF to RAG-Ready Dataset

utilityforgelab/pdf-to-rag-ready-dataset

Convert public text-based PDFs into clean, token-aware RAG chunks with Markdown, page references, metadata, stable document IDs, and ready-to-use Apify Dataset output.

Split Markdown or plain text into retrieval-ready chunks for RAG pipelines: cuts at headings, packs whole sentences up to a size limit with optional overlap, and tags every chunk with its heading breadcrumb. Handles Japanese sentence boundaries. No LLM cost.

RAG Post Processor - Text Cleaner & Chunker for LLM Pipelines

jalicia/rag-post-processor

Clean and chunk scraped text into embedding-ready chunks, sized in tokens (tiktoken) rather than characters. Preserves headings, tables, code blocks and links. Every chunk carries its token count, heading path and content hash. Chain after any scraper. Billed per KB of text processed.

AI / RAG Web Crawler

groupoject/ai-rag-web-crawler

Crawl any website and extract clean, LLM-ready Markdown chunks to feed AI agents, chatbots, and RAG pipelines. One row per embeddable chunk.

RAG Dataset Builder

zinin/sitemap-to-knowledge

Turn buyer-authorized public website sitemaps into bounded, source-linked text chunks for RAG, search, migration, and review. Each delivered chunk includes provenance, hashes, crawl-gap flags, confidence, and billing metadata. Partial and homepage-fallback chunks are explicitly labeled.

Web-to-Markdown Generator for AI & RAG Pipelines

profitstack/web-to-markdown-generator-for-ai-rag-pipelines

Convert any website into clean, heading-based chunking, LLM-ready Markdown for RAG and AI agents.