Pricing

Pay per event

🧩 LLM Dataset Builder - Site to Embedding-Ready Chunks

⚡ Turn any site into a RAG dataset in one run. ✅ Clean markdown (code blocks, tables, headings kept; nav and cookie banners stripped), split into semantic chunks on heading boundaries with token estimates, configurable overlap and a source URL on every chunk. ✅ Ready to embed, no post-processing.

Pricing

Pay per event

Rating

0.0

(0)

Developer

mohamed alaya

Actor stats

Bookmarked

Total users

Monthly active users

6 days ago

Last modified

🧩 RAG Dataset Builder - Crawl to Embedding-Ready Chunks

that_red_bird/rag-dataset-builder

🧩 Crawl a site and get back a FINISHED, embedding-ready RAG dataset — not just clean text. ✅ Semantic chunking on heading boundaries, never mid-code-block or mid-table. ✅ Token estimator, configurable overlap, near-duplicate removal, full provenance per chunk.

mohamed alaya

RAG Docs Extractor - Documentation to Chunks

ambitious_door/ragdocs-extractor

Turn any documentation site into clean, RAG-ready chunks in a single call. Semantic boundaries, preserved structure, per-chunk metadata.

C. K.

RAG Text Chunker — heading & sentence aware, Japanese ready

shoebill-dev27/rag-text-chunker

Split Markdown or plain text into retrieval-ready chunks for RAG pipelines: cuts at headings, packs whole sentences up to a size limit with optional overlap, and tags every chunk with its heading breadcrumb. Handles Japanese sentence boundaries. No LLM cost.

Shinobu Otani

Website to Markdown for LLMs

agentictools/website-to-markdown-llm

Crawl a site and export clean Markdown with token counts and chunks, ready for RAG.

Ken Agland

RAG Post Processor - Text Cleaner & Chunker for LLM Pipelines

jalicia/rag-post-processor

Clean and chunk scraped text into embedding-ready chunks, sized in tokens (tiktoken) rather than characters. Preserves headings, tables, code blocks and links. Every chunk carries its token count, heading path and content hash. Chain after any scraper. Billed per KB of text processed.

Jordan Wagner

PDF → RAG Chunks (Token-Aware, Vector-Ready)

gochujang/pdf-rag-chunker

Download any PDF and chunk into semantically coherent segments ready for embedding/RAG. Configurable chunk size + overlap. Returns one row per chunk with page, char count, token estimate. Feed directly into OpenAI text-embedding-3 / Voyage / Cohere. $0.005 per PDF + $0.0002 per chunk.

Hojun Lee

PDF to RAG-Ready Dataset

utilityforgelab/pdf-to-rag-ready-dataset

Convert public text-based PDFs into clean, token-aware RAG chunks with Markdown, page references, metadata, stable document IDs, and ready-to-use Apify Dataset output.

UtilityForgeLab

PDF to RAG Markdown Chunks for Embeddings

awesome_highboy/docforge

Convert PDFs into token-bounded Markdown chunks for RAG, embeddings, and vector databases (Pinecone, Chroma, Weaviate, Qdrant). Set maxTokens + overlap; get clean chunks with page number, token count, and SHA-256 content hash for dedup. JSON dataset ready for any LLM pipeline.

Adam

LLM-Ready Web Content Extractor

beyondxl/llm-ready-content-extractor

Turn any public web page or shallow site crawl into clean, LLM-ready Markdown + metadata + optional RAG chunks. Boilerplate/nav/ads removed. Honors robots.txt.