Semantic RAG Document Chunker and Token Slicer
Pricing
from $1.00 / 1,000 document chunkeds
Deterministic document chunker that splits text, markdown, and tables without cutting sentences or code blocks in half.
List of documents containing document text or markdown to segment into token exact chunks.
[ { "id": "doc-001", "text": "# Introduction to Autonomous RAG\n\nRetrieval Augmented Generation combines information retrieval with language modeling.\n\n## Vector Embeddings\nEmbeddings convert semantic meaning into dense mathematical vectors.\n\n```python\ndef get_embedding(text):\n return model.encode(text)\n```\n\n## Conclusion\nChunking preservation avoids truncating code blocks or tables." }]Maximum token limit per chunk using cl100k_base tokenizer encoding.