RAG Text Chunker (Markdown to Chunks, Token Counts)

Created by

Murat Uzun

Actor

RAG Text Chunker (Markdown to Chunks, Token Counts)

Split Markdown or text into RAG-ready chunks with exact OpenAI token counts (o200k/cl100k): heading-aware, code blocks kept whole, sentence-boundary overlap, heading path on every chunk. Chunks any dataset from Website to Markdown, PDF to Markdown or your own texts.

Try for free

RAG Text Chunker (Markdown to Chunks, Token Counts)webdatatools/rag-text-chunker

Document

Section

Tokens

+1 field

Input

Texts:# Guide First section text. ## Details More text here.

Chunk size (tokens):500

Overlap (tokens):50

Tokenizer:o200k_base

Start a chunk at each section:true

Section heading level:2

Heading path in chunk text:true

Output fields

Document

Section

Tokens

Text

How it works

Sign up on Apify or log in

The RAG Text Chunker (Markdown to Chunks, Token Counts) Actor is ready to run with the inputs already set for "RAG Text Chunker (Markdown to Chunks, Token Counts)". Sign up or log in and you'll land in Apify Console with a copy of this exact task.

Start the "RAG Text Chunker (Markdown to Chunks, Token Counts)" task

The RAG Text Chunker (Markdown to Chunks, Token Counts) Actor will start running based on the input automatically. You can also change all the input fields, like texts, chunk size (tokens), and more, to match your specific use case.

Receive the "RAG Text Chunker (Markdown to Chunks, Token Counts)" output

Monitor the progress in real time. You will be notified as soon as your dataset is complete and ready for review. Each result includes data such as document, #, section, and more.

Integrate into your workflow

The final output is delivered in JSON, CSV, or Excel format, ready to be plugged into your workflow. You can also run the "RAG Text Chunker (Markdown to Chunks, Token Counts)" task via API.