Text Chunker & Embeddings for RAG (LangChain, LlamaIndex)

Split Markdown, plain text and web pages into heading-aware, token-sized chunks with overlap, and optionally turn each chunk into a vector embedding with a small model running inside this Actor on CPU. No OpenAI key and no third-party API. The output drops straight into LangChain, LlamaIndex, n8n, Make, Zapier and vector databases (Pinecone, Qdrant, Weaviate, Chroma, pgvector).

Pairs with Document to Markdown: convert PDFs/DOCX first, then pass that run's dataset ID here.

Quick start

1. Click Start. The prefill chunks a short help-center article and embeds it with bge-small-en-v1.5.
2. Paste your own text into Documents, add URLs, or give a dataset ID from a document-to-markdown run.
3. Open the Output tab, or download every chunk as one JSONL file (chunks.jsonl in the key-value store).

What you get (one dataset item per chunk)

{
  "id": "5f525e2c5caa87629dd2",
  "text": "# Refund policy\n\nCustomers can request a full refund within 30 days of purchase. ...",
  "metadata": {"source": "help-center/refunds.md", "headings": ["Refund policy"], "section": "Refund policy",
               "chunkIndex": 0, "chunkCount": 3, "startChar": 0, "tokenCount": 35, "lang": "en"},
  "embeddingModel": "BAAI/bge-small-en-v1.5", "embeddingDim": 384,
  "embedding": [-0.033255, 0.030728, 0.014127, "... 384 numbers"]
}

id is stable for the same source, position and text, so re-running upserts instead of duplicating. text + metadata is exactly a LangChain Document / LlamaIndex TextNode. Vectors are L2-normalised: use cosine or dot product.

How chunking works

• Splits at Markdown headings (code fences are respected) and stores the heading path as section.
• Packs paragraphs, then sentences, then words up to Max tokens per chunk (default 400 approximate subword tokens).
• Repeats about Overlap tokens (default 50) from the end of the previous chunk.
• The heading path is prepended to the text that gets embedded (not to text), which improves retrieval.

Embedding models (all local, CPU)

Option	Languages	Dim	Notes
none	–	–	Chunks only (cheapest)
BAAI/bge-small-en-v1.5	English	384	Best quality for English retrieval
sentence-transformers/all-MiniLM-L6-v2	English	384	About 3x faster and cheaper; reads the first 256 tokens of a chunk (use chunk size ≤ 256)
sentence-transformers/paraphrase-multilingual-MiniLM-L12-v2	50+	384	Multilingual; reads the first 128 tokens (use chunk size ≤ 128)

Important: embed your search queries with the same model. With bge-small, that's FastEmbedEmbeddings(model_name="BAAI/bge-small-en-v1.5") in LangChain or FastEmbedEmbedding in LlamaIndex, both free and local.

Pricing (pay per event)

Event	Price	When
Document chunked	$0.001	Per source document that produced chunks
Chunk created	$0.00002 ($0.02 per 1,000)	Per chunk in the output
Chunk embedded	$0.00022 ($0.22 per 1,000)	Per chunk embedded with bge-small or the multilingual model
Chunk embedded (fast)	$0.00007 ($0.07 per 1,000)	Per chunk embedded with all-MiniLM-L6-v2

Bronze, Silver and Gold Apify plans get 10%, 20% and 30% off. Example: 1,000 chunks from 10 documents with bge-small cost $0.25 ($0.10 with MiniLM, $0.03 chunks only). Failed sources and empty input are never charged. Set Maximum cost per run to cap spending.

Integrations

Steps follow Apify's integration docs: Zapier, Make, n8n, LangChain, LlamaIndex.

n8n: new Drive file → searchable knowledge base

Google Drive trigger (new file) → Apify: Run an Actor and get dataset → document-to-markdown → Apify: Run an Actor and get dataset → this Actor with {"documents": {{ $json }}, "embeddingModel": "BAAI/bge-small-en-v1.5"} → Qdrant / Pinecone / Postgres (pgvector) node: upsert id, embedding, payload text + metadata.

Make: help-center pages → vector DB every night

Schedule → Apify Run an Actor ({"urls": ["https://example.com/help/a", "..."]}) → a second scenario: Watch Actor Runs → Get Dataset Items → HTTP upsert into your vector DB's REST API.

Zapier: new Notion page → chunks

Notion New Database Item → Apify Run Actor (async) with {"documents": [{"text": "{{Content}}", "source": "{{Page URL}}"}]} → second Zap: Finished Actor run → Fetch dataset items → store wherever you search from.

LangChain

from langchain_apify import ApifyWrapper
from langchain_core.documents import Document
from langchain_community.embeddings.fastembed import FastEmbedEmbeddings
from langchain_community.vectorstores import FAISS

loader = ApifyWrapper().call_actor(
    actor_id="steady_kahikatea/text-chunker-embeddings",
    run_input={"urls": ["https://docs.apify.com/platform/integrations/langchain"], "embeddingModel": "BAAI/bge-small-en-v1.5"},
    dataset_mapping_function=lambda r: Document(page_content=r["text"] or "", metadata={**r["metadata"], "embedding": r.get("embedding")}),
)
docs = [d for d in loader.load() if d.page_content]
emb = FastEmbedEmbeddings(model_name="BAAI/bge-small-en-v1.5")   # same model, for queries
pairs = [(d.page_content, d.metadata.pop("embedding")) for d in docs]
store = FAISS.from_embeddings(pairs, emb, metadatas=[d.metadata for d in docs])
print(store.similarity_search("How do I load Apify data into LangChain?", k=3))

LlamaIndex

from llama_index.readers.apify import ApifyActor
from llama_index.core import Document
reader = ApifyActor("<APIFY_TOKEN>")
docs = reader.load_data(
    actor_id="steady_kahikatea/text-chunker-embeddings",
    run_input={"datasetId": "<document-to-markdown defaultDatasetId>", "embeddingModel": "BAAI/bge-small-en-v1.5"},
    dataset_mapping_function=lambda r: Document(text=r["text"] or "", metadata=r["metadata"], embedding=r.get("embedding")),
)

Limits and safety

• Up to 500 sources and 20,000 chunks per run; files up to 20 MB.
• URLs must be public http(s); private, internal and cloud-metadata addresses are blocked.
• PDFs, Word files and images aren't parsed here: use document-to-markdown first.
• Your text is never logged.