RAG Content Delta & Stable Chunks

Update your retrieval index with only the content chunks that changed between two complete snapshots. This Actor compares documents you supply; it does not crawl sites, create embeddings, call an LLM, or write to a vector database.

Quick start

Use the prefilled demo and run. The output dataset contains one delete for the old answer and one upsert with the new answer. SUMMARY in the run's key-value store contains counts and complete: true when output writing finished. Only consume output from a successful run with that marker.

For each side provide either previousDocuments / currentDocuments or the corresponding dataset ID. Clear the prefilled inline array when using an ID. Documents must each have a unique string url and nonempty string markdown by default; change keyField or textField to use another top-level field. Dataset IDs must be accessible with the Actor's permissions. Use complete snapshots in the same scope, not incremental crawler results.

Output and indexing

Each row has operation (upsert or delete), chunkId, sourceId, and contentHash. Upserts also contain text. Apply upserts by chunkId and delete by chunkId, in your own integration. Store the exact chunks created by this Actor when initially indexing: an empty previous snapshot emits all current chunks as upserts. The Actor does not apply changes on your behalf.

It normalizes Unicode to NFC and line endings, splits on blank lines, then splits oversized paragraphs at the configured character count with no overlap. Each chunk ID hashes the exact source ID, content hash, and occurrence number for repeated identical paragraphs. Reordering distinct paragraphs does not change IDs, but edits to a long paragraph can affect multiple fixed-length slices. New chunk text is included in output. No metadata, title, ordering, semantic meaning, token limits or embedding changes are tracked. Empty content is rejected. Keep chunkSize, source identity, and field choices consistent when comparing against a previously indexed snapshot. Changing the algorithm or chunk settings requires a full reindex.

Safety and limits

A missing source causes failure by default. To intentionally remove absent sources, set allowMissingDocuments to true and set maxMissingPercent to an appropriate percentage. This is a guard, not proof of completeness: failed pages that still contain partial nonempty content can produce chunk deletions. Check crawler success and review deletes before applying them. Explicit empty current snapshots can remove everything only with both settings enabled and the threshold at 100. A duplicate or missing source key, missing or empty text, oversized input, inaccessible dataset or incomplete dataset fetch causes failure before output writes. A failed output write can still leave partial rows; ignore output from failed or incomplete runs.

Each snapshot is limited to 5,000 documents and 20 million characters total, and each document to 2 million characters. Dataset input uses dataset IDs. Requests are limited to 85 MiB after JSON encoding. Cloudflare Worker CPU and memory limits can cause large otherwise valid snapshots to fail; split your workload into complete, consistently scoped batches. The Cloudflare service applies an approximate shared limit of 30 comparison requests per minute per location; rate-limited runs fail and can be retried later as a new run. No automatic state, scheduling, deletion, HTTP fetch or change tracking across runs is included.

Data and pricing

The source ID and content fields from both snapshots are sent over HTTPS to a private Cloudflare Worker for in-memory comparison. Cloudflare also processes request metadata under its platform policies. Inline input and output chunks are stored in Apify storages under the run and subject to the account's retention and access controls; this Actor does not delete them. Source IDs, including URLs, may contain private data. Avoid supplying secrets and restrict access to storage. The Actor writes only counts to application logs. Hashes are not anonymization.

Price and billing

The Actor event price is $0.10 per completed comparison, $0.001 per output chunk change (both upserts and deletes), and $0.00005 per start for runs up to 1 GB. Platform usage is included in the configured Actor pricing; see the live Pricing tab for the effective offer and any taxes. A successful run with two changes costs $0.10205. A successful comparison with no changes costs $0.10005. The start fee is charged even if input validation fails or a run is stopped. The complete remaining charge must fit the run's budget after the start fee; otherwise the Actor fails before writing output. Once output starts, interruptions or errors can leave partial output and charges: do not apply output unless the run succeeded and SUMMARY.complete is true. Check billing state and contact support before retrying an incomplete run. The start, output and completion charges are not atomic, and a new run incurs a new start charge. The Actor refuses to resume a run that already contains SUMMARY or BILLING_STATE. Report problems through the Actor's Apify Issues tab.