# RAG Content Delta & Stable Chunks (`emredurukan/rag-content-delta`) Actor

Compare two content snapshots and produce stable chunk upserts and deletes for RAG indexing, with missing-document safety checks.

- **URL**: https://apify.com/emredurukan/rag-content-delta.md
- **Developed by:** [Emre Durukan](https://apify.com/emredurukan) (community)
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 chunk changes

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## RAG Content Delta & Stable Chunks

Update your retrieval index with only the content chunks that changed between two complete snapshots. This Actor compares documents you supply; it does not crawl sites, create embeddings, call an LLM, or write to a vector database.

### Quick start

Use the prefilled demo and run. The output dataset contains one `delete` for the old answer and one `upsert` with the new answer. `SUMMARY` in the run's key-value store contains counts and `complete: true` when output writing finished. Only consume output from a successful run with that marker.

For each side provide either `previousDocuments` / `currentDocuments` or the corresponding dataset ID. Clear the prefilled inline array when using an ID. Documents must each have a unique string `url` and nonempty string `markdown` by default; change `keyField` or `textField` to use another top-level field. Dataset IDs must be accessible with the Actor's permissions. Use complete snapshots in the same scope, not incremental crawler results.

### Output and indexing

Each row has `operation` (`upsert` or `delete`), `chunkId`, `sourceId`, and `contentHash`. Upserts also contain `text`. Apply upserts by `chunkId` and delete by `chunkId`, in your own integration. Store the exact chunks created by this Actor when initially indexing: an empty previous snapshot emits all current chunks as upserts. The Actor does not apply changes on your behalf.

It normalizes Unicode to NFC and line endings, splits on blank lines, then splits oversized paragraphs at the configured character count with no overlap. Each chunk ID hashes the exact source ID, content hash, and occurrence number for repeated identical paragraphs. Reordering distinct paragraphs does not change IDs, but edits to a long paragraph can affect multiple fixed-length slices. New chunk text is included in output. No metadata, title, ordering, semantic meaning, token limits or embedding changes are tracked. Empty content is rejected. Keep `chunkSize`, source identity, and field choices consistent when comparing against a previously indexed snapshot. Changing the algorithm or chunk settings requires a full reindex.

### Safety and limits

A missing source causes failure by default. To intentionally remove absent sources, set `allowMissingDocuments` to true and set `maxMissingPercent` to an appropriate percentage. This is a guard, not proof of completeness: failed pages that still contain partial nonempty content can produce chunk deletions. Check crawler success and review deletes before applying them. Explicit empty current snapshots can remove everything only with both settings enabled and the threshold at 100. A duplicate or missing source key, missing or empty text, oversized input, inaccessible dataset or incomplete dataset fetch causes failure before output writes. A failed output write can still leave partial rows; ignore output from failed or incomplete runs.

Each snapshot is limited to 5,000 documents and 20 million characters total, and each document to 2 million characters. Dataset input uses dataset IDs. Requests are limited to 85 MiB after JSON encoding. Cloudflare Worker CPU and memory limits can cause large otherwise valid snapshots to fail; split your workload into complete, consistently scoped batches. The Cloudflare service applies an approximate shared limit of 30 comparison requests per minute per location; rate-limited runs fail and can be retried later as a new run. No automatic state, scheduling, deletion, HTTP fetch or change tracking across runs is included.

### Data and pricing

The source ID and content fields from both snapshots are sent over HTTPS to a private Cloudflare Worker for in-memory comparison. Cloudflare also processes request metadata under its platform policies. Inline input and output chunks are stored in Apify storages under the run and subject to the account's retention and access controls; this Actor does not delete them. Source IDs, including URLs, may contain private data. Avoid supplying secrets and restrict access to storage. The Actor writes only counts to application logs. Hashes are not anonymization.

### Price and billing

The Actor event price is $0.10 per completed comparison, $0.001 per output chunk change (both upserts and deletes), and $0.00005 per start for runs up to 1 GB. Platform usage is included in the configured Actor pricing; see the live Pricing tab for the effective offer and any taxes. A successful run with two changes costs $0.10205. A successful comparison with no changes costs $0.10005. The start fee is charged even if input validation fails or a run is stopped. The complete remaining charge must fit the run's budget after the start fee; otherwise the Actor fails before writing output. Once output starts, interruptions or errors can leave partial output and charges: do not apply output unless the run succeeded and SUMMARY.complete is true. Check billing state and contact support before retrying an incomplete run. The start, output and completion charges are not atomic, and a new run incurs a new start charge. The Actor refuses to resume a run that already contains SUMMARY or BILLING_STATE. Report problems through the Actor's Apify Issues tab.

# Changelog

This Actor's version history is a separate document: https://apify.com/emredurukan/rag-content-delta/changelog.md

# Actor input Schema

## `previousDocuments` (type: `array`):

Previous full snapshot as JSON objects. Use this or Previous dataset ID, not both. An empty array is a valid initial baseline.

## `currentDocuments` (type: `array`):

Current complete snapshot as JSON objects. Use this or Current dataset ID, not both.

## `previousDatasetId` (type: `string`):

Dataset ID accessible to the Actor. Clear the corresponding inline input to use it.

## `currentDatasetId` (type: `string`):

Dataset ID accessible to the Actor. Clear the corresponding inline input to use it.

## `keyField` (type: `string`):

A unique, stable string field present in both snapshots. URL changes are treated as deletions and additions.

## `textField` (type: `string`):

Nonempty string content field in both snapshots. Empty or missing content causes the run to fail.

## `chunkSize` (type: `integer`):

Paragraph chunks are split at this character length. Keep this unchanged between comparisons.

## `allowMissingDocuments` (type: `boolean`):

Enable only when the current snapshot is complete and absent sources should be deleted. Also set the maximum missing percentage.

## `maxMissingPercent` (type: `integer`):

Safety limit for missing previous source IDs. Set 100 only when intentionally deleting every previous source.

## Actor input object example

```json
{
  "previousDocuments": [
    {
      "url": "https://example.com/help",
      "markdown": "# Help\n\nOld answer.\n\nUnchanged details."
    }
  ],
  "currentDocuments": [
    {
      "url": "https://example.com/help",
      "markdown": "# Help\n\nNew answer.\n\nUnchanged details."
    }
  ],
  "keyField": "url",
  "textField": "markdown",
  "chunkSize": 2000,
  "allowMissingDocuments": false,
  "maxMissingPercent": 0
}
```

# Actor output Schema

## `changes` (type: `string`):

Upsert and delete operations keyed by source-scoped chunk IDs. Upserts include content text.

## `summary` (type: `string`):

Counts, completion marker, and acknowledged charges. Start fee is separate.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "previousDocuments": [
        {
            "url": "https://example.com/help",
            "markdown": "# Help\n\nOld answer.\n\nUnchanged details."
        }
    ],
    "currentDocuments": [
        {
            "url": "https://example.com/help",
            "markdown": "# Help\n\nNew answer.\n\nUnchanged details."
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("emredurukan/rag-content-delta").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "previousDocuments": [{
            "url": "https://example.com/help",
            "markdown": """# Help

Old answer.

Unchanged details.""",
        }],
    "currentDocuments": [{
            "url": "https://example.com/help",
            "markdown": """# Help

New answer.

Unchanged details.""",
        }],
}

# Run the Actor and wait for it to finish
run = client.actor("emredurukan/rag-content-delta").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "previousDocuments": [
    {
      "url": "https://example.com/help",
      "markdown": "# Help\\n\\nOld answer.\\n\\nUnchanged details."
    }
  ],
  "currentDocuments": [
    {
      "url": "https://example.com/help",
      "markdown": "# Help\\n\\nNew answer.\\n\\nUnchanged details."
    }
  ]
}' |
apify call emredurukan/rag-content-delta --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,emredurukan/rag-content-delta"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/1J3hb6lduZjiyO6Mr/builds/kdUMNaJj0Mm8uCqkT/openapi.json