# Dataset Diff & Change Detector (`nerolabs/dataset-diff-detector`) Actor

Returns what was added, removed or changed between two Apify datasets or JSON arrays, matched by key, field by field, with an optional CSV or Excel diff report. Inputs: old and new dataset IDs or inline data, key fields. Agent-ready: pay per event (x402 and MCP), per difference found and file.

- **URL**: https://apify.com/nerolabs/dataset-diff-detector.md
- **Developed by:** [Adam Pearce](https://apify.com/nerolabs) (community)
- **Categories:** Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $5.00 / 1,000 difference detecteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

**Compare two datasets and get exactly what changed**, no manual spreadsheet comparison, no writing your own diff script. Point it at two Apify datasets (or paste in two JSON arrays), give it a key field like `id`, `sku`, or `email`, and it tells you precisely which rows were **added**, **removed**, or **changed**, down to which fields on each row actually moved. Works with the output of any scraper, any monitor Actor, or your own exported data.

### Why use Dataset Diff & Change Detector?

If you scrape or export the same source more than once, you already have this problem: two big JSON files, and no fast way to answer "what actually changed since last time?" without eyeballing thousands of rows.

- **Re-scraping the same source on a schedule?** Feed it yesterday's dataset and today's, and skip building your own comparison logic.
- **Reconciling two exports from different systems?** Match them on a shared key (email, SKU, ID) and see exactly where they diverge.
- **Auditing a data migration?** Confirm nothing silently dropped or changed during a transfer.
- **QA-checking a scraper you just changed?** Diff its old output against its new output to see the real impact of your change, not just "does it run."

No scraping involved at all, it only processes data you already have, so there's nothing to break when a website changes and nothing to worry about on data-source terms.

### How to use it

1. Pick your two snapshots: either an existing Apify dataset for **Old dataset** and **New dataset**, or paste JSON arrays directly into **Old data (inline)** / **New data (inline)**.
2. Set **Key field(s)** to whatever uniquely identifies a row (e.g. `sku`, `id`, `email`). This is what lets the Actor tell "this row changed" apart from "this row was removed and a different one was added".
3. Optionally list fields to ignore (like a `scrapedAt` timestamp that always differs) under **Fields to ignore**, so it doesn't flag every row as changed for no real reason.
4. Run it. Check the **added** / **removed** / **changed** rows in the dataset, or turn on a CSV/Excel export for a shareable report.

### Input

- **Old dataset / New dataset** — pick existing Apify datasets via the resource picker (limited-permissions safe, it can only read the dataset you point at).
- **Old data / New data (inline)** — paste a JSON array directly instead, for one-off comparisons.
- **Key field(s)** — the field(s) that identify a row across both snapshots. Leave empty to match by full row content instead (still works, but then a changed row shows as a remove+add pair, not a single "changed" row).
- **Fields to compare** — optionally restrict comparison to specific fields only.
- **Fields to ignore** — field names to exclude from comparison (timestamps, run IDs, anything that always differs).
- **Include unchanged rows in the output** — off by default, since most people only want the deltas.
- **Export diff report as file** — optionally generate a real downloadable CSV and/or Excel file on top of the dataset.

### Output

One row per difference, for example:

```json
{
  "status": "changed",
  "key": { "sku": "A100" },
  "oldValues": { "sku": "A100", "title": "Blue Widget", "price": 19.99, "stock": 42 },
  "newValues": { "sku": "A100", "title": "Blue Widget", "price": 17.99, "stock": 30 },
  "changedFields": ["price", "stock"]
}
```

`status` is one of `added`, `removed`, `changed`, or (if you turned that option on) `unchanged`. You can download the dataset in various formats such as JSON, CSV, or Excel directly from the Output tab, or turn on the built-in export for a ready-to-open diff report file.

### Pricing

Pay-per-event, anchored the same way as every low-maintenance data tool in this line: you're charged **$0.005 per real difference found** (an added, removed, or changed row), **$0.0005 per unchanged row confirmed** (only if you turn that option on), and **$0.01 per file export**. Comparing two 1,000-row datasets where 50 rows actually changed costs about $0.25, not $5, since you're only billed for the parts of the answer that are actually useful. There's no separate platform-usage charge; it's included.

### Tips

- Always set **Key field(s)** if your data has any kind of stable identifier. Without one, the Actor still works, but "changed" rows show up as a remove+add pair instead, which is technically correct but less useful for spotting exactly what moved.
- Add any always-changing field (timestamps, scrape IDs, "last checked" dates) to **Fields to ignore**, otherwise every single row will look "changed" every time.
- Duplicate key values within one snapshot are handled gracefully (the last-seen row wins) and flagged in the run's warnings, so you'll know if your chosen key field isn't actually unique.

### FAQ

**Does this work with any Apify dataset?** Yes, point "Old dataset" and "New dataset" at any two dataset IDs from your own account, including the output of another Actor's run.

**What if I don't have two dataset IDs, just two JSON files?** Use the inline "Old data" / "New data" fields instead and paste the arrays directly.

**Is my data safe?** This Actor never sends your data anywhere outside your own Apify account's storage. It doesn't scrape, doesn't call any external API, and doesn't retain anything beyond the run's own output.

Found a bug or want a feature? Use the Issues tab, replies come from a real person, usually within hours.

# Actor input Schema

## `oldDatasetId` (type: `string`):

Pick an existing Apify dataset holding the earlier version of your data. Use this OR 'Old data (inline)' below, not both. Declaring it this way is what lets this Actor run with limited permissions: it may read the dataset you point at, and nothing else on your account.

## `oldData` (type: `array`):

A JSON array of records for the 'before' snapshot, for ad-hoc data instead of a dataset ID.

## `newDatasetId` (type: `string`):

Pick an existing Apify dataset holding the newer version of your data. Use this OR 'New data (inline)' below, not both.

## `newData` (type: `array`):

A JSON array of records for the 'after' snapshot, for ad-hoc data instead of a dataset ID.

## `keyFields` (type: `array`):

The field name(s) that uniquely identify a row across both snapshots (e.g. 'sku', 'id', or 'email'). This is what lets the Actor tell 'this row changed' apart from 'this row was removed and a different one was added'. Leave empty to match rows by their full content instead (fine for simple lists, but then any change to any field makes a row look like a remove+add pair rather than a 'changed' row).

## `compareFields` (type: `array`):

Optional. Only compare these specific fields when deciding if a matched row changed. Leave empty to compare every field except any listed in 'Fields to ignore' below.

## `ignoreFields` (type: `array`):

Optional. Field names to exclude from comparison, useful for things like a 'scrapedAt' or 'lastChecked' timestamp that always differs between runs and would otherwise mark every row as changed.

## `includeUnchanged` (type: `boolean`):

When on, rows present in both snapshots with no real difference are also written to the output dataset (charged at a much lower rate, see Pricing). Off by default, since a diff tool's whole point is usually just the things that changed.

## `exportFormats` (type: `array`):

Optional. Generate a real downloadable diff report on top of the dataset. Leave empty to skip file export entirely.

## `maxItems` (type: `integer`):

Safety cap on how many rows from the larger of the two snapshots to process. Leave at 0 for no cap (up to the Actor's own 100,000-row hard limit).

## Actor input object example

```json
{
  "oldData": [
    {
      "sku": "A100",
      "title": "Blue Widget",
      "price": 19.99,
      "stock": 42
    },
    {
      "sku": "A101",
      "title": "Red Widget",
      "price": 24.99,
      "stock": 0
    },
    {
      "sku": "A102",
      "title": "Green Widget",
      "price": 15.5,
      "stock": 8
    }
  ],
  "newData": [
    {
      "sku": "A100",
      "title": "Blue Widget",
      "price": 17.99,
      "stock": 30
    },
    {
      "sku": "A102",
      "title": "Green Widget",
      "price": 15.5,
      "stock": 8
    },
    {
      "sku": "A103",
      "title": "Yellow Widget",
      "price": 12,
      "stock": 100
    }
  ],
  "keyFields": [
    "sku"
  ],
  "ignoreFields": [
    "scrapedAt",
    "lastChecked",
    "timestamp"
  ],
  "includeUnchanged": false,
  "exportFormats": [],
  "maxItems": 0
}
```

# Actor output Schema

## `diffRows` (type: `string`):

One row per difference found (added, removed, changed), plus unchanged rows if that option was enabled.

## `csvFile` (type: `string`):

A ready-to-open CSV file of the diff report, if requested.

## `xlsxFile` (type: `string`):

A ready-to-open Excel (.xlsx) file of the diff report, if requested.

## `diffSummary` (type: `string`):

Counts of added, removed, changed and unchanged rows, and any warnings from this run.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "oldData": [
        {
            "sku": "A100",
            "title": "Blue Widget",
            "price": 19.99,
            "stock": 42
        },
        {
            "sku": "A101",
            "title": "Red Widget",
            "price": 24.99,
            "stock": 0
        },
        {
            "sku": "A102",
            "title": "Green Widget",
            "price": 15.5,
            "stock": 8
        }
    ],
    "newData": [
        {
            "sku": "A100",
            "title": "Blue Widget",
            "price": 17.99,
            "stock": 30
        },
        {
            "sku": "A102",
            "title": "Green Widget",
            "price": 15.5,
            "stock": 8
        },
        {
            "sku": "A103",
            "title": "Yellow Widget",
            "price": 12,
            "stock": 100
        }
    ],
    "keyFields": [
        "sku"
    ],
    "ignoreFields": [
        "scrapedAt",
        "lastChecked",
        "timestamp"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("nerolabs/dataset-diff-detector").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "oldData": [
        {
            "sku": "A100",
            "title": "Blue Widget",
            "price": 19.99,
            "stock": 42,
        },
        {
            "sku": "A101",
            "title": "Red Widget",
            "price": 24.99,
            "stock": 0,
        },
        {
            "sku": "A102",
            "title": "Green Widget",
            "price": 15.5,
            "stock": 8,
        },
    ],
    "newData": [
        {
            "sku": "A100",
            "title": "Blue Widget",
            "price": 17.99,
            "stock": 30,
        },
        {
            "sku": "A102",
            "title": "Green Widget",
            "price": 15.5,
            "stock": 8,
        },
        {
            "sku": "A103",
            "title": "Yellow Widget",
            "price": 12,
            "stock": 100,
        },
    ],
    "keyFields": ["sku"],
    "ignoreFields": [
        "scrapedAt",
        "lastChecked",
        "timestamp",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("nerolabs/dataset-diff-detector").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "oldData": [
    {
      "sku": "A100",
      "title": "Blue Widget",
      "price": 19.99,
      "stock": 42
    },
    {
      "sku": "A101",
      "title": "Red Widget",
      "price": 24.99,
      "stock": 0
    },
    {
      "sku": "A102",
      "title": "Green Widget",
      "price": 15.5,
      "stock": 8
    }
  ],
  "newData": [
    {
      "sku": "A100",
      "title": "Blue Widget",
      "price": 17.99,
      "stock": 30
    },
    {
      "sku": "A102",
      "title": "Green Widget",
      "price": 15.5,
      "stock": 8
    },
    {
      "sku": "A103",
      "title": "Yellow Widget",
      "price": 12,
      "stock": 100
    }
  ],
  "keyFields": [
    "sku"
  ],
  "ignoreFields": [
    "scrapedAt",
    "lastChecked",
    "timestamp"
  ]
}' |
apify call nerolabs/dataset-diff-detector --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,nerolabs/dataset-diff-detector"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/LSGrUPrLWIt2rmhho/builds/IjaN2SWS1iYXliNFm/openapi.json