# Dataset Diff: Only New & Changed Items From Any Actor (`fayoussef/dataset-diff`) Actor

Point it at any Apify Actor, Task or dataset and get only what changed since its last run: new rows, edited rows, rows that disappeared. Push the delta to Slack, Discord, Telegram or a webhook. No target website, so nothing breaks.

- **URL**: https://apify.com/fayoussef/dataset-diff.md
- **Developed by:** [youssef farhan](https://apify.com/fayoussef) (community)
- **Categories:**
- **Stats:** 2 total users, 1 monthly users, 0.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.02 / 1,000 item compareds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

### What does Dataset Diff do?

**Dataset Diff turns any Apify Actor into a change monitor.** Point it at an Actor, a Task or a dataset and it returns **only the items that are new, changed or gone since that source last ran**, instead of the full snapshot you get every time. It compares this run against the previous one, writes the delta to its own dataset, and pushes an alert to **Slack, Discord, Telegram or any webhook**.

Every scheduled scraper produces a complete dump on each run. You almost never want the dump. You want the three listings that appeared overnight, the price that dropped, the job posting that disappeared. This Actor is that missing step, and it works with **any Actor in the [Apify Store](https://apify.com/store)** or your own, because it reads datasets through the Apify API rather than scraping a website.

That is also why it does not break. There is no target site, no HTML parser, no anti-bot wall. The only dependency is the Apify API itself.

### Why use Dataset Diff?

- **Get only new results from any scraper.** Chain it after any listings, jobs, reviews or products Actor and stop re-reading rows you saw yesterday.
- **Price and status change alerts.** Watch a `price` or `availability` field and get pinged with the old value next to the new one.
- **Competitor monitoring.** Schedule a competitor scrape daily and let Dataset Diff tell you what moved, rather than reading 4,000 rows.
- **Cut downstream cost.** Feed 12 real changes into your AI enrichment, CRM or Google Sheet instead of the whole dataset.
- **Lead alerts in the channel you already use.** New leads land in Slack the moment the source run finishes.
- **Stop maintaining custom n8n dedupe logic.** The dedupe, the hashing, the state between runs and the digest are all handled here.

### How to use Dataset Diff

1. Run the Actor you want to watch at least once, so there is a successful run to read.
2. Paste it into **Actor, Task or dataset to watch**, in the form `username/actor-name`, or paste the link from your Apify Console.
3. Optionally add a Slack, Discord, Telegram or webhook destination under **Notifications**.
4. Click **Start**. The first run records a baseline and reports nothing, because there is nothing yet to compare against.
5. Run it again after the source Actor next runs. From now on you get only the changes.
6. Put it on a **schedule** a few minutes after the source Actor's schedule, or trigger it automatically with an Apify webhook when the source run finishes.

Everything else has a sensible default. The item key is detected automatically, volatile timestamp fields are ignored automatically, and the watchlist is named after the source automatically.

#### Try it first, with no setup

Leave the word `demo` in **Actor, Task or dataset to watch** and click **Start**. The run compares a small built-in sample dataset against its own previous snapshot and returns one new item, two changed ones (with the old price and stock status beside the new ones) and, if you also tick "Gone since last run", one removed item. It reads nothing from your account, costs nothing, and finishes in seconds. Every sample row is marked `"_demo": true` so it can never be mistaken for a real comparison.

### How do I run this automatically after another Actor?

Two ways, both native to Apify:

- **Schedule:** create a schedule for Dataset Diff that fires a few minutes after the source Actor's own schedule. It always reads the newest **successful** run.
- **Webhook:** on the source Actor, add an integration for the **Run succeeded** event pointing at Dataset Diff. Every source run then triggers a diff on its own.

### Input

The only required field is the source. A minimal input is one line:

```json
{
    "source": "apify/instagram-scraper"
}
```

A price-watch setup with a Slack alert:

```json
{
    "source": "fayoussef/autoscout24",
    "keyFields": ["url"],
    "watchFields": ["price"],
    "emit": ["new", "changed", "removed"],
    "slackWebhookUrl": "https://hooks.slack.com/services/XXX/YYY/ZZZ",
    "notifyMaxItems": 20
}
```

| Input | What it does |
|---|---|
| `source` | The Actor handle, Task, run link or dataset ID to watch. The only required field. |
| `emit` | Which changes to report: `new`, `changed`, `removed`. |
| `keyFields` | The field identifying an item across runs. Auto-detected when empty. |
| `watchFields` | Restrict `changed` to fields you care about, such as `price`. |
| `notifyMaxItems` | Changes listed inside one message before it says "and N more". |
| `slackWebhookUrl`, `discordWebhookUrl`, `telegramBotToken` + `telegramChatId`, `webhookUrl` | Where to send the alert. All optional. |
| `outputMode` | `deltaOnly` (default) or `annotated`, the whole dataset with a change column. |
| `firstRunBehaviour` | `baselineOnly` (default) or `emitAll`. |
| `ignoreFields` | Fields that change every run on their own and would otherwise mark everything as changed. |
| `monitorKey` | Names the watchlist history. Defaults to the source. |
| `maxItems` | Safety cap on how much of the source dataset to read. |

### Output

Each result row is the original item with the change columns added, so anything downstream keeps working unchanged:

```json
{
    "url": "https://www.example.com/listing/8891",
    "title": "2019 Volkswagen Golf 1.5 TSI",
    "price": 14990,
    "_change": "changed",
    "_key": "https://www.example.com/listing/8891",
    "_changedFields": ["price"],
    "_previousValues": { "price": 16250 },
    "_source": "fayoussef/autoscout24",
    "_seenAt": "2026-09-05T06:12:44Z"
}
```

Download the results as **JSON, CSV, Excel, HTML or XML**, or read them from the Apify API.

#### Data fields

| Field | Description |
|---|---|
| `_change` | `new`, `changed` or `removed`. |
| `_key` | The identity used to match this item across runs. |
| `_changedFields` | Exactly which fields moved, for a `changed` item. |
| `_previousValues` | The previous values, when `watchFields` is set. |
| `_source` | The Actor, Task or dataset the item came from. |
| `_seenAt` | When this run observed the change. |
| *(all original fields)* | Every field of the source item is carried through. |

A run summary with the counts per change type is written to the key-value store under `OUTPUT`.

### How much does it cost to monitor a scraper?

Dataset Diff is **pay per event**, so you pay for the comparison and the alerts, not for time. Comparing a 5,000-item dataset daily and sending a Slack message costs a few cents per run. Because only the delta is stored and only the delta is charged as output, watching a large dataset stays cheap even when almost nothing changes.

Costs of the **source** Actor are separate and billed by whoever owns it. This Actor never starts your source Actor; it reads the run that already happened.

**Free plan limits:** free runs compare up to 1,000 items and report up to 100 changes per run. [Subscribe to an Apify plan](https://apify.com/pricing?fpr=youssef) to lift both.

### How does it know which items are the same across runs?

By an **item key**: one field whose value identifies an item, such as `url`, `id` or `vin`. Dataset Diff samples the source dataset and picks that field automatically, choosing a field that is present and distinct on effectively every row.

If your dataset has no such field, set `keyFields` yourself. With no usable key, additions and removals are still exact, but nothing can be reported as `changed`, because an edited item then looks like one removal plus one addition. The run log says so plainly when this happens.

### What is the "demo" source?

It is a built-in sample comparison, there so you can see the output shape before wiring anything up. It is used only when the source is left as the placeholder word `demo`. Any other value is resolved for real, so a mistyped Actor handle still fails with an error naming the typo rather than quietly returning sample data.

### Why did my first run report nothing?

That is by design. The first run has no previous run to compare against, so it records a baseline and emits nothing rather than dumping the entire dataset on you and charging for it. Run it again after the source next runs and you will get the real delta. To get everything as `new` once, set **On the very first run** to "Report every item as new".

### Can it watch datasets from other Apify accounts?

Yes, if you have a token that can read them. Paste that token into **Apify API token** under "Bring your own API access". On the platform with your own datasets you never need it.

### Can an AI agent call this Actor?

Yes. Dataset Diff is available through the **Apify MCP server**, so an agent can ask what changed in a dataset since it last looked and get a compact delta instead of a full dump. That is the shape agents want: fewer tokens, only the news.

### Tips

- **Set `watchFields`** to the two or three fields you actually care about. It cuts noise, and it is what makes the old value appear next to the new one in your alert.
- **Keep `maxItems` above the size of your source dataset.** Items past the cap are not read, so they look removed.
- **Use `monitorKey`** to keep two watchlists on the same source apart, for example one for prices and one for new arrivals.
- **Use `annotated` output** when you feed a Google Sheet you keep looking at, and `deltaOnly` when you feed a workflow.

### FAQ

**Does this Actor run the source Actor for me?**
No. It reads the dataset from a run that already finished. Schedule it after the source, or trigger it from the source's "Run succeeded" webhook.

**Does it work with any Actor in the Apify Store?**
Yes, as long as the Actor writes items to a dataset, which effectively all of them do. It compares **your own runs** of that Actor, so run it at least once yourself before pointing Dataset Diff at it. Pointing at a Store Actor you have never run returns a message saying exactly that.

**Where is the previous run's data kept?**
In a named key-value store on your own account. Only a fingerprint is stored, never a copy of your items, so watching a large dataset costs a few megabytes rather than a second copy of the dataset.

**What happens if my Slack webhook fails?**
The run still succeeds and the delta is still written. The failure is reported as a line in the run's error block, never as a failed run.

**Is this legal?**
Yes. Dataset Diff reads data that is already in your own Apify account through the official API. It does not access any website. You remain responsible for how the source Actor collects its data.

**What are the limits?**
Up to 500,000 items per run. Free plan runs are capped at 1,000 items compared and 100 changes reported.

### Support

Found a bug or need a field this Actor does not return? Open an issue on the **Issues** tab and it gets looked at.

💼 Need a custom solution? Reach out at youssefarhan24@gmail.com
🌐 Got a site in mind? [Suggest it here](https://automationbyexperts.com/apify)
⚡ [Subscribe to an Apify plan](https://apify.com/pricing?fpr=youssef) to lift the free-tier limits.

# Actor input Schema

## `source` (type: `string`):

What to compare against its own previous run. Paste any one of these: an Actor handle such as apify/instagram-scraper, an Actor or Task link copied from the Apify Console, a run link, or a dataset ID. For an Actor or a Task the newest successful run of your own account is used automatically, so a scheduled setup needs nothing else. Run the source Actor at least once before pointing this at it. Leave the word "demo" here to see what the output looks like on a built-in sample dataset, at no charge.

## `emit` (type: `array`):

Which changes end up in the results. Leave the default to get appearances and edits, and add "Gone since last run" to also be told what disappeared.

## `keyFields` (type: `array`):

The field that identifies one item across runs, such as url, id or vin. Leave this empty and it is detected automatically, which is right for almost every Actor. Set it only when the automatic choice is wrong, or when identity needs two fields together.

## `watchFields` (type: `array`):

Narrow "changed" to the fields you actually care about, such as price or status. Leave empty and any edit to any field counts. Filling this in also lets the report show the old value next to the new one.

## `notifyMaxItems` (type: `integer`):

How many changes to list inside one Slack, Discord or Telegram message before it says "and N more". The full set is always in the results.

## `slackWebhookUrl` (type: `string`):

An incoming webhook URL from your Slack workspace. Create one under Slack apps, Incoming Webhooks, and pick the channel there.

## `discordWebhookUrl` (type: `string`):

A channel webhook URL from Discord, under Channel Settings, Integrations, Webhooks.

## `telegramBotToken` (type: `string`):

The token BotFather gave your own bot. Needs the chat ID below as well.

## `telegramChatId` (type: `string`):

The chat, group or channel your bot posts to.

## `webhookUrl` (type: `string`):

Any HTTPS endpoint, for n8n, Make, Zapier or your own service. It receives the changes as JSON rather than as a message.

## `outputMode` (type: `string`):

"Only what changed" is the usual choice and keeps results small. "Every item, labelled" returns the whole dataset with a change column added, which suits a Google Sheet you keep looking at.

## `firstRunBehaviour` (type: `string`):

There is nothing to compare against the first time, so the default records a baseline and reports nothing. Choose the other option to get the whole dataset as "new" once.

## `ignoreFields` (type: `array`):

Fields that change every run on their own, such as timestamps and row positions, and would otherwise mark every item as changed. A sensible list is already applied; anything you enter here replaces it.

## `monitorKey` (type: `string`):

Names the history this run compares against. Leave empty and the source names it, which is what you want. Set it to keep two separate watchlists on the same source apart.

## `maxItems` (type: `integer`):

Safety cap on how much of the source dataset to read. Items past the cap look missing, so keep this above the size of the dataset you are watching.

## `apifyToken` (type: `string`):

Optional. Leave empty: on the Apify platform the run already has the access it needs. Fill this in only to read a dataset that belongs to a different Apify account, or when running this Actor outside the platform.

## Actor input object example

```json
{
  "source": "demo",
  "emit": [
    "new",
    "changed"
  ],
  "notifyMaxItems": 20,
  "outputMode": "deltaOnly",
  "firstRunBehaviour": "baselineOnly",
  "ignoreFields": [
    "scrapedAt",
    "crawledAt",
    "timestamp",
    "position",
    "rank"
  ],
  "maxItems": 50000
}
```

# Actor output Schema

## `changes` (type: `string`):

Every new, changed and removed item found in the source dataset, with the original item's fields carried through.

## `summary` (type: `string`):

Counts per change type, the source that was read, and notification delivery status.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "source": "demo",
    "ignoreFields": [
        "scrapedAt",
        "crawledAt",
        "timestamp",
        "position",
        "rank"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fayoussef/dataset-diff").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "source": "demo",
    "ignoreFields": [
        "scrapedAt",
        "crawledAt",
        "timestamp",
        "position",
        "rank",
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("fayoussef/dataset-diff").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "source": "demo",
  "ignoreFields": [
    "scrapedAt",
    "crawledAt",
    "timestamp",
    "position",
    "rank"
  ]
}' |
apify call fayoussef/dataset-diff --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,fayoussef/dataset-diff"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/ZUxBkbycCeGE82UvO/builds/CqvZMCgCZVcFQkGaM/openapi.json