Dataset Diff: Only New & Changed Items From Any Actor

What does Dataset Diff do?

Dataset Diff turns any Apify Actor into a change monitor. Point it at an Actor, a Task or a dataset and it returns only the items that are new, changed or gone since that source last ran, instead of the full snapshot you get every time. It compares this run against the previous one, writes the delta to its own dataset, and pushes an alert to Slack, Discord, Telegram or any webhook.

Every scheduled scraper produces a complete dump on each run. You almost never want the dump. You want the three listings that appeared overnight, the price that dropped, the job posting that disappeared. This Actor is that missing step, and it works with any Actor in the Apify Store or your own, because it reads datasets through the Apify API rather than scraping a website.

That is also why it does not break. There is no target site, no HTML parser, no anti-bot wall. The only dependency is the Apify API itself.

Why use Dataset Diff?

• Get only new results from any scraper. Chain it after any listings, jobs, reviews or products Actor and stop re-reading rows you saw yesterday.
• Price and status change alerts. Watch a price or availability field and get pinged with the old value next to the new one.
• Competitor monitoring. Schedule a competitor scrape daily and let Dataset Diff tell you what moved, rather than reading 4,000 rows.
• Cut downstream cost. Feed 12 real changes into your AI enrichment, CRM or Google Sheet instead of the whole dataset.
• Lead alerts in the channel you already use. New leads land in Slack the moment the source run finishes.
• Stop maintaining custom n8n dedupe logic. The dedupe, the hashing, the state between runs and the digest are all handled here.

How to use Dataset Diff

1. Run the Actor you want to watch at least once, so there is a successful run to read.
2. Paste it into Actor, Task or dataset to watch, in the form username/actor-name, or paste the link from your Apify Console.
3. Optionally add a Slack, Discord, Telegram or webhook destination under Notifications.
4. Click Start. The first run records a baseline and reports nothing, because there is nothing yet to compare against.
5. Run it again after the source Actor next runs. From now on you get only the changes.
6. Put it on a schedule a few minutes after the source Actor's schedule, or trigger it automatically with an Apify webhook when the source run finishes.

Everything else has a sensible default. The item key is detected automatically, volatile timestamp fields are ignored automatically, and the watchlist is named after the source automatically.

Try it first, with no setup

Leave the word demo in Actor, Task or dataset to watch and click Start. The run compares a small built-in sample dataset against its own previous snapshot and returns one new item, two changed ones (with the old price and stock status beside the new ones) and, if you also tick "Gone since last run", one removed item. It reads nothing from your account, costs nothing, and finishes in seconds. Every sample row is marked "_demo": true so it can never be mistaken for a real comparison.

How do I run this automatically after another Actor?

Two ways, both native to Apify:

• Schedule: create a schedule for Dataset Diff that fires a few minutes after the source Actor's own schedule. It always reads the newest successful run.
• Webhook: on the source Actor, add an integration for the Run succeeded event pointing at Dataset Diff. Every source run then triggers a diff on its own.

Input

The only required field is the source. A minimal input is one line:

{
    "source": "apify/instagram-scraper"
}

A price-watch setup with a Slack alert:

{
    "source": "fayoussef/autoscout24",
    "keyFields": ["url"],
    "watchFields": ["price"],
    "emit": ["new", "changed", "removed"],
    "slackWebhookUrl": "https://hooks.slack.com/services/XXX/YYY/ZZZ",
    "notifyMaxItems": 20
}

Input	What it does
source	The Actor handle, Task, run link or dataset ID to watch. The only required field.
emit	Which changes to report: new, changed, removed.
keyFields	The field identifying an item across runs. Auto-detected when empty.
watchFields	Restrict changed to fields you care about, such as price.
notifyMaxItems	Changes listed inside one message before it says "and N more".
slackWebhookUrl, discordWebhookUrl, telegramBotToken + telegramChatId, webhookUrl	Where to send the alert. All optional.
outputMode	deltaOnly (default) or annotated, the whole dataset with a change column.
firstRunBehaviour	baselineOnly (default) or emitAll.
ignoreFields	Fields that change every run on their own and would otherwise mark everything as changed.
monitorKey	Names the watchlist history. Defaults to the source.
maxItems	Safety cap on how much of the source dataset to read.

Output

Each result row is the original item with the change columns added, so anything downstream keeps working unchanged:

{
    "url": "https://www.example.com/listing/8891",
    "title": "2019 Volkswagen Golf 1.5 TSI",
    "price": 14990,
    "_change": "changed",
    "_key": "https://www.example.com/listing/8891",
    "_changedFields": ["price"],
    "_previousValues": { "price": 16250 },
    "_source": "fayoussef/autoscout24",
    "_seenAt": "2026-09-05T06:12:44Z"
}

Download the results as JSON, CSV, Excel, HTML or XML, or read them from the Apify API.

Data fields

Field	Description
_change	new, changed or removed.
_key	The identity used to match this item across runs.
_changedFields	Exactly which fields moved, for a changed item.
_previousValues	The previous values, when watchFields is set.
_source	The Actor, Task or dataset the item came from.
_seenAt	When this run observed the change.
(all original fields)	Every field of the source item is carried through.

A run summary with the counts per change type is written to the key-value store under OUTPUT.

How much does it cost to monitor a scraper?

Dataset Diff is pay per event, so you pay for the comparison and the alerts, not for time. Comparing a 5,000-item dataset daily and sending a Slack message costs a few cents per run. Because only the delta is stored and only the delta is charged as output, watching a large dataset stays cheap even when almost nothing changes.

Costs of the source Actor are separate and billed by whoever owns it. This Actor never starts your source Actor; it reads the run that already happened.

Free plan limits: free runs compare up to 1,000 items and report up to 100 changes per run. Subscribe to an Apify plan to lift both.

How does it know which items are the same across runs?

By an item key: one field whose value identifies an item, such as url, id or vin. Dataset Diff samples the source dataset and picks that field automatically, choosing a field that is present and distinct on effectively every row.

If your dataset has no such field, set keyFields yourself. With no usable key, additions and removals are still exact, but nothing can be reported as changed, because an edited item then looks like one removal plus one addition. The run log says so plainly when this happens.

What is the "demo" source?

It is a built-in sample comparison, there so you can see the output shape before wiring anything up. It is used only when the source is left as the placeholder word demo. Any other value is resolved for real, so a mistyped Actor handle still fails with an error naming the typo rather than quietly returning sample data.

Why did my first run report nothing?

That is by design. The first run has no previous run to compare against, so it records a baseline and emits nothing rather than dumping the entire dataset on you and charging for it. Run it again after the source next runs and you will get the real delta. To get everything as new once, set On the very first run to "Report every item as new".

Can it watch datasets from other Apify accounts?

Yes, if you have a token that can read them. Paste that token into Apify API token under "Bring your own API access". On the platform with your own datasets you never need it.

Can an AI agent call this Actor?

Yes. Dataset Diff is available through the Apify MCP server, so an agent can ask what changed in a dataset since it last looked and get a compact delta instead of a full dump. That is the shape agents want: fewer tokens, only the news.

Tips

• Set watchFields to the two or three fields you actually care about. It cuts noise, and it is what makes the old value appear next to the new one in your alert.
• Keep maxItems above the size of your source dataset. Items past the cap are not read, so they look removed.
• Use monitorKey to keep two watchlists on the same source apart, for example one for prices and one for new arrivals.
• Use annotated output when you feed a Google Sheet you keep looking at, and deltaOnly when you feed a workflow.

FAQ

Does this Actor run the source Actor for me? No. It reads the dataset from a run that already finished. Schedule it after the source, or trigger it from the source's "Run succeeded" webhook.

Does it work with any Actor in the Apify Store? Yes, as long as the Actor writes items to a dataset, which effectively all of them do. It compares your own runs of that Actor, so run it at least once yourself before pointing Dataset Diff at it. Pointing at a Store Actor you have never run returns a message saying exactly that.

Where is the previous run's data kept? In a named key-value store on your own account. Only a fingerprint is stored, never a copy of your items, so watching a large dataset costs a few megabytes rather than a second copy of the dataset.

What happens if my Slack webhook fails? The run still succeeds and the delta is still written. The failure is reported as a line in the run's error block, never as a failed run.

Is this legal? Yes. Dataset Diff reads data that is already in your own Apify account through the official API. It does not access any website. You remain responsible for how the source Actor collects its data.

What are the limits? Up to 500,000 items per run. Free plan runs are capped at 1,000 items compared and 100 changes reported.

Support

Found a bug or need a field this Actor does not return? Open an issue on the Issues tab and it gets looked at.

💼 Need a custom solution? Reach out at youssefarhan24@gmail.com 🌐 Got a site in mind? Suggest it here ⚡ Subscribe to an Apify plan to lift the free-tier limits.