Dataset Diff & Alert Monitor

Turn any Apify Actor or dataset into a change monitor. Run it on a schedule: it runs your source Actor (or reads a dataset you already have), compares the result with the last run by a key you choose, and sends the added, changed and removed rows to a webhook, Slack, Discord or Telegram. Every change is also written to the run's dataset, so n8n, Make, Zapier or your own code can pick it up.

No scraping of its own, no browser, no proxies. You pay per row compared and per message sent; the source Actor is billed at its own price.

What it is for

• New job postings — run a LinkedIn, Indeed, Dice or Greenhouse jobs Actor daily and get only the new postings in Slack.
• Price and stock watch — run a product scraper twice a week; watchFields: ["price", "inStock"] reports only those changes, with the previous value.
• New tenders, permits, listings, grants — any registry scraper becomes an alert feed.
• Review and rating watch — new reviews on Google Maps, Trustpilot or the app stores.
• Compare two datasets once — pass previousDatasetId to diff any two datasets without a schedule.

How it works

1. Source — set sourceActorId (username/actor-name) plus sourceInput, or sourceTaskId, or an existing datasetId.
2. Key — keyFields identify an item across runs: ["url"], ["id"], ["title","company"]. Nested paths like product.sku work.
3. Diff — items are hashed on their watched fields. ignoreFields drops fields that change every run (scrapedAt, rank); watchFields restricts changes to the fields you care about.
4. Snapshot — the current state is saved in a named key-value store (snapshotStoreName, default dataset-diff-monitor) under snapshotKey. Leave the key empty and one is derived from the source and its input, so each schedule gets its own.
5. Notify — a digest goes to every configured channel; the generic webhook gets a JSON body with the summary and the rows.

The first run stores the baseline silently (firstRunBehavior: "snapshot") or reports every item as added ("notify").

Re-posts (optional)

Listings get taken down and published again under a new id: a job re-opened, a classified renewed, a whole careers board re-published. A diff by id reports each of those as one item removed and one item added. Set repostKeyFields to the fields that stay the same when that happens — ["title", "location"] for job listings, or ["roleKey"] with mdcurt/ats-job-board-scraper — and a removed item and an added item with equal values in all of them are paired and reported once as reposted.

• Pairing is one-to-one: three removals and one addition with the same values leave two removals.
• Values are compared case-insensitively with whitespace collapsed. An item missing any of the fields is never paired.
• Add "reposted" to notifyOn to list the pairs (each row carries previousKey and previous). Leave it out and they are dropped from the added and removed lists and only counted in the digest header: 2 added, 1 removed, 14 reposted (not listed).
• Choose fields that separate different items. On nine company job boards (November 2026), 189 of 1,281 live listings shared a board and title with another live listing; with the location added it was 6.
• It needs the previous items, so nothing is paired against a hashes-only snapshot (see Limits).

Measured on those nine boards over five days: 89 listings removed, 92 added, and 15 of the 89 were re-posts — 14 of them one company re-publishing most of its board.

Guards against incomplete runs (optional)

A source that was down looks exactly like a source that removed everything. Two checks stop a bad run from being reported as mass removals; both are off until you set them.

• guardGroupField — the field that says which source an item came from (board, company, shop, source.site). If a group that had 3 or more items last time has none at all in this run, its removals are held back. Example: ten job boards in one run, one board answers with a maintenance page — the other nine are diffed normally and the tenth is reported as held, not as 78 closed roles.
• maxRemovedPercent — if more than this share of the previous items is missing, the whole run is treated as incomplete (source blocked, timed out or cut short) and all removals are held back. 50 is a sensible value for most monitors.

Held-back removals are not written to the dataset and not listed in the digest; they stay in the snapshot, the digest opens with a warning line (⚠ 78 removals held back: no items at all this run for board "adobe" …), and OUTPUT.guard and the webhook body carry the same information. A message is sent for a held-back removal even when nothing else changed. If the items come back, nothing was ever reported. If they are still missing after 3 runs they are reported as removed on the next one, so something that is really gone is delayed, never lost. Added and changed items are reported as usual on a guarded run.

The group check needs the previous items, so it is skipped against a hashes-only snapshot (see Limits); maxRemovedPercent still works there. A group that merely shrinks — even to one item — is ordinary change and is reported.

Input example

{
  "sourceActorId": "curious_coder/linkedin-jobs-scraper",
  "sourceInput": { "title": "data engineer", "location": "Chicago", "rows": 100 },
  "keyFields": ["id"],
  "ignoreFields": ["scrapedAt", "postedAt"],
  "notifyOn": ["added"],
  "labelField": "title",
  "urlField": "link",
  "slackWebhookUrl": "https://hooks.slack.com/services/…"
}

Schedule this Actor, not the source Actor: the monitor runs the source for you.

Output

Dataset — one row per change the run reports:

{
  "changeType": "changed",
  "key": "https://example.com/item/42",
  "label": "Item 42",
  "url": "https://example.com/item/42",
  "changedFields": ["price"],
  "item": { "url": "https://example.com/item/42", "price": 19.99 },
  "previous": { "url": "https://example.com/item/42", "price": 24.99 },
  "detectedAt": "2026-10-24T19:47:51.899Z",
  "snapshotKey": "snap-3f1c9a2b7e4d1c0a",
  "sourceDatasetId": "…",
  "sourceRunId": "…"
}

Key-value store OUTPUT — the run summary: counts, what it was compared against, which channels were notified, and the snapshot key.

Webhook body — { title, detectedAt, summary: {added, changed, removed, reposted, held, unchanged, compared}, guard, added: [...], changed: [...], removed: [...], reposted: [...], truncated } (guard is null unless removals were held back). Rows per type are capped at maxItemsPerMessage (default 20); the dataset always holds all of them.

Chat digest (Slack, Discord, Telegram):

Diff monitor: curious_coder/linkedin-jobs-scraper: 4 added, 2 changed, 3 removed (44 items compared)

ADDED (4)
• Senior Data Engineer https://…
…
CHANGED (2)
• Data Engineer II — salary https://…

Discord messages are split at 2,000 characters, Telegram at 4,000.

Pricing

Pay per event, no platform usage charges on top:

Event	Price
Actor start	$0.02
Row compared (current items + removed items)	$0.001
Notification message sent	$0.01

A daily monitor over 500 items with one Slack digest costs about $0.53 per run plus the source Actor's own price. Reading an existing dataset instead of running a source Actor costs only the comparison.

Use from n8n, Make, MCP or code

• n8n / Make — add the Apify node, pick this Actor, paste the input JSON. Or let the Actor call your workflow's webhook URL directly through webhookUrl.
• MCP — https://mcp.apify.com/?tools=mdcurt/dataset-diff-monitor
• API — POST https://api.apify.com/v2/acts/mdcurt~dataset-diff-monitor/runs?token=… with the input as the body.

Permissions

This Actor runs with limited permissions: it never sees your account-wide storages or token. That has two consequences:

• A dataset you want to diff must be selected in the input form (the dataset picker grants read access), or produced by the source Actor this monitor runs. A dataset ID pasted as plain text through the API only works for datasets you own.
• The source Actor must itself be a limited-permissions Actor (most Store Actors are). If yours needs full permissions, schedule it yourself and point this monitor at its dataset.

The snapshot store is created by this Actor on its first run and stays readable to it afterwards.

Limits and notes

• Items with no value in any key field are skipped and counted in OUTPUT.counts.skippedNoKey.
• Duplicate keys keep the last item; the count is reported.
• Snapshots above ~8 MB keep hashes only, so previous is empty in changed rows for very large datasets and neither repostKeyFields nor guardGroupField can work on the next run. Use watchFields to shrink what is compared.
• A failing channel does not block the others; failures are listed in OUTPUT.notifications.failed.
• dryRun: true computes and writes the diff but sends nothing and leaves the snapshot untouched.

Why this exists

Freelance boards in October 2026 kept asking for the same thing with different sources: a multi-site stock alert bot, a daily digest of CIO job postings across four boards, a procurement watcher that alerts on new solicitations. Each of those is a source Actor plus this monitor.

Changelog

• 1.2.0 (2026-11-14) — re-post pairing: optional repostKeyFields; a removed and an added item with equal values in those fields are reported once as reposted (new notifyOn value, previousKey on those rows, reposted in the summary counts and the webhook body). Guards against incomplete runs: optional guardGroupField (a group with no items at all this run is treated as a source that was down) and maxRemovedPercent (too much of the previous run missing); the removals they catch are held back for up to 3 runs instead of reported, with a warning line in the digest and OUTPUT.guard. Nothing changes for inputs that set none of the three, except that summary.reposted: 0, summary.held: 0, guard: null and previousKey: null now appear in the output.
• 1.1.1 (2026-10-31) — webhookUrl is now a secret input like the Slack, Discord and Telegram fields (encrypted, hidden from logs); no functional change.
• 1.1.0 (2026-10-30) — dataset inputs became resource pickers (works under limited permissions); permission and not-found errors end the run with a plain explanation; prefilled example input runs out of the box.
• 1.0.0 (2026-10-24) — first release.