Change & Alert Engine — Webhook on Change (any URL / RSS / JSON)

A generic change-detection engine: point it at any URL — a JSON feed, an RSS/Atom feed, or a plain HTML page — and it diffs each poll against the last seen state (stored in the actor's key-value store) and emits ONLY the changes, with before and after values, to the dataset and/or your webhook.

Turn any batch poller into a push actor: no more re-downloading an entire feed to find what's new — the actor tells you exactly which items changed, and nothing else.

Why a change & alert engine?

• Event-driven beats batch. Most data jobs re-pull the whole source every run and waste time/money on unchanged data. A delta engine wakes up, checks what actually moved, and pushes only the diff.
• One actor, every source. Recalls lists, government notices, price tables, news feeds, release notes — if it's a URL, this engine watches it.
• Structured before/after. Consumers get

  {item_id, change_type, before, after, changed_at}
  — enough to render "what changed" without keeping their own copy of the previous state.

How it works

1. Poll — the actor fetches sourceUrl once per run (schedule it with your chosen interval; the actor is a single-shot poll).
2. Parse — content is sniffed automatically:
   • JSON — arrays are used directly; object feeds auto-detect common list keys (items, results, data, records, entries…) or use itemsPath for a dot path (data.records).
   • RSS / Atom — items are parsed with stable ids (guid or link).
   • HTML — the page becomes a single monitored item keyed on a content hash (page-level change detection).
3. Diff — each item gets a stable item_id (idField, natural keys, or a content hash fallback) and a content hash. The current snapshot is compared to the persisted one:
   • new id → added (before null)
   • same id, different hash → modified (before + after full values)
   • id missing from the current fetch → removed (only with includeRemovals: true)
4. Emit — changed items are pushed to the dataset, one per item, and (optionally) one batched webhook POST is sent per run that has changes. Nothing is emitted on a clean poll.
5. Persist — the new state is saved to the key-value store so the next poll diffs against it.

Input

Field	Type	Description
sourceUrl	string	URL to monitor (JSON feed, RSS/Atom, or HTML page). When empty, the actor polls the Hacker News front page feed by default.
interval	string	Informational — how often you schedule the actor (e.g. 15m, 1h, 0 9 * * *). Included in the webhook payload.
webhook	string	Optional URL. One POST with the full changes payload per run that has changes; never fired on a clean run.
dedupe	bool	true (default): persist state and emit only diffs. false: emit the full snapshot every run without persisting.
itemsPath	string	Dot path to the item list inside a JSON object (e.g. data.records). Auto-detected when empty.
idField	string	Field to use as the stable item id. Natural keys (id, guid, itemId, permalink, slug, link, url, title, name) are tried when empty.
ignoreFields	string	Comma-separated top-level fields stripped before hashing (e.g. updatedAt) so volatile metadata doesn't cause false modifications.
includeRemovals	bool	Report items that disappeared from the feed as removed (after null). Off by default.
maxItems	int	Max items considered per poll (default 500, max 20 000).

Example input:

{
  "sourceUrl": "https://recalls-rappels.canada.ca/en/search?search_api_fulltext=",
  "interval": "15m",
  "webhook": "https://hooks.example.com/alerts",
  "dedupe": true
}

Output

One dataset item per change:

Field	Description
item_id	Stable id of the changed item.
change_type	added, modified, or removed.
before	Previous value (full item) — null for additions.
after	New value (full item) — null for removals.
changed_at	When the change was detected (ISO-8601 UTC).
sourceUrl / polledAt	Monitoring context.

Webhook payload (one POST per run with changes):

{
  "sourceUrl": "https://…",
  "polledAt": "2026-08-13T12:00:00.000000+00:00",
  "requestedInterval": "15m",
  "changeCount": 2,
  "changes": [
    {"item_id": "r1", "change_type": "modified",
     "before": {"id": "r1", "risk": "Fire"},
     "after":  {"id": "r1", "risk": "Fire + Shock"},
     "changed_at": "2026-08-13T12:00:00.000000+00:00"}
  ]
}

Run it from the API

Trigger a poll and get the changes back in one call:

curl -X POST "https://api.apify.com/v2/acts/darknezz~change-alert-engine/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"sourceUrl": "https://feeds.bbci.co.uk/news/rss.xml", "webhook": "https://hooks.example.com/alerts"}'

Or from Python with the official SDK:

from apify_client import ApifyClient

client = ApifyClient("YOUR-APIFY-TOKEN")
run = client.actor("darknezz/change-alert-engine").call(
    run_input={"sourceUrl": "https://feeds.bbci.co.uk/news/rss.xml"})
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["change_type"], item["item_id"], "-", item["after"])

Scheduling: the actor is a single-shot poll — attach an Apify Schedule (*/15 * * * * for a 15-minute monitor) and it becomes a push pipeline: the run emits only what changed since the previous poll, and your webhook fires only when there is something to say.

Worked example (verified live)

Watching the BBC News RSS feed (https://feeds.bbci.co.uk/news/rss.xml):

• Run 1 — 41 added items (the full first snapshot; every item is new because there is no prior state).
• Run 2 (minutes later, nothing changed) — 0 items; no webhook fired.
• After the feed gained a new story — exactly 1 added item for the new entry, with before: null and the full story in after.

A controlled-source test (a dataset with pre-signed items URL) confirmed the modified path too: editing an existing item's price field emitted change_type: modified with the old value in before and the new one in after — then 0 on the next clean poll.

FAQ

Can I watch several URLs in one run? The engine polls one sourceUrl per run. For N sources, run it N times (an Apify Schedule with a single URL per actor run, or one actor instance per source). Each source keeps its own state, so nothing cross-contaminates.

What happens if my webhook is down? The run FAILS and nothing is persisted — the next run re-emits the same changes, so an alert is never silently lost (at-least-once delivery).

Why do I get added + removed pairs instead of modified? The item lacks a stable id, so the engine fell back to a content hash. Set idField to a real field (e.g. id, guid, link) and edits become clean modified records.

Everything looks "modified" every poll. Volatile fields (timestamps, view counters) change every fetch. List them in ignoreFields (e.g. updatedAt,viewCount) so hashing ignores them.

Does a clean poll cost anything? Only the one-time actor-start charge. No changes, no result events, no dataset items.

Use cases

• Government / safety monitors — watch a recalls search page or a regulatory feed; get a webhook the moment a new notice appears.
• Price & inventory alerts — poll a product JSON endpoint; emit only rows whose price/stock changed, with old and new values side by side.
• News / press-release watchers — diff an RSS feed and forward only the new entries to a chat webhook.
• Status pages — monitor an HTML status page by content hash; you are notified only when the page actually changes.

Delivery semantics

• At-least-once. State is persisted only after a successful webhook delivery (or immediately when no webhook is configured). If the POST fails, the run FAILS and the next run re-emits the same changes — an alert is never silently lost.
• Within-run dedupe. Duplicate item ids in one fetch collapse to the last occurrence, so a feed that repeats entries won't double-notify.

Pricing

Pay per event — you only pay for what you extract:

• apify-actor-start — $0.01 one-time charge per run.
• result — $0.01 per change emitted (primary event, verified against the live pricing record and the code's Actor.charge("result") call).

No monthly fee, no hidden costs. A clean poll (nothing changed) extracts nothing and charges nothing beyond the actor start.

Limitations

• The engine compares current fetch vs last persisted fetch — items that leave the feed are only reported with includeRemovals: true (off by default because first-page rotation is often noise).
• JSON items without any natural id fall back to a content hash as the id — with no stable id, any edit looks like an add+remove pair; set idField for clean modified detection.
• The actor polls once per run; the interval is your schedule on the actor (or an external scheduler). It does not stay resident between polls.
• Feeds with per-item volatile timestamps can look "modified" every poll — list those fields in ignoreFields to hash only what matters.