Change Watcher

Watch a URL. Get an event when it really changes. Pay only for the events.

An Apify Actor. Declare one or more URLs with an optional selector; it fetches them, extracts only the block you named, strips the rendering noise, compares against the value it stored last time, and emits a normalized event when — and only when — the watched block actually moved.

A run where nothing changed writes nothing and charges nothing.

Why this exists

The Model Context Protocol has no push. Issues #179 and #611 have been open for over a year. Until that changes, every agent that needs to react to something changing has exactly one option: poll it. Polling means pulling a page into the context window on a schedule, paying tokens to read it, and paying again to conclude that nothing happened. It is expensive, and it is slow in exactly the case that matters — the interval between the change and the next poll.

This Actor inverts it. It does the polling once, off the agent's clock and off the agent's token budget, and hands back a small structured event when there is something to say. The fetch is the cost, and one fetch serves every subscriber watching the same target, so the marginal cost of the second subscriber to a URL is close to zero.

The hard part is not the diff

A naive diff of any live page fires on every single check. The header clock moved. The CSRF token rotated. The view counter ticked. The CDN handed out a fresh cache-busting query string. The server-side renderer emitted the attributes in a different order. None of that is a change, and a watcher that reports all of it is worse than useless, because a subscriber learns to ignore it.

So most of this Actor is the part that decides what not to report:

Problem	What it does
Page-level noise	Hashes the selected block, never the whole document
Timestamps, dates, relative times	Masked with stable placeholders before hashing
Session ids, CSRF tokens, nonces, JWTs, UUIDs, hex and base64 blobs	Masked
View counters, "12 people are viewing this"	Masked
Cache-busting and tracking query parameters	Masked
HTML attribute order, build-hashed class names, framework-generated ids	Normalized away
<script>, <style>, HTML comments	Removed before the text is taken
JSON key order	Serialized in sorted key order
A/B buckets and half-deployed clusters	Stability window: N consecutive checks
Residual jitter no generic rule catches	minChangeRatio noise floor, plus custom patterns

What is deliberately not masked: bare numbers. A price, a stock level, a version and a score are all bare numbers and they are exactly what people watch. Masking them would make the Actor detect nothing at all.

Minimal input

{
  "watchers": [
    { "url": "https://example.com", "selector": "h1" }
  ]
}

The first run records a baseline and emits nothing — there is nothing to compare against yet. Every run after that emits an event if the block moved.

A fuller example

{
  "watchers": [
    {
      "label": "Pricing table",
      "url": "https://example.com/pricing",
      "selectorType": "css",
      "selector": "#pricing .plan-price"
    },
    {
      "label": "Service status",
      "url": "https://api.example.com/v1/status",
      "mode": "json",
      "selectorType": "jsonpath",
      "selector": "$.components[?(@.name == 'API')].status"
    },
    {
      "label": "Release tag",
      "url": "https://api.example.com/v1/releases/latest",
      "mode": "json",
      "selector": "$.tag_name",
      "headers": { "authorization": "Bearer REPLACE_ME" }
    }
  ],
  "stabilityChecks": 3,
  "minChangeRatio": 0.01,
  "topicPrefix": "change"
}

What a watcher is

Field	Meaning
url	Required, absolute http/https
selector	The query. Omit it to watch the whole body.
selectorType	css, xpath, regex, jsonpath or wholeBody. Inferred only when unambiguous: no selector means wholeBody, and a selector in json mode means jsonpath.
mode	text (default) or json
extractAs	text (default) or html. html compares markup structure, not just the words.
label	Used in logs and in the topic. Defaults to the hostname.
id	Stable identifier. Derived from host and selector when omitted.
topic	Overrides the generated topic entirely.
method, headers, body	For APIs that need a POST or an auth header.
joinWith	Separator when the selector matches several nodes. Defaults to a newline.

Selector support, honestly

• CSS — full support, via cheerio.
• XPath — a documented subset, translated to CSS: //tag, /a/b, //*, [@attr], [@attr='v'], [n], contains(@attr,'v'), starts-with(@attr,'v'), trailing /text() and /@attr. Axes, unions, position() arithmetic and [text()='x'] are rejected with an error, not silently mistranslated. A silently wrong selector would compare the wrong block forever, which is much worse than a loud failure.
• Regex — capture group 1 when the pattern has one, the whole match otherwise.
• JSONPath — $, .name, ['name'], [n], [-1], [*], [a:b], ..name, and filters with ==, !=, >, >=, <, <=, =~ and bare presence.

The event

{
  "topic": "change.example-com.pricing-table",
  "schema": "apify.change-watcher/change-event/v1",
  "timestamp": "2026-07-27T15:00:37.455Z",
  "url": "https://example.com/pricing",
  "watcherId": "example-com-pricing-plan-price",
  "label": "Pricing table",
  "changeType": "updated",
  "oldValue": "49.99 EUR",
  "newValue": "54.99 EUR",
  "diff": {
    "similarity": 0.818,
    "addedLines": ["54.99"],
    "removedLines": ["49.99"],
    "unifiedDiff": "-49.99\n+54.99",
    "changedCharacters": 10,
    "truncated": false
  },
  "confidence": 0.78,
  "meta": {
    "checkCount": 42,
    "confirmations": 3,
    "requiredConfirmations": 3,
    "hashBefore": "535fa30d...",
    "hashAfter": "6b51d431...",
    "selectorType": "css",
    "selector": "#pricing .plan-price",
    "mode": "text",
    "appliedIgnoreRules": ["unicode", "isoTimestamp", "..."],
    "candidateFirstSeenAt": "2026-07-27T14:58:31.002Z",
    "previousChangeAt": "2026-07-19T09:12:00.000Z",
    "confidenceFactors": {
      "base": 0.9,
      "stabilityFactor": 1,
      "magnitudeFactor": 1,
      "integrityFactor": 1
    }
  }
}

changeType is created when the block appears (or the selector starts matching), removed when it stops matching, updated otherwise.

Confidence

Confidence is a product of four independent factors, all of them reported in meta.confidenceFactors so nobody has to take the number on faith:

• stability — how many consecutive checks the value survived. One check is a guess; three is evidence.
• magnitude — a change of a handful of characters in a large block is more likely to be noise that slipped past the filters than real news (scored down), and a change that replaces almost the entire block usually means the page broke rather than that the content was rewritten (also scored down).
• integrity — the block collapsing to nothing, or the selector losing its match entirely, are reported as changes because they are, but they are the outcomes most likely to be an upstream problem.

The stability window

A differing value does not become an event. It becomes a candidate, and it has to show up again unchanged on stabilityChecks consecutive checks before it is confirmed.

• A candidate replaced by yet another value resets the counter to one.
• A candidate that reverts to the confirmed value is dropped outright — that is the A/B bucket and rolling-deploy case, and it is the single most common source of phantom events.

The window can close across scheduled runs (state persists) or inside one run by setting checksPerRun above 1. Note that a run with checksPerRun above 1 stays alive for (checksPerRun - 1) x checkIntervalSeconds and pays compute for the wait.

State

State lives in a named key-value store, change-watcher-state by default. The default store of a run is scoped to that run, so state written there would be gone by the next check; the named store is what turns a series of one-shot runs into a watcher with a memory.

Each watcher gets its own record, keyed by its id and a fingerprint of its extraction settings (url, mode, selectorType, selector, extractAs, joinWith, method, body, headers). Two consequences, both deliberate:

• Two watchers on the same URL with different selectors never share a baseline. Sharing one would make each report the other's value as a change on every single check.
• Editing a selector starts a clean baseline instead of producing one guaranteed false change against a value that was extracted a different way.

Set resetState: true to drop the baselines and start over — do that after changing the noise filters, since the stored baseline was normalized under the old rules.

Output

Where	What
Dataset	One item per confirmed change event. Signal only — an empty dataset is an unambiguous "nothing happened".
OUTPUT (key-value store)	Run summary: per-watcher checks performed, baseline registered, change detected, pending candidate and its confirmation count, 304s, errors.
EVENTS (key-value store)	The same events as one JSON array, for consumers that want a single request.
change-watcher-state store	The persisted baselines. Inspect these to see the exact normalized value a watcher is comparing against.

The dataset has three views: Change events (what moved), Before and after (the values), and Detection details (hashes, confirmations, confidence factors).

Cost behaviour

Two pay-per-event events:

Event	Charged when
watcher-registered	The first time a target is baselined. Cheap.
change-event-emitted	Per confirmed change event. This is the one that pays.

A check that finds nothing is not charged at all — not at a reduced rate, not a page fee. That is the whole proposition: polling costs the agent tokens on every check whether or not anything happened; this costs nothing until there is something to say.

Conditional requests are on by default. When the server supports ETag or Last-Modified, a 304 answers "did this change?" for the price of the headers — no body, no parsing, no diff.

Running it

npm install
npm test          # builds, then runs the suite
npm start         # builds and runs once against storage/key_value_stores/default/INPUT.json
apify run         # same, through the Apify CLI
apify push        # deploy to your account

To watch continuously, schedule the Actor on the Apify platform at whatever interval suits the target. State carries over between runs automatically.

Limits and known gaps

• No JavaScript rendering. The Actor fetches HTML; it does not run a browser. A value that only exists after client-side hydration will not be seen. Watch the API the page calls instead — it is cheaper and more stable anyway.
• No proxy support. Targets that block datacenter traffic will need one; it is not wired up yet.
• XPath is a translated subset, as described above.
• epochMillis masking is off by default. It would swallow any 10- or 13-digit number, and some of those are content.
• clockTime masks ratios that look like times. 16:9 reads as a clock. Turn the rule off if that matters for your target.
• Requests are made sequentially. Fifty watchers on slow hosts make for a slow run.

Layout

src/
  main.ts              entry point, run summary, charging
  engine.ts            the check loop
  input.ts             input parsing and validation
  fetcher.ts           HTTP with retries and conditional requests
  extract/
    index.ts           extractor dispatch, stable JSON serialization
    xpath.ts           XPath subset to CSS translation
    jsonpath.ts        JSONPath evaluator
  normalize/
    rules.ts           the anti-noise catalogue
    text.ts            normalization pipeline and hashing
    html.ts            markup-level de-noising
  diff.ts              LCS diff and the confidence score
  stability.ts         the stability window (pure, no I/O)
  store.ts             persistent state and key namespacing
  event.ts             event construction and topics
  billing.ts           pay-per-event charging
tests/                 106 tests across normalization, extraction, diff, window, input, engine