# Web Watchdog - Website Change Monitor & Diff API for AI Agents (`santhej/web-watchdog`) Actor

Check if any URL changed since a given content hash, or watch it on a schedule. Diffs web pages, JSON APIs, RSS feeds and sitemaps and returns what changed, old vs new values, numeric deltas and a one-line summary. Billed per change detected, not per check. Strongest on APIs, docs and changelogs.

- **URL**: https://apify.com/santhej/web-watchdog.md
- **Developed by:** [Santhej Kallada](https://apify.com/santhej) (community)
- **Categories:** Other, Developer tools
- **Stats:** 2 total users, 2 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: 5.00 out of 5 stars

## Pricing

from $0.30 / 1,000 target checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Web Watchdog — Website Change Monitor & Diff API for AI Agents

Check whether any URL changed since a given content hash, or watch it on a schedule.

Web Watchdog diffs **web pages, JSON APIs, RSS/Atom feeds and XML sitemaps** and returns what changed:
old vs new values, added and removed lines, pre-extracted numeric deltas, and one deterministic
human- and LLM-readable sentence. Every row is flat JSON with the same 34 keys, every time.

**You are billed per change detected, not per check.** A quiet run costs three tenths of a cent plus
$0.0003 per target. A run that finds nothing produces an empty dataset and zero webhook calls.

***

### What this actor is good at — and where it is not

This is a **change detector for machine-readable and text-stable sources**. It is strongest on:

- **JSON APIs** — package registries, release feeds, status APIs, your own endpoints
- **Documentation and changelogs** — release notes, `CHANGELOG.md`, docs pages
- **Registries and indexes** — npm, PyPI, Node dist, GitHub repo/release metadata
- **Status pages** — "All Systems Operational" → "Degraded Performance"
- **Pricing pages** — a card rate moving `2.9% → 2.7%`
- **RSS/Atom feeds and XML sitemaps** — new items, new or updated URLs

#### The measured numbers, published rather than implied

The economics rest on HTTP conditional GET: if a server answers `If-None-Match` / `If-Modified-Since`
with **304 Not Modified**, the check costs ~1.5 KB on the wire and zero parse CPU, versus ~30 KB gzip
for a full body — a **20× bandwidth reduction per target**. But not every server plays along, and we
measured how many actually do:

| Target class | Servers returning a real 304 |
|---|---|
| JSON APIs (`registry.npmjs.org`, `api.github.com`, `pypi.org`, `nodejs.org/dist`) | **~100%** (4/4) |
| A realistic API / registry / docs / CDN mix | **67%** |
| Arbitrary consumer HTML | **33%** |

Re-verification found this **drifts**: 4/4 JSON APIs still return real 304s, but 0/4 of the HTML pages
retested did — `stripe.com`, `status.openai.com` and `anthropic.com/pricing` send no validator at all,
and `docs.apify.com/platform` **stopped sending an ETag or Last-Modified between probes**, having
previously sent both.

**So, plainly: dynamic marketing pages land on the full-scrape cost profile, not the cheap one.**
They still work — the normalisation pipeline and the diff do not care — but the conditional-GET
saving is not there, and you should expect to pay the per-check price with no shortcut. If you want
the cheap path, watch the API behind the page rather than the page.

Two more traps we handle rather than hide:

- **"Validator present but revalidation ignored."** `bbc.com/news` and `ycombinator.com/companies`
  both send a valid `ETag` and then answer `If-None-Match` with a **full 200**. Naive implementations
  assume "has validator therefore cheap" and silently revert to full-scrape economics. We count
  consecutive misses per target, stop sending validators after 3, and surface
  `revalidationHonoured: false` on the row so the cost profile is visible.
- **JavaScript-rendered pages.** There is **no browser in v1**. `openai.com/api/pricing/` and
  `ycombinator.com/companies` yield 0.0 KB of extractable static text while their raw HTML
  demonstrably changes — a hash that is stable forever and meaningless forever. Rather than report
  "no change" indefinitely, an unscoped watch whose normalised text is under 200 characters fails
  loudly with `errorReason: "js_rendered_no_static_content"` and `confidence: 0.3`. (This guard does
  **not** apply to scoped watches — `"In stock"` is 8 characters and perfectly valid.)

***

### Billed per change detected, not per check

Every subscription incumbent bills for **checks performed**, so what you pay is fully decoupled from
what you get: the same monthly bill whether a page changed 400 times or zero times. Here the
expensive event fires only when something actually changed and passed the significance triggers.

| Event | Price (FREE tier) | When |
|---|---|---|
| `actor-start` | **$0.003** | Once per run |
| `target-checked` | **$0.0003** | Per target that returned a **definitive answer** — a 304, a parsed 200, or a real 4xx |
| `change-detected` | **$0.008** → $0.006 at DIAMOND | Only when a real change is found and reported |

Volume tiers on `change-detected`: FREE $0.0080 · BRONZE $0.0076 · SILVER $0.0072 · GOLD $0.0068 ·
PLATINUM $0.0064 · DIAMOND $0.0060.

**What is deliberately NOT billed:**

- Timeouts, DNS failures, TLS failures and blocked hosts — no definitive answer, no `target-checked`
  charge. Point this at 1,000 gated pages and you pay for the run, not for 1,000 error rows.
- `baseline_captured` — the first sighting of a target, and any hash produced under a different
  normaliser version. Never billed.
- `no_change` rows, quarantined targets, and anything past `maxChangeEvents`.

Every row carries an explicit **`billed: true|false`**, so the dataset reconciles against the invoice
without inference.

#### Worked example

100 targets, checked hourly, 730 runs a month, roughly one change per run:

```
730 x ($0.003 start + 100 x $0.0003 per check + 1 x $0.008 per change) = $29.93 / month
```

For context on the same 72,000-check workload: Distill Professional is $35/mo; Visualping's page cap
(not its check cap) forces a $140/mo tier for 100 pages regardless of frequency, and 100 pages hourly
exceeds even its $350 50K plan. There is **no page cap and no subscription floor here** — a month
with zero runs costs $0.00.

**One honest caveat we would rather state than have you discover:** Distill's Flexi pay-as-you-go
rate is $0.00025/check, which is cheaper per check than `target-checked` before a single change
event — but it requires an $80/mo plan to unlock. The accurate claim is **the cheapest metered price
with no subscription floor**, not "the cheapest price".

#### Runaway-bill protection

`change-detected` is the one uncapped term, so four brakes ship with it:

| Control | Effect |
|---|---|
| `maxChangeEvents` (default 500) | Hard cap on billable change events per run. Worst case $4.00/run. Once hit the run keeps checking and stops charging, and the summary reports `budgetCapped: true`. |
| `maxBudgetUsd` | Pre-flight abort: `0.003 + targets × 0.0003 + min(maxChangeEvents, targets) × 0.008` is computed **before any fetching** and the run refuses to start if it exceeds your number. The `min()` is deliberate — each target is checked once per run, so a run can never emit more change events than it has targets. |
| Flap detection (always on) | A page that swaps **back to content it has already served** — an A/B test, a rotating banner — is reported as `changeType: "flap_suppressed"` and **not billed**, from the third sighting onward. A permanent A/B test costs $0.008 once, not $0.008 per run. Stateful mode only. |
| `noiseQuarantineRuns` (default 5) | The backstop for churn that is *not* a clean oscillation. A target that changes on 5 consecutive runs is auto-quarantined: still checked, still reported as `changeType: "quarantined"`, **not billed**, until it goes one full run quiet. |
| `sensitivity` | Semantic triggers rather than raw diff, so a cookie-banner reshuffle is not a billable event. Thresholds are on the triggers themselves — **never on how long the page is** — so a price move, an availability flip, anything inside your `selector`, and a genuine text edit are all billed the same whether the page is 100 lines or 6,000. See [What counts as significant](#what-counts-as-significant). |

> ⚠️ **Stateless mode has no quarantine and no flap detection.** `quarantined`, `changeStreak`,
> revalidation-miss tracking and the per-target hash history all live in the state blob. A stateless
> caller passes back one hash, not a set, so the actor cannot know whether a state has been seen
> before and **cannot suppress a flap** — that is a real limitation, not an omission from this list.
> In stateless mode the only spend limits are `maxChangeEvents` and `maxBudgetUsd`. Treat
> `maxBudgetUsd` as mandatory when `autoSchedule` is on.

***

### Two ways to use it

#### 1. Stateless one-shot: "has this changed since `<hash>`?"

Pass `knownHash` on **every** target and the run writes nothing, opens no store, works under any
token scope, and hands back `newHash` for you to persist. This is the shape an MCP tool call or an
agent loop wants — the agent holds the state, so the tool is idempotent and replayable, with no
remote monitor object to create, own or clean up.

```json
{
  "targets": [
    {
      "url": "https://registry.npmjs.org/apify",
      "watchId": "apify-latest",
      "jsonPath": "dist-tags.latest",
      "knownHash": "wd2:3f2a9c8e1b47d605e2c1a9f8b3d4e5061728394a5b6c7d8e9f0a1b2c3d4e5f60",
      "knownEtag": "\"6c1f-5f2a9c8e1b4\""
    }
  ]
}
```

Round-trip `newHash` → `knownHash`, `etag` → `knownEtag`, `lastModified` → `knownLastModified` and
your next call earns the 304 too.

#### 2. Scheduled watch with durable state

Give a list of URLs and let the actor keep the state. It opens one **named** key-value store
(`wd-state-{stateKey}`) holding exactly **one record** with every target's hash, validators,
check counts and streaks. Leave `stateKey` empty and it is derived from the sorted target URL set, so
re-running the same list resumes the same watch with zero bookkeeping.

```json
{
  "targets": [
    "https://registry.npmjs.org/apify",
    "https://nodejs.org/dist/index.json",
    { "url": "https://status.openai.com/", "selector": ".status-index-page .component-status" },
    { "url": "https://api.github.com/repos/apify/crawlee/releases/latest", "jsonPath": "tag_name" }
  ],
  "sensitivity": "normal",
  "intervalMinutes": 60,
  "webhookUrl": "https://n8n.example.com/webhook/watchdog",
  "maxChangeEvents": 200,
  "maxBudgetUsd": 2.5
}
```

Set `autoSchedule: true` to have the actor create or update exactly one Apify Schedule named
`watch-{stateKey}` at `intervalMinutes`. It is **off by default** because schedules are your quota
(100 on Starter) and silently consuming it would break your other automations. When it is off the run
summary still contains a ready-to-POST `scheduleRequestBody` so an agent can create it in one call.
Apify creates schedules **disabled** — enable it once and it runs.

If the named store cannot be created (restricted scoped token), the actor logs the required scope,
sets `state_store_unavailable` in the summary, and **degrades to stateless** rather than crashing.
You get usable hashes instead of a permissions error.

***

### Detection methods

`mode` is `auto` by default and dispatches on the response `Content-Type`. You can pin it globally or
per target. `method` on every row tells you how the answer was actually derived.

| Method | What it compares | Notes |
|---|---|---|
| `conditional_get_hash` | `ETag`/`Last-Modified`, then a normalised page hash | The cheap path. A 304 ends the check with zero parsing. |
| `json_path` | One value at a dotted/bracket path | The strongest primitive by far: `registry.npmjs.org/apify` is 5,216,807 bytes; watching `dist-tags.latest` reduces the watched state to **9 bytes**. `pypi.org` 192,960 → 10 B. `nodejs.org/dist` 327,629 → 11 B. |
| `css_selector` | The text inside a CSS selector | Real HTML parsing via cheerio, not regex. Scoped watches are significant by construction. |
| `rss_guid` | The **set** of item GUIDs/ids | Never a document hash — feeds reorder constantly. Emits `item_added` / `item_removed` with guids and titles. |
| `sitemap_lastmod` | `<lastmod>` values, falling back to a `<loc>` set diff | Root sitemaps are 196–1,148 bytes, so polling one is essentially free. |
| `full_body_hash` | The raw body hash, unnormalised | Escape hatch. Expect false positives on anything with a nonce or a timestamp. |

#### Normalisation is a billing-integrity feature

Raw SHA-256 over HTML flapped on **8 of 15 real pages fetched seconds apart — a 53% false-positive
rate**. Under per-change billing, every false positive is a wrongly billed event. The pipeline runs,
in this order: strip `script/style/noscript/svg/template/iframe` and preload/CSRF tags → delete
volatile attributes (`nonce`, `integrity`, `data-csrf`, `data-timestamp`, `data-testid`, `style`, …)
→ strip query strings from every `src`/`href` (kills cache-busting hashes) → scope to your selector
or `body` → extract text → mask hex blobs, UUIDs, ISO-8601 timestamps, epochs, clock times,
"3 minutes ago" and RFC-822 dates to stable sentinels → collapse whitespace.

After the pipeline: **1 of 15**, and that one was a genuine news-homepage change.

#### The hash is versioned: `wd2:<sha256>`

`newHash` always carries a `wd<version>:` prefix. Any change to the normaliser inputs changes every
hash for every URL, which would otherwise make **100% of a stateless caller's targets report
`content_changed`** against a hash that is merely stale — on the uncapped pricing term. So on a
prefix mismatch the actor emits **`baseline_captured`, never `content_changed`, and never bills**.

#### What counts as significant

`sensitivity` drives a discrete trigger set, not a magnitude float. A change is emitted, delivered and
billed if **any** trigger fires, and `triggerReasons` on the row says which:

| Trigger | Fires when |
|---|---|
| `T_SCOPED` | The change was inside your `selector` or `jsonPath` — you already told us what matters |
| `T_NUMERIC` | Any number moved by at least the sensitivity's percentage threshold |
| `T_LINES` | `changedLineCount` reached the sensitivity's line threshold |
| `T_ITEMS` | `item_added` or `item_removed` |
| `T_AVAIL` | `became_available` or `became_unavailable` |

| `sensitivity` | Numeric threshold | Lines (whole page) | Lines (scoped) |
|---|---|---|---|
| `low` | 1.0% | 10 | 1 |
| **`normal`** (default) | **0.5%** | **2** | **1** |
| `high` | any | 1 | 1 |

**There is no page-length term in that table, and that is deliberate.** The same edit gets the same
verdict whether it lands on a 100-line page or a 6,000-line one. An earlier build filtered
`T_LINES`-only changes by the *fraction* of the document that moved; because that fraction shrinks as
pages grow, it silently swallowed a genuine three-paragraph rewrite on any page over ~1,300 lines and
a twenty-line addition on a 6,000-line page. It was removed. A monitor that quietly stops reporting
is worse than a noisy one.

`changedLineCount` is `added + removed`, so **a one-line MODIFICATION counts as 2** (the old line is
removed, the new line is added) while a one-line **INSERTION** counts as 1. At the default `normal`:

| What the page did | `changedLineCount` | Emitted and billed at `normal`? |
|---|---|---|
| A price moved, e.g. `2.9%` → `2.7%` — anywhere, on a page of any size | 2 | **Yes.** `T_NUMERIC` |
| Anything at all inside your `selector` or `jsonPath` | ≥1 | **Yes.** `T_SCOPED` |
| `In stock` → `Out of stock`, `operational` → `degraded` | ≥1 | **Yes.** `T_AVAIL` |
| A new RSS item or sitemap URL | ≥1 | **Yes.** `T_ITEMS` |
| Three new changelog entries — on a 400-line page or a 4,000-line one | 3 | **Yes**, identically. `T_LINES` |
| A three-paragraph policy rewrite on a 6,000-line terms page | 6 | **Yes.** Page length is not evidence |
| One line's text was edited, first time you have seen that text | 2 | **Yes.** `T_LINES` |
| …and then the page **swapped back** to text it has served before | 2 | **No** — `changeType: "flap_suppressed"`, `billed: false` |
| An A/B copy swap: 7 lines of marketing text, no number moved, once it starts alternating | 7 | **No** — same, from the third sighting |
| One new line was inserted, nothing removed | 1 | No — it does not reach the line threshold |

#### Flap detection: the page has to go somewhere *new*

The one thing that can stop a `T_LINES`-only change from being billed is **recurrence**, not size.

The actor remembers the last **8 distinct** content hashes per watched target (inside the same single
state record — it costs no extra storage operations). If a page changes back to a state it has
already served, that is an A/B test or a rotating banner, not news:

- **A content hash never seen before always emits.** Always. Whatever the page length, whatever the
  sensitivity. This is what makes the feature safe: a real edit is always novel.
- **A hash already in the target's history does not** — unless a price moved, a stock or status state
  flipped, an item appeared or disappeared, or the change was inside your `selector`. Those four
  bypass flap detection completely, because a price that oscillates has moved *twice* and a status
  page that recovers and relapses is doing exactly what you are watching it for.
- It takes effect on the **third** sighting: baseline, then one unavoidable report the first time the
  page swaps, then silence. A permanent A/B test costs **$0.008 once**, not per run.
- `sensitivity: "high"` does **not** turn it off, because there is nothing to turn off: `high` still
  reports every novel state on every changed line. It just does not re-announce a state you hold.
- Not available in **stateless mode** — there is no history there to compare against.

A suppressed flap is **still written to the dataset**, with `triggerReasons: ["T_LINES"]`,
`significance: "trivial"`, `billed: false` and a `diffSummary` naming the hash it recognised. You can
always see what the page did; you are simply not charged or woken for it.

If a target is still noisier than you want, pick `sensitivity: "low"`, scope the watch with a
`selector` so the noise is outside it, or let `noiseQuarantineRuns` catch it after 5 consecutive
changed runs — that is the backstop for churn that never repeats itself, such as a view counter.

#### Equivalent price renderings are collapsed before hashing

`$0.30`, `30¢`, `0.30 USD` and `USD 0.30` are the same amount, and a page that A/B tests its own
formatting must not look like a price change. Before hashing, the actor canonicalises USD/EUR/GBP
amounts: thousands separators and insignificant trailing zeros are dropped, and `¢` is converted to
the major unit. Every rewrite is **value-preserving** — `$0.30` and `$0.35` stay different, `$0.305` is
never rounded to `$0.30`, and `$0.30` never becomes `€0.30`. Ambiguous forms (`30p`, `30c`, "30 cents",
and `.` used as a thousands separator as in `€1.234,56`) are deliberately left alone, because a wrong
rewrite would hide a real price change.

***

### Output — 34 keys, every key on every row

Every key is present on every row: `null`, `[]`, `false` or `0` — **never absent**. Errors are rows
with the same shape, not exceptions.

```json
{
  "eventId": "9c41e0b7a3d5f218",
  "watchId": "apify-latest",
  "url": "https://registry.npmjs.org/apify",
  "changeType": "value_changed",
  "significance": "major",
  "triggerReasons": ["T_SCOPED", "T_NUMERIC"],
  "diffSummary": "dist-tags.latest changed from 3.4.4 to 3.5.0 on https://registry.npmjs.org/apify",
  "method": "json_path",
  "selector": "dist-tags.latest",
  "oldValue": "3.4.4",
  "newValue": "3.5.0",
  "diffAdded": ["3.5.0"],
  "diffRemoved": ["3.4.4"],
  "changedLineCount": 2,
  "numbersChanged": [
    { "from": 4, "to": 5, "delta": 1, "pctChange": 25, "label": "dist-tags.latest" }
  ],
  "magnitude": 1,
  "newHash": "wd2:8b1a0c4e7f2d95360a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f6071",
  "previousHash": "wd2:3f2a9c8e1b47d605e2c1a9f8b3d4e5061728394a5b6c7d8e9f0a1b2c3d4e5f60",
  "etag": "\"7d2a-6b3c9f1e2a4\"",
  "lastModified": "Fri, 14 Aug 2026 09:12:44 GMT",
  "revalidationHonoured": true,
  "httpStatus": 200,
  "detectedAt": "2026-08-14T10:00:03.412Z",
  "previousCheckAt": "2026-08-14T09:00:02.887Z",
  "checkCount": 37,
  "confidence": 1,
  "bytesTransferred": 30184,
  "latencyMs": 268,
  "quarantined": false,
  "billed": true,
  "errorReason": null,
  "error": null,
  "webhookDelivered": true,
  "stateKey": "a91c3f7d2b04"
}
```

| # | Field | Type | Meaning |
|---|---|---|---|
| 1 | `eventId` | string | Idempotency key — see below |
| 2 | `watchId` | string | Your name, or the derived id (see below). The join key across runs |
| 3 | `url` | string | The exact URL requested |
| 4 | `changeType` | string | `content_changed \| value_changed \| item_added \| item_removed \| became_available \| became_unavailable \| baseline_captured \| no_change \| flap_suppressed \| quarantined \| rate_limited \| error` — the one field to branch on. `flap_suppressed` = the page went back to content it has already served; reported, never billed |
| 5 | `significance` | string | `trivial \| minor \| major` |
| 6 | `triggerReasons` | array | Which triggers fired. `[]` on non-emitted rows |
| 7 | `diffSummary` | string | One deterministic templated sentence. **No LLM call.** `""` when nothing changed |
| 8 | `method` | string | How the answer was derived |
| 9 | `selector` | string|null | The CSS selector or JSON path the comparison was scoped to |
| 10 | `oldValue` | string|null | Previous watched value, truncated to 2,000 chars |
| 11 | `newValue` | string|null | Current watched value, truncated to 2,000 chars |
| 12 | `diffAdded` | array | Lines present now and absent before, capped at 50 |
| 13 | `diffRemoved` | array | Lines gone, capped at 50 |
| 14 | `changedLineCount` | integer | Uncapped count, before the 50-line display cap |
| 15 | `numbersChanged` | array | `{from, to, delta, pctChange, label}` pre-extracted so you do no math |
| 16 | `magnitude` | number | `changedLineCount / (oldLines + newLines)` — the fraction of the document that moved. **Diagnostic only:** nothing in the emit or billing decision reads it, because a fraction of a long page is not a measure of significance |
| 17 | `newHash` | string | `wd2:<sha256>` — the field a stateless caller persists |
| 18 | `previousHash` | string|null | What this check compared against. `null` on `baseline_captured` |
| 19 | `etag` | string|null | Round-trip as `knownEtag` |
| 20 | `lastModified` | string|null | Round-trip as `knownLastModified` |
| 21 | `revalidationHonoured` | boolean | False when the origin sent a validator and answered 200 anyway |
| 22 | `httpStatus` | integer | 304 = cheap path fired, 200 = full body, 0 = transport failure |
| 23 | `detectedAt` | string | ISO-8601 UTC |
| 24 | `previousCheckAt` | string|null | So you know the observation window |
| 25 | `checkCount` | integer | Consecutive checks since the last change. 0 in stateless mode |
| 26 | `confidence` | number | 0–1. Drops below 0.5 when unscoped static extraction looks too thin to trust |
| 27 | `bytesTransferred` | integer | Wire bytes, compressed. ~1.5 KB on a 304 vs ~30 KB gzip for a body |
| 28 | `latencyMs` | integer | Round-trip for the final request, including retries |
| 29 | `quarantined` | boolean | Reported but not billed until the target goes quiet for a run |
| 30 | `billed` | boolean | Whether this row generated a `change-detected` charge |
| 31 | `errorReason` | string|null | `js_rendered_no_static_content \| http_error \| timeout \| dns_error \| tls_error \| rate_limited \| blocked \| selector_not_found \| json_path_not_found \| invalid_url \| body_too_large \| state_store_unavailable \| ssrf_blocked` |
| 32 | `error` | string|null | Actionable message naming the fix where one exists |
| 33 | `webhookDelivered` | boolean|null | `true` on 2xx, `false` if every in-run retry failed, `null` when no webhook was configured |
| 34 | `stateKey` | string | The resolved state key, or the literal `"stateless"` |

#### `watchId` — how the derived id is computed

Supply `watchId` on a target object and that string is used verbatim. Leave it out — which is what a
bare URL string does — and it is derived from the **canonical** form of the URL, not the raw one:

```
canonicalUrlKey(url) = the URL with
                         scheme and host lower-cased,
                         any user:password credentials removed,
                         the fragment (#...) removed,
                         a default port (:80 on http, :443 on https) removed,
                         query parameters sorted by name, then by value,
                         and a bare trailing "/" removed when there is no query

watchId = sha256(canonicalUrlKey(url)).slice(0, 12)
```

Canonicalising first is deliberate: `?b=2&a=1` and `?a=1&b=2` are the same endpoint, and a watch must
not split into two histories because a caller reordered its query string. The cost is that you cannot
reproduce the id from the raw URL you typed — so if you need to compute `watchId` on your side,
canonicalise the same way, or just **set `watchId` explicitly**, which is always exact:

```json
{ "url": "https://registry.npmjs.org/apify", "watchId": "apify-latest" }
```

Worked examples:

| Input URL | Canonical key | `watchId` |
|---|---|---|
| `https://registry.npmjs.org/apify` | `https://registry.npmjs.org/apify` | `fd0871697c5c` |
| `HTTPS://Registry.NPMjs.ORG:443/apify#top` | `https://registry.npmjs.org/apify` | `fd0871697c5c` |
| `https://example.com/api?b=2&a=1` | `https://example.com/api?a=1&b=2` | `2c92d744698a` |

`stateKey`, when you leave it empty, is derived the same way: `sha256` over the sorted, de-duplicated
set of canonical keys, truncated to 16 characters.

#### `eventId` — the idempotency key, and how to dedupe

```
eventId = sha256(watchId + "|" + previousHash + "|" + newHash + "|" + runId).slice(0, 16)
```

It is **event-addressed, not content-addressed**: `runId` and `previousHash` are both in the key.
That matters more than it sounds. A content-addressed id (`sha256(watchId + newHash)`) is byte-identical
for every recurrence of the same content state — so a target flapping `A → B → A → B` (an A/B test, a
rotating banner, a stock badge, a queue depth) emits the **same id** for run 1's `A→B` and run 3's
`A→B`, and a set-membership dedupe silently throws the second real change away. A `became_available`
event would even collide with the original `baseline_captured` for that hash.

**Recommended dedupe pattern** — safe with this key:

```js
const seen = new Set(await loadSeenEventIds());   // your store, capped/TTL'd as you like

for (const row of datasetItems) {
    if (seen.has(row.eventId)) continue;          // a duplicate DELIVERY of one event
    seen.add(row.eventId);
    await handle(row);                            // a genuinely new event, always
}
```

Use `eventId` for delivery deduplication (webhook retry, dataset re-read, run resurrection) and
`watchId` as the join key for "what is the current state of this target". Do not use `newHash` as an
event key — two different events can legitimately share it.

***

### Webhook delivery

Set `webhookUrl` and the actor POSTs **one request per emitted change event, and nothing at all when
nothing changed**. Firing is the signal. Apify's native webhooks are run-scoped, cannot carry dataset
items in their payload (the receiver has to make a second authenticated API call), and fire on every
run regardless — which wakes your agent up to learn that nothing happened.

Each POST carries the full flat 34-key body and these headers:

| Header | Value |
|---|---|
| `Content-Type` | `application/json` |
| `X-Watchdog-Event-Id` | The row's `eventId` |
| `X-Watchdog-Delivery-Attempt` | `1`, `2` or `3` |
| `X-Watchdog-Signature` | `sha256=<HMAC-SHA256 of the exact raw body>`, when `webhookSecret` is set |

10 s timeout, 3 in-run attempts at 1 s / 4 s / 12 s with jitter. Failures are persisted in state
(capped at 200) and retried at the **start of the next run before any fetching**, then dropped after
24 hours. Because the actor delivers its own events there is no platform-level retry net, so
`deliveryFailures` and `webhookDropped` are both surfaced in the run summary rather than swallowed.

Verify the signature over the **raw bytes**, before parsing:

```js
const expected = 'sha256=' + crypto.createHmac('sha256', secret).update(rawBody).digest('hex');
const ok = crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(receivedHeader));
```

***

### Run summary

Written to the key-value store record `OUTPUT` on every run:

`stateKey`, `mode` (`stateful`/`stateless`), `normalizerVersion`, `targetsChecked`,
`definitiveAnswers`, `changesDetected`, `notModified304`, `fullBodies`, `errorsByReason`,
`bytesTransferred`, `wallMs`, `estimatedChargeUsd`, `budgetCapped`, `quarantinedTargets`,
`deliveryFailures`, `webhookDropped`, `stateRecordCount`, `sameHostWarning`, `scheduleId`,
`scheduleRequestBody`.

`notModified304` versus `fullBodies` is your live cost profile: the first number is the cheap path,
the second is not.

***

### Input reference

One field is enough. `{"targets": ["https://registry.npmjs.org/apify"]}` is a complete, valid call.

| Field | Type | Default | Purpose |
|---|---|---|---|
| `targets` | array | `["https://registry.npmjs.org/apify"]` | URL strings or objects `{url, watchId, selector, jsonPath, mode, knownHash, knownEtag, knownLastModified, headers}`. Max 5,000. A bare string derives its `watchId` from the canonical URL — see [`watchId`](#watchid--how-the-derived-id-is-computed) |
| `mode` | string | `auto` | Global detection mode; per-target `mode` overrides |
| `sensitivity` | string | `normal` | `low` / `normal` / `high` — drives the trigger set |
| `webhookUrl` | string | `""` | One POST per change event, silence otherwise |
| `webhookSecret` | string | — | HMAC-SHA256 signing key (secret) |
| `maxChangeEvents` | integer | `500` | Hard cap on billable change events per run, 0–5,000 |
| `maxBudgetUsd` | number | — | Pre-flight abort. Worst case = `0.003 + targets × 0.0003 + min(maxChangeEvents, targets) × 0.008` |
| `emitUnchanged` | boolean | `false` | Also push `no_change` rows. Never affects billing |
| `stateKey` | string | `""` | Named store suffix; empty derives it from the URL set |
| `statelessMode` | boolean | `false` | Force stateless. Auto-enabled when every target has `knownHash` |
| `intervalMinutes` | integer | `60` | Cadence for the cron and staleness gap, 5–10,080 |
| **Advanced** | | | |
| `noiseQuarantineRuns` | integer | `5` | Auto-quarantine after N consecutive changed runs, 0–1,000. 0 disables. Stateful only |
| `concurrency` | integer | `50` | Global parallel cap, 1–100. Per-host is separately clamped to 2 |
| `maxBodyBytes` | integer | `5242880` | Abort past this → `body_too_large` |
| `githubToken` | string | — | Sent as Bearer on `api.github.com` targets only (secret) |
| `proxyConfiguration` | object | `{"useApifyProxy": false}` | Optional IP rotation. RESIDENTIAL is downgraded to DATACENTER |
| `autoSchedule` | boolean | `false` | Create/update exactly one Schedule `watch-{stateKey}` |

***

### Limits, safety and honest caveats

- **No JavaScript rendering in v1.** SPA-only pages fail loudly (see the SPA guard above) instead of
  reporting "no change" forever.
- **Egress guard.** Every target URL, every redirect hop (max 3, `redirect: manual`) and the
  `webhookUrl` itself are validated: `http`/`https` only, default ports only, and any host resolving
  to loopback, link-local (`169.254.0.0/16`, `fe80::/10`), RFC1918, CGNAT (`100.64/10`), unique-local
  or `0.0.0.0/8` is refused with `errorReason: "ssrf_blocked"`. This actor fetches user-supplied URLs
  on a schedule from shared cloud infrastructure, so this is a hard control, not a setting.
- **Blocked domains.** A small registrable-domain blocklist (major retail, social and
  `reddit.com`) is refused with `errorReason: "blocked"` and **never charged**. Those sites either
  hard-block datacenter traffic or forbid automated collection in their terms; a generic watcher
  pointed at them is a terms problem, not a technical one.
- **Per-host politeness costs wall time.** Global concurrency is up to 100, but per-host concurrency
  is clamped to **2** with jitter no matter what you set. 1,000 URLs spread across many hosts is
  fast; 1,000 URLs on a single host is not, and the summary raises `sameHostWarning` when the sweep
  is at risk of overrunning `intervalMinutes`.
- **Rate limits.** Unauthenticated GitHub is 60 requests/hour per IP and each run may present a
  different egress IP — set `githubToken` if you watch more than a handful of repos, or expect
  `rate_limited` rows.
- **One state record, by design.** The named store holds exactly one blob. If you ever see
  `stateRecordCount` above 2 in the summary, something is wrong — open an issue.
- **Residential proxy is excluded** on purpose: its bandwidth cost would exceed the entire rest of
  the run. Sites that require it are refused rather than served badly.

***

### Support

Issues, feature requests and target classes that misbehave: use the Issues tab on the Actor page.
Include the `watchId`, `errorReason` and `method` from the row — they are on every row precisely so a
report is one copy-paste.

# Actor input Schema

## `targets` (type: `array`):

URLs to watch — one per line. For per-URL control (a CSS selector, a JSON path, your own knownHash) use the Advanced targets field instead; entries there are added to these. A bare URL becomes {url, watchId, mode: "auto"}, where watchId is the first 12 hex characters of sha256 over the CANONICAL url — lower-cased scheme and host, default port and any user:password credentials removed, fragment removed, query parameters sorted by name — so two spellings of the same URL join the same watch. Pass your own `watchId` if you would rather not reproduce that. Max 5,000 per run. Supplying knownHash on EVERY target switches the run to stateless mode: no key-value store is touched, it works under any token scope, and the caller holds the state — this is the shape an MCP tool call wants.

## `mode` (type: `string`):

How to derive the watched value. `auto` picks per response Content-Type: JSON → JSON path / whole-document hash, RSS/Atom → item-set diff, XML sitemap → lastmod, HTML → normalised page hash. Per-target `mode` overrides this.

## `sensitivity` (type: `string`):

Drives the semantic trigger set, not a magnitude float. A change is emitted, delivered and billed if ANY trigger fires: the change was inside your selector/jsonPath (always significant), a number moved by at least the percentage threshold, at least N lines changed, an RSS/sitemap item was added or removed, or availability flipped. `triggerReasons` on every row tells you which fired. A line-count-only change on an UNSCOPED watch must also clear a magnitude floor — the fraction of the document that moved — so an A/B copy swap on a long page is not billed; every other trigger ignores the floor. low = 1.0% / 10 lines / 2% of the document, normal = 0.5% / 2 lines / 0.3% of the document, high = any changed line, no floor.

## `webhookUrl` (type: `string`):

The actor POSTs one flat JSON body per emitted change event and stays completely silent when nothing changed — so firing IS the signal. Point it at an n8n Webhook node or an agent callback. Validated by the egress guard: public hosts only, https strongly recommended.

## `webhookSecret` (type: `string`):

When set, each POST carries `X-Watchdog-Signature: sha256=<HMAC-SHA256 of the exact raw body>` so the receiver can verify the request came from this actor.

## `maxChangeEvents` (type: `integer`):

Runaway protection for the one uncapped pricing term. Once hit, the run stops charging, keeps checking, and sets `budgetCapped: true` in the run summary. Default 500 → a worst-case $4.00 per run.

## `maxBudgetUsd` (type: `number`):

Pre-flight abort. Before any fetching the actor computes actor-start + targets × target-checked + min(maxChangeEvents, targets) × change-detected; if that worst case exceeds this number the run aborts immediately with a clear message rather than part-way through a sweep. The min() is deliberate: each target is checked once per run, so a run can never emit more change events than it has targets. Strongly recommended whenever `autoSchedule` is on.

## `emitUnchanged` (type: `boolean`):

Push a `no_change` row per unchanged target too. Off by default so a quiet run produces an empty dataset — cheap to poll, unambiguous to branch on. Never affects billing.

## `stateKey` (type: `string`):

Name suffix for the durable named key-value store `wd-state-{stateKey}`. Leave empty and it is derived from the sha256 of the sorted target URL set, so re-running the same list resumes the same watch with zero bookkeeping. Ignored in stateless mode.

## `statelessMode` (type: `boolean`):

Never open a named store: each target's `knownHash` is the sole prior state and `newHash` is returned on every row for you to persist. Auto-enabled when every target carries `knownHash`, and used automatically as a fallback when named-store creation fails under a restricted scoped token. Note: quarantine, changeStreak and revalidation-miss tracking do not exist in this mode — `maxChangeEvents` and `maxBudgetUsd` are your only spend limits.

## `intervalMinutes` (type: `integer`):

Cadence used to build the cron expression for `autoSchedule` and for the ready-to-POST schedule request body in the run summary. Also used to compute the expected gap between checks for staleness warnings.

## `advancedTargets` (type: `array`):

Targets that need more than a URL, as JSON objects, ADDED to the plain URLs above. Each accepts {"url", "watchId", "selector", "jsonPath", "mode", "knownHash", "knownEtag", "knownLastModified", "headers"} — for example {"url":"https://registry.npmjs.org/crawlee","jsonPath":"dist-tags.latest"} to watch one field instead of the whole document, or {"url":"https://example.com/pricing","selector":"#price-table"} to watch one element. Supplying knownHash on EVERY target across both fields switches the run to stateless mode: no key-value store is touched, it works under any token scope, and the caller holds the state — this is the shape an MCP tool call wants.

## `noiseQuarantineRuns` (type: `integer`):

A target that changes on this many consecutive runs is auto-quarantined: still checked and still reported as `changeType: "quarantined"`, but NOT billed, until it goes one full run without changing. Set 0 to disable. Stateful mode only. Anything above 1000 is clamped to 1000.

## `concurrency` (type: `integer`):

Parallel request cap across all hosts. 50 was measured at 109 MB peak RSS. Per-host concurrency is separately clamped to 2 with jitter regardless of this value, so 1,000 URLs on one host is slow by design.

## `maxBodyBytes` (type: `integer`):

Abort a download past this and emit `body_too_large`, suggesting a `jsonPath` or `selector`. Caps the transfer line, which is the largest cost component. The 5 MB default accommodates large package-registry documents.

## `githubToken` (type: `string`):

Sent as `Authorization: Bearer` on api.github.com targets ONLY. Unauthenticated GitHub is 60 requests/hour per IP and every run may present a different egress IP, so watching more than a handful of repos without this produces intermittent `rate_limited` rows.

## `proxyConfiguration` (type: `object`):

Optional IP rotation if repeat-polling trips a host rate limit. Every recommended target class was verified reachable from a bare datacenter IP, so this is off by default. RESIDENTIAL groups are downgraded to DATACENTER with a warning — residential bandwidth would cost more than the entire rest of the run.

## `autoSchedule` (type: `boolean`):

Create or update exactly one idempotent Apify Schedule named `watch-{stateKey}` at `intervalMinutes`, and return its id. Off by default because schedules are your quota (100 on Starter) and silently consuming it would break your other automations. When off, the run summary still contains a ready-to-POST schedule request body so an agent can create it in one call.

## Actor input object example

```json
{
  "targets": [
    "https://registry.npmjs.org/apify",
    "https://registry.npmjs.org/crawlee"
  ],
  "mode": "auto",
  "sensitivity": "normal",
  "webhookUrl": "",
  "maxChangeEvents": 500,
  "emitUnchanged": false,
  "stateKey": "",
  "statelessMode": false,
  "intervalMinutes": 60,
  "advancedTargets": [],
  "noiseQuarantineRuns": 5,
  "concurrency": 50,
  "maxBodyBytes": 5242880,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "autoSchedule": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

One row per detected change. Empty when nothing changed.

## `summary` (type: `string`):

Counts, bytes, estimated charge, state key and the ready-to-POST schedule body.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "targets": [
        "https://registry.npmjs.org/apify",
        "https://registry.npmjs.org/crawlee"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("santhej/web-watchdog").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "targets": [
        "https://registry.npmjs.org/apify",
        "https://registry.npmjs.org/crawlee",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("santhej/web-watchdog").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "targets": [
    "https://registry.npmjs.org/apify",
    "https://registry.npmjs.org/crawlee"
  ]
}' |
apify call santhej/web-watchdog --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,santhej/web-watchdog"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/WNZpn949WOGc3QuLL/builds/LOkjhuoLqmekBV4Wx/openapi.json