Web Watchdog - Website Change Monitor & Diff API for AI Agents
Pricing
from $0.30 / 1,000 target checkeds
Web Watchdog - Website Change Monitor & Diff API for AI Agents
Check if any URL changed since a given content hash, or watch it on a schedule. Diffs web pages, JSON APIs, RSS feeds and sitemaps and returns what changed, old vs new values, numeric deltas and a one-line summary. Billed per change detected, not per check. Strongest on APIs, docs and changelogs.
Pricing
from $0.30 / 1,000 target checkeds
Rating
5.0
(4)
Developer
Santhej Kallada
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
2
Monthly active users
12 days ago
Last modified
Categories
Share
Web Watchdog — Website Change Monitor & Diff API for AI Agents
Check whether any URL changed since a given content hash, or watch it on a schedule.
Web Watchdog diffs web pages, JSON APIs, RSS/Atom feeds and XML sitemaps and returns what changed: old vs new values, added and removed lines, pre-extracted numeric deltas, and one deterministic human- and LLM-readable sentence. Every row is flat JSON with the same 34 keys, every time.
You are billed per change detected, not per check. A quiet run costs three tenths of a cent plus $0.0003 per target. A run that finds nothing produces an empty dataset and zero webhook calls.
What this actor is good at — and where it is not
This is a change detector for machine-readable and text-stable sources. It is strongest on:
- JSON APIs — package registries, release feeds, status APIs, your own endpoints
- Documentation and changelogs — release notes,
CHANGELOG.md, docs pages - Registries and indexes — npm, PyPI, Node dist, GitHub repo/release metadata
- Status pages — "All Systems Operational" → "Degraded Performance"
- Pricing pages — a card rate moving
2.9% → 2.7% - RSS/Atom feeds and XML sitemaps — new items, new or updated URLs
The measured numbers, published rather than implied
The economics rest on HTTP conditional GET: if a server answers If-None-Match / If-Modified-Since
with 304 Not Modified, the check costs ~1.5 KB on the wire and zero parse CPU, versus ~30 KB gzip
for a full body — a 20× bandwidth reduction per target. But not every server plays along, and we
measured how many actually do:
| Target class | Servers returning a real 304 |
|---|---|
JSON APIs (registry.npmjs.org, api.github.com, pypi.org, nodejs.org/dist) | ~100% (4/4) |
| A realistic API / registry / docs / CDN mix | 67% |
| Arbitrary consumer HTML | 33% |
Re-verification found this drifts: 4/4 JSON APIs still return real 304s, but 0/4 of the HTML pages
retested did — stripe.com, status.openai.com and anthropic.com/pricing send no validator at all,
and docs.apify.com/platform stopped sending an ETag or Last-Modified between probes, having
previously sent both.
So, plainly: dynamic marketing pages land on the full-scrape cost profile, not the cheap one. They still work — the normalisation pipeline and the diff do not care — but the conditional-GET saving is not there, and you should expect to pay the per-check price with no shortcut. If you want the cheap path, watch the API behind the page rather than the page.
Two more traps we handle rather than hide:
- "Validator present but revalidation ignored."
bbc.com/newsandycombinator.com/companiesboth send a validETagand then answerIf-None-Matchwith a full 200. Naive implementations assume "has validator therefore cheap" and silently revert to full-scrape economics. We count consecutive misses per target, stop sending validators after 3, and surfacerevalidationHonoured: falseon the row so the cost profile is visible. - JavaScript-rendered pages. There is no browser in v1.
openai.com/api/pricing/andycombinator.com/companiesyield 0.0 KB of extractable static text while their raw HTML demonstrably changes — a hash that is stable forever and meaningless forever. Rather than report "no change" indefinitely, an unscoped watch whose normalised text is under 200 characters fails loudly witherrorReason: "js_rendered_no_static_content"andconfidence: 0.3. (This guard does not apply to scoped watches —"In stock"is 8 characters and perfectly valid.)
Billed per change detected, not per check
Every subscription incumbent bills for checks performed, so what you pay is fully decoupled from what you get: the same monthly bill whether a page changed 400 times or zero times. Here the expensive event fires only when something actually changed and passed the significance triggers.
| Event | Price (FREE tier) | When |
|---|---|---|
actor-start | $0.003 | Once per run |
target-checked | $0.0003 | Per target that returned a definitive answer — a 304, a parsed 200, or a real 4xx |
change-detected | $0.008 → $0.006 at DIAMOND | Only when a real change is found and reported |
Volume tiers on change-detected: FREE $0.0080 · BRONZE $0.0076 · SILVER $0.0072 · GOLD $0.0068 ·
PLATINUM $0.0064 · DIAMOND $0.0060.
What is deliberately NOT billed:
- Timeouts, DNS failures, TLS failures and blocked hosts — no definitive answer, no
target-checkedcharge. Point this at 1,000 gated pages and you pay for the run, not for 1,000 error rows. baseline_captured— the first sighting of a target, and any hash produced under a different normaliser version. Never billed.no_changerows, quarantined targets, and anything pastmaxChangeEvents.
Every row carries an explicit billed: true|false, so the dataset reconciles against the invoice
without inference.
Worked example
100 targets, checked hourly, 730 runs a month, roughly one change per run:
730 x ($0.003 start + 100 x $0.0003 per check + 1 x $0.008 per change) = $29.93 / month
For context on the same 72,000-check workload: Distill Professional is $35/mo; Visualping's page cap (not its check cap) forces a $140/mo tier for 100 pages regardless of frequency, and 100 pages hourly exceeds even its $350 50K plan. There is no page cap and no subscription floor here — a month with zero runs costs $0.00.
One honest caveat we would rather state than have you discover: Distill's Flexi pay-as-you-go
rate is $0.00025/check, which is cheaper per check than target-checked before a single change
event — but it requires an $80/mo plan to unlock. The accurate claim is the cheapest metered price
with no subscription floor, not "the cheapest price".
Runaway-bill protection
change-detected is the one uncapped term, so four brakes ship with it:
| Control | Effect |
|---|---|
maxChangeEvents (default 500) | Hard cap on billable change events per run. Worst case $4.00/run. Once hit the run keeps checking and stops charging, and the summary reports budgetCapped: true. |
maxBudgetUsd | Pre-flight abort: 0.003 + targets × 0.0003 + min(maxChangeEvents, targets) × 0.008 is computed before any fetching and the run refuses to start if it exceeds your number. The min() is deliberate — each target is checked once per run, so a run can never emit more change events than it has targets. |
| Flap detection (always on) | A page that swaps back to content it has already served — an A/B test, a rotating banner — is reported as changeType: "flap_suppressed" and not billed, from the third sighting onward. A permanent A/B test costs $0.008 once, not $0.008 per run. Stateful mode only. |
noiseQuarantineRuns (default 5) | The backstop for churn that is not a clean oscillation. A target that changes on 5 consecutive runs is auto-quarantined: still checked, still reported as changeType: "quarantined", not billed, until it goes one full run quiet. |
sensitivity | Semantic triggers rather than raw diff, so a cookie-banner reshuffle is not a billable event. Thresholds are on the triggers themselves — never on how long the page is — so a price move, an availability flip, anything inside your selector, and a genuine text edit are all billed the same whether the page is 100 lines or 6,000. See What counts as significant. |
⚠️ Stateless mode has no quarantine and no flap detection.
quarantined,changeStreak, revalidation-miss tracking and the per-target hash history all live in the state blob. A stateless caller passes back one hash, not a set, so the actor cannot know whether a state has been seen before and cannot suppress a flap — that is a real limitation, not an omission from this list. In stateless mode the only spend limits aremaxChangeEventsandmaxBudgetUsd. TreatmaxBudgetUsdas mandatory whenautoScheduleis on.
Two ways to use it
1. Stateless one-shot: "has this changed since <hash>?"
Pass knownHash on every target and the run writes nothing, opens no store, works under any
token scope, and hands back newHash for you to persist. This is the shape an MCP tool call or an
agent loop wants — the agent holds the state, so the tool is idempotent and replayable, with no
remote monitor object to create, own or clean up.
{"targets": [{"url": "https://registry.npmjs.org/apify","watchId": "apify-latest","jsonPath": "dist-tags.latest","knownHash": "wd2:3f2a9c8e1b47d605e2c1a9f8b3d4e5061728394a5b6c7d8e9f0a1b2c3d4e5f60","knownEtag": "\"6c1f-5f2a9c8e1b4\""}]}
Round-trip newHash → knownHash, etag → knownEtag, lastModified → knownLastModified and
your next call earns the 304 too.
2. Scheduled watch with durable state
Give a list of URLs and let the actor keep the state. It opens one named key-value store
(wd-state-{stateKey}) holding exactly one record with every target's hash, validators,
check counts and streaks. Leave stateKey empty and it is derived from the sorted target URL set, so
re-running the same list resumes the same watch with zero bookkeeping.
{"targets": ["https://registry.npmjs.org/apify","https://nodejs.org/dist/index.json",{ "url": "https://status.openai.com/", "selector": ".status-index-page .component-status" },{ "url": "https://api.github.com/repos/apify/crawlee/releases/latest", "jsonPath": "tag_name" }],"sensitivity": "normal","intervalMinutes": 60,"webhookUrl": "https://n8n.example.com/webhook/watchdog","maxChangeEvents": 200,"maxBudgetUsd": 2.5}
Set autoSchedule: true to have the actor create or update exactly one Apify Schedule named
watch-{stateKey} at intervalMinutes. It is off by default because schedules are your quota
(100 on Starter) and silently consuming it would break your other automations. When it is off the run
summary still contains a ready-to-POST scheduleRequestBody so an agent can create it in one call.
Apify creates schedules disabled — enable it once and it runs.
If the named store cannot be created (restricted scoped token), the actor logs the required scope,
sets state_store_unavailable in the summary, and degrades to stateless rather than crashing.
You get usable hashes instead of a permissions error.
Detection methods
mode is auto by default and dispatches on the response Content-Type. You can pin it globally or
per target. method on every row tells you how the answer was actually derived.
| Method | What it compares | Notes |
|---|---|---|
conditional_get_hash | ETag/Last-Modified, then a normalised page hash | The cheap path. A 304 ends the check with zero parsing. |
json_path | One value at a dotted/bracket path | The strongest primitive by far: registry.npmjs.org/apify is 5,216,807 bytes; watching dist-tags.latest reduces the watched state to 9 bytes. pypi.org 192,960 → 10 B. nodejs.org/dist 327,629 → 11 B. |
css_selector | The text inside a CSS selector | Real HTML parsing via cheerio, not regex. Scoped watches are significant by construction. |
rss_guid | The set of item GUIDs/ids | Never a document hash — feeds reorder constantly. Emits item_added / item_removed with guids and titles. |
sitemap_lastmod | <lastmod> values, falling back to a <loc> set diff | Root sitemaps are 196–1,148 bytes, so polling one is essentially free. |
full_body_hash | The raw body hash, unnormalised | Escape hatch. Expect false positives on anything with a nonce or a timestamp. |
Normalisation is a billing-integrity feature
Raw SHA-256 over HTML flapped on 8 of 15 real pages fetched seconds apart — a 53% false-positive
rate. Under per-change billing, every false positive is a wrongly billed event. The pipeline runs,
in this order: strip script/style/noscript/svg/template/iframe and preload/CSRF tags → delete
volatile attributes (nonce, integrity, data-csrf, data-timestamp, data-testid, style, …)
→ strip query strings from every src/href (kills cache-busting hashes) → scope to your selector
or body → extract text → mask hex blobs, UUIDs, ISO-8601 timestamps, epochs, clock times,
"3 minutes ago" and RFC-822 dates to stable sentinels → collapse whitespace.
After the pipeline: 1 of 15, and that one was a genuine news-homepage change.
The hash is versioned: wd2:<sha256>
newHash always carries a wd<version>: prefix. Any change to the normaliser inputs changes every
hash for every URL, which would otherwise make 100% of a stateless caller's targets report
content_changed against a hash that is merely stale — on the uncapped pricing term. So on a
prefix mismatch the actor emits baseline_captured, never content_changed, and never bills.
What counts as significant
sensitivity drives a discrete trigger set, not a magnitude float. A change is emitted, delivered and
billed if any trigger fires, and triggerReasons on the row says which:
| Trigger | Fires when |
|---|---|
T_SCOPED | The change was inside your selector or jsonPath — you already told us what matters |
T_NUMERIC | Any number moved by at least the sensitivity's percentage threshold |
T_LINES | changedLineCount reached the sensitivity's line threshold |
T_ITEMS | item_added or item_removed |
T_AVAIL | became_available or became_unavailable |
sensitivity | Numeric threshold | Lines (whole page) | Lines (scoped) |
|---|---|---|---|
low | 1.0% | 10 | 1 |
normal (default) | 0.5% | 2 | 1 |
high | any | 1 | 1 |
There is no page-length term in that table, and that is deliberate. The same edit gets the same
verdict whether it lands on a 100-line page or a 6,000-line one. An earlier build filtered
T_LINES-only changes by the fraction of the document that moved; because that fraction shrinks as
pages grow, it silently swallowed a genuine three-paragraph rewrite on any page over ~1,300 lines and
a twenty-line addition on a 6,000-line page. It was removed. A monitor that quietly stops reporting
is worse than a noisy one.
changedLineCount is added + removed, so a one-line MODIFICATION counts as 2 (the old line is
removed, the new line is added) while a one-line INSERTION counts as 1. At the default normal:
| What the page did | changedLineCount | Emitted and billed at normal? |
|---|---|---|
A price moved, e.g. 2.9% → 2.7% — anywhere, on a page of any size | 2 | Yes. T_NUMERIC |
Anything at all inside your selector or jsonPath | ≥1 | Yes. T_SCOPED |
In stock → Out of stock, operational → degraded | ≥1 | Yes. T_AVAIL |
| A new RSS item or sitemap URL | ≥1 | Yes. T_ITEMS |
| Three new changelog entries — on a 400-line page or a 4,000-line one | 3 | Yes, identically. T_LINES |
| A three-paragraph policy rewrite on a 6,000-line terms page | 6 | Yes. Page length is not evidence |
| One line's text was edited, first time you have seen that text | 2 | Yes. T_LINES |
| …and then the page swapped back to text it has served before | 2 | No — changeType: "flap_suppressed", billed: false |
| An A/B copy swap: 7 lines of marketing text, no number moved, once it starts alternating | 7 | No — same, from the third sighting |
| One new line was inserted, nothing removed | 1 | No — it does not reach the line threshold |
Flap detection: the page has to go somewhere new
The one thing that can stop a T_LINES-only change from being billed is recurrence, not size.
The actor remembers the last 8 distinct content hashes per watched target (inside the same single state record — it costs no extra storage operations). If a page changes back to a state it has already served, that is an A/B test or a rotating banner, not news:
- A content hash never seen before always emits. Always. Whatever the page length, whatever the sensitivity. This is what makes the feature safe: a real edit is always novel.
- A hash already in the target's history does not — unless a price moved, a stock or status state
flipped, an item appeared or disappeared, or the change was inside your
selector. Those four bypass flap detection completely, because a price that oscillates has moved twice and a status page that recovers and relapses is doing exactly what you are watching it for. - It takes effect on the third sighting: baseline, then one unavoidable report the first time the page swaps, then silence. A permanent A/B test costs $0.008 once, not per run.
sensitivity: "high"does not turn it off, because there is nothing to turn off:highstill reports every novel state on every changed line. It just does not re-announce a state you hold.- Not available in stateless mode — there is no history there to compare against.
A suppressed flap is still written to the dataset, with triggerReasons: ["T_LINES"],
significance: "trivial", billed: false and a diffSummary naming the hash it recognised. You can
always see what the page did; you are simply not charged or woken for it.
If a target is still noisier than you want, pick sensitivity: "low", scope the watch with a
selector so the noise is outside it, or let noiseQuarantineRuns catch it after 5 consecutive
changed runs — that is the backstop for churn that never repeats itself, such as a view counter.
Equivalent price renderings are collapsed before hashing
$0.30, 30¢, 0.30 USD and USD 0.30 are the same amount, and a page that A/B tests its own
formatting must not look like a price change. Before hashing, the actor canonicalises USD/EUR/GBP
amounts: thousands separators and insignificant trailing zeros are dropped, and ¢ is converted to
the major unit. Every rewrite is value-preserving — $0.30 and $0.35 stay different, $0.305 is
never rounded to $0.30, and $0.30 never becomes €0.30. Ambiguous forms (30p, 30c, "30 cents",
and . used as a thousands separator as in €1.234,56) are deliberately left alone, because a wrong
rewrite would hide a real price change.
Output — 34 keys, every key on every row
Every key is present on every row: null, [], false or 0 — never absent. Errors are rows
with the same shape, not exceptions.
{"eventId": "9c41e0b7a3d5f218","watchId": "apify-latest","url": "https://registry.npmjs.org/apify","changeType": "value_changed","significance": "major","triggerReasons": ["T_SCOPED", "T_NUMERIC"],"diffSummary": "dist-tags.latest changed from 3.4.4 to 3.5.0 on https://registry.npmjs.org/apify","method": "json_path","selector": "dist-tags.latest","oldValue": "3.4.4","newValue": "3.5.0","diffAdded": ["3.5.0"],"diffRemoved": ["3.4.4"],"changedLineCount": 2,"numbersChanged": [{ "from": 4, "to": 5, "delta": 1, "pctChange": 25, "label": "dist-tags.latest" }],"magnitude": 1,"newHash": "wd2:8b1a0c4e7f2d95360a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f6071","previousHash": "wd2:3f2a9c8e1b47d605e2c1a9f8b3d4e5061728394a5b6c7d8e9f0a1b2c3d4e5f60","etag": "\"7d2a-6b3c9f1e2a4\"","lastModified": "Fri, 14 Aug 2026 09:12:44 GMT","revalidationHonoured": true,"httpStatus": 200,"detectedAt": "2026-08-14T10:00:03.412Z","previousCheckAt": "2026-08-14T09:00:02.887Z","checkCount": 37,"confidence": 1,"bytesTransferred": 30184,"latencyMs": 268,"quarantined": false,"billed": true,"errorReason": null,"error": null,"webhookDelivered": true,"stateKey": "a91c3f7d2b04"}
| # | Field | Type | Meaning |
|---|---|---|---|
| 1 | eventId | string | Idempotency key — see below |
| 2 | watchId | string | Your name, or the derived id (see below). The join key across runs |
| 3 | url | string | The exact URL requested |
| 4 | changeType | string | content_changed | value_changed | item_added | item_removed | became_available | became_unavailable | baseline_captured | no_change | flap_suppressed | quarantined | rate_limited | error — the one field to branch on. flap_suppressed = the page went back to content it has already served; reported, never billed |
| 5 | significance | string | trivial | minor | major |
| 6 | triggerReasons | array | Which triggers fired. [] on non-emitted rows |
| 7 | diffSummary | string | One deterministic templated sentence. No LLM call. "" when nothing changed |
| 8 | method | string | How the answer was derived |
| 9 | selector | string|null | The CSS selector or JSON path the comparison was scoped to |
| 10 | oldValue | string|null | Previous watched value, truncated to 2,000 chars |
| 11 | newValue | string|null | Current watched value, truncated to 2,000 chars |
| 12 | diffAdded | array | Lines present now and absent before, capped at 50 |
| 13 | diffRemoved | array | Lines gone, capped at 50 |
| 14 | changedLineCount | integer | Uncapped count, before the 50-line display cap |
| 15 | numbersChanged | array | {from, to, delta, pctChange, label} pre-extracted so you do no math |
| 16 | magnitude | number | changedLineCount / (oldLines + newLines) — the fraction of the document that moved. Diagnostic only: nothing in the emit or billing decision reads it, because a fraction of a long page is not a measure of significance |
| 17 | newHash | string | wd2:<sha256> — the field a stateless caller persists |
| 18 | previousHash | string|null | What this check compared against. null on baseline_captured |
| 19 | etag | string|null | Round-trip as knownEtag |
| 20 | lastModified | string|null | Round-trip as knownLastModified |
| 21 | revalidationHonoured | boolean | False when the origin sent a validator and answered 200 anyway |
| 22 | httpStatus | integer | 304 = cheap path fired, 200 = full body, 0 = transport failure |
| 23 | detectedAt | string | ISO-8601 UTC |
| 24 | previousCheckAt | string|null | So you know the observation window |
| 25 | checkCount | integer | Consecutive checks since the last change. 0 in stateless mode |
| 26 | confidence | number | 0–1. Drops below 0.5 when unscoped static extraction looks too thin to trust |
| 27 | bytesTransferred | integer | Wire bytes, compressed. ~1.5 KB on a 304 vs ~30 KB gzip for a body |
| 28 | latencyMs | integer | Round-trip for the final request, including retries |
| 29 | quarantined | boolean | Reported but not billed until the target goes quiet for a run |
| 30 | billed | boolean | Whether this row generated a change-detected charge |
| 31 | errorReason | string|null | js_rendered_no_static_content | http_error | timeout | dns_error | tls_error | rate_limited | blocked | selector_not_found | json_path_not_found | invalid_url | body_too_large | state_store_unavailable | ssrf_blocked |
| 32 | error | string|null | Actionable message naming the fix where one exists |
| 33 | webhookDelivered | boolean|null | true on 2xx, false if every in-run retry failed, null when no webhook was configured |
| 34 | stateKey | string | The resolved state key, or the literal "stateless" |
watchId — how the derived id is computed
Supply watchId on a target object and that string is used verbatim. Leave it out — which is what a
bare URL string does — and it is derived from the canonical form of the URL, not the raw one:
canonicalUrlKey(url) = the URL withscheme and host lower-cased,any user:password credentials removed,the fragment (#...) removed,a default port (:80 on http, :443 on https) removed,query parameters sorted by name, then by value,and a bare trailing "/" removed when there is no querywatchId = sha256(canonicalUrlKey(url)).slice(0, 12)
Canonicalising first is deliberate: ?b=2&a=1 and ?a=1&b=2 are the same endpoint, and a watch must
not split into two histories because a caller reordered its query string. The cost is that you cannot
reproduce the id from the raw URL you typed — so if you need to compute watchId on your side,
canonicalise the same way, or just set watchId explicitly, which is always exact:
{ "url": "https://registry.npmjs.org/apify", "watchId": "apify-latest" }
Worked examples:
| Input URL | Canonical key | watchId |
|---|---|---|
https://registry.npmjs.org/apify | https://registry.npmjs.org/apify | fd0871697c5c |
HTTPS://Registry.NPMjs.ORG:443/apify#top | https://registry.npmjs.org/apify | fd0871697c5c |
https://example.com/api?b=2&a=1 | https://example.com/api?a=1&b=2 | 2c92d744698a |
stateKey, when you leave it empty, is derived the same way: sha256 over the sorted, de-duplicated
set of canonical keys, truncated to 16 characters.
eventId — the idempotency key, and how to dedupe
eventId = sha256(watchId + "|" + previousHash + "|" + newHash + "|" + runId).slice(0, 16)
It is event-addressed, not content-addressed: runId and previousHash are both in the key.
That matters more than it sounds. A content-addressed id (sha256(watchId + newHash)) is byte-identical
for every recurrence of the same content state — so a target flapping A → B → A → B (an A/B test, a
rotating banner, a stock badge, a queue depth) emits the same id for run 1's A→B and run 3's
A→B, and a set-membership dedupe silently throws the second real change away. A became_available
event would even collide with the original baseline_captured for that hash.
Recommended dedupe pattern — safe with this key:
const seen = new Set(await loadSeenEventIds()); // your store, capped/TTL'd as you likefor (const row of datasetItems) {if (seen.has(row.eventId)) continue; // a duplicate DELIVERY of one eventseen.add(row.eventId);await handle(row); // a genuinely new event, always}
Use eventId for delivery deduplication (webhook retry, dataset re-read, run resurrection) and
watchId as the join key for "what is the current state of this target". Do not use newHash as an
event key — two different events can legitimately share it.
Webhook delivery
Set webhookUrl and the actor POSTs one request per emitted change event, and nothing at all when
nothing changed. Firing is the signal. Apify's native webhooks are run-scoped, cannot carry dataset
items in their payload (the receiver has to make a second authenticated API call), and fire on every
run regardless — which wakes your agent up to learn that nothing happened.
Each POST carries the full flat 34-key body and these headers:
| Header | Value |
|---|---|
Content-Type | application/json |
X-Watchdog-Event-Id | The row's eventId |
X-Watchdog-Delivery-Attempt | 1, 2 or 3 |
X-Watchdog-Signature | sha256=<HMAC-SHA256 of the exact raw body>, when webhookSecret is set |
10 s timeout, 3 in-run attempts at 1 s / 4 s / 12 s with jitter. Failures are persisted in state
(capped at 200) and retried at the start of the next run before any fetching, then dropped after
24 hours. Because the actor delivers its own events there is no platform-level retry net, so
deliveryFailures and webhookDropped are both surfaced in the run summary rather than swallowed.
Verify the signature over the raw bytes, before parsing:
const expected = 'sha256=' + crypto.createHmac('sha256', secret).update(rawBody).digest('hex');const ok = crypto.timingSafeEqual(Buffer.from(expected), Buffer.from(receivedHeader));
Run summary
Written to the key-value store record OUTPUT on every run:
stateKey, mode (stateful/stateless), normalizerVersion, targetsChecked,
definitiveAnswers, changesDetected, notModified304, fullBodies, errorsByReason,
bytesTransferred, wallMs, estimatedChargeUsd, budgetCapped, quarantinedTargets,
deliveryFailures, webhookDropped, stateRecordCount, sameHostWarning, scheduleId,
scheduleRequestBody.
notModified304 versus fullBodies is your live cost profile: the first number is the cheap path,
the second is not.
Input reference
One field is enough. {"targets": ["https://registry.npmjs.org/apify"]} is a complete, valid call.
| Field | Type | Default | Purpose |
|---|---|---|---|
targets | array | ["https://registry.npmjs.org/apify"] | URL strings or objects {url, watchId, selector, jsonPath, mode, knownHash, knownEtag, knownLastModified, headers}. Max 5,000. A bare string derives its watchId from the canonical URL — see watchId |
mode | string | auto | Global detection mode; per-target mode overrides |
sensitivity | string | normal | low / normal / high — drives the trigger set |
webhookUrl | string | "" | One POST per change event, silence otherwise |
webhookSecret | string | — | HMAC-SHA256 signing key (secret) |
maxChangeEvents | integer | 500 | Hard cap on billable change events per run, 0–5,000 |
maxBudgetUsd | number | — | Pre-flight abort. Worst case = 0.003 + targets × 0.0003 + min(maxChangeEvents, targets) × 0.008 |
emitUnchanged | boolean | false | Also push no_change rows. Never affects billing |
stateKey | string | "" | Named store suffix; empty derives it from the URL set |
statelessMode | boolean | false | Force stateless. Auto-enabled when every target has knownHash |
intervalMinutes | integer | 60 | Cadence for the cron and staleness gap, 5–10,080 |
| Advanced | |||
noiseQuarantineRuns | integer | 5 | Auto-quarantine after N consecutive changed runs, 0–1,000. 0 disables. Stateful only |
concurrency | integer | 50 | Global parallel cap, 1–100. Per-host is separately clamped to 2 |
maxBodyBytes | integer | 5242880 | Abort past this → body_too_large |
githubToken | string | — | Sent as Bearer on api.github.com targets only (secret) |
proxyConfiguration | object | {"useApifyProxy": false} | Optional IP rotation. RESIDENTIAL is downgraded to DATACENTER |
autoSchedule | boolean | false | Create/update exactly one Schedule watch-{stateKey} |
Limits, safety and honest caveats
- No JavaScript rendering in v1. SPA-only pages fail loudly (see the SPA guard above) instead of reporting "no change" forever.
- Egress guard. Every target URL, every redirect hop (max 3,
redirect: manual) and thewebhookUrlitself are validated:http/httpsonly, default ports only, and any host resolving to loopback, link-local (169.254.0.0/16,fe80::/10), RFC1918, CGNAT (100.64/10), unique-local or0.0.0.0/8is refused witherrorReason: "ssrf_blocked". This actor fetches user-supplied URLs on a schedule from shared cloud infrastructure, so this is a hard control, not a setting. - Blocked domains. A small registrable-domain blocklist (major retail, social and
reddit.com) is refused witherrorReason: "blocked"and never charged. Those sites either hard-block datacenter traffic or forbid automated collection in their terms; a generic watcher pointed at them is a terms problem, not a technical one. - Per-host politeness costs wall time. Global concurrency is up to 100, but per-host concurrency
is clamped to 2 with jitter no matter what you set. 1,000 URLs spread across many hosts is
fast; 1,000 URLs on a single host is not, and the summary raises
sameHostWarningwhen the sweep is at risk of overrunningintervalMinutes. - Rate limits. Unauthenticated GitHub is 60 requests/hour per IP and each run may present a
different egress IP — set
githubTokenif you watch more than a handful of repos, or expectrate_limitedrows. - One state record, by design. The named store holds exactly one blob. If you ever see
stateRecordCountabove 2 in the summary, something is wrong — open an issue. - Residential proxy is excluded on purpose: its bandwidth cost would exceed the entire rest of the run. Sites that require it are refused rather than served badly.
Support
Issues, feature requests and target classes that misbehave: use the Issues tab on the Actor page.
Include the watchId, errorReason and method from the row — they are on every row precisely so a
report is one copy-paste.