SiteVerdict — Website Change & Uptime Monitor

Know within the hour when a page goes down, a price changes, a competitor edits their pricing table, or a deploy quietly adds noindex, without false alarms from timestamps, rotating tokens or cookie banners.

Put it on an Apify Schedule (every 15 min, hourly or daily). Each run checks your URLs. It reports only what really changed, with a plain-English summary and a unified diff, and can alert you by webhook, Slack or e-mail.

Who it is for

• Site owners and agencies: catch downtime, slow responses, expiring TLS certificates, broken redirects, accidental noindex and lost canonicals on client sites.
• E-commerce and pricing teams: watch competitor pricing tables, plan pages and stock status ("sold out" / "in stock").
• SEO teams: changes to title, meta description, canonical, H1, robots directives, robots.txt and sitemap URLs.
• Compliance and procurement: changes to terms, policies, tender pages and status pages.
• AI agents (MCP): "tell me what changed on this page since yesterday".

What each run checks

Area	Details
Uptime	HTTP status, total response time and TTFB, DNS resolution and IPs, the full redirect chain (every hop and status), TLS certificate expiry date, days left, issuer and whether it is trusted. Failures are confirmed with a retry before anything is reported as down.
Content change	The whole page or one region, chosen by CSS selector (e.g. #pricing) or XPath (e.g. //table[@id='plans']). Text is extracted one block per line, with table cells separated by |. The result is a line-based unified diff with +/− line counts and the change %.
Noise filtering	Built-in filter for dates, times, "5 minutes ago", Unix timestamps, UUIDs, long random tokens (CSRF/nonces) and copyright years. You can add your own regular expressions (Visitors today: \d+) and remove elements (nav, footer, ads, chat widgets) by CSS selector. Ignore patterns apply retroactively, because the raw text is stored.
Structured extraction	Prices (£ $ € ₹ ¥, ISO codes) with "£49 → £59" changes. A list of items by CSS selector (text or attribute, e.g. data-sku, href), reported as added or removed.
Keywords	Alert when "sold out", "discontinued" or /only \d+ left/i appears or disappears.
SEO-critical	Title, meta description, canonical, H1s, lang, final URL, meta robots and the X-Robots-Tag header. noindex appearing is flagged as critical.
robots.txt and sitemap	Rule changes (comments and whitespace ignored). Sitemap URLs added or removed (sitemap indexes and .gz supported; lastmod-only edits ignored).
Visual diff (browser mode)	A screenshot of the viewport, the full page or just the selected region, compared pixel by pixel (anti-aliasing ignored) against a % threshold. Dynamic areas can be masked. A diff image is saved for every visual change.

Alerts

Every alert carries a short human-readable summary and a unified diff excerpt:

Acme pricing: CHANGED — Text changed: +2 / -1 lines (12.5% of lines). NOINDEX APPEARED ("noindex, follow"): the page asks search engines to drop it.
title changed: "Pricing — Acme" -> "Pricing (new) — Acme". Keyword "sold out" appeared. Price changed: £49 -> £59.

@@ -2,3 +2,4 @@
 Basic | £19/mo
-Pro | £49/mo
+Pro | £59/mo
+Business | £199/mo

• Webhook: JSON POST to your URL (Zapier, Make, n8n, your API), retried with backoff.
• Slack: an incoming-webhook message with the summary and the diff.
• E-mail: one digest per run, sent through Apify's official apify/send-mail Actor in your account. It uses a few seconds of your platform usage. You can also use Apify's built-in integrations (Slack, e-mail, Zapier, Make) on the Actor's runs.
• Pick which kinds of change alert you (alertOn). All changes are still recorded in the dataset and in the history.
• Downtime alerts fire once when a site goes down and once when it recovers, never on every run while it stays down. TLS alerts fire when the certificate crosses your threshold, and again at 7, 3, 1 and 0 days.

Quick start

1. Add URLs under URLs to monitor (or per-page settings under Advanced monitors).
2. Optional: set a region (CSS/XPath), keywords, price tracking and alert channels.
3. Run once to save the baseline, then create a Schedule (e.g. hourly) with the same input.

Example input: competitor pricing table plus uptime

{
  "urls": [{ "url": "https://www.example-shop.com/" }],
  "monitors": [
    {
      "url": "https://competitor.example/pricing",
      "name": "Competitor pricing",
      "selector": "#pricing-table",
      "trackPrices": true,
      "keywords": ["sold out", "limited time"],
      "excludeSelectors": [".countdown"]
    },
    {
      "url": "https://www.example-shop.com/landing",
      "name": "Landing page (visual)",
      "mode": "browser",
      "visualDiff": true,
      "visualThresholdPercent": 2,
      "visualMaskSelectors": [".carousel", "#chat-widget"]
    }
  ],
  "checkRobotsTxt": true,
  "checkSitemap": true,
  "slackWebhookUrl": "https://hooks.slack.com/services/XXX/YYY/ZZZ",
  "stateStoreName": "my-shop-monitors"
}

Output (one dataset row per URL per run)

{
  "name": "Competitor pricing",
  "url": "https://competitor.example/pricing",
  "status": "up",
  "httpStatus": 200,
  "responseTimeMs": 412,
  "ttfbMs": 188,
  "redirectChain": [{ "url": "http://competitor.example/pricing", "status": 301 }],
  "tlsDaysToExpiry": 44,
  "changeStatus": "changed",
  "changeTypes": ["content", "extraction", "keywords"],
  "summary": "Competitor pricing: CHANGED — Text changed: +2 / -1 lines ... Price changed: £49 -> £59.",
  "diffExcerpt": "@@ -2,3 +2,4 @@\n Basic | £19/mo\n-Pro | £49/mo\n+Pro | £59/mo ...",
  "changes": { "text": {...}, "seo": [...], "keywords": [...], "prices": {...}, "items": {...}, "visual": {...}, "robotsTxt": {...}, "sitemap": {...} },
  "alertSent": true,
  "alertChannels": { "slack": "sent" },
  "chargedEvents": ["url-checked", "change-alert"]
}

Dataset views: Overview, Changes and diffs, Uptime and TLS and SEO snapshot. A RUN_SUMMARY record holds the run's counts and charged events.

Statuses

status	Meaning	Charged?
up	2xx/3xx response; content compared	yes
down	4xx/5xx, timeout, DNS or TLS failure (confirmed by retry)	yes (it is a monitoring result)
blocked	Bot protection, challenge page, WAF 403 or 429. We never bypass it. Allow-list the user agent SiteVerdict/change-uptime-monitor, or monitor a page you control.	no
robots-disallowed	robots.txt disallows the URL for SiteVerdict or *	no
error	Internal problem (e.g. browser crashed)	no

changeStatus is one of baseline (first run), no-change, changed or not-compared (when the page was not up).

State and history

Snapshots live in a named key-value store (stateStoreName, default siteverdict-monitor), so they survive between scheduled runs. Each monitor keeps:

• the last snapshot (raw region text, SEO fields, items, prices, keyword presence, robots/sitemap hashes, screenshot);
• its last up/down status and when that started;
• the last N changes (historySize, default 20), each with its summary and diff.

Use a different store name per client or project. resetBaseline starts a monitor fresh. A monitor's identity is its URL + mode + region. Changing the selector or XPath therefore starts a new history; editing ignore patterns, keywords or alerts does not.

Pricing (pay per event)

• url-checked: one HTTP check ($0.002)
• url-checked-browser: one browser check with optional screenshot diff ($0.006)
• change-alert: once per monitor per run when an alert-worthy change is found ($0.01)

You are never charged for blocked, robots-disallowed or errored URLs. The run never exceeds your Max total charge. If the budget runs out before an alert, the change is not saved, so it is detected again next run. See PRICING.md for examples.

FAQ

Why didn't a change trigger an alert? Check changeTypes against alertOn. Small text changes below minChangePercent, matches of ignore patterns, and elements removed by excludeSelectors are ignored on purpose.

I get alerts for a ticker, counter or carousel. Add an excludeSelectors entry (e.g. .ticker), an ignore pattern, or a visualMaskSelectors entry for screenshots.

The page is built by JavaScript. Use "mode": "browser", with waitForSelector if needed.

Does it log in, solve CAPTCHAs or rotate proxies? No. It fetches public pages with an honest user agent. It respects robots.txt by default and stops at bot protection.

Can I monitor JSON APIs or text files? Yes. Non-HTML responses are compared as text (JSON is pretty-printed first, so the diff is per field).

How fast? HTTP checks take about 0.1–1 s per URL. Browser checks take about 3–8 s. Requests to the same site run one at a time, 1 s apart.

Limits

• Up to 5 MB of HTML analysed per page. Stored text is capped at 300k characters. Diff excerpts are capped at 60 lines / 4,000 characters, and to 2,000 characters in history.
• Full-page screenshots are cut at maxScreenshotHeight (default 8,000 px).
• The sitemap check reads the sitemap file itself (or an index's list of child sitemaps), not every child sitemap.
• XPath runs on a static parse of the HTML (jsdom; scripts never execute). Use browser mode for JavaScript-built content.
• The TLS check reads the certificate presented on port 443 (or the URL's port) via SNI.
• Uptime checks run from Apify's cloud location. This is not a multi-region probe network.

Compliance and sources

• Only public pages are fetched. The user agent is SiteVerdict/change-uptime-monitor (+aman.thakran@bharatfare.com) (browser mode adds it to Chromium's UA). robots.txt is respected by default; turn this off only for sites you own or are authorised to monitor. Follow each site's terms of service. Some sites (e.g. example.com) ask not to be used for monitoring.
• Bot protection, CAPTCHAs, logins and paywalls are never bypassed. Such pages are reported as blocked and are not charged.
• No personal data is collected on purpose. If a monitored page shows personal data, it may appear in snapshots and diffs. Under UK GDPR you are the controller of what you choose to monitor and store. Use regions and exclusions to avoid it, and do not monitor pages about private individuals.
• Webhook and Slack URLs are secret inputs (stored encrypted) and are never logged.
• This tool provides technical monitoring information, not legal advice (e.g. on terms changes).
• Where data is kept (please read): to compare runs, every run saves each monitored page's state in a named key-value store in your account, siteverdict-monitor by default (input stateStoreName). The state includes the watched region's text (capped), its hash, SEO fields, extracted items/prices, robots.txt/sitemap snapshots, a short change history and, when screenshots are on, the latest screenshot. Apify keeps named stores until you delete them, so delete the store (or use a new name) when you stop monitoring a page.
• Per-run results (dataset rows and reports) go to the run's default dataset and key-value store, which are unnamed. How long Apify keeps them is set by Apify and your plan, not by SiteVerdict (Apify data retention).
• What SiteVerdict keeps: nothing. The Actor runs in your Apify account and sends no data to SiteVerdict; our contact address appears only in the User-Agent sent to the sites it checks.