Sitemap Monitor & Website Health API

Monitor public websites from their XML sitemaps and receive only meaningful changes between runs. The Actor discovers nested and compressed sitemaps, checks each bounded page URL, stores a snapshot under your monitor ID, and emits added, removed, or changed records on later runs.

Use it for SEO monitoring, migration checks, release monitoring, broken-page detection, and scheduled automation in Apify, n8n, Make, or your own API client.

What it checks

• HTTP status and request errors
• Final URL and redirect chain
• HTML canonical URL
• noindex directives from HTML and X-Robots-Tag
• Optional bounded content hash
• URLs added to or removed from the sitemap

The first run creates a quiet baseline. It does not report every discovered URL as newly added. Reuse the same monitorId on later runs to compare against that baseline.

Input

{
  "monitorId": "production-site",
  "startUrls": ["https://example.com"],
  "maxUrls": 100,
  "concurrency": 10,
  "timeoutSecs": 20,
  "detectContentChanges": false,
  "includeUnchanged": false
}

Field	Description
monitorId	Stable ID for an independent persisted snapshot. Use letters, numbers, _, and -; maximum 64 characters.
startUrls	One or more public HTTP(S) website roots or explicit sitemap URLs. Maximum 20 entries.
maxUrls	Maximum discovered page URLs checked in one run. Default 100; hard limit 5,000.
concurrency	Concurrent page checks. Default 10; maximum 50.
timeoutSecs	Per-request timeout. Default 20 seconds; range 2-60.
detectContentChanges	Hash bounded response bodies to detect content changes. Disabled by default.
includeUnchanged	Emit unchanged records as well as deltas. Disabled by default.

For a root website URL, the Actor checks robots.txt for sitemap declarations and tries /sitemap.xml. Explicit .xml and .xml.gz inputs are fetched directly. Nested sitemap indexes and gzip payloads are supported.

Output

Every successful run writes a summary to the default dataset:

{
  "recordType": "summary",
  "monitorId": "production-site",
  "baseline": false,
  "urlCount": 100,
  "changeCount": 2,
  "errorCount": 1,
  "checkedAt": "2026-07-16T09:15:00+00:00",
  "discoveryErrors": []
}

Later runs also write one record for each change:

{
  "recordType": "change",
  "changeType": "changed",
  "url": "https://example.com/pricing",
  "status": 404,
  "previousStatus": 200,
  "changedFields": ["status"],
  "previous": { "status": 200 },
  "current": { "status": 404 },
  "checkedAt": "2026-07-16T09:15:00+00:00"
}

changeType is one of added, removed, changed, or unchanged. Unchanged records are only emitted when includeUnchanged is enabled.

Scheduling and automation

Create an Apify schedule that runs the Actor with the same monitorId. Read the default dataset after each run and route records where recordType is change to email, Slack, n8n, Make, a webhook, or another Actor.

API clients can run the Actor synchronously and receive dataset items:

POST https://api.apify.com/v2/acts/believing_amelanchier_qbv~sitemap-health-monitor/run-sync-get-dataset-items
Authorization: Bearer YOUR_APIFY_TOKEN
Content-Type: application/json

Send the same JSON input shown above as the request body. Keep Apify tokens in a credential manager rather than workflow fields or source code.

Pricing

The Actor is free during its initial Store launch. Users pay only the platform usage shown by Apify.

Pay-per-event pricing is planned at $0.001 per checked page URL after developer payout setup is complete. The implementation already respects a run's maximum total charge: if the remaining budget cannot cover every discovered URL, it checks only the chargeable prefix; if no URL can be charged, it stops before page checks.

Security and privacy

• Only public HTTP(S) targets are accepted.
• URL credentials, localhost, and non-public, loopback, link-local, multicast, unspecified, or reserved addresses are rejected.
• Every redirect target is resolved and validated before it is followed.
• Responses, redirects, nested sitemaps, concurrency, timeouts, and decompressed payloads are bounded.
• The Actor does not use a browser, proxy, LLM, external database, or third-party analytics service.
• Snapshots are stored in an Apify named key-value store derived from monitorId.
• Run results and snapshots remain subject to the retention and access settings of the Apify account running the Actor.

Do not place secrets, private URLs, or personal data in startUrls or monitorId.

Limitations

• JavaScript-rendered pages are not rendered; checks use HTTP responses only.
• A website must expose a valid XML sitemap, either explicitly or through robots.txt or /sitemap.xml.
• Content hashes detect any bounded response-body difference and do not distinguish meaningful editorial changes from dynamic markup.
• Canonical and indexability checks apply to HTML responses.
• Network failures and rate limits are reported as URL errors; they are not automatically retried indefinitely.
• The Actor does not send notifications itself. Use Apify schedules, webhooks, or an automation platform.

Verified behavior

The Actor has been exercised on Apify infrastructure with limited permissions. Verified scenarios include first-run baselines, persisted second runs without false changes, nested and gzip sitemap parsing, added/removed/changed classification, separate snapshot state for different monitor IDs, redirect validation, and private-address rejection.

Support

For reproducible issues, open an issue from the Actor page and include the Apify run ID, sanitized input, expected result, and affected public URL. Do not include API tokens or private data.

This Actor monitors website behavior; it does not provide legal, security-audit, or uptime guarantees.