Sitemap Diff — Added, Removed & Changed URLs

Turn XML sitemap updates into ready-to-route URL events for SEO monitoring, crawl queues and incremental RAG updates. Get added, removed, changed and unchanged records in structured JSON or CSV.

• Up to 50,000 matching URLs and 100 sitemap files per run.
• Nested indexes, gzip and XML namespaces supported.
• Launch price: $0.50 per 1,000 output rows, plus $0.001 run start; platform usage included.
• Reusable snapshots make comparisons reproducible in your API workflow.
• Incomplete-scan protection keeps temporary source failures out of removal decisions.

Start with an inventory

{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"maxUrls":10000,"maxSitemaps":30}

Each dataset row contains url, lastmod, changefreq, priority, sourceSitemap, change and observedAt. A first run uses change: "current".

Download SNAPSHOT from the run's key-value store. Pass that JSON object as previousSnapshot on the next call, with the same sitemap URLs and urlContains filter. Your workflow owns the snapshot and can automate the entire comparison.

import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["APIFY_TOKEN"])
actor = client.actor("gertner-data/sitemap-delta")
first = actor.call(run_input={"sitemapUrls": ["https://www.sitemaps.org/sitemap.xml"]})
snapshot = client.key_value_store(first["defaultKeyValueStoreId"]).get_record("SNAPSHOT")["value"]
second = actor.call(run_input={"sitemapUrls": ["https://www.sitemaps.org/sitemap.xml"], "previousSnapshot": snapshot})
for row in client.dataset(second["defaultDatasetId"]).iterate_items():
    if row["change"] != "unchanged":
        print(row["change"], row["url"])

Route the changes

• added: enqueue a newly listed URL for crawling or indexing.
• changed: re-check a URL with updated lastmod, changefreq or priority.
• removed: a URL left the sitemap inventory; both snapshots must be complete and cover the same scope.
• unchanged: retain the existing inventory record.

The comparison operates on sitemap membership and metadata. Page availability and page-body changes belong to separate URL checks.

API coverage and run settings

Supply 1–10 root sitemaps. Configure 1–100 sitemap files and up to 50,000 unique matching URLs. Downloaded or expanded XML is capped at 8 MB per file; inventory output at 6 MB. Nested sitemap hosts must match a supplied root. Use the final host directly when a root redirects between hosts. Supports sources compatible with identity HTTP encoding and explicit .gz sitemap files.

urlContains is a case-sensitive substring filter. Deduplication removes fragments and preserves exact URL identities. Dates retain the sitemap's original values.

SUMMARY.complete, SUMMARY.errors and SUMMARY.removalsEvaluated describe scan coverage. Source errors, malformed XML, conflicting duplicates and inventory limits suppress removal evaluation. A complete empty inventory can remove every URL from a previous complete inventory. At the spending cap, SNAPSHOT is withheld to preserve the last complete baseline.

Inputs and results use your Apify account's retention settings. Logs contain aggregates; snapshots contain the inventory URLs needed for comparison. Use public sources you are authorized to inspect.

Pricing

Launch price: $0.001 per run + $0.0005 per emitted URL row, including unchanged and removed rows. Platform usage is included.

Output rows	Event price
84	$0.043
1,000	$0.501

Set a maximum run charge to control output spending. The run-start event applies independently of output count.

Connect your workflow

Use the Apify API, tasks, schedules, webhooks or MCP integration. Pass the prior snapshot automatically between runs. Default memory: 512 MB. Recommended timeout: 300 seconds. Network failures, 429 and 5xx responses receive bounded retries.

Support: Evgeny Gertner.