Sitemap Change Monitor: New, Removed & Broken URLs
Pricing
from $0.10 / 1,000 url rows
Sitemap Change Monitor: New, Removed & Broken URLs
Sitemap change monitor: compare any XML sitemap, sitemap index or robots.txt with the last snapshot. Get new URLs, removed URLs, changed lastmod and optional HTTP status checks for broken links. Built for scheduled monitoring of sitemap changes.
Pricing
from $0.10 / 1,000 url rows
Rating
0.0
(0)
Developer
Ferrolark
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Categories
Features
Share
Sitemap Delta: New, Removed & Broken URLs Since Last Run
Point it at a sitemap or a website. Each run reads every sitemap (sitemap indexes, .xml.gz and plain-text sitemaps included), compares the URL list with the snapshot saved by the previous run and writes only what changed:
- added: URLs that are new in the sitemap
- removed: URLs that disappeared from the sitemap
- lastmod-changed: URLs whose
<lastmod>date changed - optional HTTP status checks to find new URLs that are already broken (4xx/5xx) or redirecting
Schedule it daily or weekly to watch your own site after deploys, track a competitor's new pages, or feed new URLs into indexing, translation or content workflows. The first run saves a baseline, and after that you pay only for changes.
Real example
Run 1 (no snapshot yet), with this input:
{"urls": ["https://www.sitemaps.org/"],"checkStatus": "added","maxStatusChecks": 20}
The Actor found the sitemap through robots.txt, read 84 URLs and saved the baseline. It wrote 84 baseline rows and status-checked 20 of them (all returned 200).
- Run time: 4.5 s on Apify at 512 MB.
- Your cost: 1 sitemap + 84 rows + 20 checks = $0.0194.
Run 2, same site, a few minutes later: status No changes. 84 URLs in 1 sitemap(s); 0 rows.
- Run time: 4.9 s.
- Your cost: $0.001 (one sitemap).
A dataset row:
{"url": "https://www.sitemaps.org/da/","change": "baseline","lastmod": "2016-11-21","sitemap": "https://www.sitemaps.org/sitemap.xml","statusCode": 200,"broken": false,"snapshotName": "www.sitemaps.org","checkedAt": "2026-11-08T23:16:05Z"}
Rows for the other change types look like this:
{"url": "https://example.com/pricing", "change": "added", "lastmod": null, "statusCode": 404, "broken": true}{"url": "https://example.com/about", "change": "removed", "previousLastmod": null}{"url": "https://example.com/docs", "change": "lastmod-changed", "lastmod": "2026-02-01", "previousLastmod": "2026-01-01", "statusCode": 200, "broken": false}
Every run also writes a summary to the OUTPUT record of its key-value store:
{"snapshotName": "www.sitemaps.org","snapshotSaved": true,"previous": {"source": "stored snapshot \"www.sitemaps.org.json.gz\"", "createdAt": "2026-11-08T23:16:05Z", "urlCount": 84},"sitemapsFetched": 1,"sitemapErrors": [],"urlsInSitemaps": 84,"changes": {},"urlsEmitted": 0,"statusChecked": 0,"broken": 0,"partial": false,"notes": []}
Price and limits
Pay per event, with no monthly fee:
| Event | Price |
|---|---|
| Sitemap file downloaded and parsed (including index files) | $0.001 |
| URL row written to the dataset | $0.0001 |
| HTTP status check (optional) | $0.0005 |
Examples:
- A daily check of a 2,000-URL site with one sitemap and 5 new pages costs about $0.0015 per day.
- The baseline run for the same site costs about $0.20, or $0.001 with
firstRunOutput: "silent".
Sitemaps that fail to download are not charged. Set Max total charge on the run to cap the cost: the Actor stops writing rows when it reaches the cap.
Limits:
- Up to 500,000 URLs per run (default 50,000).
- Up to 1,000 sitemap files (default 200).
- Up to 50 MB per sitemap (gzip expansion is capped too).
- Index nesting is followed up to 4 levels.
- Up to 5,000 status checks per run (default 500).
Input
| Field | Default | What it does |
|---|---|---|
urls | required | Sitemap URLs, or website URLs. For a website, sitemaps come from robots.txt, with /sitemap.xml as fallback. |
snapshotName | host of the first URL | Which saved snapshot to compare with and update. Use different names to track sections separately. |
compareWithStoredSnapshot | true | Compare with the last snapshot saved under that name. |
previousSnapshotUrl | none | Compare with a JSON file instead: a snapshot from this Actor, a JSON array of URLs, or [{"url", "lastmod"}]. Gzip is fine. |
previousDatasetId | none | Compare with the dataset of an earlier inventory run (baseline or unchanged rows). |
saveSnapshot | true | Save this run as the new snapshot. |
firstRunOutput | inventory | With no previous snapshot, inventory writes every URL as a baseline row; silent only saves the snapshot. |
includeUnchanged | false | Also write unchanged URLs, for a full inventory every run. |
checkStatus | none | added, changes (added, changed and removed) or all rows. |
maxStatusChecks | 500 | Cap on status checks. |
includePattern / excludePattern | none | Regular expressions to keep or drop URLs, for example /blog/. |
maxUrls / maxSitemaps | 50,000 / 200 | Safety limits. |
Snapshots are stored in your own Apify account, in the named key-value store sitemap-delta-snapshots, as gzipped JSON. You can download them or delete them there.
How it decides what changed
- URLs are compared exactly as written in
<loc>.https://a.com/xandhttps://a.com/x/are different URLs, as they are for search engines. lastmod-changedcompares the raw<lastmod>text.- Partial runs are safe. If any sitemap fails or a limit is hit:
removedrows are not reported, because they could be false positives;- the stored snapshot is not overwritten;
- the
OUTPUTrecord lists the errors.
- Status checks send a HEAD request, falling back to GET on 403, 405 and 501, with a 20 s timeout.
- Redirects are reported in
redirectTo, not followed. brokenis true for 4xx, 5xx and network errors.
- Redirects are reported in
- Only public addresses are fetched; private and local IPs are refused.
Call it from code, n8n or an AI agent
API (synchronous, returns the changed URLs):
curl -X POST "https://api.apify.com/v2/acts/ferrolark~sitemap-delta/run-sync-get-dataset-items?maxTotalChargeUsd=0.5" \-H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \-d '{"urls": ["https://www.example.com/"], "checkStatus": "added"}'
Scheduled monitoring in Apify:
- Create a task with your input.
- Add a schedule, for example daily at 07:00.
- Add an integration (Slack, email, webhook) that fires when a run succeeds.
The task keeps its snapshotName, so every run compares with the previous one.
n8n: paste this workflow on the canvas (Ctrl+V). Then add a Header Auth credential with the name Authorization and the value Bearer <your Apify token>. It runs every morning and passes only changed rows on, to a Slack, Google Sheets or email node.
{"name": "Sitemap Delta: daily new and removed URLs","nodes": [{"parameters": {"rule": {"interval": [{"triggerAtHour": 7}]}}, "name": "Every day 07:00", "type": "n8n-nodes-base.scheduleTrigger", "typeVersion": 1.2, "position": [0, 0]},{"parameters": {"method": "POST", "url": "https://api.apify.com/v2/acts/ferrolark~sitemap-delta/run-sync-get-dataset-items?maxTotalChargeUsd=0.5", "authentication": "genericCredentialType", "genericAuthType": "httpHeaderAuth", "sendBody": true, "specifyBody": "json", "jsonBody": "{\"urls\": [\"https://www.example.com/\"], \"firstRunOutput\": \"silent\", \"checkStatus\": \"added\"}", "options": {"timeout": 300000}}, "name": "Run Sitemap Delta", "type": "n8n-nodes-base.httpRequest", "typeVersion": 4.2, "position": [220, 0]},{"parameters": {"conditions": {"string": [{"value1": "={{ $json.change }}", "operation": "notEqual", "value2": "unchanged"}]}}, "name": "Only changes", "type": "n8n-nodes-base.if", "typeVersion": 1, "position": [440, 0]}],"connections": {"Every day 07:00": {"main": [[{"node": "Run Sitemap Delta", "type": "main", "index": 0}]]},"Run Sitemap Delta": {"main": [[{"node": "Only changes", "type": "main", "index": 0}]]}}}
Make / Zapier: use the Apify app's "Run Actor" action with ferrolark/sitemap-delta and the JSON input above, then "Get dataset items".
AI agents (MCP): add ferrolark/sitemap-delta to the Apify MCP server (https://mcp.apify.com?actors=ferrolark/sitemap-delta). The agent can then ask "what pages did example.com add this week?"
Use cases
- After every deploy:
- check that no important URL dropped out of the sitemap;
- check that new URLs return 200 (
checkStatus: "changes").
- Competitor watch: new product, pricing, landing or blog pages as soon as they reach the sitemap.
- SEO and indexing: send
addedURLs to an indexing or translation queue, andremovedURLs to a redirect review. - Content inventory:
includeUnchanged: truegives a full, dated URL inventory you can export to CSV or Excel.
Troubleshooting
- "no sitemap could be read": the site has no
Sitemap:line inrobots.txtand no/sitemap.xml. Pass the sitemap URL directly. You were not charged. - "Got an HTML page, not a sitemap": the URL returns a web page, often a soft 404. Check the URL in a browser.
- Everything shows as
baselineagain: thesnapshotNamechanged, or the snapshot store was deleted. SetsnapshotNameexplicitly in scheduled tasks. - No
removedrows although pages were deleted: the run was partial (seenotesinOUTPUT). Fix the failing sitemap, or raisemaxUrlsandmaxSitemaps. - Many
403statuses: some sites block automated HEAD and GET requests. The sitemap diff is still correct; only the status columns are affected.
Limitations
- Reads sitemaps only. It does not crawl pages to discover URLs that are missing from the sitemap.
- Does not render JavaScript. Sitemaps are static files, so this is rarely an issue.
<lastmod>is only as reliable as the site's CMS. Some sites update it on every build.- News, image and video sitemap extensions are read for
<loc>and<lastmod>only.
Changelog
- 1.0 (November 2026): first release. Sitemap index, gzip and text sitemaps,
robots.txtdiscovery, stored snapshots, partial-run protection and optional status checks.
More Ferrolark tools
- Brand Name Availability Checker: domains, GitHub, npm, PyPI and more in one report.
- Domain Intelligence Report: WHOIS, DNS, email security, SSL and hosting for any domain.
- Launch Asset Pack: favicons, PWA icons, OG image and head tags from one logo.
- Live Deploy QA: meta tags, canonical, robots, OG image and favicon of your deployed routes, with diffs between deploys.
- RSS Delta: only new RSS, Atom and JSON Feed items, de-duplicated.
- PDF Merge API: merge PDFs from URLs or base64 with page ranges and a SHA-256 manifest.
- PDF Split API: split a PDF by page ranges or every N pages, or extract pages, with a ZIP and manifest.
- Image Convert: PNG, JPG, WebP, AVIF and HEIC conversion (SVG input too) from URLs or base64.
- Image Compress & Resize: resize images and compress them to a target size in KB, WebP or AVIF.
- PDF to Image: PDF pages to PNG, JPG or WebP at the DPI you choose.
- Image to PDF: JPG, PNG, HEIC and WebP images to one PDF, JPEGs embedded without recompression.
- PDF Compress: smaller PDFs, lossless or with image downsampling; charged only when smaller.
- HTML to PDF & URL to PDF: web pages or raw HTML to PDF with headless Chrome.
- Markdown to PDF: Markdown with tables, code and footnotes to a clean PDF.
- DOCX to PDF: Word, Excel, PowerPoint and ODT to PDF with LibreOffice.
- Data Convert: CSV, JSON, NDJSON and Excel XLSX in any direction.
Built by Ferrolark.