Sitemap Change Detector avatar
Sitemap Change Detector

Pricing

Pay per usage

Go to Store
Sitemap Change Detector

Sitemap Change Detector

Developed by

Tri⟁angle

Tri⟁angle

Maintained by Apify

Identify and monitor sitemaps for specified websites. Retrieve only the new, updated, or removed URLs since the last crawl.

0.0 (0)

Pricing

Pay per usage

2

Total users

13

Monthly users

13

Runs succeeded

89%

Last modified

24 days ago

Monitor changes in website sitemaps by comparing current entries against a previous snapshot stored in a named key-value store. Detect new, updated, removed, or unchanged URLs and react to changes automatically.

What is Sitemap Change Detector?

This Apify actor fetches sitemap URLs (or discovers them via robots.txt), parses <loc> and <lastmod> entries, and stores a snapshot of URLs in a key-value store named sitemap-change-detector-snapshots. On subsequent runs, it retrieves the previous snapshot and compares the lists to identify changes.

How it Detects Changes

  • NEW: URLs present in the current sitemap but missing from the previous snapshot.
  • REMOVED: URLs present in the previous snapshot but missing from the current sitemap.
  • UPDATED: URLs present in both snapshots where the current <lastmod> timestamp is later than the previous one.
  • SAME: URLs present in both snapshots with identical <lastmod> timestamps.

How to Use

  1. Go to the Sitemap Change Detector actor on the Apify Store.
  2. Enter one or more Start URLs (direct sitemap URLs or root domains).
  3. Enable Discover sitemaps to fetch robots.txt and enqueue sitemap URLs.
  4. Select which Change Types to detect: NEW, UPDATED, REMOVED, SAME.
  5. (Optional) Provide a URL filter regex to restrict monitored URLs.
  6. (Optional) Enable Add removed URLs to KVS to log deleted URLs separately.
  7. Save the input and click Run.
  8. View the results in the default dataset or fetch via API or webhooks.

Example Input

{
"startUrls": [
{
"url": "https://www.apify.com",
"method": "GET"
}
],
"changeTypes": ["NEW", "UPDATED"],
"addRemovedUrlsToKvs": false,
"discoverSitemaps": true,
"snapshotKeyPrefix": "APIFY"
}

Output

Each record in the output dataset represents a URL change:

{
"url": "https://example.com/page",
"changeType": "UPDATED"
}

If Add removed URLs to KVS is enabled, check the REMOVED-URLS KVS record for an array of removed URLs.

FAQ

Can I export data using API?

Yes, you can access this actor using your own applications through the Apify API. Click on the API tab for code examples or check out the Apify API reference docs at https://docs.apify.com/api/v2 for full details.

Can I use Sitemap Change Detector through an MCP Server?

This actor, like all Apify actors, works on the Apify MCP server. For more information and instructions, read the Apify MCP server integration guide at https://docs.apify.com/platform/integrations/mcp.

Can I integrate data from Sitemap Change Detector with other apps?

Yes. Sitemap Change Detector can be connected with almost any cloud service or web app. Read more about the possibilities on our integrations page at https://apify.com/integrations.

This actor only extracts publicly available data. It does not collect private user data. However, you should ensure your reason for scraping is legitimate. Consult legal counsel if unsure. For more on scraping legality and ethics, see:

Your feedback

We welcome feedback to improve this actor. If you encounter issues or have suggestions, please create an issue on the actor’s Issues tab.