# Sitemap URL Inventory - New, Changed & Removed URLs (`maydit/sitemap-url-inventory-monitor`) Actor

Extract XML and gzip sitemap inventories, follow bounded sitemap indexes, and track URL additions, metadata changes and removals only after complete comparable checks.

- **URL**: https://apify.com/maydit/sitemap-url-inventory-monitor.md
- **Developed by:** [Brandt May](https://apify.com/maydit) (community)
- **Categories:** SEO tools, Developer tools
- **Features:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.30 / 1,000 saved observations

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Sitemap URL Inventory — a dependable feed of published URL changes

Find newly listed product pages, documentation URLs and competitor content from XML sitemaps. Re-run the same scope to see additions, metadata changes and URLs absent from the next complete inventory.

**For SEO agencies, content researchers and AI ingestion teams.** Each observation keeps the exact URL, sitemap evidence, publisher-supplied metadata and comparison status. A removal means absent from these sitemaps; it does not prove the web page was deleted.

### Try it

Empty input reads the official sitemaps.org XML sitemap. A smaller first run:

```json
{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"urlPrefixes":["https://www.sitemaps.org/protocol.html","https://www.sitemaps.org/faq.html","https://www.sitemaps.org/terms.html"],"maxResults":10}
```

1. Paste your XML sitemap URLs or sitemap-index URLs.
2. Start with a small scope. Open the dataset and the readable run report.
3. For recurring monitoring, save the input as an Apify task, enable comparison and reuse the snapshot name. Run a snapshot sequentially.

### What makes the changes usable

- Follows bounded same-origin sitemap indexes and supports gzip sitemap files.
- Deduplicates page URLs across maps while retaining every source map.
- Saves one URL observation per result, with no fabricated page titles or dates.
- Emits removals only after a complete inventory. A network error, truncated map, output cap or spending stop leaves the previous baseline intact.
- Supplies the same structured output to the Apify API, MCP tools, scheduled tasks and spreadsheet exports.

### Three input recipes

**English documentation pages:** the first-run input above.

**German documentation inventory:**

```json
{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"urlPrefixes":["https://www.sitemaps.org/de/"],"maxResults":20}
```

**Full official sitemap:**

```json
{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"maxResults":100}
```

### Inputs

| Input | Default | Purpose |
|---|---|---|
| sitemapUrls | Official sitemaps.org map | Up to10 public XML maps or indexes |
| urlPrefixes | none | Select output by absolute URL prefix; all maps still read |
| maxSitemaps |20| Root and child sitemap request cap, up to100 |
| maxUrls |5000| Unique URLs inventoried before filters, up to10000 |
| maxResults |100| Saved output rows including removals, up to10000 |
| comparePrevious |false| Compare only complete inventories |
| onlyChanges |false| Implies comparison; unchanged repeats can return zero rows |
| snapshotName |default| Independent history scope; letters/numbers/underscore/hyphen |
| maxRunSeconds |90| Shared deadline,30–900seconds, also limited by the platform |

### Output

One saved row is one URL observation. Fields: url, title (the URL itself), recordId, lastmod, changefreq, priority, sitemapUrls, sourceUrl, observedAt, presence, inventoryComplete, comparisonStatus, changedFields, previousObservedAt and previous. Optional metadata is null when absent. Publisher metadata is kept as text, without inventing a date, priority or page title.

Comparison statuses: not_requested, baseline, added, changed, unchanged, removed, or not_compared_incomplete. The SUMMARY record reports exact coverage, failures, limits and whether history was updated. RUN_REPORT is a self-contained HTML view. JSON preserves nested evidence; CSV is useful for flat URL inventories.

### Pricing

Base price: **$0.50 per1,000 saved URL observations**, plus the configured $0.00005 Actor Start event. Saved removal observations have the same price. An unchanged row is charged when saved; onlyChanges skips it. Failed source requests and summary/report records are not dataset result events. Check the Pricing tab for your plan's discounts. Use the run maximum-charge control to bound spending.

### Coverage and limits

XML urlset and sitemapindex, optionally gzip, only. RSS/Atom/text feeds, images/video extension extraction and cross-origin child maps are unsupported. Each response has a4MiB wire and decoded body cap; one XML map supports at most10000 entries. Public HTTP(S) only with robots.txt, DNS and redirect validation. No login, proxies or browser rendering. The Actor reads maps; it never visits every listed web page or claims to know Google indexing. lastmod, changefreq and priority are publisher statements, not verified changes or ranking scores. The same root-map and prefix scope identifies a saved baseline; changing it starts independent history. Concurrent writes are checked but the platform store is not transactional: run a snapshot sequentially.

### Use in workflows

Schedule the saved task, read SUMMARY after success, then route complete added/changed rows into your crawler, research queue or AI index. Route removed rows for review before deleting downstream content. Partial source observations remain explicitly marked. Use the Apify REST API or the Actor's MCP integration with your own Apify authentication.

Independent tool by May'd It. Source format: https://www.sitemaps.org/protocol.html . Support: open an issue with the sitemap URL, relevant input and expected result; omit credentials.

# Actor input Schema

## `sitemapUrls` (type: `array`):

Up to10 absolute XML sitemap or sitemap-index URLs. Empty uses the official sitemaps.org sitemap.

## `urlPrefixes` (type: `array`):

Optional absolute URL prefixes. Empty selects every inventoried URL. Filtering does not limit sitemap reading.

## `maxSitemaps` (type: `integer`):

Includes root maps and same-origin child maps. A reached limit prevents a complete baseline.

## `maxUrls` (type: `integer`):

Unique inventory cap before filters. Partial inventories never produce removal claims or replace history.

## `maxResults` (type: `integer`):

Global output cap, including removal observations. If output is truncated, saved history is not advanced.

## `comparePrevious` (type: `boolean`):

Only a complete inventory with complete output can become a baseline. Removed means absent from the sitemap, not deleted from the website.

## `onlyChanges` (type: `boolean`):

Enables comparison. Baseline, additions, changes and removals are saved; unchanged repeats can return zero rows.

## `snapshotName` (type: `string`):

1-80 letters, numbers, underscores or hyphens. Reuse the same map URLs and filters. Run this snapshot sequentially.

## `maxRunSeconds` (type: `integer`):

Shared request and output deadline; platform timeout can be shorter.

## Actor input object example

```json
{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ],
  "urlPrefixes": [],
  "maxSitemaps": 20,
  "maxUrls": 5000,
  "maxResults": 100,
  "comparePrevious": false,
  "onlyChanges": false,
  "snapshotName": "default",
  "maxRunSeconds": 90
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `report` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "sitemapUrls": [
        "https://www.sitemaps.org/sitemap.xml"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("maydit/sitemap-url-inventory-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "sitemapUrls": ["https://www.sitemaps.org/sitemap.xml"] }

# Run the Actor and wait for it to finish
run = client.actor("maydit/sitemap-url-inventory-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ]
}' |
apify call maydit/sitemap-url-inventory-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,maydit/sitemap-url-inventory-monitor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/RGfTsivyHveCZmq47/builds/wFQKAHibDeJcJvtjt/openapi.json