# Sitemap Monitor & Website Health API (`kingii98/sitemap-health-monitor`) Actor

Monitor XML sitemaps, check public URL health, persist snapshots, and return only added, removed, or changed pages for SEO and automation.

- **URL**: https://apify.com/kingii98/sitemap-health-monitor.md
- **Developed by:** [kingii98](https://apify.com/kingii98) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 80.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$1.00 / 1,000 url checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Sitemap Monitor & Website Health API

Monitor public websites from their XML sitemaps and receive only meaningful changes between runs. The Actor discovers nested and compressed sitemaps, checks each bounded page URL, stores a snapshot under your monitor ID, and emits added, removed, or changed records on later runs.

Use it for SEO monitoring, migration checks, release monitoring, broken-page detection, and scheduled automation in Apify, n8n, Make, or your own API client.

### What it checks

- HTTP status and request errors
- Final URL and redirect chain
- HTML canonical URL
- `noindex` directives from HTML and `X-Robots-Tag`
- Optional bounded content hash
- URLs added to or removed from the sitemap

The first run creates a quiet baseline. It does not report every discovered URL as newly added. Reuse the same `monitorId` on later runs to compare against that baseline.

### Input

```json
{
  "monitorId": "production-site",
  "startUrls": ["https://example.com"],
  "maxUrls": 100,
  "concurrency": 10,
  "timeoutSecs": 20,
  "detectContentChanges": false,
  "includeUnchanged": false
}
```

| Field | Description |
|---|---|
| `monitorId` | Stable ID for an independent persisted snapshot. Use letters, numbers, `_`, and `-`; maximum 64 characters. |
| `startUrls` | One or more public HTTP(S) website roots or explicit sitemap URLs. Maximum 20 entries. |
| `maxUrls` | Maximum discovered page URLs checked in one run. Default 100; hard limit 5,000. |
| `concurrency` | Concurrent page checks. Default 10; maximum 50. |
| `timeoutSecs` | Per-request timeout. Default 20 seconds; range 2-60. |
| `detectContentChanges` | Hash bounded response bodies to detect content changes. Disabled by default. |
| `includeUnchanged` | Emit unchanged records as well as deltas. Disabled by default. |

For a root website URL, the Actor checks `robots.txt` for sitemap declarations and tries `/sitemap.xml`. Explicit `.xml` and `.xml.gz` inputs are fetched directly. Nested sitemap indexes and gzip payloads are supported.

### Output

Every successful run writes a summary to the default dataset:

```json
{
  "recordType": "summary",
  "monitorId": "production-site",
  "baseline": false,
  "urlCount": 100,
  "changeCount": 2,
  "errorCount": 1,
  "checkedAt": "2026-07-16T09:15:00+00:00",
  "discoveryErrors": []
}
```

Later runs also write one record for each change:

```json
{
  "recordType": "change",
  "changeType": "changed",
  "url": "https://example.com/pricing",
  "status": 404,
  "previousStatus": 200,
  "changedFields": ["status"],
  "previous": { "status": 200 },
  "current": { "status": 404 },
  "checkedAt": "2026-07-16T09:15:00+00:00"
}
```

`changeType` is one of `added`, `removed`, `changed`, or `unchanged`. Unchanged records are only emitted when `includeUnchanged` is enabled.

### Scheduling and automation

Create an Apify schedule that runs the Actor with the same `monitorId`. Read the default dataset after each run and route records where `recordType` is `change` to email, Slack, n8n, Make, a webhook, or another Actor.

API clients can run the Actor synchronously and receive dataset items:

```text
POST https://api.apify.com/v2/acts/believing_amelanchier_qbv~sitemap-health-monitor/run-sync-get-dataset-items
Authorization: Bearer YOUR_APIFY_TOKEN
Content-Type: application/json
```

Send the same JSON input shown above as the request body. Keep Apify tokens in a credential manager rather than workflow fields or source code.

### Pricing

The Actor is free during its initial Store launch. Users pay only the platform usage shown by Apify.

Pay-per-event pricing is planned at `$0.001` per checked page URL after developer payout setup is complete. The implementation already respects a run's maximum total charge: if the remaining budget cannot cover every discovered URL, it checks only the chargeable prefix; if no URL can be charged, it stops before page checks.

### Security and privacy

- Only public HTTP(S) targets are accepted.
- URL credentials, localhost, and non-public, loopback, link-local, multicast, unspecified, or reserved addresses are rejected.
- Every redirect target is resolved and validated before it is followed.
- Responses, redirects, nested sitemaps, concurrency, timeouts, and decompressed payloads are bounded.
- The Actor does not use a browser, proxy, LLM, external database, or third-party analytics service.
- Snapshots are stored in an Apify named key-value store derived from `monitorId`.
- Run results and snapshots remain subject to the retention and access settings of the Apify account running the Actor.

Do not place secrets, private URLs, or personal data in `startUrls` or `monitorId`.

### Limitations

- JavaScript-rendered pages are not rendered; checks use HTTP responses only.
- A website must expose a valid XML sitemap, either explicitly or through `robots.txt` or `/sitemap.xml`.
- Content hashes detect any bounded response-body difference and do not distinguish meaningful editorial changes from dynamic markup.
- Canonical and indexability checks apply to HTML responses.
- Network failures and rate limits are reported as URL errors; they are not automatically retried indefinitely.
- The Actor does not send notifications itself. Use Apify schedules, webhooks, or an automation platform.

### Verified behavior

The Actor has been exercised on Apify infrastructure with limited permissions. Verified scenarios include first-run baselines, persisted second runs without false changes, nested and gzip sitemap parsing, added/removed/changed classification, separate snapshot state for different monitor IDs, redirect validation, and private-address rejection.

### Support

For reproducible issues, open an issue from the Actor page and include the Apify run ID, sanitized input, expected result, and affected public URL. Do not include API tokens or private data.

This Actor monitors website behavior; it does not provide legal, security-audit, or uptime guarantees.

# Actor input Schema

## `monitorId` (type: `string`):

Stable ID used to isolate the persisted snapshot.

## `startUrls` (type: `array`):

Public HTTP(S) websites or explicit XML sitemap URLs.

## `maxUrls` (type: `integer`):

Hard cap across discovered page URLs.

## `concurrency` (type: `integer`):

Maximum number of page checks performed concurrently.

## `timeoutSecs` (type: `integer`):

Per-request timeout applied to sitemap and page fetches.

## `detectContentChanges` (type: `boolean`):

Hash bounded HTML bodies in addition to checking HTTP metadata.

## `includeUnchanged` (type: `boolean`):

Emit an output record for URLs whose monitored fields did not change.

## Actor input object example

```json
{
  "monitorId": "default",
  "startUrls": [
    "https://example.com/"
  ],
  "maxUrls": 100,
  "concurrency": 10,
  "timeoutSecs": 20,
  "detectContentChanges": false,
  "includeUnchanged": false
}
```

# Actor output Schema

## `dataset` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {};

// Run the Actor and wait for it to finish
const run = await client.actor("kingii98/sitemap-health-monitor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {}

# Run the Actor and wait for it to finish
run = client.actor("kingii98/sitemap-health-monitor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{}' |
apify call kingii98/sitemap-health-monitor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,kingii98/sitemap-health-monitor"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/Aqfbjf9Wb8owFf8cR/builds/eeefWA3SbcRPLR3ht/openapi.json