# Sitemap & Feed Freshness Monitor (`silicon1235/sitemap-feed-freshness-monitor`) Actor Monitor public sitemaps and RSS/Atom feeds for new, stale, changed, and unreachable URLs. - **URL**: https://apify.com/silicon1235/sitemap-feed-freshness-monitor.md - **Developed by:** [X L](https://apify.com/silicon1235) (community) - **Categories:** SEO tools, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing Pay per usage This Actor is paid per platform usage. The Actor is free to use, and you only pay for the Apify platform usage, which gets cheaper the higher subscription plan you have. Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-usage ## What's an Apify Actor? Actors are a software tools running on the Apify platform, for all kinds of web data extraction and automation use cases. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. In JavaScript/TypeScript projects, use official [JavaScript/TypeScript client](https://docs.apify.com/api/client/js.md): ```bash npm install apify-client ``` In Python projects, use official [Python client library](https://docs.apify.com/api/client/python.md): ```bash pip install apify-client ``` In shell scripts, use [Apify CLI](https://docs.apify.com/cli/docs.md): ````bash # MacOS / Linux curl -fsSL https://apify.com/install-cli.sh | bash # Windows irm https://apify.com/install-cli.ps1 | iex ```bash In AI frameworks, you might use the [Apify MCP server](https://docs.apify.com/platform/integrations/mcp.md). If your project is in a different language, use the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Sitemap & Feed Freshness Monitor Monitor public `sitemap.xml`, sitemap index, RSS, and Atom URLs for new, seen, changed, stale, dead, and error rows. This Actor is built for SEO teams, AI/RAG ingestion workflows, agencies, founders, and content teams that need scheduled freshness checks without maintaining a custom crawler. ### What It Does Paste one or more public sitemap or feed URLs and get a structured dataset plus a Markdown report. Scheduled runs automatically compare against the previous saved URL snapshot, so users do not need to manually pass yesterday's URL list. Saved snapshots include URL fingerprints, so changes to title, date, or source metadata are reported as `CHANGED`. Typical workflows: - monitor new blog, docs, changelog, or product URLs - watch competitor feeds - validate sitemap freshness before SEO indexing or AI ingestion - schedule lightweight content freshness checks - catch stale or unreachable URLs before they break downstream pipelines The Actor reads public URLs only. It does not log in, does not use residential proxies, and is designed for low-cost scheduled runs. ### Input ```json { "sourceUrls": [ "https://blog.apify.com/rss/" ], "previousUrls": [], "maxSources": 10, "maxItems": 25, "staleAfterDays": 90, "checkHttpStatus": false, "snapshotKey": "URL_SNAPSHOT", "saveSnapshot": true } ```` Fields: - `sourceUrls`: public sitemap, sitemap index, RSS, or Atom URLs. - `previousUrls`: optional URL list from a previous run. URLs not in this list are marked `NEW`. - `maxSources`: maximum source URLs to process in one run. Default is `10`; hard cap is `25`. - `maxItems`: maximum entries to process across all sources. Default is `500`; hard cap is `1000`. Keep first remote tests around `25`. - `staleAfterDays`: entries older than this threshold are marked `STALE` when a date is available. - `checkHttpStatus`: checks discovered URLs and marks unreachable entries as `DEAD`. Keep this off for the cheapest scheduled runs. - `snapshotKey`: key-value store record used to load and save the previous URL snapshot. - `saveSnapshot`: save the current URL list so the next scheduled run can detect `NEW` and `SEEN` automatically. ### Output Each dataset row includes: - `status`: `NEW`, `SEEN`, `CHANGED`, `STALE`, `DEAD`, or `ERROR` - `sourceType`: sitemap or feed - `sourceUrl` - `url` - `title` when available - `date` and `ageDays` when available - optional `httpStatus` - `fingerprint` for snapshot comparison - `reason` explaining the decision The run also writes: - `SUMMARY`: machine-readable status totals - `REPORT`: human-readable Markdown report - `URL_SNAPSHOT`: saved URL snapshot for the next scheduled run ### Why This Exists Raw sitemap extraction is useful, but scheduled users usually care about the answer after extraction: - what changed? - what is stale? - what disappeared? - what is broken? - is this source still safe to ingest? This Actor focuses on that monitoring layer instead of being only another URL list exporter. The first release is intentionally capped for predictable scheduled-run cost. It is not a broad crawler and should not be positioned as a full-site crawling replacement. ### Pricing Direction First release should be free or usage-cost-only while real usage is validated. If organic usage appears, review pay-per-usage or pay-per-event pricing based on measured run cost. A later paid version should charge only for actionable rows such as `NEW`, `CHANGED`, `STALE`, `DEAD`, or `ERROR`, not plain `SEEN` rows. Do not use new rental pricing for this Actor. ### Local Live Smoke Evidence Latest local public-source smoke test: - source: `https://blog.apify.com/rss/` - result: `15` rows - status: passed - artifact: `_target_checks/live-smoke/2026-07-01T10-35-38-168Z.json` # Actor input Schema ## `sourceUrls` (type: `array`): Public sitemap.xml, sitemap index, RSS, or Atom URLs. ## `previousUrls` (type: `array`): Optional URLs from a previous run. URLs missing from this list are marked NEW. ## `maxSources` (type: `integer`): Maximum sitemap/feed source URLs to process in one run. Keep this low for first remote tests and scheduled runs. ## `maxItems` (type: `integer`): Maximum sitemap/feed entries to process across all sources. Values above 1000 are capped by the Actor to protect scheduled-run cost. ## `staleAfterDays` (type: `integer`): Entries older than this threshold are marked STALE when a date is available. ## `checkHttpStatus` (type: `boolean`): Fetch each discovered URL with HEAD/GET and mark dead URLs. Keep off for cheapest runs. ## `snapshotKey` (type: `string`): Key-value store record used to load and save the previous URL snapshot for scheduled monitoring. ## `saveSnapshot` (type: `boolean`): Save the current URL list to the key-value store so the next run can detect NEW versus SEEN URLs automatically. ## Actor input object example ```json { "sourceUrls": [ "https://blog.apify.com/sitemap.xml", "https://blog.apify.com/rss/" ], "previousUrls": [], "maxSources": 10, "maxItems": 500, "staleAfterDays": 90, "checkHttpStatus": false, "snapshotKey": "URL_SNAPSHOT", "saveSnapshot": true } ``` # Actor output Schema ## `items` (type: `string`): Default dataset rows with source, URL, date, status, and recommended action. ## `summary` (type: `string`): No description ## `report` (type: `string`): No description ## `snapshot` (type: `string`): No description # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "sourceUrls": [ "https://blog.apify.com/sitemap.xml", "https://blog.apify.com/rss/" ] }; // Run the Actor and wait for it to finish const run = await client.actor("silicon1235/sitemap-feed-freshness-monitor").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "sourceUrls": [ "https://blog.apify.com/sitemap.xml", "https://blog.apify.com/rss/", ] } # Run the Actor and wait for it to finish run = client.actor("silicon1235/sitemap-feed-freshness-monitor").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"]) for item in client.dataset(run["defaultDatasetId"]).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "sourceUrls": [ "https://blog.apify.com/sitemap.xml", "https://blog.apify.com/rss/" ] }' | apify call silicon1235/sitemap-feed-freshness-monitor --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "command": "npx", "args": [ "mcp-remote", "https://mcp.apify.com/?tools=silicon1235/sitemap-feed-freshness-monitor", "--header", "Authorization: Bearer " ] } } } ``` ## OpenAPI specification ```json { "openapi": "3.0.1", "info": { "title": "Sitemap & Feed Freshness Monitor", "description": "Monitor public sitemaps and RSS/Atom feeds for new, stale, changed, and unreachable URLs.", "version": "0.3", "x-build-id": "Zsj0kM9cArCExNqk2" }, "servers": [ { "url": "https://api.apify.com/v2" } ], "paths": { "/acts/silicon1235~sitemap-feed-freshness-monitor/run-sync-get-dataset-items": { "post": { "operationId": "run-sync-get-dataset-items-silicon1235-sitemap-feed-freshness-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for its completion, and returns Actor's dataset items in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } }, "/acts/silicon1235~sitemap-feed-freshness-monitor/runs": { "post": { "operationId": "runs-sync-silicon1235-sitemap-feed-freshness-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor and returns information about the initiated run in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK", "content": { "application/json": { "schema": { "$ref": "#/components/schemas/runsResponseSchema" } } } } } } }, "/acts/silicon1235~sitemap-feed-freshness-monitor/run-sync": { "post": { "operationId": "run-sync-silicon1235-sitemap-feed-freshness-monitor", "x-openai-isConsequential": false, "summary": "Executes an Actor, waits for completion, and returns the OUTPUT from Key-value store in response.", "tags": [ "Run Actor" ], "requestBody": { "required": true, "content": { "application/json": { "schema": { "$ref": "#/components/schemas/inputSchema" } } } }, "parameters": [ { "name": "token", "in": "query", "required": true, "schema": { "type": "string" }, "description": "Enter your Apify token here" } ], "responses": { "200": { "description": "OK" } } } } }, "components": { "schemas": { "inputSchema": { "type": "object", "required": [ "sourceUrls" ], "properties": { "sourceUrls": { "title": "Sitemap or feed URLs", "type": "array", "description": "Public sitemap.xml, sitemap index, RSS, or Atom URLs.", "items": { "type": "string" } }, "previousUrls": { "title": "Previous URLs", "type": "array", "description": "Optional URLs from a previous run. URLs missing from this list are marked NEW.", "default": [], "items": { "type": "string" } }, "maxSources": { "title": "Maximum source URLs", "minimum": 1, "maximum": 25, "type": "integer", "description": "Maximum sitemap/feed source URLs to process in one run. Keep this low for first remote tests and scheduled runs.", "default": 10 }, "maxItems": { "title": "Maximum items", "minimum": 1, "maximum": 1000, "type": "integer", "description": "Maximum sitemap/feed entries to process across all sources. Values above 1000 are capped by the Actor to protect scheduled-run cost.", "default": 500 }, "staleAfterDays": { "title": "Stale after days", "minimum": 1, "maximum": 3650, "type": "integer", "description": "Entries older than this threshold are marked STALE when a date is available.", "default": 90 }, "checkHttpStatus": { "title": "Check HTTP status", "type": "boolean", "description": "Fetch each discovered URL with HEAD/GET and mark dead URLs. Keep off for cheapest runs.", "default": false }, "snapshotKey": { "title": "Snapshot key", "type": "string", "description": "Key-value store record used to load and save the previous URL snapshot for scheduled monitoring.", "default": "URL_SNAPSHOT" }, "saveSnapshot": { "title": "Save URL snapshot", "type": "boolean", "description": "Save the current URL list to the key-value store so the next run can detect NEW versus SEEN URLs automatically.", "default": true } } }, "runsResponseSchema": { "type": "object", "properties": { "data": { "type": "object", "properties": { "id": { "type": "string" }, "actId": { "type": "string" }, "userId": { "type": "string" }, "startedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "finishedAt": { "type": "string", "format": "date-time", "example": "2025-01-08T00:00:00.000Z" }, "status": { "type": "string", "example": "READY" }, "meta": { "type": "object", "properties": { "origin": { "type": "string", "example": "API" }, "userAgent": { "type": "string" } } }, "stats": { "type": "object", "properties": { "inputBodyLen": { "type": "integer", "example": 2000 }, "rebootCount": { "type": "integer", "example": 0 }, "restartCount": { "type": "integer", "example": 0 }, "resurrectCount": { "type": "integer", "example": 0 }, "computeUnits": { "type": "integer", "example": 0 } } }, "options": { "type": "object", "properties": { "build": { "type": "string", "example": "latest" }, "timeoutSecs": { "type": "integer", "example": 300 }, "memoryMbytes": { "type": "integer", "example": 1024 }, "diskMbytes": { "type": "integer", "example": 2048 } } }, "buildId": { "type": "string" }, "defaultKeyValueStoreId": { "type": "string" }, "defaultDatasetId": { "type": "string" }, "defaultRequestQueueId": { "type": "string" }, "buildNumber": { "type": "string", "example": "1.0.0" }, "containerUrl": { "type": "string" }, "usage": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "integer", "example": 1 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } }, "usageTotalUsd": { "type": "number", "example": 0.00005 }, "usageUsd": { "type": "object", "properties": { "ACTOR_COMPUTE_UNITS": { "type": "integer", "example": 0 }, "DATASET_READS": { "type": "integer", "example": 0 }, "DATASET_WRITES": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_READS": { "type": "integer", "example": 0 }, "KEY_VALUE_STORE_WRITES": { "type": "number", "example": 0.00005 }, "KEY_VALUE_STORE_LISTS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_READS": { "type": "integer", "example": 0 }, "REQUEST_QUEUE_WRITES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_INTERNAL_GBYTES": { "type": "integer", "example": 0 }, "DATA_TRANSFER_EXTERNAL_GBYTES": { "type": "integer", "example": 0 }, "PROXY_RESIDENTIAL_TRANSFER_GBYTES": { "type": "integer", "example": 0 }, "PROXY_SERPS": { "type": "integer", "example": 0 } } } } } } } } } } ```