Sitemap URL Extractor & New Page Monitor

Pull every URL, lastmod and priority from any site's XML sitemaps (robots.txt discovery, indexes, .gz), with no login, no browser and no code. Export to JSON, CSV or Excel, or use the API.

Get every URL a website lists in its XML sitemaps, with lastmod, changefreq and priority. Then schedule it with monitor mode to get only the new, updated and removed pages since the last run.

Use cases

• Track competitor product launches, landing pages, pricing pages and blog posts
• Get alerts when a news site or blog publishes
• SEO audits: count indexed URLs and find stale sections
• Feed new URLs into another scraper

How it works

It reads robots.txt for Sitemap: lines, falling back to /sitemap.xml and /sitemap_index.xml. It follows sitemap indexes, handles gzip, and filters by an optional regex. It never downloads page HTML.

Input

domains, urlFilter, maxUrlsPerDomain, monitorMode, outputMode (all or changesOnly).

Output

One row per URL: domain, url, lastmod, changefreq, priority, sitemap, status (current/new/modified/removed). A per-domain summary is saved to the RUN_SUMMARY key-value record.

Example output

{
  "domain": "www.sitemaps.org",
  "url": "https://www.sitemaps.org/",
  "lastmod": "2016-11-21",
  "changefreq": null,
  "priority": null,
  "sitemap": "https://www.sitemaps.org/sitemap.xml",
  "status": "current",
  "checkedAt": "2026-10-12T01:26:27.634Z"
}

Pricing

Pay per result, with no monthly rental or subscription:

• URL record: $0.0002
• New/modified/removed URL (monitor): $0.002

Example: 1,000 results cost about $0.20 in fees. Apify's free plan includes $5 of monthly credit, which is enough to test this Actor.

FAQ

Do I need an account, cookies or a login for the source site? No. Sitemap URL Extractor & New Page Monitor uses public data sources only, so there is nothing to connect and no account to risk.

How much will a run cost? You pay only for results delivered (see Pricing above), plus Apify platform usage, which the free plan's monthly credit easily covers for test runs. Start with a small limit to see the exact cost per run.

Can I export to CSV, Excel or Google Sheets? Yes. Every run's dataset can be downloaded as JSON, CSV, Excel, XML or HTML, or pushed to Google Sheets, Make, Zapier, n8n, Slack or a webhook through Apify integrations.

Can I run it on a schedule or from my own code? Yes. Add an Apify Schedule (hourly, daily, weekly) or call it through the Apify API / Python & JavaScript clients. It also works with AI agents through the Apify MCP server.

What happens if an input is invalid or a source is down? The run keeps going. The bad input gets a row with an error field so you can see what failed, and you are not charged for results that were not delivered.

Is this legal to use? It collects publicly available data. You are responsible for using the output in line with the source's terms and privacy laws such as GDPR/CCPA; don't use it for spam.

Something missing or broken? Open an issue on the Issues tab. Feature requests are usually added within a few days.

Related tools

• Broken Link Checker & Redirect Chain Auditor
• On-Page SEO Audit & Metadata Extractor (bulk + crawl)
• Product Price Monitor for Any Store (schema.org) + Alerts
• Wayback Machine History: Site Age, Snapshots & Archived URLs

Pipeline recipe: sitemap-monitor → link-auditor → seo-page-audit: list every URL, find broken links and redirects, then audit titles, meta and schema.

REST API

GET https://api.mediaprotools.com/v1/sitemap-monitor?items=... (header X-API-Key). Up to 3 items per call, separated by ,. Or POST {"items":[...],"options":{...}}.

Parameter	Type	Notes
items	token	Sites to read. Sitemaps are found via robots.txt, then /sitemap.xml and /sitemap_index.xml.
urlFilter	string	Keep only URLs matching this, e.g. /blog/ or /products/.
maxUrlsPerDomain	integer 1-5000	Stops reading sitemaps after this many URLs. Removed-page detection is skipped when truncated.
maxSitemapsPerDomain	integer 1-100	Safety cap on sitemap files fetched.
outputMode	string	changesOnly needs a previous run.

Output fields: changefreq, checkedAt, domain, error, lastmod, priority, sitemap, status, url.