Sitemoss: Sitemap URL Extractor — Get All Website URLs from Sitemaps

Every page URL of a website, from its own sitemaps, in seconds

Give it domains or sitemap URLs; get every page URL the site publishes, with lastmod, changefreq, priority, hreflang alternates and image counts. Sitemoss finds the sitemaps itself (robots.txt Sitemap: lines, /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml), follows sitemap indexes and reads .xml.gz and .txt sitemaps. $0.40 per 1,000 URLs, no start fee.

▶ Try it now with a prefilled input · $0.0004 per URL (10,000 URLs = $4) · sites with no sitemap and filtered-out URLs are free · Apify's free plan includes $5 of credit a month

Built for: SEO teams auditing or migrating a site · content teams tracking a competitor's new pages · developers feeding a crawler, link checker or RAG pipeline · AI agents that need "all URLs of example.com"

Real output (real run on Apify, 25 Oct 2026: 2 sites, 20 URLs, 2.6 s, $0.008):

URL	Found in	Site
https://apify.com/	apify.com/sitemap/pages.xml (from the sitemap index)	apify.com
https://apify.com/about	apify.com/sitemap/pages.xml	apify.com
https://apify.com/actors	apify.com/sitemap/pages.xml	apify.com
https://crawlee.dev/blog	crawlee.dev/sitemap.xml (found via robots.txt)	crawlee.dev
https://crawlee.dev/blog/archive	crawlee.dev/sitemap.xml	crawlee.dev

{"urls": ["https://apify.com/sitemap.xml", "crawlee.dev"], "maxUrlsPerSite": 10}

Ready-made inputs (one click)

Job	Input	Result
First try	{"urls": ["https://apify.com/sitemap.xml"], "maxUrlsPerSite": 20}	20 URLs, about $0.008.
All URLs of a site	{"urls": ["example.com"], "maxUrlsPerSite": 50000}	Every URL in every sitemap the site lists.
Only blog posts changed this month	{"urls": ["example.com"], "urlPattern": "/blog/", "lastmodAfter": "2026-10-01"}	Recently updated posts only; the rest is free.
Competitor new-page alerts	{"urls": ["competitor.com"], "monitor": true, "monitorKey": "competitor"}	On a schedule: only pages that are new or have a new lastmod.

Use cases

• Site audits and migrations: the full URL inventory in one table, ready for a redirect map or a status check (pipe it into Bouncefern).
• Competitor content tracking: monitor mode on a daily schedule returns only the pages a competitor added or updated.
• Crawler seeds and RAG: start a crawl from the URLs a site wants indexed instead of guessing links.
• International SEO: every hreflang alternate the sitemap declares, per URL.
• Image and video SEO: image and video counts per URL from image/video sitemap extensions.

Why Sitemoss

	Sitemoss	Typical sitemap extractor on the Store
Finds the sitemap for you	robots.txt, then 4 well-known paths	Often needs the exact sitemap URL
Sitemap indexes, .gz, .txt	All followed and read	Varies
Fields	lastmod, changefreq, priority, hreflang alternates, image/video counts, source sitemap, index depth	Often URL only
New/updated pages only	Built-in monitor mode, unchanged URLs free	Not offered
Price	$0.40 per 1,000 URLs, no start fee	$0.35 to $100 per 1,000 (rival median about $1.50, Store prices on 25 Oct 2026), some with start fees

Input

Field	What it does
urls	Domains, any page of a site, or sitemap URLs (required).
maxUrlsPerSite	Stop after this many URLs per site (default 1,000).
urlPattern, excludePattern	Regex include/exclude on the URL.
lastmodAfter	Only URLs whose sitemap lastmod is on/after this date.
sameSiteOnly	Drop entries on other domains.
monitor, monitorKey	Only new or updated URLs since the last monitor run.

Output

One row per URL: url, lastmod (ISO), changefreq, priority, imageCount, videoCount, alternates (hreflang + href), newsTitle, site, sitemap (the file it came from), sitemapDepth, and change in monitor mode. The run's OUTPUT record has per-site counts, the sitemap source (input, robots.txt or well-known paths) and any errors.

Pricing

• $0.0004 per URL delivered ($0.40 per 1,000). No start fee.
• Free: duplicates, filtered-out URLs, unchanged URLs in monitor mode, sites without a sitemap.
• Set a maximum charge per run in Apify's run options; Sitemoss stops cleanly when it is reached.

Run it on a schedule

Apify Console → Schedules → Create new → cron 0 6 * * * → Add Actor → Sitemoss with your sites and "monitor": true. Through the API:

POST https://api.apify.com/v2/schedules?token=YOUR_TOKEN
{
  "name": "sitemoss-competitor-pages",
  "cronExpression": "0 6 * * *",
  "timezone": "America/New_York",
  "isEnabled": true,
  "actions": [{
    "type": "RUN_ACTOR",
    "actorId": "cybermax/sitemap-url-extractor",
    "runInput": {
      "body": "{\"urls\":[\"competitor.com\"],\"monitor\":true,\"monitorKey\":\"competitor\",\"maxUrlsPerSite\":50000}",
      "contentType": "application/json"
    }
  }]
}

Add an email, Slack or webhook notification on "Run succeeded" under the Actor's Integrations tab.

Integrations

API (synchronous): POST https://api.apify.com/v2/acts/cybermax~sitemap-url-extractor/run-sync-get-dataset-items?token=YOUR_TOKEN with the input JSON returns the URL rows.

Make / Zapier: Apify app → "Run an Actor" (cybermax/sitemap-url-extractor, run synchronously) → "Get dataset items" → map url and lastmod into Google Sheets, Airtable or Slack.

n8n: Apify node ("Run an Actor and get dataset"), or HTTP Request:

POST https://api.apify.com/v2/acts/cybermax~sitemap-url-extractor/run-sync-get-dataset-items?token=YOUR_TOKEN
{"urls": ["example.com"], "maxUrlsPerSite": 5000}

Google Sheets:

=IMPORTDATA("https://api.apify.com/v2/acts/cybermax~sitemap-url-extractor/runs/last/dataset/items?format=csv&fields=url,lastmod,sitemap&token=YOUR_TOKEN")

Python (pip install apify-client):

from apify_client import ApifyClient
client = ApifyClient("YOUR_TOKEN")
run = client.actor("cybermax/sitemap-url-extractor").call(run_input={"urls": ["example.com"], "maxUrlsPerSite": 5000})
urls = [r["url"] for r in client.dataset(run["defaultDatasetId"]).iterate_items()]

AI agents (MCP): https://mcp.apify.com/?tools=cybermax/sitemap-url-extractor (Authorization: Bearer YOUR_TOKEN). Typical prompts: "List every blog URL on example.com updated since October", "Which pages did competitor.com add this week?".

When NOT to use it

• Sites without a sitemap: Sitemoss reads what the site publishes; it doesn't crawl links. Those sites are reported in OUTPUT and cost nothing.
• Pages a site leaves out of its sitemap (private, noindex or forgotten pages) won't appear.

FAQ

Why not open sitemap.xml in a browser? Fine for one small file. Real sites split sitemaps into indexes of dozens of gzipped files; Sitemoss follows all of them, de-duplicates, filters and hands you one table, on a schedule if you like.

Why not a full website crawler? A crawler fetches every page (slow, costly, and heavier on the site). The sitemap is the site's own list of pages and comes back in seconds.

Can an AI agent use it (MCP)? Yes. Add https://mcp.apify.com/?tools=cybermax/sitemap-url-extractor as an MCP server in Claude, Cursor, VS Code or any MCP client.

How many URLs can it handle? Up to 1,000,000 per site per run (maxUrlsPerSite), streamed in batches with flat memory.

Is it legal to use? Sitemaps are published by site owners precisely so automated tools can read them, and Sitemoss reads only the sitemap files and robots.txt. This is general information, not legal advice.

Start now

▶ Try Sitemoss with a prefilled input · $0.40 per 1,000 URLs. Questions or a sitemap it can't read? Open an issue on the Store page and it gets answered the same day.

Also from CyberMax

• Bouncefern — bulk URL status & redirect checker: status code, final URL and redirect chain for every URL Sitemoss finds.
• Unfurlwren — meta tags & Open Graph extractor: title, description, OG image and schema.org per page, with SEO issues.
• Eldwren — Wayback Machine scraper: a site's archive history and every URL it ever had.
• Keyfern — long-tail keyword ideas: Google Autocomplete keyword research.

Changelog

• 2026-10-25: First release: robots.txt and well-known sitemap discovery, sitemap indexes, .gz and .txt sitemaps, hreflang/image/video fields, filters, monitor mode.

© 2026 CyberMax. All rights reserved.