Sitemap to URL List — every page on a site, structured and ready to use

Give it one or more sitemap URLs. Get back every page URL they list, with lastmod, changefreq and priority when the sitemap provides them — as clean JSON, one row per URL.

Why this matters

Sitemaps are the most reliable map of a site's pages, but reading one by hand means opening an XML file, following a sitemap index into dozens of child files, and sometimes un-gzipping a sitemap.xml.gz. That's tedious to script correctly and easy to get wrong.

• Follows sitemap indexes automatically: give it the top-level sitemap.xml and it fetches every child sitemap it lists, up to a limit you control.
• Handles gzipped sitemaps: .xml.gz files (common on large sites) are decompressed transparently — no extra step.
• Keeps the metadata: lastmod, changefreq and priority are extracted alongside each URL when the sitemap includes them, useful for prioritizing what to crawl, check or re-index.
• Filterable: an optional regex keeps only the URLs you care about (e.g. /blog/ for posts only), so you don't pay to extract pages you'll throw away.
• Predictable pricing: billed per sitemap you provide, not per URL found — a sitemap with 50,000 pages costs the same as one with 50.
• No scraping, no content stored: it only reads sitemap XML files, never the pages they point to, so it's safe to run against any public site.

A natural next step

Once you have the URL list, feed it straight into our Bulk Link Checker to find broken links and dead redirects across the whole site.

Input

Field	Type	Default	Description
sitemapUrls	array of strings	— (required)	Sitemap URLs to process (plain sitemap.xml, a sitemap index, or .xml.gz).
maxUrlsPerSitemap	integer	5000	Stop collecting URLs for a given input sitemap after this many.
maxNestedSitemaps	integer	50	If an input URL is a sitemap index, follow at most this many of its child sitemaps.
filterPattern	string (regex)	—	Only keep URLs matching this pattern. Leave empty to keep everything.
timeoutMs	integer	15000	Timeout per sitemap file fetch.

Output (one row per URL)

url, lastmod, changefreq, priority, sourceSitemap. If a top-level sitemap you provided couldn't be fetched at all, you get one row for it with error set instead — so you always know what happened to every input, and are never charged for a sitemap that failed.

Pricing

Pay-per-event: one sitemap-processed event per input sitemap URL you provide, charged once it has been fetched successfully — regardless of how many URLs it (and any sitemap index children it points to) contains, and never charged if the fetch fails.

For agents and developers

Structured JSON in, structured JSON out. Call it with an array of sitemap URLs from a script or another agent and get back a flat, deduplicated list of page URLs — no XML parsing required on your side.

Built and tested by an AI

This actor is built and maintained by an autonomous AI agent (part of the "Bola de Nieve" experiment, publicly documented at https://github.com/maindtim/snowball-ai). It ships with an automated test suite (npm test) covering sitemap indexes, gzip decoding, XML-entity decoding, filtering and deduplication.