Sitemap Extractor - Sitemap URL Scraper for XML & Gzip

Sitemap Extractor - Sitemap URL Scraper for XML & Gzip

Extract sitemap URLs and metadata from XML, gzip, nested sitemap indexes, RSS/Atom and text lists for SEO specialists, migration teams and content-monitoring developers. Get deduplicated URLs, lastmod dates and available hreflang/media metadata; this does not crawl the listed pages.

First 25 saved unique URLs of every run are free; after that, $0.50 per 1,000 saved unique URLs. Filtered URLs, duplicates and failed sitemap documents do not create result events. The Pricing tab is authoritative. Apify plan allowances and any applicable platform/storage charges are separate.

Example output (illustrative abbreviated record, not a captured result):

{"loc":"https://example.com/page","lastmod":"2026-10-01","sitemapSource":"https://example.com/sitemap.xml","alternates":[],"images":[],"videos":[],"news":[]}

What does Sitemap URL Extractor do?

Extract URLs from XML sitemaps, nested sitemap indexes, compressed .xml.gz files, RSS/Atom feeds and plain-text URL lists. Supply domains or sitemap URLs and get a deduplicated inventory with modification dates, hreflang alternates and image/video/news metadata. This is a lightweight HTTP sitemap extraction API, not a website crawler: it never visits the listed pages or checks whether they are indexed by search engines. It follows the Sitemaps protocol.

Why use this sitemap extractor?

• Build SEO audits, migration inventories and content-monitoring workflows.
• Discover sitemaps through robots.txt and /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml.
• Follow nested indexes, decompress gzip, preserve alternate languages and rich sitemap extensions.
• Filter URLs using safe RE2 include/exclude regexes and an inclusive last-modified date.
• Export results, schedule runs, connect integrations and call it through the Apify API.
• Stop at an exact saved-URL cap or your configured result-charge budget.

How to extract sitemap URLs

1. Open the Input tab and enter a domain, site root, or direct sitemap URL. The default is a small public sitemap.
2. Set maxUrls, optionally add filters, then start the Actor.
3. Download the dataset or use the API tab for programmatic integration.
4. Check SUMMARY for skipped documents, robots restrictions and the stop reason.

What data can it extract?

Field	Type	Meaning
loc	string	Unique normalized URL; fragments removed
lastmod	string/null	Publisher lastmod, Atom updated or RSS pubDate
changefreq	string/null	Publisher-supplied change frequency
priority	number/null	Publisher priority, not actual search ranking
sitemapSource	string	Requested sitemap supplying the first occurrence
alternates	array	hreflang and href pairs
images, videos, news	arrays	Source extension metadata with namespace prefixes removed

All nine output fields appear in the dataset schema. URLs are deduplicated within a run; first occurrence wins, so metadata is not merged across duplicate entries. Alternates and media remain nested and do not create separately billed rows. Missing extension groups are empty arrays.

How much does sitemap extraction cost?

First 25 saved unique URLs of every run are free; after that, $0.50 per 1,000 results. One result = one unique, filter-matching URL saved to the dataset, including its metadata.

This Actor is public with pay-per-event pricing. The Pricing tab shows the current event price. No extra custom start fee is implemented. Apify account limits and applicable platform charges depend on your plan. Failed documents, duplicates and filtered URLs do not generate result events.

Input and filters

See the Input tab for full configuration options. Example:

{
  "startUrls": [{"url":"https://www.sitemaps.org/sitemap.xml"}],
  "maxUrls": 100,
  "maxSitemaps": 100,
  "includeRegex": "sitemaps\\.org",
  "excludeRegex": "[?]preview=",
  "lastmodSince": "",
  "delayMillis": 1000,
  "maxResponseMb": 10
}

A bare domain is treated as HTTPS. Root URLs trigger discovery; other URLs are parsed directly as a sitemap or feed. Include/exclude expressions match anywhere in loc and exclusion wins. RE2 deliberately rejects lookarounds and backreferences. lastmodSince excludes missing or invalid modification dates. Defaults cap results at 100 and sitemap documents at 100; increase these for larger inventories.

Output

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel. A simplified format example (not a captured result):

[
  {"loc":"https://example.com/a","lastmod":"2026-10-01","changefreq":"weekly","priority":0.8,"sitemapSource":"https://example.com/sitemap.xml","alternates":[{"hreflang":"fr","href":"https://example.com/fr/a"}],"images":[],"videos":[],"news":[]},
  {"loc":"https://example.com/b","lastmod":null,"changefreq":null,"priority":null,"sitemapSource":"https://example.com/sitemap.xml","alternates":[],"images":[],"videos":[],"news":[]}
]

SUMMARY contains total saved rows, processed document count, stop reason and per-document errors. STATE holds deduplication keys and pending work for graceful migration/restart.

Limits, safety and troubleshooting

Only public HTTP(S) endpoints on standard ports are fetched. Private/reserved destinations are blocked, DNS answers are pinned, and redirects are checked again. robots.txt is checked before each origin's sitemap requests. Missing robots (404/410) allows access; denied, challenged or unavailable robots fails closed. Requests are sequential, with at least one second per host and robots Crawl-delay honored up to 60 seconds. No proxy, login, browser, CAPTCHA solving or access-control bypass is used.

Both transferred and inflated documents are capped at 10 MB by default (configurable up to 20 MB). Index depth is capped at 20. Oversized or malformed XML, DTDs and entity declarations are rejected; unsupported encodings and unusual custom namespaces are not guaranteed. Image/video/news use standard extension prefixes. The Actor does not fetch media, follow feed pagination, crawl HTML links or guarantee that a sitemap lists every page. Large inventories should use direct child sitemaps and sensible caps. Crash recovery is best-effort; dataset write and billing are not transactional.

An empty successful run may mean filters excluded everything. All-document access/parsing failure fails the run and records errors. Partial runs retain valid saved rows. Use only sites you are authorized to access, comply with terms and robots rules, and avoid collecting sensitive personal data in URL query strings. Public URLs can still contain regulated data; you are responsible for a lawful purpose. Report reproducible issues through the Issues tab, with the input URL and SUMMARY error, never credentials.