Sitemap URL Extractor: XML Sitemap Parser, Sitemap Index Crawler & URL List Exporter

Sitemap extractor and XML sitemap parser for any website. Paste a homepage or a sitemap.xml URL and get every published page as a clean list: loc, lastmod, changefreq, priority, the sitemap it came from, and (optionally) the live HTTP status. Nested sitemap indexes, .xml.gz files and Sitemap: lines in robots.txt are followed automatically.

Use it as a sitemap scraper for SEO audits, crawl seeding, RAG URL discovery, redirect maps and site migrations. No login, no API key, no headless browser and no paid proxy.

Why this sitemap extractor

	This Actor	Typical sitemap scrapers
Auto-discover from homepage + robots.txt	✅	Often needs the exact sitemap URL
Nested sitemap indexes + .xml.gz	✅	Partial
lastmod, changefreq, priority	✅	Sometimes URL only
Optional HTTP status check	✅ opt-in, off by default so large sites stay cheap	Often forced, slow and expensive
Image sitemap <image:loc>	✅ optional	Rare
Path include / exclude filters	✅	Rare
Price	$0.09 per 1,000 URLs	Official Store extractor starts at $0.50 / 1,000 on the free plan

Who uses it

• SEO audits: export every URL a site declares, then find stale lastmod values or pages that now 404.
• Crawl seeding: feed Website Content Crawler, RAG pipelines or custom scrapers a complete, de-duplicated URL list.
• Site migrations and 301 maps: compare an old sitemap with the live site.
• Competitor research: see how a competitor structures blog, product and locale URLs.
• AI / agents: give an LLM a real inventory of pages instead of guessing from the homepage.

Example input

{
  "startUrls": ["https://docs.apify.com", "https://www.sitemaps.org/sitemap.xml"],
  "checkPageStatus": false,
  "pathContains": "",
  "pathExcludes": "/tag/,/author/",
  "maxUrls": 5000
}

Accepted input: a homepage (https://example.com), a bare domain (example.com), or a direct sitemap (https://example.com/sitemap.xml, sitemap_index.xml, sitemap.xml.gz).

Example output

{
  "url": "https://docs.apify.com/platform/actors",
  "lastmod": "2026-10-12T00:00:00+00:00",
  "changefreq": "weekly",
  "priority": "0.8",
  "sitemapUrl": "https://docs.apify.com/sitemap_base.xml",
  "source": "docs.apify.com",
  "extractedAt": "2026-11-06T07:30:00+00:00"
}

When Check HTTP status is on, each row also includes status, finalUrl, responseMs and error (if the request failed).

A SUMMARY record in the default key-value store reports how many URLs were extracted and how many sitemap files were parsed.

Pricing

Pay per event:

Event	Price
Actor start	$0.00005
URL extracted	$0.00009 ($0.09 per 1,000 URLs)

You only pay for unique URLs written to the dataset. Failed sitemap downloads are free. Set a maximum charge per run to cap spend.

How it works

1. For a homepage, the Actor reads robots.txt for Sitemap: directives and also tries common paths (/sitemap.xml, /sitemap_index.xml, …).
2. It parses urlset and sitemapindex documents, including gzip-compressed sitemaps, and walks nested indexes.
3. URLs are de-duplicated, optionally filtered, and exported.
4. If you turn on status checks, each URL is verified with HEAD (GET fallback) — no JavaScript rendering and no proxy required.

API example

curl -X POST "https://api.apify.com/v2/acts/classier_wholemeal~sitemap-url-extractor/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://docs.apify.com"], "maxUrls": 200}'

Export results as JSON, CSV or Excel, or send them to Google Sheets, Zapier, Make, n8n or a webhook. It also works as a tool for AI agents through the Apify MCP server.

FAQ

Do I need the exact sitemap URL? No. A homepage is enough in most cases. If a site hides its sitemap, paste the sitemap URL directly.

Does it follow sitemap indexes? Yes. Nested indexes are crawled up to maxSitemapDepth (default 10; 0 means no limit).

Will this get blocked? It only downloads public sitemap and robots files with a clear User-Agent. It does not render pages or bypass bot protections. Status checks use simple HEAD/GET requests; turn them off for the lightest run.

What about huge sitemaps (100k+ URLs)? Set maxUrls and a maximum charge per run. Increase memory on very large sites if the XML files themselves are big.

Is this legal? Yes. Sitemaps and robots.txt are published so crawlers can find pages. Use the URL list in line with the site's terms and applicable law.

Does it replace a full website crawler? It lists the URLs the site declares. Pages that are not in any sitemap will not appear. Combine it with a crawler if you need undiscovered URLs.

Can I filter to a section of the site? Yes. Use pathContains (e.g. /blog/) and pathExcludes (comma-separated, e.g. /tag/,/author/).