Sitemap URL Extractor – Get All Website URLs from XML Sitemaps

Extract every page URL of any website from its XML sitemaps in seconds. Paste a domain and this Actor finds the sitemaps for you (robots.txt Sitemap: lines, /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml), follows sitemap index files and gzipped .xml.gz sitemaps, and returns each URL with lastmod, changefreq, priority, image count, video flag, news title and hreflang alternates. No browser, no proxies, no login.

What can I use it for?

• SEO audits & content inventories – list all indexable pages of your site or a competitor's.
• Change monitoring – set Only URLs modified after and schedule daily to get just the new or updated pages.
• Crawl seeding – feed the URL list into Website Content Crawler, a link checker or an AI/RAG pipeline.
• Migration checks – compare URLs before/after a redesign or CMS migration.
• Find all PDFs, blog posts or product pages – filter with regex like \.pdf$, /blog/ or /products/.

Features

• Auto-discovery from robots.txt and common sitemap locations (WordPress, Yoast, Shopify, Next.js, Docusaurus…)
• Sitemap index recursion (up to 5 levels), .gz sitemaps, plain-text and RSS/Atom sitemaps
• Include / exclude regex filters and lastmod date filter
• Per-website cap so one huge site doesn't eat your budget
• Polite: retries with backoff, honours Retry-After, per-host rate limiting
• Clear error report per website in the RUN_SUMMARY record (no silent failures)

Input example

{
  "startUrls": ["apify.com", "https://crawlee.dev/sitemap.xml", "https://www.gov.uk"],
  "maxUrls": 5000,
  "maxUrlsPerSite": 2000,
  "includePatterns": ["/blog/"],
  "excludePatterns": ["/tag/"],
  "lastmodAfter": "2026-01-01"
}

Output example

{
  "url": "https://www.gov.uk/government/news/example",
  "lastmod": "2026-10-22T09:14:03+01:00",
  "changefreq": null,
  "priority": null,
  "imageCount": 0,
  "hasVideo": false,
  "newsTitle": null,
  "alternates": [],
  "sitemapUrl": "https://www.gov.uk/sitemaps/sitemap_1.xml",
  "domain": "www.gov.uk",
  "startUrl": "https://www.gov.uk"
}

Download as JSON, CSV, Excel, XML or HTML, or fetch via the Apify API.

Pricing

Pay per event: $1.00 per 1,000 URLs extracted (you only pay for URLs returned). Set a max cost per run and the Actor stops cleanly at your limit.

Use with AI agents (MCP)

This Actor is available as a tool in the Apify MCP server. Add it with https://mcp.apify.com?tools=<username>/sitemap-url-extractor in Claude, Cursor, VS Code or any MCP client.

FAQ

The site has no sitemap? The run summary lists it under errors. Use a crawler instead for that site. Is it legal? Sitemaps are published by site owners specifically for automated consumption. The Actor only reads sitemap files and robots.txt, never page content. Why are some URLs missing with a date filter? URLs without <lastmod> are excluded when Only URLs modified after is set.

Found a problem? Open an issue on the Actor's Issues tab – we respond fast.