Sitemap URL Extractor

Get every URL a website publishes in its sitemaps — as a clean table you can download as CSV, JSON or Excel, or feed straight into another Actor or workflow.

Paste a website address and the Actor finds the sitemaps for you. No need to know where they live.

What it does

• Finds sitemaps automatically from robots.txt and the usual locations (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, …).
• Follows sitemap indexes to any depth, so large sites with hundreds of nested sitemaps work.
• Reads every format: XML sitemaps, gzipped .xml.gz, plain-text sitemaps, RSS and Atom feeds.
• Keeps the metadata: lastmod, changefreq, priority, image URLs, hreflang alternates and Google News titles.
• Filters as it goes: include/exclude URL patterns and a Modified since date. Schedule it daily to get only new or updated pages.
• Optional status check: records the HTTP status code and redirect target of each URL, so you can find broken (404) and redirected pages listed in your sitemap.
• Never loses a whole run to one bad file. Unreadable sitemaps are skipped and listed in the SUMMARY record, and everything else is saved.

Use cases

• SEO audits: find 404s, redirects and orphaned URLs in your sitemap.
• Content monitoring: track new pages, products or articles on competitor sites with a daily schedule and Modified since.
• Crawl planning: get a clean list of URLs to feed into a content or product scraper, instead of crawling blindly.
• AI / RAG pipelines: enumerate every docs or blog page before indexing.
• Site migrations: snapshot every URL before and after a move.

Input

Field	Description
Start URLs	Websites or direct sitemap/feed URLs
Include / Exclude URL patterns	Regular expressions, e.g. /blog/, /products/, /tag/
Modified since	Keep only URLs with lastmod on or after this date
Check HTTP status	Adds statusCode and finalUrl for every URL
Max URLs / Max URLs per website	Cost and size limits

Output

{
  "url": "https://example.com/blog/hello-world",
  "lastmod": "2026-09-21T10:00:00Z",
  "changefreq": "weekly",
  "priority": 0.8,
  "statusCode": 200,
  "sitemapUrl": "https://example.com/post-sitemap.xml",
  "site": "https://example.com"
}

A SUMMARY record in the key-value store lists the sitemaps found per website and any that could not be read.

Pricing

Pay only for results: you are charged per URL saved, plus per URL when the optional status check is on. Set Max URLs or a maximum cost per run to stay in budget. The Actor stops cleanly at your limit and keeps everything collected so far.

FAQ

The site has no sitemap. The run tells you so (it checks robots.txt and common locations). If you know where the sitemap lives, paste its URL directly.

Some URLs have no lastmod. The site didn't publish one. With Modified since set, those URLs are skipped because their age can't be confirmed.

Is this legal? Sitemaps are published by site owners specifically so automated tools can read them.

Found a problem or need a feature? Open an issue on the Issues tab. Issues are answered within 24 hours.