Sitemap Extractor: All URLs + Status Check

Give it a website, get every URL from its sitemaps, with last-modified dates, priorities, images and hreflang alternates. Optionally check every URL's HTTP status to find broken (404) and redirected pages that your sitemap still lists.

Why this one

• 🔍 Auto-discovery: reads robots.txt, then tries common locations (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, …)
• 🪆 Nested sitemap indexes followed automatically (tested with 30+ nested sitemaps)
• 🗜️ Gzip (.xml.gz) and plain-text sitemaps supported
• 🖼️ Images and hreflang alternates extracted per URL
• 🩺 Status check (optional): status code + redirect target for every URL
• 🎯 Filters: include/exclude by regex, "modified since" date
• 🔁 De-duplicated URLs; a site without a sitemap never fails the whole run
• ⚡ No browser, so runs are fast and efficient

Use cases

• SEO audits: find 404s, redirects and stale pages listed in the sitemap
• Content inventory: list every blog post or product page of a site
• Competitor monitoring: schedule daily with modified since to see new or updated pages
• Crawl planning: feed the URL list into another scraper

Input example

{
  "startUrls": [{ "url": "https://apify.com" }, { "url": "https://example.com/sitemap_index.xml" }],
  "includePattern": "/blog/",
  "modifiedSince": "2026-09-01",
  "checkStatus": true
}

Output example

{
  "site": "https://apify.com",
  "url": "https://apify.com/about",
  "lastmod": "2026-09-30",
  "changefreq": "weekly",
  "priority": 0.8,
  "imagesCount": 1,
  "images": ["https://apify.com/img/team.jpg"],
  "alternates": [{ "hreflang": "fr", "href": "https://apify.com/fr/about" }],
  "sitemap": "https://apify.com/sitemap/pages.xml",
  "statusCode": 200,
  "redirectTo": null
}

statusCode and redirectTo only appear when Check HTTP status is on.

Tips

• Sitemaps are discovered per domain: entering https://site.com/blog/ reads the whole site's sitemaps. Use Only URLs matching (/blog/) to narrow down.
• Very large sites: raise Max URLs per site (default 50,000).

FAQ

The site has no sitemap. You'll get a row explaining that no sitemap was found; other sites in the run continue normally.

Is it legal? Sitemaps are published by sites specifically so they can be read by crawlers.