Sitemap URL Extractor & Checker avatar

Sitemap URL Extractor & Checker

Pricing

from $0.30 / 1,000 urls

Go to Apify Store
Sitemap URL Extractor & Checker

Sitemap URL Extractor & Checker

Get every URL from a website's XML sitemaps: finds sitemaps via robots.txt, follows sitemap indexes, reads .xml.gz and text sitemaps. Returns lastmod, priority and hreflang, optionally checks each URL for 404s and redirects. No proxies.

Pricing

from $0.30 / 1,000 urls

Rating

0.0

(0)

Developer

Tim Tonat

Tim Tonat

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

4 days ago

Last modified

Share

What does Sitemap URL Extractor & Checker do?

This Actor extracts every URL from a website's XML sitemaps. Enter a homepage, and it finds the sitemaps on its own (via robots.txt or the usual paths like /sitemap.xml), follows sitemap indexes, unpacks .xml.gz files and reads text sitemaps. You get one row per URL with lastmod, changefreq, priority, hreflang alternates and image count.

Optionally it checks the HTTP status of every URL and finds 404s, redirects and server errors that should never be in a sitemap.

No proxies and no browser, so runs are fast and cheap, even for sitemaps with hundreds of thousands of URLs.

Why use this sitemap extractor?

  • Get a complete URL list of any website in seconds, without crawling it
  • SEO audits: find sitemap URLs that return 404, redirect or are blocked. Google treats these as sitemap errors
  • Content inventories and migrations: export all URLs with their last modification date to Excel or Google Sheets
  • Monitor competitors: schedule the Actor and see which pages a site adds, using lastmod
  • Feed other tools: pass the URL list to a PageSpeed, SEO or scraping Actor
  • Filter by section with include/exclude globs, e.g. only /blog/**
  • Pay per URL: you control the cost with Max URLs

What data does it return?

FieldDescription
urlURL listed in the sitemap
lastmod, changefreq, priorityValues from the sitemap, null if missing
alternateshreflang alternates (xhtml:link)
imagesNumber of image entries for the URL
sitemapSitemap file the URL was found in
statusCodeHTTP status code (only with Check HTTP status)
redirectToRedirect target, if the URL redirects
issuesREDIRECT, HTTP_404, HTTP_500, … or UNREACHABLE
errorReason if a sitemap could not be read

The Summary (key-value store record SUMMARY) lists every sitemap file found with its URL count and errors, plus duplicates, URLs without lastmod, filtered URLs and status code counts.

How to extract all URLs from a sitemap

  1. Click Try for free.
  2. Enter one or more homepages or sitemap URLs in Websites or sitemap URLs.
  3. Set Max URLs.
  4. Enable Check HTTP status of every URL if you want to find broken URLs.
  5. Click Start and download the results as JSON, CSV, Excel or HTML.

How much does it cost?

This Actor uses pay per event, and platform usage is included:

  • a small price per extracted URL
  • plus a price per checked URL, only if you enable the status check

The current prices are shown in the Pricing tab. URLs that can't be reached during the status check are not charged for the check. With Max URLs you always know the maximum cost in advance.

Input

See the Input tab for all options.

{
"startUrls": ["https://example.com", "https://example.org/sitemap_index.xml"],
"maxUrls": 10000,
"checkStatus": true,
"includeGlobs": ["https://example.com/blog/**"],
"excludeGlobs": ["**/tag/**"]
}

Output

You can download the dataset in various formats such as JSON, HTML, CSV, or Excel.

{
"url": "https://example.com/blog/sitemap-guide/",
"lastmod": "2026-10-01T08:30:00+00:00",
"changefreq": "weekly",
"priority": 0.8,
"alternates": [{ "lang": "de", "url": "https://example.com/de/blog/sitemap-guide/" }],
"images": 2,
"sitemap": "https://example.com/post-sitemap.xml",
"statusCode": 301,
"redirectTo": "https://example.com/blog/sitemaps/",
"issues": ["REDIRECT"],
"error": null
}

Use the URL list as input for the other audits:

FAQ

How does it find the sitemap?

It reads all Sitemap: lines in robots.txt. If there are none, it tries /sitemap.xml, /sitemap_index.xml and /wp-sitemap.xml. You can always enter the sitemap URL directly.

Why do some URLs show HTTP 403 in the status check?

Some large sites block automated requests with a bot protection and answer 403 to everything that isn't a browser. The URL is probably fine for visitors. The status check is most useful for your own sites and client sites.

Are there limits?

Sitemap files can be up to 50 MB (the limit of the sitemap protocol), and sitemap indexes are followed up to 5 levels deep. Each URL is returned only once, duplicates are counted in the summary.

Something does not work

Please open an issue in the Issues tab with the website and the run ID. You can also start the Actor programmatically from the API tab.