Sitemap URL Extractor: All Pages, Lastmod & Status

Get every page URL a website lists in its sitemaps, with lastmod dates, change frequency, priority, hreflang alternates, image counts and Google News titles. Just type a domain: sitemaps are found automatically from robots.txt and common paths, and sitemap indexes (including .xml.gz) are followed all the way down.

Fast and cheap: a 3,000-URL site takes about a second.

What you can do with it

• Crawl planning: feed a clean, de-duplicated URL list into a content scraper instead of crawling blindly.
• Change monitoring: schedule it with Changed on or after to get only new or updated pages (new blog posts, products, job pages, news articles).
• SEO audits: find broken links and redirects listed in the sitemap (Check HTTP status), pages with missing lastmod, and hreflang coverage.
• Competitor research: see how many product, blog or landing pages a competitor has, and how fast they publish.
• News tracking: read news sitemaps for headline + publication time, minutes after publishing.

Input

Field	What it does
urls	Domains, homepages, section URLs (https://www.bbc.com/news) or sitemap URLs. One per line.
onlyUnderPath	For a section URL, return only pages under that path (on by default).
includeUrlPatterns / excludeUrlPatterns	Keep or drop URLs containing these strings.
lastmodAfter	Only pages changed on or after this date.
checkStatus	Adds statusCode, finalUrl and httpLastModified per URL (broken links, redirects, server-side change date).
maxUrlsPerSite, maxItems	Caps to control cost. maxItems is shared fairly: every site gets an equal share, and sites with fewer URLs pass the rest on.

Example:

{ "urls": ["apify.com", "https://www.bbc.com/sitemaps/https-index-com-news.xml"], "includeUrlPatterns": ["/news/"], "lastmodAfter": "2026-10-01" }

Output

One item per URL:

{
  "url": "https://wordpress.org/news/2026/09/some-post/", "lastmod": "2026-09-01T12:37:54Z",
  "changefreq": "weekly", "priority": 0.8, "alternates": [{"hreflang": "de", "url": "https://..."}],
  "imageCount": 2, "newsTitle": null, "newsPublishedAt": null,
  "statusCode": 200, "finalUrl": "https://...", "httpLastModified": "Tue, 01 Sep 2026 12:37:54 GMT",
  "site": "https://wordpress.org", "sitemap": "https://wordpress.org/news/wp-sitemap-posts-post-1.xml"
}

statusCode/finalUrl/httpLastModified appear only with Check HTTP status. A per-site summary (URLs returned, sitemaps read, errors, and whether Max URLs in total cut a site short) is saved to the run's key-value store as SUMMARY.

Pricing

Pay per URL, with lower prices on higher Apify plans: $0.40 per 1,000 URLs on the free plan, down to $0.08 per 1,000 on Gold and above. Filters and caps apply before you are charged. A run where no sitemap can be found fails clearly instead of silently returning an empty dataset. A filtered run with no new pages succeeds with 0 results.

Limits and notes

• Reads only what a site publishes in its sitemaps. Pages not listed there won't appear (it doesn't crawl links).
• Up to 300,000 URLs and 1,000 sitemap files per site, and 60 MB per sitemap file (the sitemap spec allows 50 MB). Memory use stays flat whatever the site size.
• Section URLs: if a site lists nothing under the path in 100 sitemap files in a row, the search for that section stops there (noted in SUMMARY).
• With Changed on or after, URLs without a lastmod date are skipped.
• Public data only. Status checks send one light request per URL; use caps for very large sites.