Sitemap URL Extractor – All Page URLs from Any Website

Get the complete list of page URLs of any website from its XML sitemaps, in seconds and without a browser.

• Automatic discovery: finds sitemaps through robots.txt, then tries /sitemap.xml and /sitemap_index.xml. You can also enter a sitemap URL directly.
• Complete: follows sitemap indexes recursively and reads gzip-compressed (.xml.gz) and plain-text sitemaps.
• Metadata: returns lastmod, changefreq and priority for every URL.
• Filters: keep only URLs that contain a pattern (for example /blog/) or that were modified after a given date.
• Delta mode: schedule it and receive only new or changed pages. This is ideal for monitoring competitors' new products, articles or landing pages.
• Polite and compliant: at most 1 request per second per website. It respects robots.txt and identifies itself with a clear user agent.

Use cases

• SEO audits: compare the URLs in the sitemap with the indexed pages, and find stale lastmod dates.
• Competitor monitoring: get a daily list of new pages on competitor websites (delta mode).
• Crawl seeding: feed the URL list into other actors, such as content extractors or link checkers.
• Migrations: export every URL before a relaunch to build redirect maps.

Input examples

All URLs of two websites:

{ "startUrls": ["example.com", "https://www.another-site.com"] }

Only new blog posts since the last run (schedule daily):

{ "startUrls": ["competitor.com"], "includePatterns": ["/blog/"], "mode": "delta" }

A direct sitemap with a date filter:

{ "startUrls": ["https://www.gov.uk/sitemap.xml"], "modifiedSince": "2026-10-01" }

Output (one item)

{
  "id": "https://apify.com/about",
  "url": "https://apify.com/about",
  "lastmod": "2026-10-01T08:00:00Z",
  "changefreq": "weekly",
  "priority": 0.8,
  "sitemapUrl": "https://apify.com/sitemap/pages.xml",
  "website": "https://apify.com",
  "source": { "name": "XML sitemaps of the websites you provide" },
  "fetchedAt": "2026-10-08T14:00:00.000Z"
}

changeType (new or changed) is present only in delta mode. The run log reports how many sitemaps and URLs were found per website, and warns when a site has no sitemap.

Pricing

Pay per event: USD 1.00 per 1,000 URLs (event result), with lower prices on higher Apify subscription plans. There is no start fee, and websites without a sitemap cost nothing. Users on the Apify Free plan are limited to 50 results per run.

Speed: about 50,000 URLs per sitemap file. A site with 200,000 URLs typically finishes in a few minutes.

Limits

• The actor reads only sitemaps. It does not crawl pages, so pages that are missing from the sitemap are not found.
• Sitemaps that robots.txt disallows for all user agents are skipped and reported in the log.
• Websites that block automated requests (for example with a bot challenge) cannot be read. The actor does not try to bypass such protection.
• Each sitemap file can be up to 60 MB, and by default the actor reads up to 1,000 sitemap files per website (maxSitemaps).

Responsibility

The actor only reads files that website operators publish for automated reading (robots.txt and sitemaps), and it returns the URLs listed in them. You are responsible for using the results in line with the terms of the websites you choose. This actor is not affiliated with any of these websites.

Impressum / Legal notice

Anbieter dieses Actors (Angaben gemäß § 5 DDG) / Provider of this actor:

Friedrich Köhler
Spatzenweg 7
04720 Döbeln
Deutschland
E-Mail: actorcreater_010@web.de