Sitemap URL Extractor: All Pages from sitemap.xml

Give it a website or a sitemap address. Get back every page URL the site lists in its sitemaps, one clean row per URL, with the last-modified date, change frequency and priority. Useful for SEO audits, content inventories, migration checks, change monitoring and as the URL list for a crawler or a RAG pipeline.

It finds the sitemaps for you: first the Sitemap: lines in the site's robots.txt, then the usual locations (/sitemap.xml, /sitemap_index.xml, /sitemap-index.xml, /wp-sitemap.xml). Sitemap index files are followed down to every child sitemap, and gzip (.xml.gz) and plain-text sitemaps are read too.

It only reads files that websites publish for machines, over plain HTTP, and it obeys robots.txt for every file. No browser, no proxy, so it is fast (50,000 URLs in about 4 seconds in our tests) and cheap: $0.30 per 1,000 URLs, and duplicates are free.

What you get

One row per unique page URL:

• url: the page address (<loc>), made absolute
• lastmod, changefreq, priority: exactly as the sitemap states them (null when the sitemap leaves them out)
• section: the first folder of the path, such as /blog or /products, so you can group and count pages
• images: image URLs from image sitemaps (up to 50 per page)
• newsTitle, newsPublishedAt: from Google News sitemaps
• sitemapUrl: the sitemap file the URL was found in
• inputUrl: the website or sitemap you sent

Plus a SUMMARY record (in the run's key-value store, linked on the Output tab) with, for each website: the sitemaps read, URLs found, returned, duplicates and filtered out, the URL count per section, and every sitemap that could not be read with the reason (missing, blocked by robots.txt, not a sitemap, ...).

Input

Field	Required	Example	Notes
urls	yes	["https://www.sitemaps.org"]	Websites or sitemap addresses, up to 1,000 per run. example.com without https:// works too
maxUrls	no	10000	Most URLs per website (or per sitemap you send). Default 10,000, up to 500,000
urlMustContain	no	["/blog/"]	Keep only URLs containing one of these texts (not case sensitive)
urlMustNotContain	no	["/tag/", "?page="]	Drop URLs containing any of these texts
lastmodFrom	no	"2026-10-01" or "7 days"	Keep only URLs modified on or after this date. URLs with no lastmod are left out when set

A website address (https://example.com) starts discovery from its robots.txt. An address that ends in .xml, .xml.gz or .txt, or has sitemap in its path, is read directly.

{
    "urls": ["https://www.sitemaps.org", "https://developer.mozilla.org/sitemaps/ko/sitemap.xml.gz"],
    "maxUrls": 10000,
    "urlMustNotContain": ["/tag/"],
    "lastmodFrom": "30 days"
}

Tip for schedules: run it daily with lastmodFrom set to "1 day" to get only the pages a site added or changed since yesterday.

Output example

A real row (from a run on nodejs.org):

{
    "url": "https://nodejs.org/en/blog/release/v010.28",
    "lastmod": "2026-10-30T16:17:53.645Z",
    "changefreq": "always",
    "priority": null,
    "section": "/en",
    "images": [],
    "newsTitle": null,
    "newsPublishedAt": null,
    "sitemapUrl": "https://nodejs.org/sitemap.xml",
    "inputUrl": "https://nodejs.org"
}

And part of the SUMMARY record for docs.apify.com from the same run (lists shortened):

{
    "input": "https://docs.apify.com",
    "sitemapsRead": ["https://docs.apify.com/sitemap.xml", "https://docs.apify.com/sitemap_base.xml", "..."],
    "urlsFound": 4153,
    "urlsReturned": 4145,
    "duplicates": 8,
    "filteredOut": 0,
    "sections": { "/": 16, "/academy": 201, "/api": 1920, "/legal": 19 },
    "stoppedAtLimit": false,
    "problems": []
}

Pricing

Pay per event:

Event	Price	When
URL extracted	$0.0003	Every unique URL in your dataset
Actor start	$0.00005	Once per run

Example: a site with 12,000 URLs, run with maxUrls 20,000, costs 12,000 x $0.0003 + $0.00005 = $3.60005.

Free: a URL found in several sitemaps (charged once), URLs your filters drop, and sitemaps that cannot be read. The actor checks your run's maximum charge before it starts and stops at the number of URLs that charge covers, so a run never goes over the limit you set.

Limits

• Reads sitemaps, not pages. A site with no sitemap (or one that blocks bots from it) returns no URLs, and the SUMMARY record says why. To discover pages by following links, use a crawler.
• Up to 2,000 sitemap files per run and 500,000 URLs per run. One sitemap file is read up to 60 MB after unzipping (the protocol's limit is 50 MB).
• RSS and Atom feeds listed as sitemaps are not read.
• Sites that refuse bots (HTTP 403 or a challenge page) are reported, not bypassed. There is no proxy rotation.
• lastmod, changefreq and priority are the site's own claims. Some sites set every lastmod to the day the sitemap was built, which makes the date filter less useful on those sites.
• Up to 4 sitemap files are read at a time.

FAQ

Does it respect robots.txt? Yes, always, and it cannot be turned off. It identifies itself as SitemapUrlExtractorBot with a link to this page, reads each site's robots.txt once per run, and skips any sitemap file the site disallows, following RFC 9309 (the internet standard for robots.txt).

Why did I get fewer URLs than maxUrls? The site lists fewer pages, your filters dropped some, the same URL appeared in several sitemaps (it is returned once), or your run's maximum charge covered fewer URLs. The SUMMARY record has the counts for each site.

Can I send several websites in one run? Yes, up to 1,000. maxUrls applies to each website separately, so one huge site cannot use up the run.

Can I call it from an AI agent, MCP client or the API? Yes. The input is a list of addresses plus optional filters, every output field has a fixed type, and it runs with limited permissions, so it needs no access to your other Apify data.

Is there personal data in the results? The rows are page addresses and dates that the website itself publishes for search engines. The actor adds nothing of its own. You are responsible for how you use the URLs of the sites you choose.

Something broke or a sitemap was read badly. What do I do? Open an issue on the Issues tab with your run link and the sitemap address. Issues are answered within a few days.