Sitemap URL Extractor: Get All Website URLs from sitemap.xml

Sitemap URL Extractor pulls every URL from a website's XML sitemap into a clean JSON, CSV or Excel list. Give it a domain and it auto-discovers sitemaps from robots.txt (and common paths like /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml), follows sitemap indexes, reads gzipped .xml.gz and plain-text sitemaps, and returns each page's lastmod, changefreq, priority, hreflang alternates, image count and Google News title/date. Filter by regex or last-modified date. No login, no proxies, no browser. Pay per result from $0.08 per 1,000 URLs, no start fee.

Why use this sitemap extractor

• ✅ Just type a domain: sitemaps are found for you via robots.txt Sitemap: lines, with fallbacks.
• ✅ Handles real-world sitemaps: nested sitemap indexes, .gz files, text sitemaps, image/news/hreflang extensions.
• ✅ Filters that save money: include/exclude regex (/blog/, /products/) and modified since date, applied before anything is saved.
• ✅ Reliable and fast: plain HTTP, no headless browser; thousands of URLs in seconds, with safety caps so runs don't hang.
• ✅ Polite by design: obeys robots.txt and Crawl-delay, identifies itself with a clear user agent.
• ✅ Pay only for results: $0.40 per 1,000 URLs on the Free plan, down to $0.08 on Gold+. No start fee.

Use cases

• SEO audits: list every indexed page, find stale pages by lastmod, check hreflang coverage.
• Content inventory and migrations: export all URLs before a redesign or CMS move.
• Competitor monitoring: schedule it daily with modified since to see new or updated pages.
• Crawler seeding: feed the URL list into Website Content Crawler, RAG pipelines or your own scraper.
• Ecommerce: list all product or category URLs with an include pattern.

How to extract URLs from a sitemap

1. Add one or more websites (https://example.com) or direct sitemap URLs.
2. Optional: add include/exclude regex patterns or a modified since date.
3. Click Start and export the results as JSON, CSV or Excel, or use the API.

An empty input {} runs a demo on https://www.usa.gov (public U.S. government site).

Input example

{
  "startUrls": [{"url": "https://www.irs.gov"}],
  "includePatterns": ["/newsroom/"],
  "modifiedSince": "2026-01-01",
  "maxUrls": 3000
}

Field	What it does
startUrls	Domains/homepages (auto-discovery) or direct sitemap URLs (.xml, .xml.gz, .txt, index)
maxUrls	Total URL cap for the run (0 = no limit)
includePatterns / excludePatterns	Regex filters on the URL
modifiedSince	Keep only URLs with lastmod on/after this date
includeAlternates	Add hreflang alternate URLs
maxSitemaps	Safety cap on sitemap files per site (default 200)

Sample results (real rows from our test run on irs.gov, Oct 24, 2026)

URL	lastmod	changefreq	priority	hreflang alternates
https://www.irs.gov/newsroom	2026-09-25T14:32:07-04:00	weekly	1.0	3
https://www.irs.gov/newsroom/irs-doing-a-paycheck-checkup-is-a-good-idea-for-workers-with-multiple-jobs	2026-03-26T18:28:53-04:00	monthly	0.8	6
https://www.irs.gov/newsroom/tax-professionals-should-review-data-safeguards	2026-08-24T23:44:14-04:00	monthly	0.8	2
https://www.irs.gov/zh-hans/newsroom/tax-basics-understanding-the-difference-between-standard-and-itemized-deductions	2026-08-24T23:48:57-04:00	monthly	0.8	3

That run read 12 sitemap files (found via robots.txt) and saved 3,000 matching URLs in about 5 seconds.

Output example

{
  "url": "https://www.irs.gov/es/newsroom",
  "domain": "www.irs.gov",
  "path": "/es/newsroom",
  "lastmod": "2026-09-25T14:32:07-04:00",
  "changefreq": "weekly",
  "priority": 1.0,
  "imageCount": 0,
  "alternates": [{"hreflang": "en", "url": "https://www.irs.gov/newsroom"}, {"hreflang": "es", "url": "https://www.irs.gov/es/newsroom"}],
  "newsTitle": null,
  "newsPublished": null,
  "sitemapUrl": "https://www.irs.gov/sitemap.xml?page=8",
  "site": "www.irs.gov"
}

A RUN_SUMMARY record lists, per site, where sitemaps were discovered, how many files were read, URLs saved, anything skipped by robots.txt, and errors.

Pricing

Pay per result, live since Oct 24, 2026: you pay only for URLs saved to the dataset. No start fee, no monthly rental.

Apify plan	Price per 1,000 URLs
Free	$0.40
Bronze	$0.25
Silver	$0.15
Gold, Platinum, Diamond	$0.08

Apify platform usage is included. Set Maximum cost per run in the run options to cap spend; the run stops charging at your limit.

FAQ

The site has no sitemap. What happens? The run finishes with 0 URLs and the RUN_SUMMARY says which locations were checked. This Actor reads sitemaps only; it does not crawl pages.

Does it respect robots.txt? Yes. Sitemap files disallowed for crawlers are skipped and Crawl-delay is obeyed.

Can I process many websites? Yes, add as many start URLs as you like; maxUrls is a total cap for the run.

Is this legal? Sitemaps are files that site owners publish specifically so automated crawlers can find their pages. You are responsible for how you use the URLs and any content you later fetch from them; check each site's terms.

Found a bug or need a field? Open an issue on the Issues tab.

More free tools from the same developer

• Career Page Jobs Scraper: every open job from company career pages on Greenhouse, Lever, Ashby and Workable in one dataset.
• SAM.gov Contract Opportunities: federal bids and solicitations from SAM.gov's official public data, no API key.
• Remote Jobs Aggregator: remote jobs from We Work Remotely and Remote OK, deduplicated, with salary and a new-jobs monitor.
• PDF Text Extractor: text, per-page text, keyword hits and metadata from PDF URLs in bulk, no start fee.