Sitemap URL Extractor – All URLs, Auto-Discovery, $0.30/1K

What does Sitemap URL Extractor do?

Sitemap URL Extractor finds and extracts every URL listed in a website's XML sitemaps – fast, cheap and reliable. Just enter a domain like example.com: the Actor auto-discovers sitemaps from robots.txt and common locations (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, .xml.gz, .txt), follows nested sitemap indexes of any depth, and returns a clean list of URLs with lastmod, changefreq and priority.

No browser, no crawling – it reads sitemaps directly, so a typical run with tens of thousands of URLs finishes in seconds.

Why use it?

• 🔎 SEO audits – get the full list of indexable pages of your site or a competitor's.
• 🆕 Monitor new content – schedule it daily with Only URLs modified since to get just the new/updated pages (new products, articles, job posts).
• 🕷️ Feed other scrapers & RAG pipelines – use the URL list as start URLs for crawlers or LLM ingestion.
• 🛒 E-commerce intelligence – list all product URLs of a Shopify/WooCommerce/Magento store.
• 🔁 Migrations & redirects – export every URL before a site migration.

Features

• Auto-discovery via robots.txt + common sitemap paths
• Sitemap indexes, gzip (.xml.gz), plain-text sitemaps, news/image/video sitemaps
• Include / exclude regex filters
• Modified-since filter (skips whole child sitemaps that are older – very efficient)
• Optional image URLs from image sitemaps
• Automatic retry through Apify Proxy if a site blocks direct requests
• De-duplicated output, per-site limit
• A SUMMARY record in the key-value store with URL and sitemap counts per site

Input

Field	Description
startUrls	Domains (https://example.com, example.com) or direct sitemap URLs
maxUrlsPerSite	Stop after N URLs per site (0 = unlimited)
includePattern / excludePattern	Regex filters on the URL
modifiedSince	ISO date – return only URLs with lastmod on/after it
includeImages	Add image URLs from image sitemaps
proxyConfiguration	Optional proxy

Example:

{
  "startUrls": ["https://www.allbirds.com", "https://www.bbc.com/sitemaps/https-index-com-news.xml"],
  "includePattern": "/products/",
  "modifiedSince": "2026-01-01"
}

Output

{
  "url": "https://www.allbirds.com/products/mens-tree-runners",
  "lastmod": "2026-10-05T08:12:44-04:00",
  "changefreq": "daily",
  "priority": 0.8,
  "sitemap": "https://www.allbirds.com/sitemap_products_1.xml?from=1&to=9999",
  "domain": "www.allbirds.com"
}

Download as JSON, CSV, Excel, XML or HTML, or get it via the Apify API, webhooks, Make, Zapier or n8n.

How much does it cost?

Pay only for results: $0.30 per 1,000 URLs plus a tiny start fee per run. Extracting a 10,000-page site costs about $3; a daily "new pages only" monitor typically costs cents per day.

FAQ

The Actor found no sitemap. Some sites don't publish one or block bots. Try entering the sitemap URL directly or enable Apify Proxy (residential) in the input.

Is it legal? Sitemaps are public files that websites publish specifically so that machines can read them.

Need a feature? Open an issue on the Issues tab – I usually respond within 24 hours.