Sitemap URL Extractor: All Pages, Lastmod & Hreflang

Extract every URL from any website sitemap: auto-discovers sitemaps via robots.txt, follows nested sitemap indexes, reads .xml.gz files and returns lastmod, changefreq, priority, image counts and hreflang alternates. Filter by regex or modified date. Only $0.05 per 1,000 URLs.

Data source: each website's own public XML sitemaps, discovered through robots.txt Sitemap lines or the standard /sitemap.xml, /sitemap_index.xml and /wp-sitemap.xml locations. It needs no login, proxy or browser. Results are returned as clean, flat JSON that is ready for spreadsheets, databases, BI tools and AI agents.

What can you use it for?

• Build a full URL list before crawling or auditing a site
• Track new and updated pages on competitor sites (filter by lastmod)
• Find all product, blog or job URLs with include patterns
• Feed URL lists to other Actors (SEO audits, contact scrapers, RAG crawlers)
• Use it as a tool from AI agents through the Apify MCP server, or schedule it and send new results to Slack, Google Sheets or a webhook

Input

Field	Name	Description
startUrls	Websites or sitemap URLs	Homepages (sitemaps auto-discovered) or direct sitemap URLs
includePatterns	Include URL patterns (regex)	Only URLs matching any of these regexes, e.g. /products/
excludePatterns	Exclude URL patterns (regex)	Skip URLs matching any of these regexes
modifiedSince	Modified since (date)	Only URLs whose lastmod is on or after this date, e.g. 2026-10-01
maxUrlsPerSite	Max URLs per site	Stop after this many URLs per website
maxSitemapsPerSite	Max sitemap files per site	Safety cap on nested sitemaps fetched
concurrency	Concurrency	Sites in parallel

Example input

{
  "startUrls": [
    "https://www.allbirds.com",
    "https://apify.com"
  ],
  "maxUrlsPerSite": 10
}

Output

Each result is one dataset item with these fields: site, url, lastmod, changefreq, priority, imageCount, alternates, newsTitle, sitemapUrl.

Example output (real result, long fields shortened)

{
  "site": "https://www.allbirds.com",
  "url": "https://www.allbirds.com/products/mens-wool-runners",
  "lastmod": "2026-11-01T09:42:11-08:00",
  "changefreq": "daily",
  "priority": null,
  "imageCount": 1,
  "alternates": [],
  "newsTitle": "Men's Wool Runner - Natural Grey (Light Grey Sole)",
  "sitemapUrl": "https://www.allbirds.com/sitemap_products_1.xml?from=1878193471557&to=7369944137808"
}

Download the results as JSON, CSV, Excel or XML, or read them through the Apify API.

Pricing

This Actor uses pay-per-event pricing. You pay $5e-05 per URL returned, plus standard Apify platform usage, which is small because the Actor uses lightweight HTTP requests (no browser). You are never charged for results that are not returned. To cap spending, set a maximum cost per run in the run options.

Tips

• Start with a small max-results value to check your filters, then scale up.
• Schedule the Actor (for example daily) to monitor changes over time.
• The data comes straight from the source, so its freshness and completeness depend on that publisher.

Disclaimer

This Actor is not affiliated with or endorsed by any website, platform or data publisher named here. Use it in line with the target sites' terms and applicable privacy laws (e.g. GDPR/CAN-SPAM for contact data). Verify critical decisions against the official source.