Sitemap URL Extractor - Expand Sitemaps to URLs avatar

Sitemap URL Extractor - Expand Sitemaps to URLs

Pricing

from $0.17 / 1,000 url extracteds

Go to Apify Store
Sitemap URL Extractor - Expand Sitemaps to URLs

Sitemap URL Extractor - Expand Sitemaps to URLs

Expand one or more sitemaps into a flat list of URLs. Follows nested sitemap-index trees, decompresses .gz sitemaps, and returns each URL with its lastmod, changefreq and priority. No browser.

Pricing

from $0.17 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Scrape Sage

Scrape Sage

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

5 days ago

Last modified

Share

Turn one or more sitemaps into a flat, deduplicated list of URLs - the fast way to get every page of a site before crawling it. Point it at a sitemap.xml and it follows the whole index tree (sitemaps of sitemaps), decompresses .xml.gz files, and returns each URL with its lastmod, changefreq and priority where the sitemap provides them.

Pure HTTP + XML parsing, no browser - fast and cents per URL. The output feeds straight into another scraper (paste the URLs into any actor's Start-URLs, or link the dataset).

What you get per URL

loc (the URL) · lastmod · changefreq · priority · sourceSitemap (which sitemap it came from).

Input

{
"sitemapUrls": ["https://example.com/sitemap.xml"],
"includeMetadata": true,
"maxItems": 10000
}
  • Sitemap URLs - one per line. An index sitemap is followed into its children automatically; .gz is decompressed. Leave empty for a small free sample.
  • Import from a file - paste a list, or link a public .txt/.csv, a Google Sheet/Drive link, or an Apify key-value-store record.
  • Max sitemaps / Max URLs - bound how much of a large index tree to expand.
  • Output fields - tick only the columns you need (e.g. just loc).

Notes

  • Handles standard XML sitemaps and sitemap indexes (the sitemaps.org protocol), including gzipped ones. A sitemap that cannot be fetched or parsed is reported and skipped, never crashing the run.
  • URLs are deduplicated across every sitemap in the tree.

Use with AI assistants (MCP)

Available through the Apify MCP server - an agent can expand a site's sitemap into a work-list of URLs and hand them to a crawler or content pipeline.

Agent-ready: autonomous payments (x402 & Skyfire)

This actor is agent-ready - AI agents can discover it, run it, and pay for it autonomously, with no Apify account and no human in the loop. It uses pay-per-event pricing and limited permissions, so it qualifies for Apify's agentic-payment standards:

  • x402 - an open, HTTP-native payment protocol. Agents pay per run in USDC on the Base network directly through the Apify MCP server - no account, no API key.
  • Skyfire - agent-to-service payments for fully autonomous AI-agent workflows.

Building an AI agent, MCP tool, or autonomous data pipeline? This scraper is ready to plug in and pay as it goes.