Sitemap URL Extractor - Expand Sitemaps to URLs
Pricing
from $0.17 / 1,000 url extracteds
Sitemap URL Extractor - Expand Sitemaps to URLs
Expand one or more sitemaps into a flat list of URLs. Follows nested sitemap-index trees, decompresses .gz sitemaps, and returns each URL with its lastmod, changefreq and priority. No browser.
Pricing
from $0.17 / 1,000 url extracteds
Rating
0.0
(0)
Developer
Scrape Sage
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
5 days ago
Last modified
Categories
Share
Turn one or more sitemaps into a flat, deduplicated list of URLs - the fast way to get every page of a
site before crawling it. Point it at a sitemap.xml and it follows the whole index tree (sitemaps
of sitemaps), decompresses .xml.gz files, and returns each URL with its lastmod, changefreq
and priority where the sitemap provides them.
Pure HTTP + XML parsing, no browser - fast and cents per URL. The output feeds straight into another scraper (paste the URLs into any actor's Start-URLs, or link the dataset).
What you get per URL
loc (the URL) · lastmod · changefreq · priority · sourceSitemap (which sitemap it came from).
Input
{"sitemapUrls": ["https://example.com/sitemap.xml"],"includeMetadata": true,"maxItems": 10000}
- Sitemap URLs - one per line. An index sitemap is followed into its children automatically;
.gzis decompressed. Leave empty for a small free sample. - Import from a file - paste a list, or link a public
.txt/.csv, a Google Sheet/Drive link, or an Apify key-value-store record. - Max sitemaps / Max URLs - bound how much of a large index tree to expand.
- Output fields - tick only the columns you need (e.g. just
loc).
Notes
- Handles standard XML sitemaps and sitemap indexes (the
sitemaps.orgprotocol), including gzipped ones. A sitemap that cannot be fetched or parsed is reported and skipped, never crashing the run. - URLs are deduplicated across every sitemap in the tree.
Use with AI assistants (MCP)
Available through the Apify MCP server - an agent can expand a site's sitemap into a work-list of URLs and hand them to a crawler or content pipeline.
Agent-ready: autonomous payments (x402 & Skyfire)
This actor is agent-ready - AI agents can discover it, run it, and pay for it autonomously, with no Apify account and no human in the loop. It uses pay-per-event pricing and limited permissions, so it qualifies for Apify's agentic-payment standards:
- x402 - an open, HTTP-native payment protocol. Agents pay per run in USDC on the Base network directly through the Apify MCP server - no account, no API key.
- Skyfire - agent-to-service payments for fully autonomous AI-agent workflows.
Building an AI agent, MCP tool, or autonomous data pipeline? This scraper is ready to plug in and pay as it goes.