Sitemap URL Extractor

Extract every URL from any XML sitemap — including sitemap indexes, gzip-encoded sitemaps, robots.txt discovery, and optional metadata (lastmod, changefreq, priority). Returns clean structured output: one URL per row, LLM/MCP-ready.

Why this actor?

• Reliable — built for production use with retry logic and transparent error handling
• Fast — HTTP-first, 256MB, no headless browser. A typical sitemap extracts in seconds
• Complete — supports sitemap indexes (recursive), gzip, robots.txt discovery, and domain convention
• Cheap — $0.001 per extracted URL, 60x less than the reliable competitor
• Clean output — flat, typed JSON with url, sourceSitemap, lastModified, changeFrequency, and priority

Input

Provide a sitemap URL or a domain name:

{
  "url": "https://apify.com/sitemap.xml"
}

Or let the actor discover the sitemap automatically:

{
  "url": "https://apify.com",
  "followIndexes": true,
  "maxDepth": 3,
  "maxUrls": 10000,
  "extractMetadata": true
}

Input fields

Field	Type	Default	Description
url	string	required	Sitemap URL or domain name
followIndexes	boolean	true	Recurse into sitemap index children
maxDepth	integer	3	Maximum nesting depth for indexes (1-10)
maxUrls	integer	10000	Maximum URLs to extract (0 = no limit, max 100000)
extractMetadata	boolean	true	Extract lastmod, changefreq, priority
checkRobotsTxt	boolean	true	Check robots.txt when given a domain
timeoutSecs	integer	60	HTTP timeout per request

Output

One row per extracted URL:

{
  "url": "https://apify.com/store",
  "sourceSitemap": "https://apify.com/sitemap/pages.xml",
  "lastModified": "2026-07-15",
  "changeFrequency": "daily",
  "priority": 0.8,
  "depth": 0,
  "error": null,
  "errorType": null,
  "scraped_at": "2026-07-31T12:00:00.000Z"
}

Error items (for invalid input, not-found, network errors) carry a non-null error and errorType:

{
  "url": "",
  "sourceSitemap": "https://example.invalid/sitemap.xml",
  "lastModified": null,
  "changeFrequency": null,
  "priority": null,
  "depth": 0,
  "error": "Sitemap not found (HTTP 404)",
  "errorType": "not_found",
  "scraped_at": "2026-07-31T12:00:00.000Z"
}

Pricing

Pay per event, and the platform usage is on us — the price you see is the price you pay, with no compute bill on top.

Event	Price	What one charge buys
Sitemap URL extracted	$0.001	Charged once per extracted URL. Error items (invalid sitemaps, network failures, empty results) are never charged.
Actor Start	$0.002	One run, whatever it returns.

Higher Apify subscription tiers pay less on every event (Silver −20%, Gold −35%).

Use cases

• SEO analysis — extract all URLs from a site's sitemap for bulk analysis
• Site migration — get a complete URL inventory before restructuring
• Sitemap validation — verify all URLs in a sitemap are accessible
• Agent workflows — feed structured URL lists into downstream actors

FAQ

Does it follow sitemap indexes? Yes — when followIndexes is true, the actor recursively follows <sitemapindex> child references up to maxDepth levels.

Can I extract URLs from just a domain name? Yes — the actor will check robots.txt for Sitemap: directives and fall back to /sitemap.xml.

What about gzip-encoded sitemaps? Fully supported — the actor transparently decompresses gzip content.

Are error items charged? No — invalid inputs, not-found sitemaps, and network errors are delivered free.

---

The Sitemap URL Extractor is an independent tool for the Apify platform. Sitemaps are public XML files; this actor parses them without authentication or personal data.