Sitemap URL Extractor - every URL from sitemap.xml, priced per file not per URL

Give it a sitemap URL, a sitemap index, or just a bare domain - get back every URL the site declares, with lastmod, changefreq, priority and the sitemap each one came from. Nested sitemap indexes are followed automatically. Duplicates are removed across every file.

The pricing shape is the whole point: $0.005 per sitemap file parsed, plus $0.015 per 1,000 URLs extracted (rounded up per run). Actors that bill per URL turn an ordinary 5,000-URL sitemap into $2.50 to $150. Here it is $0.08.

---

What you get

• One record per URL: loc, lastmod, changefreq, priority, sourceSitemap.
• Sitemap index recursion - index files are followed up to 5 levels deep, or turn it off with followNested: false.
• Domain mode - pass example.com and it discovers sitemaps from the Sitemap: lines in robots.txt, falling back to /sitemap.xml.
• Deduplicated across every nested file, so a URL listed in two sitemaps appears once.
• Fail-soft: a dead or malformed sitemap returns { "ok": false, "error": ... } and is never charged; the rest of the run keeps going.

Input

{
  "url": "https://www.example.com/sitemap.xml",
  "urls": ["example.org", "https://example.net/sitemap_index.xml"],
  "maxUrls": 5000,
  "followNested": true
}

Field	Type	Default	Notes
url	string	-	A sitemap URL, a sitemap-index URL, or a bare domain
urls	array of strings	-	Several of any of the above in one run
maxUrls	integer	5000	Hard cap on URLs returned, and therefore on the volume charge (max 50,000)
followNested	boolean	true	Follow child sitemaps inside a sitemap index

Hard caps per run: 50,000 URLs, 200 sitemap files, 5 levels of index nesting.

Output

One dataset item per URL (real run, 2026-08-07):

{
  "loc": "https://www.google.com/intl/en/about/",
  "lastmod": "2026-07-30",
  "changefreq": null,
  "priority": null,
  "sourceSitemap": "https://www.google.com/about/sitemap.xml"
}

A sitemap that cannot be fetched or parsed comes back as a diagnostic record, uncharged:

{ "sitemap": "https://example.com/dead.xml", "ok": false, "error": "HTTP 404" }

Filter on ok !== false if you only want URLs.

---

Pricing

$0.005 per sitemap file parsed (one event per index or urlset file that actually fetched and parsed) plus $0.015 per started 1,000 URLs extracted across the run. Fetch errors, broken XML and documents that are not sitemaps are recorded free.

Worked examples:

Job	Files parsed	URLs	Cost
One small sitemap	1	300	$0.02
One large sitemap	1	5,000	$0.08
An index plus 5 children	6	5,000	$0.105
A big site, capped	12	50,000	$0.81

Honest comparison

Measured from each actor's live pricing on 2026-08-15. Note that store actors price in tiers by your Apify plan - the "Free tier" column is the list price a new buyer sees, and the "Best paid tier" column is the cheapest that actor ever gets. This actor has a single flat price with no tiers.

Actor	Price shape (free tier)	5,000 URLs, free tier	5,000 URLs, their best tier
This actor	$0.005/file + $0.015 per 1,000 URLs	$0.08	$0.08
apify/sitemap-extractor	$0.0005 per URL	$2.50	$0.50
crawlerbros/sitemap-url-extractor	$0.005 start + $0.002 per URL	$10.01	$5.01
onescales/sitemap-url-extractor	$0.00005 start + $0.03 per URL	$150.00	$13.00

Where they win: on a tiny sitemap. Ten URLs costs $0.005 at apify/sitemap-extractor versus $0.02 here, because our first per-1,000 event fires at URL number one. Against that actor's free tier the crossover is about 40 URLs; against its cheapest paid tier, about 200. If your job is a handful of URLs and nothing else, that is the cheaper tool and you should use it.

Where this wins: every site above a couple of hundred pages, and the gap widens from there - over 30x cheaper at 5,000 URLs, and it never scales with URL count the way per-URL pricing does. The volume event exists so a 50,000-URL run does not cost us more to serve than it earns; a listing that quietly loses money on its own happy path does not stay up, and we would rather say that than surprise you later.

When NOT to use this

• You want the page content, not the URL list. This reads sitemaps only. Feed the loc values into Webpage Text Extractor for the content.
• The site has no sitemap. There is no crawler here, no link following. If robots.txt and /sitemap.xml come up empty, you get a diagnostic record, not a discovered URL list.
• Gzipped sitemaps. .xml.gz files are not decompressed yet - they come back as an unrecognized document, uncharged.
• Plain-text or RSS sitemap variants. Only XML <urlset> and <sitemapindex> are parsed.
• You need to know whether those URLs work. A sitemap is a claim, not a guarantee. Check the URLs with Broken Link Checker or Redirect Chain Checker.
• Sites that block bots at the CDN. Those fetches fail, and are free.

---

FAQ

How do I get a list of every URL on a website? If the site publishes a sitemap, pass the domain - example.com - and this finds the sitemaps via robots.txt, follows any index files, and returns one record per declared URL. That is the complete list the site itself claims, which is the closest thing to a definitive URL inventory without crawling.

What is a sitemap index, and do I have to handle it myself? No. A sitemap index is a sitemap of sitemaps - large sites split their URLs across many files because a single sitemap is capped at 50,000 URLs. Pass the index; child sitemaps are fetched recursively up to 5 levels and the results are merged and deduplicated.

Can I pass just a domain instead of the sitemap URL? Yes. Anything that does not look like a sitemap path is treated as a domain: robots.txt is read for Sitemap: directives, and /sitemap.xml is tried as a fallback.

What does lastmod actually tell me? The date the site claims the page last changed. It is self-reported and often wrong or templated, but when it is maintained it is the cheapest possible signal for "what changed since my last crawl" - sort by it and re-fetch only the top.

Why is this priced per sitemap file instead of per URL? Because a sitemap is one HTTP request whether it holds 10 URLs or 50,000, so per-URL pricing charges you for work that did not happen. The small per-1,000 volume event covers the real cost of writing that many result rows. See the comparison table above for what the difference adds up to.

What happens if a sitemap is broken or returns a 404? You get a record with ok: false and the reason, the run continues with the other inputs, and that file is not charged.

Does it respect robots.txt? It reads robots.txt to discover sitemaps. It does not crawl the site, so there is nothing to disallow - it fetches only the sitemap files themselves.

How many URLs can one run return? 50,000, from at most 200 sitemap files, at up to 5 levels of index nesting. Use maxUrls to set a lower cap when you want to bound the spend.

---

For agents and automation

• Capability: enumerate every URL a site declares in its sitemaps
• Required input: url (sitemap URL or domain) or urls (array)
• Returns: one JSON record per URL; records with ok: false are diagnostics, not URLs
• Bounded: maxUrls, a 200-file cap and a 5-level depth cap keep every run finite
• Side effects: none - GET requests only

curl -X POST "https://api.apify.com/v2/acts/EliAI~sitemap-url-extractor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "content-type: application/json" \
  -d '{"url":"example.com","maxUrls":5000}'

Agents: connect Apify MCP and call the EliAI/sitemap-url-extractor tool.

Related actors

• Sitemap Analyzer - validate a sitemap's health, not just its contents
• Robots.txt Analyzer - see what you are actually blocking
• Webpage Links Extractor - every link on a single page
• Broken Link Checker - find the dead ones
• Webpage Text Extractor - turn those URLs into clean text

Built by Broke to Built.

For AI agents

This Actor is built to be called by software, not just by people.

• Mount it directly as an MCP tool — no Store search, no ranking, just this one tool: https://mcp.apify.com/?actors=eliai/sitemap-url-extractor
• Or call it over HTTP and get the results in the same request: POST https://api.apify.com/v2/acts/eliai~sitemap-url-extractor/run-sync-get-dataset-items
• Pay with x402, without an Apify account. This Actor is whitelisted for agentic payments, so an agent holding USDC on Base can buy a prepaid token and spend it here. The minimum purchase is $1, the token balance is an absolute spending cap, and it expires 14 days after purchase.
• Costs are predictable before you call. Pricing is pay-per-event (see Pricing above), so an agent can budget a run in advance instead of discovering the bill afterwards.
• Send only the field you mean. If you pass the bulk field, it is used on its own; the single-value field is a fallback, never merged into your request. You are charged for the items you sent and nothing else.