Sitemap Extractor — Website → All URLs (sitemap.xml parser)

Give it a website. Get back every URL from its sitemap. For each site the Actor auto-discovers the sitemap, parses it, follows nested sitemap indexes, and returns one clean record per URL — sourceSite, url, lastmod, changefreq, priority. Perfect for SEO audits, seeding a crawler, and fast URL discovery across an entire domain.

Point it at a domain. Walk away with the site's full URL inventory — no scraping, no rendering, no guesswork.

---

✨ What it does

• Auto-discovers the sitemap — tries /sitemap.xml, reads Sitemap: directives from robots.txt, and uses direct sitemap URLs if you pass them.
• Handles sitemap indexes — recurses into nested <sitemapindex> files (and their children) to gather every child sitemap's URLs.
• One record per URL — each <url><loc> entry becomes one dataset item with its lastmod, changefreq, and priority when present.
• Robust by design — a site with no sitemap or a fetch error never kills the run: it's routed to a separate failures dataset and never charged.
• Safe by default — a maxUrls cap bounds run size; recursion depth is capped and loops are de-duplicated so a malformed index can't run away.
• Fast & lightweight — pure HTTP + a streaming XML parser (fast-xml-parser). No browser, no rendering.

---

🎯 Use cases

You want to…	This Actor gives you…
Run an SEO audit	Every indexed URL with lastmod / changefreq / priority for coverage & freshness checks.
Seed a crawler / scraper	A ready-made URL frontier for your own extraction pipeline — no link discovery needed.
Discover a site's structure	The full URL inventory of a domain, straight from its own sitemap.
Monitor content changes	lastmod timestamps to detect new or updated pages between runs.
Feed a RAG / AI pipeline	A clean list of canonical page URLs to fetch and embed downstream.

---

📥 Input

Field	Type	Default	Description
urls	string[]	— (required)	One or more website URLs (or direct sitemap.xml URLs). Each discovered URL is one result.
maxUrls	integer	5000	Safety cap on how many discovered URLs are emitted per run (across all sites).
includeLastmod	boolean	true	Include lastmod, changefreq, and priority fields in each result when present.

Example input

{
  "urls": [
    "https://www.apify.com",
    "https://example.com"
  ],
  "maxUrls": 5000,
  "includeLastmod": true
}

---

📤 Output

One dataset item per discovered URL:

{
  "sourceSite": "https://www.apify.com/",
  "url": "https://apify.com/store",
  "lastmod": "2024-01-15",
  "changefreq": "daily",
  "priority": "0.8"
}

Sites that fail (no sitemap / fetch error) go to a separate failures dataset and are never charged:

{
  "sourceSite": "https://no-sitemap-here.example/",
  "error": "No usable sitemap found (tried: https://no-sitemap-here.example/sitemap.xml)",
  "fetchedAt": "2026-07-19T02:17:19.831Z"
}

---

🔌 Call it from code (Apify API)

curl -X POST "https://api.apify.com/v2/acts/YOUR_USERNAME~sitemap-extractor/run-sync-get-dataset-items?token=YOUR_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"urls":["https://www.apify.com"]}'

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: 'YOUR_TOKEN' });
const { defaultDatasetId } = await client
    .actor('YOUR_USERNAME/sitemap-extractor')
    .call({ urls: ['https://www.apify.com'] });
const { items } = await client.dataset(defaultDatasetId).listItems();
console.log(`${items.length} URLs discovered`);

---

💸 Pricing (Pay-Per-Event)

Event	Price
Actor start	$0.05 per run
URL found	$0.0005 per discovered URL

• You only pay for URLs actually discovered — sites with no sitemap or a fetch error are never charged.
• Per-URL pricing is deliberately low because a single sitemap can yield thousands of URLs.
• 🎁 Free tier: free-plan users' platform usage is covered by Apify, so you can try it and run small jobs at no cost before scaling up.

---

⚖️ Acceptable use

This is a general-purpose sitemap-parsing tool: you supply the URLs and are responsible for having the right to crawl and use the content you submit. It only reads the public robots.txt and sitemap files each site voluntarily publishes. It does not render pages, does not target any single platform's private API, and does not harvest personal data. Please use responsibly and comply with each site's terms of service and applicable law.

---

🧱 Under the hood

Node.js · got-scraping · fast-xml-parser · Apify SDK. Stateless — nothing is stored between runs.