Sitemap Extractor: sitemap.xml Parser & URL to CSV
Pricing
from $0.10 / 1,000 url extracteds
Sitemap Extractor: sitemap.xml Parser & URL to CSV
Get every URL from a website's sitemaps. Auto-discovers sitemaps via robots.txt and common paths, follows sitemap indexes, reads .xml, .xml.gz, .txt, RSS and Atom, and de-duplicates. Filter by pattern or lastmod. Pay only per unique URL delivered.
Pricing
from $0.10 / 1,000 url extracteds
Rating
0.0
(0)
Developer
LiveFree Analytics
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
18 days ago
Last modified
Categories
Features
Share
Sitemap URL Extractor: Get Every Page URL of Any Website
Type a website. Get a clean list of every page it tells Google about, with the date each page last changed. No crawling and no browser, so it's fast and cheap: over 1 million URLs from 312 sitemaps in a single test run.
Who it's for
- SEO: list every indexable page, find pages missing from your crawl, check a migration
- Competitor and content watchers: "show me every page they published this month" on a weekly schedule
- Scraper builders: get all product or article URLs, then feed them to any other actor
- AI / RAG: get the full URL list of a docs site before you ingest it
What you get (real output row)
{"url": "https://nodejs.org/en/blog","lastmod": "2026-10-21T12:15:15.044Z","changefreq": "always","priority": null,"host": "nodejs.org","sitemapUrl": "https://nodejs.org/sitemap.xml","sitemapFormat": "xml","input": "https://nodejs.org","extractedAt": "2026-10-22T00:58:23.391Z"}
A SUMMARY record lists, for each website, where its sitemaps were found, which failed and why, and duplicate and filter counts. Export to Excel, CSV or JSON.
Try it in 1 minute
-
Put one or more websites in Start URLs (
nodejs.orgis fine; a sitemap URL works too). -
Optional: add a path filter like
/blog/or a Modified since date. -
Click Start.
{ "startUrls": ["https://www.djangoproject.com", "https://nodejs.org"], "includeUrlPatterns": ["/blog/"] }
Popular setups
| Goal | Settings |
|---|---|
| All blog posts of a competitor | includeUrlPatterns: ["/blog/"] |
| New pages this month (run weekly) | modifiedSince: "2026-10-01" + a Schedule |
| All product URLs of a Shopify store | includeUrlPatterns: ["/products/"] |
| Docs URLs for a RAG pipeline | sameDomainOnly: true |
| Every language version of a page | includeAlternates: true |
| Quick sample of a huge site | maxUrls: 1000 |
How it finds sitemaps
It reads the Sitemap: lines in robots.txt, then tries /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, /sitemap.xml.gz and /sitemap.txt. It follows nested indexes (WordPress, Yoast, Shopify, Next.js) and removes duplicates across all sitemaps. It's polite: a per-host delay, robots.txt Crawl-delay, retries on 429/5xx, and private or internal addresses are refused.
Price
$0.40 per 1,000 unique URLs on Apify's Free plan ($0.30 Starter, $0.20 Scale, $0.10 Business). Compute is included.
- No start fee from us. Only Apify's standard minimum start event (a fraction of a cent).
- Websites with no sitemap, duplicates and filtered-out URLs are free.
- If no sitemap is found at all, the run fails with a clear message and costs nothing.
- Use Max URLs or Max total charge to cap spend.
All settings
| Field | Default | What it does |
|---|---|---|
startUrls | – (required) | Websites or sitemap URLs |
discoverFromRobotsTxt / tryCommonPaths | true / true | How sitemaps are found |
respectRobotsTxt | true | Honor robots.txt rules |
sameDomainOnly | false | Drop URLs on other domains |
includeUrlPatterns / excludeUrlPatterns | – | Keep or drop URLs by text or regex |
modifiedSince | – | Only URLs whose lastmod is on or after this date |
maxUrls | 0 (no limit) | Stop after N URLs |
includeImages / includeAlternates | false | Image entries and hreflang versions |
maxSitemaps / maxDepth | 2000 / 6 | Limits for huge sites |
maxConcurrency / requestDelayMs / requestTimeoutSecs / maxRetries | 5 / 250 / 60 / 2 | Speed vs. politeness |
proxyConfiguration | off | Optional Apify Proxy |
Use it from code, no-code tools or AI agents
- API:
POST https://api.apify.com/v2/acts/blond_terrarium~sitemap-url-extractor-pro/run-sync-get-dataset-items?token=YOUR_TOKEN - Make, Zapier, n8n, Google Sheets, webhooks through Apify integrations
- AI agents (MCP):
https://mcp.apify.com?tools=blond_terrarium/sitemap-url-extractor-pro. Ask "List every blog URL on example.com".
FAQ
The site has no sitemap. Now what? You pay nothing. Use a crawler instead, such as our SEO Audit Crawler or Website to Markdown.
Is the lastmod date reliable? It's whatever the site publishes. Many sites keep it accurate, but some don't.
Does it visit every page? No, it only reads sitemap files. That's why it's fast and light on the site.
Support? Use the Issues tab. We read every one.
Related tools
Bulk Website Screenshot · Broken Link Checker · Website to Markdown for LLMs
Built and maintained by LiveFree Analytics.