Sitemap Extractor: sitemap.xml Parser & URL to CSV avatar

Sitemap Extractor: sitemap.xml Parser & URL to CSV

Pricing

from $0.10 / 1,000 url extracteds

Go to Apify Store
Sitemap Extractor: sitemap.xml Parser & URL to CSV

Sitemap Extractor: sitemap.xml Parser & URL to CSV

Get every URL from a website's sitemaps. Auto-discovers sitemaps via robots.txt and common paths, follows sitemap indexes, reads .xml, .xml.gz, .txt, RSS and Atom, and de-duplicates. Filter by pattern or lastmod. Pay only per unique URL delivered.

Pricing

from $0.10 / 1,000 url extracteds

Rating

0.0

(0)

Developer

LiveFree Analytics

LiveFree Analytics

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

18 days ago

Last modified

Features

Limited permissions

Share

Sitemap URL Extractor: Get Every Page URL of Any Website

Type a website. Get a clean list of every page it tells Google about, with the date each page last changed. No crawling and no browser, so it's fast and cheap: over 1 million URLs from 312 sitemaps in a single test run.

Who it's for

  • SEO: list every indexable page, find pages missing from your crawl, check a migration
  • Competitor and content watchers: "show me every page they published this month" on a weekly schedule
  • Scraper builders: get all product or article URLs, then feed them to any other actor
  • AI / RAG: get the full URL list of a docs site before you ingest it

What you get (real output row)

{
"url": "https://nodejs.org/en/blog",
"lastmod": "2026-10-21T12:15:15.044Z",
"changefreq": "always",
"priority": null,
"host": "nodejs.org",
"sitemapUrl": "https://nodejs.org/sitemap.xml",
"sitemapFormat": "xml",
"input": "https://nodejs.org",
"extractedAt": "2026-10-22T00:58:23.391Z"
}

A SUMMARY record lists, for each website, where its sitemaps were found, which failed and why, and duplicate and filter counts. Export to Excel, CSV or JSON.

Try it in 1 minute

  1. Put one or more websites in Start URLs (nodejs.org is fine; a sitemap URL works too).

  2. Optional: add a path filter like /blog/ or a Modified since date.

  3. Click Start.

    { "startUrls": ["https://www.djangoproject.com", "https://nodejs.org"], "includeUrlPatterns": ["/blog/"] }

GoalSettings
All blog posts of a competitorincludeUrlPatterns: ["/blog/"]
New pages this month (run weekly)modifiedSince: "2026-10-01" + a Schedule
All product URLs of a Shopify storeincludeUrlPatterns: ["/products/"]
Docs URLs for a RAG pipelinesameDomainOnly: true
Every language version of a pageincludeAlternates: true
Quick sample of a huge sitemaxUrls: 1000

How it finds sitemaps

It reads the Sitemap: lines in robots.txt, then tries /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, /sitemap.xml.gz and /sitemap.txt. It follows nested indexes (WordPress, Yoast, Shopify, Next.js) and removes duplicates across all sitemaps. It's polite: a per-host delay, robots.txt Crawl-delay, retries on 429/5xx, and private or internal addresses are refused.

Price

$0.40 per 1,000 unique URLs on Apify's Free plan ($0.30 Starter, $0.20 Scale, $0.10 Business). Compute is included.

  • No start fee from us. Only Apify's standard minimum start event (a fraction of a cent).
  • Websites with no sitemap, duplicates and filtered-out URLs are free.
  • If no sitemap is found at all, the run fails with a clear message and costs nothing.
  • Use Max URLs or Max total charge to cap spend.

All settings

FieldDefaultWhat it does
startUrls– (required)Websites or sitemap URLs
discoverFromRobotsTxt / tryCommonPathstrue / trueHow sitemaps are found
respectRobotsTxttrueHonor robots.txt rules
sameDomainOnlyfalseDrop URLs on other domains
includeUrlPatterns / excludeUrlPatterns–Keep or drop URLs by text or regex
modifiedSince–Only URLs whose lastmod is on or after this date
maxUrls0 (no limit)Stop after N URLs
includeImages / includeAlternatesfalseImage entries and hreflang versions
maxSitemaps / maxDepth2000 / 6Limits for huge sites
maxConcurrency / requestDelayMs / requestTimeoutSecs / maxRetries5 / 250 / 60 / 2Speed vs. politeness
proxyConfigurationoffOptional Apify Proxy

Use it from code, no-code tools or AI agents

  • API: POST https://api.apify.com/v2/acts/blond_terrarium~sitemap-url-extractor-pro/run-sync-get-dataset-items?token=YOUR_TOKEN
  • Make, Zapier, n8n, Google Sheets, webhooks through Apify integrations
  • AI agents (MCP): https://mcp.apify.com?tools=blond_terrarium/sitemap-url-extractor-pro. Ask "List every blog URL on example.com".

FAQ

The site has no sitemap. Now what? You pay nothing. Use a crawler instead, such as our SEO Audit Crawler or Website to Markdown. Is the lastmod date reliable? It's whatever the site publishes. Many sites keep it accurate, but some don't. Does it visit every page? No, it only reads sitemap files. That's why it's fast and light on the site. Support? Use the Issues tab. We read every one.

Bulk Website Screenshot · Broken Link Checker · Website to Markdown for LLMs

Built and maintained by LiveFree Analytics.