# XML Sitemap URL Extractor & Sitemap to CSV (`swillz/sitemap-url-extractor`) Actor

Get every URL from a website's XML sitemap as JSON, CSV or Excel. Give a domain or sitemap URL: it finds sitemaps via robots.txt, follows indexes and .gz files, and returns one row per page with lastmod, images and hreflang. Optional HTTP status check and new-page monitoring.

- **URL**: https://apify.com/swillz/sitemap-url-extractor.md
- **Developed by:** [William Jin](https://apify.com/swillz) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.10 / 1,000 url extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

**XML Sitemap URL Extractor** pulls **every URL from any website's XML sitemap** and exports it to **CSV, JSON or Excel**, with `lastmod`, `changefreq`, `priority`, images and hreflang.

It works from just a domain. Sitemaps are auto-discovered through `robots.txt` and common paths, and sitemap indexes and `.xml.gz` files are followed automatically.

It's the fastest way to **extract all URLs from a sitemap**, convert a **sitemap to CSV**, check sitemap URLs for 404s, or get alerted when a site publishes new pages. Try it free: the Apify free plan covers thousands of URLs every month.

**When to use:** you need every page URL of a website (SEO audits, crawl lists, content inventories, new-page monitoring).
**Input:** a domain or sitemap URL, e.g. `{"startUrls":[{"url":"https://example.com"}]}`.
**Output:** one row per page URL with lastmod, changefreq, priority, images and hreflang (optional HTTP status). From $0.10 per 1,000 URLs.

### What can XML Sitemap URL Extractor do?

- 🗺️ **Extract all URLs from a sitemap.** Give it a domain or a sitemap link and get a clean, deduplicated URL list in seconds. A 10,000-URL sitemap usually takes less than 30 seconds.
- 📄 **Convert sitemap XML to CSV, Excel or JSON.** Download in one click or fetch the results through the API.
- 🔎 **Check sitemap URLs.** The optional HTTP status check finds 404s, redirects and 5xx errors in your sitemap.
- 🆕 **Monitor new pages.** Schedule it with *Only new URLs* to catch competitor launches, new products, blog posts and job pages.
- 🎯 **Filter.** Use include and exclude patterns, a `lastmod` date range (`2026-01-01` or `7 days`) or a sub-path scope.
- ⚡ **No proxy and no browser.** Runs are fast and cheap, and it works on Shopify, WordPress, WooCommerce, Magento and custom sites.

### What data can you extract from a sitemap?

| Field | Description |
|---|---|
| `url` | Page URL listed in the sitemap |
| `lastmod` | Last modification date (ISO 8601) |
| `changefreq`, `priority` | Sitemap hints, when present |
| `imageCount` / images | Image sitemap entries (optional) |
| `alternates` | hreflang language alternates (optional) |
| `isNew` | `true` if the URL appeared since the last monitoring run |
| `httpStatus`, `finalUrl` | Status code and redirect target (optional status check) |
| `sitemapUrl`, `site` | Which sitemap and site the URL came from |

Supported formats: XML urlset, sitemap indexes nested to any depth, gzip `.xml.gz`, plain-text sitemaps, RSS/Atom feeds and Google News sitemaps.

### How to extract URLs from a sitemap (step by step)

1. Click **Try for free** and sign in to Apify. The free plan works.
2. Paste one or more domains or sitemap URLs into **Websites or sitemap URLs** (for example, `example.com`).
3. Optionally add URL patterns, a `lastmod` date range or **Check HTTP status**.
4. Click **Start**.
5. Open the **Output** tab and export the results as **CSV**, Excel, JSON or XML. That's your sitemap converted to CSV.

### How much does it cost to extract URLs from a sitemap?

You pay only for results, through **pay-per-event** pricing:

- **$0.10-0.40 per 1,000 URLs extracted**, depending on your Apify plan: Free $0.40, Starter $0.28, Scale $0.18, Business and up $0.10. You're charged only for URLs written to your dataset, after filters and deduplication.
- **$1.00 per 1,000 HTTP status checks**, and only if you enable the status check.
- A negligible start fee per run.

| Job | Free plan | Starter | Business and up |
|---|---|---|---|
| Blog with 500 URLs | $0.20 | $0.14 | $0.05 |
| Store with 10,000 product URLs | $4.00 | $2.80 | $1.00 |
| 10,000 URLs + status check | $14.00 | $12.80 | $11.00 |
| Daily new-page monitor (≈20 new URLs/day) | ≈ $0.25 / month | ≈ $0.17 / month | ≈ $0.06 / month |

Prices drop on bigger Apify plans (Starter = Bronze tier, Scale = Silver, Business and up = Gold).

Set a **maximum cost per run** in the run options and the actor stops cleanly when that limit is reached. Higher Apify plans may get volume discounts (see the Pricing tab).

### Input example

```json
{
  "startUrls": [{ "url": "https://www.example.com" }],
  "includeUrlPatterns": ["*/blog/*"],
  "excludeUrlPatterns": ["*/tag/*"],
  "lastmodFrom": "30 days",
  "onlyNewUrls": false,
  "checkUrlStatus": false,
  "maxUrlsPerSite": 50000
}
```

If you pass a sub-path such as `https://example.com/blog/`, only URLs inside that path are returned. To get the whole domain, set `restrictToStartPath` to `false`.

### Output example

```json
{
  "url": "https://wordpress.org/news/2026/10/wordpress-7-1-3-maintenance-and-security-release/",
  "lastmod": "2026-10-08T16:23:17.000Z",
  "changefreq": null,
  "priority": null,
  "imageCount": 0,
  "isNew": true,
  "httpStatus": 200,
  "finalUrl": null,
  "sitemapUrl": "https://wordpress.org/news/sitemap-2.xml",
  "site": "https://wordpress.org",
  "extractedAt": "2026-10-23T07:18:07.910Z"
}
```

A run summary is saved to the `OUTPUT` record. For each site it shows how the sitemap was discovered, which sitemaps failed, and how many URLs were filtered.

### Use cases

- **SEO audits:** list every indexable URL, then find broken or redirected URLs in the sitemap.
- **Content inventory and site migrations:** export the sitemap to CSV and confirm that every URL returns 200 after a migration.
- **Competitor monitoring:** get a daily list of new product, landing and blog pages.
- **Crawler and RAG seeding:** feed a complete URL list to Website Content Crawler or your own scraper.
- **Ecommerce research:** list every product URL of a Shopify or WooCommerce store.

### How to monitor a website for new pages

1. Enable **Only new URLs since the last run**. To run several independent monitors, give each one its own `stateKey`.
2. Create a **Schedule** in Apify (for example, daily). The first run saves a baseline, and each later run outputs only new URLs.
3. Add an integration to get notified: email, Slack, a webhook, Zapier, Make or n8n.

### Use with AI agents (MCP, ChatGPT, Claude, Cursor, n8n)

AI agents can find and run this Actor through the [Apify MCP server](https://mcp.apify.com). Add it to Claude Desktop, Cursor, VS Code or any MCP client:

```json
{"mcpServers":{"apify":{"url":"https://mcp.apify.com?tools=swillz/sitemap-url-extractor"}}}
```

Or run it locally: `npx -y @apify/actors-mcp-server --tools swillz/sitemap-url-extractor` with `APIFY_TOKEN` set. In n8n or Make, use the Apify node with Actor `swillz/sitemap-url-extractor`.

Example prompts:

- "List every blog URL on example.com updated in the last 30 days"
- "Find broken links in stripe.com's sitemap"
- "Give me all product URLs from this Shopify store"

Minimal input:

```json
{"startUrls": [{"url": "https://example.com"}], "maxUrls": 1000}
```

**Keep it fast:** `maxUrls` defaults to 1000, so a call finishes in seconds. Add `includeUrlPatterns` (e.g. `["/blog/"]`) instead of fetching everything; `checkUrlStatus` makes runs slower.

**Cap the cost:** pass `maxTotalChargeUsd` ("Maximum cost per run") when you call the Actor. The run stops cleanly at the cap, keeps the results so far, and says why in the status message and the OUTPUT record.

### Integrations and API

- **API and clients:** run it from Python or JavaScript with the Apify client, or call the REST API and download the dataset as CSV or JSON.
- **No-code:** use the official Apify apps for **Make**, **n8n** and **Zapier**. Choose *Run Actor*, then `swillz/sitemap-url-extractor`.
- **AI agents (MCP):** add it as a tool for Claude, Cursor or other MCP clients through `https://mcp.apify.com?tools=swillz/sitemap-url-extractor`.
- **Google Sheets, Slack and webhooks:** set these up from the Integrations tab of any run.

### XML Sitemap URL Extractor vs alternatives

| | This actor | Online sitemap extractor tools | Browser extensions |
|---|---|---|---|
| Auto-discovery (robots.txt, indexes, .gz) | ✅ | Partial | ❌ |
| Millions of URLs | ✅ | Usually capped | ❌ |
| Export CSV, Excel and JSON plus API | ✅ | CSV only | Copy and paste |
| HTTP status check | ✅ | ❌ | ❌ |
| Scheduled new-page monitoring | ✅ | ❌ | ❌ |
| Price | $0.10-0.40 / 1,000 URLs by plan | Free with limits | Free |

### FAQ

**The site has no sitemap. What happens?**
The run still succeeds. The site is reported with `discoveredVia: none` in the run summary, and you aren't charged for it. If you know the exact sitemap URL, pass it directly.

**Do I need a proxy?**
Usually not, because sitemaps are meant for bots. If a site blocks datacenter IPs, enable Apify Proxy in the **Proxy** section.

**Can it handle huge sites?**
Yes. It handles indexes with hundreds of files and millions of URLs. To keep runs small, use `maxUrlsPerSite`, `maxSitemaps` and the `lastmod` filter.

**How does monitoring know which pages are new?**
Seen URLs are stored in a named key-value store (`sitemap-url-extractor-state`), under a key derived from your start URLs or under your own `stateKey`.

**Is it legal to extract URLs from a sitemap?**
Sitemaps are public files that site owners publish so automated tools can discover their pages. This actor reads only sitemaps and `robots.txt` (plus status codes, if you enable the check). It never logs in, bypasses protections or collects personal data. You are responsible for how you use the data.

**Something doesn't work, or you need a custom solution?**
Open an issue on the **Issues** tab. Bug reports and feature requests are welcome.

### Related tools

More tools from the same developer, built to work together:

- [Career Site Jobs Scraper](https://apify.com/swillz/career-site-jobs-scraper): jobs from Greenhouse, Lever, Ashby, Workable and Recruitee boards, with salary fields, as a jobs API or CSV.
- [Workday Jobs Scraper](https://apify.com/swillz/workday-jobs-scraper): every job from any myworkdayjobs.com career site, past the 2,000-result cap.
- [Company Website Enrichment](https://apify.com/swillz/company-website-enrichment): turn a list of domains into company name, emails, phones, social links and tech stack.
- [Google Sheets Import & Export](https://apify.com/swillz/google-sheets-import-export): send any Apify dataset to a Google Sheet (append, replace or dedupe) or read a sheet as JSON.

**Popular workflows**

- **Site audits for prospects:** check a prospect's pages and status codes with this Actor, then get their contacts with [Company Website Enrichment](https://apify.com/swillz/company-website-enrichment).

Source code: https://github.com/williamjin02/apify-actors/tree/main/actors/sitemap-url-extractor

# Actor input Schema

## `startUrls` (type: `array`):

Required. Array of objects, e.g. \[{"url":"https://example.com"}] or \[{"url":"https://example.com/sitemap.xml"}]. A home page triggers sitemap auto-discovery (robots.txt + common paths); a sitemap URL is read directly.

## `maxUrls` (type: `integer`):

Stop after this many URLs across all sites (default 1000; 0 = no limit). Large sites list 100k+ URLs: raise this deliberately. Also applies to API and AI-agent calls that omit it.

## `maxUrlsPerSite` (type: `integer`):

Stop after this many URLs from each start URL (0 = no limit). Useful when one big site would use up maxUrls.

## `includeUrlPatterns` (type: `array`):

Only output URLs matching at least one pattern. Use a glob (e.g. */blog/*) or a regular expression (e.g. /products/\d+). Case-insensitive.

## `excludeUrlPatterns` (type: `array`):

Drop URLs matching any of these globs or regular expressions (e.g. */tag/*).

## `lastmodFrom` (type: `string`):

Only URLs whose lastmod date is on/after this date. Absolute (2026-01-31) or relative (7 days, 24 hours, 2 weeks). URLs without lastmod are excluded when a date filter is set.

## `lastmodTo` (type: `string`):

Only URLs whose lastmod date is on/before this date. Absolute or relative.

## `onlyNewUrls` (type: `boolean`):

Remembers URLs between runs (in a named key-value store) and outputs only URLs not seen before. The first run outputs everything and saves the baseline. Ideal with a schedule to get alerts on new pages, products or articles.

## `stateKey` (type: `string`):

Optional name for the remembered URL set. Defaults to a hash of the start URLs. Use different keys for independent monitors.

## `checkUrlStatus` (type: `boolean`):

Sends a HEAD request (GET fallback) to each output URL and adds httpStatus and finalUrl (redirect target). Finds broken (404) and redirected sitemap entries. Slower; charged as an extra event per URL.

## `statusCheckConcurrency` (type: `integer`):

How many HTTP status checks run at the same time (only with checkUrlStatus). Lower it if the site rate-limits you.

## `includeImages` (type: `boolean`):

Add the image URLs listed for each page (image:loc tags in image sitemaps).

## `includeAlternates` (type: `boolean`):

Add hreflang language alternates (xhtml:link tags) for each page.

## `restrictToStartPath` (type: `boolean`):

When a start URL has a path (e.g. https://example.com/blog/), output only URLs under that path and try that path's own sitemap first.

## `discoverFromRobotsTxt` (type: `boolean`):

Read Sitemap: lines from /robots.txt when a website (not a sitemap) is given.

## `tryCommonSitemapPaths` (type: `boolean`):

If robots.txt lists no sitemap, try /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, /sitemap.txt and similar.

## `maxSitemapDepth` (type: `integer`):

How deep to follow nested sitemap indexes.

## `maxSitemaps` (type: `integer`):

Safety limit on the number of sitemap files fetched per run.

## `sitemapConcurrency` (type: `integer`):

Sitemap files fetched in parallel per site.

## `requestTimeoutSecs` (type: `integer`):

Timeout for each sitemap or status-check HTTP request, in seconds.

## `maxRetries` (type: `integer`):

Retries on network errors, 403, 429 and 5xx with exponential backoff (honours Retry-After).

## `proxyConfiguration` (type: `object`):

Usually not needed — sitemaps are meant for bots. Enable Apify Proxy only for sites that block datacenter traffic.

## `emitStatusRows` (type: `boolean`):

Add one free row (status "no_results" plus a reason) for each start URL that returned no URLs, so you or an AI agent can see why. Not charged. Turn off for a pure URL list.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://docs.apify.com"
    }
  ],
  "maxUrls": 1000,
  "maxUrlsPerSite": 0,
  "includeUrlPatterns": [],
  "excludeUrlPatterns": [],
  "onlyNewUrls": false,
  "checkUrlStatus": false,
  "statusCheckConcurrency": 10,
  "includeImages": false,
  "includeAlternates": false,
  "restrictToStartPath": true,
  "discoverFromRobotsTxt": true,
  "tryCommonSitemapPaths": true,
  "maxSitemapDepth": 5,
  "maxSitemaps": 1000,
  "sitemapConcurrency": 5,
  "requestTimeoutSecs": 30,
  "maxRetries": 3,
  "proxyConfiguration": {
    "useApifyProxy": false
  },
  "emitStatusRows": true
}
```

# Actor output Schema

## `results` (type: `string`):

One row per URL. Example: {"url":"https://example.com/blog/post-1","lastmod":"2026-09-30","changefreq":"weekly","priority":0.8,"sitemapUrl":"https://example.com/sitemap.xml","site":"https://example.com"}. Status rows: {"status":"no_results","site":"https://example.com","reason":"No sitemap found…"}.

## `summary` (type: `string`):

JSON: totals (sites, sitemaps, urlsOutput, duplicates, filtered), per-site details, maxUrls, stoppedReason (e.g. "Reached maxUrls (1000)…") and zeroResultsReason.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://docs.apify.com"
        }
    ],
    "maxUrls": 1000
};

// Run the Actor and wait for it to finish
const run = await client.actor("swillz/sitemap-url-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "startUrls": [{ "url": "https://docs.apify.com" }],
    "maxUrls": 1000,
}

# Run the Actor and wait for it to finish
run = client.actor("swillz/sitemap-url-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://docs.apify.com"
    }
  ],
  "maxUrls": 1000
}' |
apify call swillz/sitemap-url-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,swillz/sitemap-url-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/vCAwWvNofUlwpRTBX/builds/XixtYpmuS7bL594DM/openapi.json