# Sitemap URL Extractor (`leorochasantos/sitemap-extractor`) Actor

Extract every URL from any XML sitemap — support sitemap indexes, gzip, robots.txt discovery, and metadata extraction. Returns clean structured output: one URL per row.

- **URL**: https://apify.com/leorochasantos/sitemap-extractor.md
- **Developed by:** [Leonardo Santos](https://apify.com/leorochasantos) (community)
- **Categories:** Developer tools, SEO tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.65 / 1,000 sitemap url extracteds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Sitemap URL Extractor

Extract every URL from any XML sitemap — including sitemap indexes, gzip-encoded sitemaps, robots.txt discovery, and optional metadata (lastmod, changefreq, priority). Returns clean structured output: one URL per row, LLM/MCP-ready.

### Why this actor?

- **Reliable** — built for production use with retry logic and transparent error handling
- **Fast** — HTTP-first, 256MB, no headless browser. A typical sitemap extracts in seconds
- **Complete** — supports sitemap indexes (recursive), gzip, robots.txt discovery, and domain convention
- **Cheap** — $0.001 per extracted URL, 60x less than the reliable competitor
- **Clean output** — flat, typed JSON with `url`, `sourceSitemap`, `lastModified`, `changeFrequency`, and `priority`

### Input

Provide a sitemap URL or a domain name:

```json
{
  "url": "https://apify.com/sitemap.xml"
}
```

Or let the actor discover the sitemap automatically:

```json
{
  "url": "https://apify.com",
  "followIndexes": true,
  "maxDepth": 3,
  "maxUrls": 10000,
  "extractMetadata": true
}
```

#### Input fields

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `url` | string | *required* | Sitemap URL or domain name |
| `followIndexes` | boolean | `true` | Recurse into sitemap index children |
| `maxDepth` | integer | `3` | Maximum nesting depth for indexes (1-10) |
| `maxUrls` | integer | `10000` | Maximum URLs to extract (0 = no limit, max 100000) |
| `extractMetadata` | boolean | `true` | Extract lastmod, changefreq, priority |
| `checkRobotsTxt` | boolean | `true` | Check robots.txt when given a domain |
| `timeoutSecs` | integer | `60` | HTTP timeout per request |

### Output

One row per extracted URL:

```json
{
  "url": "https://apify.com/store",
  "sourceSitemap": "https://apify.com/sitemap/pages.xml",
  "lastModified": "2026-07-15",
  "changeFrequency": "daily",
  "priority": 0.8,
  "depth": 0,
  "error": null,
  "errorType": null,
  "scraped_at": "2026-07-31T12:00:00.000Z"
}
```

Error items (for invalid input, not-found, network errors) carry a non-null `error` and `errorType`:

```json
{
  "url": "",
  "sourceSitemap": "https://example.invalid/sitemap.xml",
  "lastModified": null,
  "changeFrequency": null,
  "priority": null,
  "depth": 0,
  "error": "Sitemap not found (HTTP 404)",
  "errorType": "not_found",
  "scraped_at": "2026-07-31T12:00:00.000Z"
}
```

### Pricing

Pay per event, and **the platform usage is on us** — the price you see is the price you pay, with no compute bill on top.

| Event | Price | What one charge buys |
|---|---|---|
| Sitemap URL extracted | **$0.001** | Charged once per extracted URL. Error items (invalid sitemaps, network failures, empty results) are never charged. |
| Actor Start | **$0.002** | One run, whatever it returns. |

Higher Apify subscription tiers pay less on every event (Silver −20%, Gold −35%).

### Use cases

- **SEO analysis** — extract all URLs from a site's sitemap for bulk analysis
- **Site migration** — get a complete URL inventory before restructuring
- **Sitemap validation** — verify all URLs in a sitemap are accessible
- **Agent workflows** — feed structured URL lists into downstream actors

### FAQ

**Does it follow sitemap indexes?** Yes — when `followIndexes` is true, the actor recursively follows `<sitemapindex>` child references up to `maxDepth` levels.

**Can I extract URLs from just a domain name?** Yes — the actor will check `robots.txt` for `Sitemap:` directives and fall back to `/sitemap.xml`.

**What about gzip-encoded sitemaps?** Fully supported — the actor transparently decompresses gzip content.

**Are error items charged?** No — invalid inputs, not-found sitemaps, and network errors are delivered free.

***

*The Sitemap URL Extractor is an independent tool for the Apify platform. Sitemaps are public XML files; this actor parses them without authentication or personal data.*

# Actor input Schema

## `url` (type: `string`):

Full sitemap URL (e.g. https://apify.com/sitemap.xml) or a domain name (e.g. apify.com) to auto-discover the sitemap via robots.txt and convention.

## `followIndexes` (type: `boolean`):

When the sitemap is a sitemap index, recursively follow child sitemaps to extract all URLs. Disable to only extract the index entries themselves.

## `maxDepth` (type: `integer`):

Maximum levels of sitemap index nesting to follow. 0 = direct sitemap only.

## `maxUrls` (type: `integer`):

Maximum number of URLs to extract. Set to 0 for no limit. Capped at 100,000.

## `extractMetadata` (type: `boolean`):

Extract lastmod, changefreq, and priority per URL when available.

## `checkRobotsTxt` (type: `boolean`):

When the input is a domain, try to discover the sitemap from robots.txt before falling back to /sitemap.xml.

## `timeoutSecs` (type: `integer`):

Maximum time per HTTP request.

## `proxyConfiguration` (type: `object`):

Proxy to use for outgoing requests. Sitemaps are datacenter-open and rarely need proxy rotation. Disabled by default for speed.

## Actor input object example

```json
{
  "url": "https://apify.com/sitemap.xml",
  "followIndexes": true,
  "maxDepth": 3,
  "maxUrls": 500,
  "extractMetadata": true,
  "checkRobotsTxt": true,
  "timeoutSecs": 60,
  "proxyConfiguration": {
    "useApifyProxy": false
  }
}
```

# Actor output Schema

## `results` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "url": "https://apify.com/sitemap.xml",
    "maxUrls": 500
};

// Run the Actor and wait for it to finish
const run = await client.actor("leorochasantos/sitemap-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "url": "https://apify.com/sitemap.xml",
    "maxUrls": 500,
}

# Run the Actor and wait for it to finish
run = client.actor("leorochasantos/sitemap-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "url": "https://apify.com/sitemap.xml",
  "maxUrls": 500
}' |
apify call leorochasantos/sitemap-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=leorochasantos/sitemap-extractor",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/AbjRcpzpPdRNJZoUm/builds/niLUHnmIBpAZETsCe/openapi.json