# Sitemap URL Extractor - Get All URLs from sitemap.xml (`digital-tools/sitemap-url-extractor`) Actor

Extract every URL from a website's XML sitemaps. Auto-finds sitemaps via robots.txt, follows sitemap index files, reads .xml.gz, returns lastmod, changefreq, priority, hreflang and image counts. Fast, no browser, cheap per URL.

- **URL**: https://apify.com/digital-tools/sitemap-url-extractor.md
- **Developed by:** [凡 方](https://apify.com/digital-tools) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

### What does Sitemap URL Extractor do?

**Sitemap URL Extractor** gets **all URLs from a website's sitemap.xml** in seconds. Enter a homepage and it automatically **finds the sitemaps in robots.txt** (or at `/sitemap.xml`, `/sitemap_index.xml`, `/wp-sitemap.xml`), **follows sitemap index files**, unpacks **gzipped `.xml.gz` sitemaps**, and returns one clean row per page URL with `lastmod`, `changefreq`, `priority`, **hreflang alternates** and image counts.

Use it as a **sitemap scraper / sitemap parser API** to list every page of a site before crawling, to feed URLs into other Actors (like [Webpage to Markdown](https://apify.com/Digital-Tools/webpage-to-markdown)), or to monitor which pages were added or updated.

### Why use it?

- 🔍 **Automatic sitemap discovery** – robots.txt `Sitemap:` lines plus the common default locations, including WordPress, Shopify, Yoast and Next.js sitemaps.
- 🗂️ **Sitemap index support** – recursively follows nested sitemap indexes (news, image, video, product, blog sitemaps).
- 🗜️ **`.xml`, `.xml.gz` and `.txt` sitemaps** – handles compressed and plain-text sitemaps and tolerates broken XML.
- 🎯 **Include / exclude regex filters** – e.g. only `/blog/` or `/products/` URLs. Filtered URLs are free.
- ⚡ **Fast and cheap** – plain HTTP, no browser: 10,000 URLs in about 6 seconds.
- 📊 **Summary per site** – which sitemaps were found and parsed, and which failed, in the `SUMMARY` record.

### How to use it

1. Add website homepages (e.g. `https://apify.com`) or direct sitemap URLs to **URLs**.
2. Optionally set **Max URLs per site** and an **include / exclude pattern**.
3. Click **Start** and export the URL list as JSON, CSV, Excel or via the API.

#### Input example

```json
{
  "startUrls": [{ "url": "https://apify.com" }],
  "maxUrlsPerSite": 10000,
  "includePattern": "/blog/"
}
```

#### Output example

```json
{
  "url": "https://apify.com/blog/web-scraping-guide",
  "lastmod": "2026-10-01",
  "changefreq": "weekly",
  "priority": 0.8,
  "imageCount": 1,
  "alternates": [{ "hreflang": "de", "href": "https://example.com/de/page" }],
  "sitemapUrl": "https://apify.com/sitemap/blog.xml",
  "site": "https://apify.com"
}
```

### How much does it cost?

This Actor uses **pay-per-event (PPE)** pricing: you pay only for results, and the Apify platform compute it uses is already included.

| Event | Price | Per 1,000 |
|---|---|---|
| URL extracted | **$0.0002** per URL | **$0.20** per 1,000 URLs |
| Actor start | $0.0005 per run | – |

Duplicate URLs and URLs removed by your include/exclude filters are **not charged**. Use **Max URLs per site** to cap the cost. For example, 1,000 URLs cost about **$0.20**. The Actor start fee is charged once per run (one event per GB of memory; the default memory setting counts as one). You can set a **maximum cost per run** in the run options and the Actor stops cleanly when it is reached. Apify's free plan includes $5 of monthly credit, enough to try it at no cost.

### Use cases

- **SEO audits** – count indexable pages, find stale `lastmod` dates, check hreflang coverage.
- **Crawl seeding** – get a full URL list before scraping, instead of crawling link by link.
- **Content monitoring** – schedule daily runs and compare `lastmod` to detect new or updated pages on competitor sites.
- **AI / RAG ingestion** – list every docs page, then convert them with [Webpage to Markdown](https://apify.com/Digital-Tools/webpage-to-markdown).

### FAQ

**What if a site has no sitemap?** The run summary lists it under "No sitemap found". You can then crawl it with a link crawler instead.

**Can I pass a sitemap URL directly?** Yes – any URL ending in `.xml`, `.xml.gz` or `.txt` is parsed directly.

**Can I use it from the API, Make, Zapier or an AI agent?** Yes. Call it through the Apify API, integrations or the Apify MCP server.

### Other tools by Digital-Tools

- [Webpage to Markdown for LLMs & RAG](https://apify.com/Digital-Tools/webpage-to-markdown) – Convert any web page URL into clean, LLM-ready Markdown.
- [URL Metadata & Open Graph Link Preview Extractor](https://apify.com/Digital-Tools/url-metadata-extractor) – Get title, description, preview image, favicon, canonical URL, Open Graph, Twitter Card, RSS feeds and JSON-LD schema from any list of URLs.
- [Broken Link Checker - Find 404s & Dead Links](https://apify.com/Digital-Tools/broken-link-checker) – Crawl a website and check every internal and external link for 404s, 5xx errors, timeouts and redirects.
- [Chinese Simplified/Traditional Converter & Pinyin Toolkit](https://apify.com/Digital-Tools/chinese-text-toolkit) – Bulk convert Simplified ↔ Traditional Chinese (Taiwan, Hong Kong variants with phrase conversion), generate pinyin with tones or initials, segment Chinese words and extract keywords.
- [RSS & Atom Feed Reader - Parse Any Feed to JSON](https://apify.com/Digital-Tools/rss-feed-reader) – Read RSS, Atom and RDF feeds in bulk and get clean JSON: title, link, date, author, categories, summary, full content, images and podcast enclosures.
- [SSL Certificate & DNS Checker - Bulk Domain Expiry Monitor](https://apify.com/Digital-Tools/ssl-dns-checker) – Check SSL/TLS certificates and DNS records for many domains at once: expiry date, days left, issuer, SANs, TLS version, validity, plus A, AAAA, CNAME, MX, NS, TXT, CAA, SOA, SPF and DMARC.
- [PDF, Word, PowerPoint & Excel to Text Extractor](https://apify.com/Digital-Tools/document-to-text) – Extract text from PDF, DOCX, PPTX and XLSX files by URL.

Questions or feature requests? Open an issue in the **Issues** tab.

# Actor input Schema

## `startUrls` (type: `array`):

Website homepages (sitemaps are discovered via robots.txt and common locations) or direct sitemap URLs (.xml, .xml.gz, .txt).

## `maxUrlsPerSite` (type: `integer`):

Stop after this many URLs per input (0 = no limit). Helps control cost on huge sites.

## `includePattern` (type: `string`):

Only output URLs matching this regular expression, e.g. /blog/ or /products/. Filtered-out URLs are not charged.

## `excludePattern` (type: `string`):

Skip URLs matching this regular expression, e.g. .(jpg|png)$ or /tag/.

## `maxSitemapsPerSite` (type: `integer`):

Safety cap on how many sitemap files (including index children) are parsed per site.

## `timeoutSecs` (type: `integer`):

Timeout for each sitemap download.

## Actor input object example

```json
{
  "startUrls": [
    {
      "url": "https://apify.com"
    },
    {
      "url": "https://www.bbc.com/sitemaps/https-index-com-news.xml"
    }
  ],
  "maxUrlsPerSite": 10000,
  "maxSitemapsPerSite": 500,
  "timeoutSecs": 30
}
```

# Actor output Schema

## `results` (type: `string`):

Dataset with one item per result.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "startUrls": [
        {
            "url": "https://apify.com"
        },
        {
            "url": "https://www.bbc.com/sitemaps/https-index-com-news.xml"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("digital-tools/sitemap-url-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "startUrls": [
        { "url": "https://apify.com" },
        { "url": "https://www.bbc.com/sitemaps/https-index-com-news.xml" },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("digital-tools/sitemap-url-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "startUrls": [
    {
      "url": "https://apify.com"
    },
    {
      "url": "https://www.bbc.com/sitemaps/https-index-com-news.xml"
    }
  ]
}' |
apify call digital-tools/sitemap-url-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,digital-tools/sitemap-url-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/itoztGAEG4yYryEtj/builds/hrut0QBu9x8JQCXWz/openapi.json