# Sitemap URL Extractor: Every Page of Any Website (`leadgenguy/sitemap-url-extractor`) Actor

Extract every page URL from any website sitemap, including sitemap indexes and gzip files, with last-modified dates, section and filters. Fast, no login. $0.10 per 1,000 URLs.

- **URL**: https://apify.com/leadgenguy/sitemap-url-extractor.md
- **Developed by:** [Mokssh jaiin](https://apify.com/leadgenguy) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

Pay per event

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Sitemap URL Extractor: Every Page of Any Website — $0.10 per 1,000 URLs

Get **every page URL from any website's sitemap** in seconds: blog posts, products, categories, docs. It finds the sitemap from `robots.txt` or common locations, follows sitemap indexes, reads `.xml.gz` files and plain-text sitemaps, and returns one clean row per URL with the **last-modified date and site section**.

### Why use it

- **Very cheap:** $0.10 per 1,000 URLs, no monthly fee.
- **Handles real-world sitemaps:** nested indexes, gzip, CDATA, plain text.
- **Filters:** keep or drop URLs by text, or only pages changed after a date.
- **Fast:** no browser, no login, no pages loaded: only the sitemap files.

### Use cases

- **Lead generation:** find every blog, product or location page of a prospect, then feed them to a contact or tech scraper.
- **SEO audits and content gap analysis:** list all pages, find stale or missing content, track new pages by running daily with "changed after".
- **Competitor monitoring:** watch which new product or blog URLs appear.
- **Crawl seeds:** a ready URL list for a scraper.

### Pricing

| Event | Price |
|---|---|
| URL extracted | $0.10 per 1,000 ($0.0001 each) |

URLs are charged only when delivered. Free Apify accounts are limited to **500 URLs per run**.

### Input

```json
{
  "websites": ["apify.com", "https://example.com/sitemap.xml"],
  "maxUrlsPerSite": 10000,
  "urlContains": ["/blog/"],
  "changedAfter": "2026-10-01"
}
```

### Output

```json
{ "site": "apify.com", "url": "https://apify.com/blog/web-scraping-guide", "lastmod": "2026-09-30", "section": "blog", "sitemap": "https://apify.com/sitemap-blog.xml" }
```

A per-site summary (sitemaps found and read, URLs seen and delivered, problems) is saved in the run's key-value store under `SUMMARY`.

### Honest limits

- Only sites that publish a sitemap work. A site with no sitemap returns nothing (and is not charged).
- A sitemap lists what the site chooses to list: it may leave out pages, and `lastmod` dates are only as accurate as the site makes them.
- Very large sites can have millions of URLs. Use filters and "max URLs per website".

### Feedback and reviews

Found a bug, or need a field or filter added? Open an issue on this Actor's **Issues** tab and it will be looked at quickly. If this Actor saved you time, a short and honest review helps other people find it. Thank you!

### Legal

Reads only sitemap files that sites publish for crawlers, and honours nothing beyond what is public. Check a site's terms before using its URLs for large-scale crawling.

# Actor input Schema

## `websites` (type: `array`):

Domains (<code>apify.com</code>), site URLs, or a direct sitemap link (<code>https://example.com/sitemap.xml</code>, <code>.xml.gz</code> supported). Up to 5,000 sites per run.

## `maxUrlsPerSite` (type: `integer`):

Stop after this many matching URLs per site. Free Apify plans: 500 URLs per run in total.

## `urlContains` (type: `array`):

Keep only URLs containing any of these words, e.g. <code>/blog/</code>, <code>/product/</code>.

## `urlExcludes` (type: `array`):

Drop URLs containing any of these words, e.g. <code>/tag/</code>, <code>?page=</code>.

## `changedAfter` (type: `string`):

Keep only URLs whose last-modified date is on or after this date, e.g. <code>2026-10-01</code>. URLs without a date are dropped.

## `includeLastmod` (type: `boolean`):

Adds the date the site says each page last changed.

## Actor input object example

```json
{
  "websites": [
    "apify.com"
  ],
  "maxUrlsPerSite": 10000,
  "includeLastmod": true
}
```

# Actor output Schema

## `jobs` (type: `string`):

All scraped jobs as a dataset.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "websites": [
        "apify.com",
        "stripe.com"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("leadgenguy/sitemap-url-extractor").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "websites": [
        "apify.com",
        "stripe.com",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("leadgenguy/sitemap-url-extractor").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "websites": [
    "apify.com",
    "stripe.com"
  ]
}' |
apify call leadgenguy/sitemap-url-extractor --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,leadgenguy/sitemap-url-extractor"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/hKs7mnpLVQGDQB3kw/builds/N32Y9jSqWwP9uZ36Y/openapi.json