Sitemap Extractor: get URLs from any XML sitemap for SEO audits and crawling

Sitemap Extractor finds a website's sitemaps and returns the URLs in them as structured data (capped at 1,000 URLs per site by default; raise maxUrlsPerSite, or set it to 0 for no limit), with lastmod, changefreq, priority, image counts, Google News titles and, optionally, hreflang alternates. Paste a domain and it reads the sitemaps listed in robots.txt, falling back to the usual paths (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml and others). Or paste a sitemap URL directly. It follows sitemap indexes, reads gzipped (.gz) and plain-text sitemaps, filters by regex and last-modified date, and can return only URLs that are new since the last run. You pay $0.0002 per URL written to the dataset.

Who it's for

• SEO specialists auditing a site's indexable pages, checking lastmod freshness, or comparing sitemap URLs with crawl and Search Console data.
• Content and competitive researchers tracking which pages a competitor publishes, and when.
• Developers and data teams feeding URL lists into a crawler, a migration check or a RAG pipeline.
• Agencies pulling page inventories for many client sites at once.

What data you get

One dataset item per URL:

Field	What it holds
url	The page URL from the sitemap.
site	The site origin it came from, e.g. https://www.nasa.gov.
lastmod	Last-modified date, normalized to ISO 8601 (null if the sitemap has none).
changefreq, priority	As given in the sitemap (often null).
sitemapUrl	The sitemap file the URL was listed in.
imageCount	Number of <image:image> entries, if any.
newsTitle	Google News sitemap title, if any.
alternates	hreflang alternates, when includeAlternates is on.

Sample item (real output from nasa.gov):

{
  "url": "https://www.nasa.gov/history/glenn-orbits-the-earth/",
  "site": "https://www.nasa.gov",
  "lastmod": "2023-07-25T15:19:07+00:00",
  "changefreq": null,
  "priority": null,
  "sitemapUrl": "https://www.nasa.gov/wp-sitemap-posts-post-1.xml",
  "imageCount": null,
  "newsTitle": null
}

How to use it

Default mode: if you leave Websites or sitemap URLs empty, or keep only the sample sitemap https://www.nasa.gov/wp-sitemap.xml that the form starts with, the run reads that NASA sitemap and returns at most 40 URLs, charged at the normal per-URL price (so at most $0.008). The log says Default mode: no start URLs given (or only the sample default), returning up to 40 sample URLs from https://www.nasa.gov/wp-sitemap.xml. Replace the sample with your own sites for a normal run.

Input field	Label	Notes
startUrls	Websites or sitemap URLs	Defaults to the sample NASA sitemap (default mode, at most 40 URLs). A domain auto-discovers sitemaps; a sitemap URL is read directly.
includePatterns	Include URL patterns (regex)	Keep URLs matching any, e.g. /blog/.
excludePatterns	Exclude URL patterns (regex)	Drop URLs matching any.
sitemapPatterns	Only follow child sitemaps matching (regex)	For big indexes, e.g. post-sitemap. Saves requests.
lastmodAfter	Changed after (date)	YYYY-MM-DD.
lastmodWithinDays	Changed within last N days	Overrides lastmodAfter.
keepUndatedUrls	Keep URLs without a lastmod	Default false: with a date filter set, URLs that have no lastmod are dropped. Set true to keep them.
maxUrlsPerSite	Max URLs per site	Default 1,000. 0 = no limit.
maxTotalUrls	Max URLs in total	0 = no limit.
maxSitemapsPerSite	Max sitemap files per site	Default 1,000.
includeAlternates	Include hreflang alternates	Default false.
onlyNewUrls	Only new URLs (monitoring)	Default false. Seen URLs are remembered for 365 days.
stateStoreName	State store name	Default sitemap-extractor-state. One name per monitor.

Example: blog URLs changed in the last 30 days.

{
  "startUrls": [
    "https://www.example.com"
  ],
  "includePatterns": [
    "/blog/"
  ],
  "lastmodWithinDays": 30
}

Example: new pages on a competitor's site since the last run (schedule it daily or weekly).

{
  "startUrls": [
    "https://competitor.example.com"
  ],
  "onlyNewUrls": true,
  "stateStoreName": "competitor-new-pages"
}

Notes:

• Date filters and missing lastmod: when lastmodAfter or lastmodWithinDays is set, URLs without a lastmod are dropped by default, because their age can't be checked, and the log and RUN_SUMMARY (undatedDropped) say how many. Set keepUndatedUrls to true to keep them (undatedKept in RUN_SUMMARY). Many sites leave lastmod out, so on such a site that returns every URL, up to maxUrlsPerSite, and each one is billed.
• Duplicate URLs within a site are returned once.
• The Actor reads sitemaps only. It doesn't visit the listed pages, so it doesn't report HTTP status codes, titles or canonical tags.
• RUN_SUMMARY in the default key-value store shows, per site, where the sitemap was found (robots.txt, a fallback path, or given), sitemap files read, errors, URLs found and URLs pushed, and, when a date filter is set, how many URLs without a lastmod were dropped (undatedDropped) or kept (undatedKept). Each site's status is ok, no-sitemap, skipped (robots.txt disallows the sitemaps or can't be read) or error. A site whose sitemaps all failed to load is error, with the last error, never ok.

Pricing

This Actor uses pay-per-event pricing: $0.0002 per URL written to the dataset. That is $0.20 per 1,000 URLs. There is no separate start fee in this Actor's pricing (as listed on October 28, 2026; the Pricing tab on the Store page is the source of truth).

You are charged only for URLs that end up in your dataset. URLs removed by your filters or date limits, duplicates within a site, and URLs skipped in monitoring mode are not charged.

Example	Urls	Cost
Small site	500	$0.10
Default cap of one site (1,000 URLs)	1,000	$0.20
One site with the cap raised to 10,000	10,000	$2.00
Large site, no cap (illustration)	250,000	$50.00

With the default maxUrlsPerSite of 1,000, one site costs at most $0.20 per run. Raise it, or set 0 for no limit, to read bigger sitemaps.

Worked example: 1,000 URLs × $0.0002 = $0.20.

To cap spend, set Maximum cost per run in the run options (or maxTotalChargeUsd via the API). The Actor stops cleanly when the next url would go over the limit and records stoppedByCostLimit: true in the run summary.

How it compares

Prices are from the public Apify Store listings on October 28, 2026.

Option	Pricing	What it is good at
This Actor	$0.0002 per URL	Sitemap discovery via robots.txt and common paths, indexes, .gz and text sitemaps, lastmod/regex filters, image/news/hreflang data, only-new-URLs mode.
apify/sitemap-extractor	See listing	Apify's own sitemap Actor. Check its listing for current features.
Desktop SEO crawlers	License	Crawl the pages themselves and compare against the sitemap. Better for full technical audits.
Free online sitemap URL extractors	Free	Fine for a single small sitemap.

Integrations and API

The actor ID is fecklesspleb/sitemap-extractor (in API URLs: fecklesspleb~sitemap-extractor).

n8n, Make and Zapier. Each has an Apify integration with a "run Actor" action. Pick fecklesspleb/sitemap-extractor, paste the input JSON shown below, then read the results from the run's default dataset in the next step. To process only new URLs on a schedule, turn on the monitoring option and run it from a schedule (Apify Schedules or a trigger in your automation tool).

Webhooks and schedules. Apify Schedules can run the Actor on a cron, and a webhook on "run succeeded" can call your own endpoint with the run ID.

HTTP API. Run the Actor and get the items back in one call. This waits for the run to finish, so use it for small runs:

curl -X POST "https://api.apify.com/v2/acts/fecklesspleb~sitemap-extractor/run-sync-get-dataset-items?token=$APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"startUrls": ["https://www.example.com"], "includePatterns": ["/blog/"], "lastmodWithinDays": 30}'

Python (pip install apify-client):

from decimal import Decimal
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("fecklesspleb/sitemap-extractor").call(
    run_input={
        "startUrls": [
            "https://www.example.com"
        ],
        "includePatterns": [
            "/blog/"
        ],
        "lastmodWithinDays": 30
    },
    max_total_charge_usd=Decimal("5"),  # optional spend cap
)
# apify-client 3.x returns a Run object; on 1.x/2.x use run["defaultDatasetId"]
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

JavaScript / Node.js (npm install apify-client):

import { ApifyClient } from 'apify-client';

const client = new ApifyClient({ token: process.env.APIFY_TOKEN });
const run = await client.actor('fecklesspleb/sitemap-extractor').call(
    {
        "startUrls": [
            "https://www.example.com"
        ],
        "includePatterns": [
            "/blog/"
        ],
        "lastmodWithinDays": 30
    },
    { maxTotalChargeUsd: 5 }, // optional spend cap
);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);

Results can also be exported from the dataset as JSON, CSV, Excel, XML or RSS from the Apify Console or the dataset API.

FAQ

How do I extract all URLs from a sitemap.xml? Paste the sitemap URL (or just the domain) in startUrls and run. Every <loc> becomes one item. Set maxUrlsPerSite to 0 to remove the default 1,000 cap.

How do I find a website's sitemap? Paste the domain. The Actor reads Sitemap: lines in robots.txt, then tries /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml, /wp-sitemap.xml and /sitemap.xml.gz. RUN_SUMMARY tells you where it found the sitemap, or that none was found.

Can it handle sitemap index files and .gz sitemaps? Yes. Indexes are followed recursively (up to maxSitemapsPerSite files), and gzipped and plain-text sitemaps are read. Use sitemapPatterns to follow only some child sitemaps.

How can I monitor a competitor's new pages? Turn on onlyNewUrls, give the monitor its own stateStoreName, and schedule it. Each run returns only URLs not seen before, and you only pay for those.

Can I export sitemap URLs to CSV or Google Sheets? Yes. Export the dataset as CSV or Excel, or send it to Google Sheets through the n8n, Make or Zapier integration.

Does it check whether URLs return 200 or 404? No. It reads the sitemap only. Feed the URL list to a crawler if you need status codes.

Do I need an API key or login? No. The Actor reads public endpoints and needs no account on the source site. You only need an Apify account to run it. It obeys robots.txt and rate-limits requests per host. robots.txt is checked again on every redirect, so a URL that redirects to a disallowed path or another host is not followed. If a host's robots.txt can't be read (a server error, a network failure or a page that isn't a robots file), that host is skipped for the run and RUN_SUMMARY says why. A missing robots.txt means no rules.

Related Actors

• RSS Feed Reader: RSS and Atom feed items as clean JSON, with feed discovery from a website URL.
• Hacker News Scraper: Hacker News stories and comments, plus parsed "Who is hiring?" job posts.
• ATS Jobs Scraper: jobs from Greenhouse, Lever, Ashby, Workable, Recruitee and Workday boards in one schema.
• Remote Jobs Aggregator: remote jobs from Remote OK, We Work Remotely and Jobicy, de-duplicated.