Sitemap Extractor

Sitemap Extractor lists every URL a website publishes in its XML sitemaps, one row per URL, with the last modified date, change frequency, priority, image URLs, Google News and video titles, and the hreflang language versions when the sitemap has them. Type a domain such as stripe.com into Websites or sitemaps and click Start: sitemaps are found through robots.txt, and indexes and gzip files are opened for you. The actor reads sitemaps only; it does not visit the pages or check whether they still load. It costs $0.40 per 1,000 URLs on the Free plan ($0.20 on Business), and the $5 of monthly credit in Apify's Free plan covers about 12,000 URLs.

What data does it return?

Field	Description
url	Page address exactly as the sitemap lists it in <loc>.
site	Host name of the website, such as ahrefs.com.
sitemap	The sitemap file that lists the URL, after any redirect, such as https://ahrefs.com/writing-tools/sitemap.xml.
lastModified	The <lastmod> date as an ISO 8601 UTC timestamp, null when the site leaves it out. Many large sites do: Stripe and Ahrefs had none in October 2026.
changeFrequency, priority	<changefreq> such as daily or weekly, and <priority> from 0 to 1. Both null when not given.
images	Image URLs listed for the page in an image sitemap, up to 20 per page.
newsTitle, newsPublishedAt	Article title and publication time from a Google News sitemap.
videoTitle	Video title from a video sitemap.
alternates	Language versions from xhtml:link rel="alternate" tags, each with hreflang (such as de or en-AT) and url. Stripe lists up to 76 per page.
isNew	With a watch name, true when the URL was not in the site's sitemaps in the previous run and false when it was. null without a watch or in the first run.
checkedAt	When the sitemap was read, as an ISO 8601 UTC timestamp.

How to extract URLs from a sitemap

1. Enter websites in Websites or sitemaps, one per line. A domain such as wordpress.org makes the actor read the Sitemap: lines of its robots.txt, or try 8 usual addresses (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, /sitemap.txt and others) when robots.txt names none. To read one file only, paste its URL, such as https://wordpress.org/news/sitemap.xml.
2. Narrow the list with URL contains (any of), such as /blog/, and URL does not contain, such as /tag/. Both ignore case.
3. To get recently changed pages, set Modified on or after to a date (2026-10-01) or a moving day (today, -7). URLs without a <lastmod> date are then left out.
4. Set Maximum URLs per website (100,000 by default) and Maximum rows to control the size of the run.
5. Click Start. URLs appear in the Output tab, ready to download as CSV, Excel or JSON. The SUMMARY record in the key-value store gives per website the number of sitemap files read, URLs found, new URLs and any error.

For the API, this input returns blog and news pages that changed in the last 30 days on two sites:

{
  "websites": ["www.shopify.com", "wordpress.org"],
  "urlIncludes": ["/blog/", "/news/"],
  "modifiedAfter": "-30",
  "maxUrlsPerSite": 100000
}

Example output

One URL from Ahrefs in a run on 11 October 2026. The Ahrefs sitemap gives no dates, so lastModified is null, and the page has 12 language versions.

{
  "url": "https://ahrefs.com/fr/writing-tools/img-alt-text-generator",
  "site": "ahrefs.com",
  "sitemap": "https://ahrefs.com/writing-tools/sitemap.xml",
  "lastModified": null,
  "changeFrequency": null,
  "priority": null,
  "images": [],
  "newsTitle": null,
  "newsPublishedAt": null,
  "videoTitle": null,
  "alternates": [
    {
      "hreflang": "de",
      "url": "https://ahrefs.com/de/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "en",
      "url": "https://ahrefs.com/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "es",
      "url": "https://ahrefs.com/es/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "id",
      "url": "https://ahrefs.com/id/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "it",
      "url": "https://ahrefs.com/it/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "ja",
      "url": "https://ahrefs.com/ja/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "ko",
      "url": "https://ahrefs.com/ko/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "pl",
      "url": "https://ahrefs.com/pl/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "pt",
      "url": "https://ahrefs.com/pt/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "tr",
      "url": "https://ahrefs.com/tr/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "uk",
      "url": "https://ahrefs.com/uk/writing-tools/img-alt-text-generator"
    },
    {
      "hreflang": "zh",
      "url": "https://ahrefs.com/zh/writing-tools/img-alt-text-generator"
    }
  ],
  "isNew": null,
  "checkedAt": "2026-10-11T16:01:38.385Z"
}

How much does it cost to extract sitemap URLs?

	Free plan	Starter	Scale	Business
1,000 URLs	$0.40	$0.32	$0.26	$0.20

You pay only for the events in the table. There is no start fee, and compute time and proxies are included.

A website with 10,000 URLs costs $4.00 on the Free plan and $2.00 on Business; stripe.com listed 7,103 in October 2026. A URL listed in several sitemaps of the same site is returned and charged once, and URLs removed by your filters or skipped as already seen in new-only mode cost nothing.

Limits

• Speed depends on the number of sitemap files, which are read one at a time. In October 2026, stripe.com, ahrefs.com and apify.com gave 59,146 URLs from 21 sitemap files in 76 seconds. Shopify and WordPress.org took 67 seconds for 371 sitemap files.
• The cap in Maximum URLs per website counts every unique URL read, before the filters. With filters on a large site, raise it: the Shopify and WordPress.org run above stopped at 100,000 URLs on each site and returned the 1,520 blog and news pages modified in the last 30 days among them.
• Up to 5,000 sitemap files are opened per website. Each file request waits up to 60 seconds and is tried 3 times on a server error or HTTP 429.
• The sitemap protocol allows 50,000 URLs per file, and the actor follows indexes inside indexes. Pages that the site leaves out of its sitemaps are not found, because the actor does not crawl links.
• lastModified, changeFrequency and priority are whatever the site writes. Some sites stamp every URL with the date the sitemap was generated.
• A website without any sitemap in robots.txt or at the usual addresses is reported in SUMMARY with the error No sitemap found and costs nothing. When no website could be read, the run fails, unless robots.txt was the reason.

robots.txt

The actor follows robots.txt of each website, using the rules for all user agents. A sitemap file that robots.txt disallows is skipped and counted under sitemapsSkippedByRobots in SUMMARY, and redirects are checked the same way. A website whose robots.txt does not answer (a server error, HTTP 429 or a timeout) is skipped for that run, while a missing robots.txt (HTTP 404) allows everything.

Alerts for new pages

Give the run a Watch name, such as shop-blog, or save it as a task, and schedule it daily or weekly. From the second run on, every URL carries isNew, and SUMMARY gives newUrls per website. Turn on Return only URLs new since the last run to receive and pay for only the new ones: new products, articles or landing pages of the site. The actor stores a 64-bit fingerprint of each URL, never the page itself. When a run stops early at a limit, its fingerprints are added to the stored ones, so the unread part of the site is not forgotten. New URLs that were read but did not fit under Maximum rows or the spending limit are not stored, so they stay new and come in the next run: three runs with Maximum rows at 5 on apify.com returned 15 different new URLs.

FAQ

Is it legal to extract URLs from a sitemap?

Sitemaps are files that websites publish so that search engines and other programs can find their pages, and the actor reads them without logging in. It honors robots.txt and returns URLs and the metadata of the sitemap, no page content and no personal data. You are responsible for what you do with the pages you then visit.

Does it check the status code of each URL?

No. It returns what the sitemap lists. To find URLs that redirect or return 404, pass the output to a URL status checker.

Can it read a sitemap that is not at the usual address?

Yes. Paste the full sitemap URL, including .xml, .xml.gz or .txt files and any URL whose path contains sitemap; it is read directly, and an index found there is opened down to its last file.

Related scrapers

• URL Status Checker
• Broken Link Checker
• Tech Stack Detector
• Article Extractor
• RSS Feed Scraper

Support

This actor is built and maintained by deriverge s.r.o., a software company based in the Czech Republic. If a run fails or a field you need is missing, please open an issue in the Issues tab or write to us at info@deriverge.com. We respond in English and Czech. Runs can be scheduled in Apify Console or started from the API tab, which has examples for Python, JavaScript and cURL and works with Make, Zapier, n8n and the Apify MCP server. If the actor saves you time, a short review helps other people find it.