Sitemap Extractor: All URLs of a Site, Lastmod & Priority avatar

Sitemap Extractor: All URLs of a Site, Lastmod & Priority

Pricing

from $0.35 / 1,000 url extracteds

Go to Apify Store
Sitemap Extractor: All URLs of a Site, Lastmod & Priority

Sitemap Extractor: All URLs of a Site, Lastmod & Priority

Every URL a site lists in its sitemaps: robots.txt and sitemap indexes followed, with last modified date, change frequency, priority and images. 1,000 sites.

Pricing

from $0.35 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Open Data Tools

Open Data Tools

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

10 hours ago

Last modified

Features

Limited permissions

Share

Give a list of sites or sitemap URLs, get every page URL they list for search engines, one row each, with its last modified date, change frequency, priority, number of images and language versions. The extractor finds the sitemaps itself in robots.txt, follows sitemap indexes, and reads XML and plain-text sitemaps. Fast and cheap: $0.50 per 1,000 URLs.

What you get

One row per page URL:

  • URL, path and site.
  • Last modified date (ISO 8601), change frequency, priority.
  • Number of images and of language versions (hreflang) listed with the page.
  • The sitemap file the URL comes from.

Each URL appears once per site, even when several sitemaps list it.

Use cases

  • SEO audits: get the full list of pages a site wants indexed, then check them with other tools (status codes, redirects, broken links).
  • Content monitoring: sort by last modified date to see what a competitor published or updated recently.
  • Crawling: feed the URL list to a scraper instead of crawling the whole site.
  • Migrations: compare the sitemaps before and after a site moves.

How to use

  • Without changing anything: the first 100 URLs of apify.com and of the Apify blog sitemap.
  • Sites or sitemaps: one per line, up to 1,000 per run. For a site, the sitemaps announced in its robots.txt are read; otherwise /sitemap.xml, then /sitemap_index.xml.
  • Maximum URLs per site: from 1 to 50,000 (5,000 by default).

Input example

{
"sites": ["apify.com", "https://blog.apify.com/sitemap.xml"],
"maxUrlsPerSite": 100
}

Output example

{
"recordType": "url",
"id": "https://apify.com/store",
"url": "https://apify.com/store",
"input": "apify.com",
"site": "apify.com",
"path": "/store",
"sitemapUrl": "https://apify.com/sitemap-pages.xml",
"lastModified": "2026-11-18T00:00:00.000Z",
"changeFrequency": "daily",
"priority": 0.9,
"imageCount": 0,
"alternateCount": 0,
"status": "ok",
"sourceName": "Sitemap of apify.com",
"sourceUrl": "https://apify.com/sitemap-pages.xml",
"licence": "Sitemap the site publishes for search engines; the pages are not visited",
"sourceUpdatedAt": null,
"actorVersion": "0.1.1",
"retrievedAt": "2026-11-19T10:12:41.207Z"
}

Pricing

Pay per event: $0.50 per 1,000 URLs extracted (url-extracted, $0.0005 each), plus Apify's start fee of $0.00005 per run. A site without a sitemap and text that is not a web address give a free error row.

URLsPrice
1,000 URLs$0.50
10,000 URLs$5
100,000 URLs$50

Data source

The sitemaps each site publishes for search engines (sitemaps.org protocol): robots.txt Sitemap: lines, sitemap indexes, XML and plain-text sitemaps, up to 200 sitemap files per site. Pages themselves are not visited. Each site's robots.txt is respected. Compressed .gz sitemaps are not read yet.

Sitemaps are published so that robots can read them. The extractor reads only those files, waits between requests to the same site, follows robots.txt and never visits private addresses.

FAQ

  • Why are some dates empty? The site does not give a last modified date for that page: it is optional in sitemaps.
  • Why fewer URLs than the site has? A sitemap lists only the pages its owner chose, and the extractor stops at your maximum per site.
  • My site has no sitemap. You get a free error row: the extractor does not crawl pages.

Every Actor here also works as a tool for AI agents (Claude, ChatGPT, Cursor…) through Apify’s MCP server.

Changelog

See CHANGELOG.md.

En français

Toutes les adresses d'un site, d'après ses sitemaps

Donnez des sites ou des adresses de sitemap, obtenez une ligne par page avec sa date de mise à jour, sa fréquence, sa priorité, ses images et ses versions en d'autres langues. Les sitemaps sont trouvés tout seuls dans robots.txt.

Prix

0,50 $ les 1 000 adresses. Les lignes d'erreur sont gratuites.

Source et légalité

Les sitemaps que chaque site publie pour les moteurs de recherche. Seuls ces fichiers sont lus, robots.txt respecté.