Sitemap URL Lister avatar

Sitemap URL Lister

Pricing

Pay per usage

Go to Apify Store
Sitemap URL Lister

Sitemap URL Lister

List every URL in a public XML sitemap (follows sitemap indexes) with lastmod dates, to a dataset. Free, no JavaScript, no page scraping.

Pricing

Pay per usage

Rating

0.0

(0)

Developer

Mitchel Davis

Mitchel Davis

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

3 days ago

Last modified

Categories

Share

Sitemap URL Lister (Apify Actor)

Lists every URL in a public XML sitemap, following sitemap index files one level deep, and writes {url, lastmod, sitemap} rows to the Actor's dataset. Useful for SEO audits, link inventories, change monitoring (via lastmod), and building crawl seed lists for public sites.

Input

fielddefaultnotes
sitemapUrlhttps://www.gov.uk/sitemap.xmlpublic http(s) sitemap or sitemap index
maxUrls1001–5000

Run with {} to use the defaults.

Output

One dataset row per URL: url, lastmod (or null), sitemap (the sitemap file it came from).

Scope and limits

  • Reads only the sitemap XML files. It does not fetch or scrape page bodies.
  • Follows only one level of sitemap index nesting.
  • Does not run JavaScript; it reads XML only.
  • Respects the maxUrls cap; a root sitemap that returns HTTP errors fails the run with the HTTP status.

Local tests

npm install
npm test # 8 offline unit tests + 1 live smoke test (gov.uk)

License

MIT (see LICENSE). Clean-room code: Node built-ins plus the official apify SDK only.