Sitemap URL Extractor - every page URL with lastmod avatar

Sitemap URL Extractor - every page URL with lastmod

Pricing

from $0.25 / 1,000 url extracteds

Go to Apify Store
Sitemap URL Extractor - every page URL with lastmod

Sitemap URL Extractor - every page URL with lastmod

Extract every page URL from any website's XML sitemaps. Reads robots.txt, follows sitemap indexes and .gz files, and returns each URL with lastmod, changefreq, priority and source sitemap. Many sites per run. Fast HTTP only, no browser. Export to CSV, JSON or Excel.

Pricing

from $0.25 / 1,000 url extracteds

Rating

0.0

(0)

Developer

Forge Creations

Forge Creations

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

7 days ago

Last modified

Categories

Security

Limited permissions

Share

Sitemap URL Extractor: every page URL with lastmod

Give it a list of websites. For each one it reads the sitemaps the site publishes for crawlers and returns every page URL listed, with:

  • loc (the page URL), lastmod, changefreq, priority, when present
  • sitemap: the sitemap file it came from, and site

It finds sitemaps from robots.txt Sitemap: lines, then common locations (/sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, ...). It follows sitemap indexes up to 5 levels deep and reads .gz sitemaps.

Pricing

$0.00025 per URL returned ($0.25 per 1,000), plus Apify's standard start fee of $0.00005 per run. Sitemap files read, filtered-out URLs and the per-website summary are not charged. Set a maximum charge in run options to cap spending; the run stops cleanly at that limit.

Input

FieldDefaultNotes
websitesrequiredDomains or URLs
maxUrlsPerWebsite10000Stops each site at this many URLs (summary marks it truncated)
includeRegex / excludeRegexnoneKeep or skip URLs by pattern
maxIndexDepth3Nested sitemap-index levels

Output

Dataset: one row per URL. Key-value store SUMMARY: one row per website with status (ok, no_sitemap, invalid_input), sitemaps read, URLs found/returned, truncation.

Limits

It returns only what sites list in their sitemaps; it does not crawl links. Sites without a sitemap return no_sitemap and cost nothing beyond the start fee.

Support

Open an issue on the Issues tab with the website and run ID.