Sitemap URL Extractor - All Page URLs + Lastmod of Any Site
Pricing
from $1.00 / 1,000 result items
Sitemap URL Extractor - All Page URLs + Lastmod of Any Site
Get every URL of a website from its XML sitemaps: auto-discovers sitemaps via robots.txt and common paths, follows sitemap indexes, reads .xml.gz and .txt. Returns lastmod, changefreq, priority, hreflang. Regex & date filters. Fast, no browser.
Pricing
from $1.00 / 1,000 result items
Rating
0.0
(0)
Developer
leo zhu
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
2 days ago
Last modified
Categories
Share
Get the full list of page URLs of any website from its XML sitemaps in seconds - no browser, no crawling. Just give a homepage: the Actor finds the sitemaps in robots.txt (or at /sitemap.xml, /sitemap_index.xml, /wp-sitemap.xml, /sitemap.txt ...), follows sitemap index files, reads gzipped .xml.gz and plain-text sitemaps, and returns every URL with lastmod, changefreq, priority, image count and hreflang alternates.
Who uses it
- SEO specialists & agencies: content inventory, find new / updated pages of competitors (
lastmodAfter), check hreflang coverage, audit indexable URLs. - Scraping & AI teams: get the complete URL list first, then feed it into a crawler, Website Content Crawler or an LLM / RAG pipeline.
- E-commerce & market research: list all product or category URLs of a store with
includePatterns: ["/products/"]. - Monitoring: schedule daily with
lastmodAfterto catch newly published pages.
Input
| Field | Description |
|---|---|
startUrls | Homepages (auto-discovery) or direct sitemap URLs. |
maxUrlsPerSite | Default 1000, up to 1,000,000. |
includePatterns / excludePatterns | Regex filters on the URL. |
lastmodAfter | e.g. 2026-01-01 - only recently updated pages. |
maxSitemapsPerSite, maxConcurrency | Safety / speed limits. |
Example:
Output (one row per URL)
Websites without a readable sitemap get one no-sitemap row (free). RUN_SUMMARY in the key-value store has counts.
Pricing (pay per event)
- $0.001 per URL saved (
sitemap-url) - $1 per 1,000 URLs. - Tiny start fee per run (
apify-actor-start). - Sites without a sitemap and filtered-out URLs are free. Set "Maximum cost per run" to cap spend - the run stops gracefully and keeps saved results.
Notes
- Duplicate URLs across sitemaps of the same site are removed.
- If a site lists several sitemaps in robots.txt (e.g. multiple sub-sites), give the specific sitemap URL to limit the scope.
- Only public sitemap files are read; nothing behind logins.
