Sitemap Extractor: all page URLs from XML sitemaps
Pricing
from $0.45 / 1,000 url processeds
Sitemap Extractor: all page URLs from XML sitemaps
Finds a site's sitemaps through robots.txt or the usual locations, follows sitemap index files, and returns every page URL with its lastmod, changefreq and priority.
Pricing
from $0.45 / 1,000 url processeds
Rating
0.0
(0)
Developer
Ed Rayyis
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Categories
Security
Limited permissionsShare
Give it a site address or a sitemap URL and it returns every page URL the site lists in its sitemaps, with the lastmod, changefreq and priority values the sitemap gives.
For a site address, it reads the site's robots.txt for Sitemap: lines. When robots.txt names none, it tries /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml and /wp-sitemap.xml. Sitemap index files are followed down to the page lists. It reads XML sitemaps, gzipped sitemaps (.xml.gz) and plain text sitemaps with one URL per line. A URL listed in more than one sitemap is output once.
Input
- Sites or sitemap URLs (required)
- Maximum URLs (default 10,000), across all start URLs
- URL filter (optional regular expression, such as
/blog/)
Output
Dataset: one item per page URL with url, lastmod, changefreq and priority when the sitemap has them, sitemap (the file it came from) and startUrl. Export as JSON, CSV or Excel, or read it over the API.
The key value store holds a SUMMARY record: for each start URL, the number of sitemap files read, the URLs found, and any errors (for example, no sitemap found, or a sitemap answering HTTP 404).
Pricing
Pay per event: one event per page URL output. Set the maximum URLs to control cost. The run stops when it reaches the maximum charge you set.
Scope and conduct
Only robots.txt and sitemap files are downloaded. The pages listed in the sitemaps are never requested. No personal data is collected.
Support
Questions and problems: the Issues tab on this page. Replies are written by an AI assistant on behalf of the developer and reviewed by the developer.
