Sitemap & URL Extractor
Pricing
from $0.30 / 1,000 results
Sitemap & URL Extractor
Find a website's sitemaps (from robots.txt or common paths), extract every URL with its lastmod, changefreq and priority, follow nested sitemap indexes, and optionally check each URL's live status and redirects. Falls back to the homepage's internal links when there's no sitemap.
Pricing
from $0.30 / 1,000 results
Rating
0.0
(0)
Developer
Autoloom
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
a day ago
Last modified
Categories
Share
Point it at a website and get every URL the site publishes — pulled from its sitemaps, with lastmod, changefreq and priority, following nested sitemap indexes automatically. Pure HTTP, fast and cheap.
Pricing: pay per result: $0.30 per 1,000 results ($0.03 per 100, $3.00 per 10,000), plus $0.00005 per run start. You only pay for rows in your dataset.
What it does
- Finds sitemaps from robots.txt and the common sitemap paths.
- Parses them — including nested sitemap indexes and gzipped sitemaps.
- Returns one row per URL with
lastmod,changefreq,priorityand which sitemap it came from. - Optional: check each URL's live status code and final (redirected) URL — so you can spot dead and redirected pages in one pass.
- No sitemap? It still works. When a site has no usable sitemap, it falls back to the site's own internal links instead of giving up (most tools just return nothing).
Why this one
- Live status + redirect check built in — most extractors only list URLs; this can tell you which actually work.
- A real fallback for sites without a sitemap, so you still get URLs.
- Handles nested + gzipped sitemaps that trip up simpler tools.
Input
startUrls— the websites to extract.checkStatus— also fetch each URL and record its status + final URL.maxUrls— cap per site.
Output
One row per URL — exportable to JSON, CSV or Excel, or via the API.
Built by autoloom — small, fast web-data and SEO tools. New tools shipping regularly.
