Sitemap URL Extractor & Sitemap Checker (404s, Redirects)
Pricing
from $0.20 / 1,000 url extracteds
Sitemap URL Extractor & Sitemap Checker (404s, Redirects)
Get every URL from a website's sitemaps (found automatically via robots.txt, indexes followed), with lastmod dates, and optionally check each one's HTTP status to find the 404s, redirects and server errors listed in your sitemap. About $1 per 1,000 URLs checked.
Pricing
from $0.20 / 1,000 url extracteds
Rating
0.0
(0)
Developer
Zahid Thani
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
3 days ago
Last modified
Categories
Features
Share
Paste a domain and get every URL in its sitemaps, with last-modified dates. Sitemaps are found automatically through robots.txt and the usual locations, and sitemap indexes are followed. Turn on the status check to find the 404s, redirects and server errors hiding in your sitemap: the pages you're asking Google to index that don't work.
What it costs, in plain terms: listing 1,000 URLs costs about 20 cents. Listing and checking 1,000 URLs costs about $1. Sites without a sitemap are free. The form opens pre-filled, so your first run works with one click on Start and costs about 8 cents (list only). To see the checker, turn on Check each URL's status before you start.
Sample output (real run, 9 November 2026)
Status check on the wordpress.org news sitemap, first 150 URLs: 146 OK, 4 redirects, 0 broken. The 4 redirects are attachment pages that forward to video files, which shouldn't be in a sitemap. Two of them:
| url | lastmodDate | checkResult | statusCode | finalUrl | responseTimeMs |
|---|---|---|---|---|---|
| wordpress.org/news/toc/ | 2023-06-27 | redirect | 200 | wordpress.org/news/files/2023/06/TOC.mov | 112 |
| wordpress.org/news/showcase-trailer-i2_1/ | 2023-10-18 | redirect | 200 | wordpress.org/news/files/2023/10/showcase-trailer-i2_1.mp4 | 108 |
statusCode is the status of the final page after redirects. With Only output problem URLs on, these would be the
only rows you get.
What it checks, honestly: the URLs listed in your sitemaps. It doesn't crawl your pages to follow every link on them. For a full-site crawl of internal and external links, use a crawler-based broken link checker; use this one to make sure your sitemap only lists live, final URLs, and to export a site's URL list fast.
What can this sitemap extractor do?
- 🗺️ Finds the sitemaps for you: reads
Sitemap:lines in robots.txt, then tries/sitemap.xml,/sitemap_index.xml,/wp-sitemap.xml,/sitemap.txtand more. Or paste a sitemap URL directly. - 🪆 Follows sitemap indexes (up to 4 levels and 1,000 sitemaps per site), including
.xml.gzfiles, plain-text sitemaps, and RSS/Atom feeds. - 🧾 One row per URL with
lastmod,changefreq,priority, image and hreflang counts, and the sitemap it came from. - 🩺 Optional status check for every URL: status code, redirect chain, final URL, response time and
X-Robots-Tag: noindex. Uses lightweight HEAD requests (falls back to GET headers when a server refuses HEAD), so pages aren't downloaded. - 🧰 Forgiving parser: a stray
&or a missing closing tag doesn't lose the sitemap. URLs are recovered and the sitemap is flagged as invalid XML in the run summary. - 🔍 Filters: only URLs changed since a date (find new or updated pages), only URLs containing
/blog/, skip/tag/... - 🤝 Gentle by default: at most 4 requests at a time per website, robots.txt respected for status checks.
Who uses it
- SEO audits: find 404s, redirects and noindexed pages listed in a sitemap, so the sitemap only lists pages you want indexed.
- Site migrations: export every old URL before a move, then check that each one redirects to the right place afterwards.
- Content monitoring: schedule it weekly with Only URLs changed on or after to see a competitor's new pages and posts.
- Crawling and AI pipelines: get a clean URL list to feed into a scraper, a search index or a RAG pipeline.
How to use it
- Click Try for free.
- Paste domains (
example.com) or sitemap URLs (example.com/sitemap.xml), one per line. - Turn on Check each URL's status if you want broken links and redirects. Add Only output problem URLs to see only what needs fixing.
- Click Start. A 500-URL site takes seconds to extract, about a minute or two with status checks.
- Open the URLs table or export CSV/Excel/JSON. Per-site totals are in the summary record (Output tab).
Input example
{"websites": ["example.com", "https://www.example.org/sitemap_index.xml"],"checkStatus": true,"onlyProblems": true,"maxUrlsPerSite": 5000}
Output example (every field)
The values below are illustrative; the sample table at the top is from a real run.
{"type": "url","site": "example.com","url": "https://example.com/old-pricing","lastmod": "2025-03-14T09:12:00+00:00","lastmodDate": "2025-03-14","changefreq": "monthly","priority": "0.6","sitemap": "https://example.com/sitemap-pages.xml","statusCode": 200,"checkResult": "redirect","redirects": 1,"redirectChain": [{"url": "https://example.com/old-pricing", "status": 301}],"finalUrl": "https://example.com/pricing","responseTimeMs": 184,"contentType": "text/html"}
checkResult is one of ok, redirect, broken (404/410), restricted (401/403), client-error, server-error,
rate-limited, redirect-loop, timeout, unreachable or skipped-robots-txt.
The run's summary (key-value store record OUTPUT) lists, per site: where the sitemap was found, every sitemap
read, sitemaps that failed (with the reason), URL totals, whether the max was reached, and counts per checkResult.
How much does it cost?
| Event | Price | Per 1,000 |
|---|---|---|
| URL extracted | $0.0002 | $0.20 |
| URL status checked | $0.0008 | $0.80 |
| Run start | $0.0005 | - |
| Job | Cost |
|---|---|
| 1,000 URLs, list only | about $0.20 |
| 1,000 URLs with status checks | about $1.00 |
| 10,000 URLs with status checks, only problems output (say 300) | about $8.06 |
Sites with no readable sitemap, invalid inputs and robots.txt-skipped checks are free. With Only output problem URLs, every URL is still checked (that's the work) but healthy URLs aren't charged as extracted. Set Maximum cost per run and the Actor stops cleanly at your budget.
Try it free: a new Apify account costs nothing, needs no credit card, and includes $5 of usage every month. That covers about 5,000 URLs checked, or 25,000 URLs listed.
How it compares
| This Actor | Google Search Console | Desktop crawlers (Screaming Frog and similar) | Crawler-based broken link checkers | |
|---|---|---|---|---|
| Finds the sitemaps for you (robots.txt, indexes, .gz) | Yes | You submit them | Yes | Usually not |
| Checks every sitemap URL's status and redirect target | Yes | Partly, with a delay | Yes | Only links they crawl |
| Sites you don't own (competitors, clients) | Yes | No, verified sites only | Yes | Yes |
| Finds broken links inside page content | No | No | Yes | Yes |
| Runs in the cloud on a schedule, API and MCP | Yes | No | No (desktop app) | Varies |
| Cost | about $1 per 1,000 URLs checked | Free | Free up to 500 URLs, then a yearly licence | Varies |
Use it with AI agents (MCP) and the API
Connect it to Claude, ChatGPT, Cursor or any MCP client via
https://mcp.apify.com?tools=zahidthani/sitemap-url-extractor-status-checker and ask, for example:
List every blog post on example.com updated since October, and tell me which ones redirect.
curl -X POST "https://api.apify.com/v2/acts/zahidthani~sitemap-url-extractor-status-checker/run-sync-get-dataset-items" \-H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \-d '{"websites": ["example.com"], "urlContains": ["/blog/"]}'
Limits and tips
- A sitemap lists the pages a site wants indexed. Pages missing from the sitemap won't appear. To find every link on a site, you need a crawler instead.
- Status checks request each URL once. Sites behind bot protection may answer
403(restricted) to cloud servers even when the page works in a browser. Try the proxy setting for those. - Robots.txt rules apply to status checks: disallowed URLs are output as
skipped-robots-txtand not charged as checked. You can turn this off for your own site. - Max URLs per site is 1,000 by default. For big sites, raise it (up to 200,000). The summary flags a site that hit the limit.
FAQ
How do I find a website's sitemap? Just paste the domain. The Actor reads robots.txt and tries the common sitemap locations. The summary shows where it was found.
How do I find broken links in my sitemap? Turn on Check each URL's status and Only output problem URLs. Every 404, redirect and server error is listed with the sitemap it came from.
What will my first run cost? The pre-filled run lists up to 200 URLs from each of 2 sites, about 8 cents. With the status check on, it's about 40 cents. After that: $0.0002 per URL listed, $0.0008 per URL checked, $0.0005 per run, and no monthly fee. Set Maximum cost per run to cap any run.
Do I need an API key or a credit card? No. A free Apify account is enough, and it includes $5 of usage a month.
Can I get only new pages? Set Only URLs changed on or after to a date. It relies on the site's lastmod
values, which most CMSs (WordPress, Shopify, Wix...) fill in.
Is it legal? Sitemaps are files websites publish so that crawlers can find their pages. The Actor reads them and, if you ask, makes one lightweight request per URL, respecting robots.txt. No login, no personal data.
Something broke? Open an issue on the Issues tab.
Related Actors
- Website Tech Stack Detector: CMS, e-commerce platform, analytics, hosting and email provider for any list of domains.
- Domain Expiry, SSL & DMARC Checker: domain and SSL certificate expiry, plus SPF, DMARC and DKIM, for a list of domains.
- Google Trends Scraper: search demand for any keyword or brand.
Want it done for you?
This Actor is built and maintained by Zahid Gulgee Thani. If you would rather not run it yourself, I can set up scheduled exports into Google Sheets or Excel, a dashboard, or a custom scraper for your use case. Free how-to guides and ways to get in touch are on zahidgulgeethani.com.