Sitemap Extractor, URL Status and Broken Link Checker

What does Sitemap Extractor, URL Status and Broken Link Checker do?

It gets every URL from a website's sitemaps and tells you which ones work. Give it a website or a sitemap, and for every URL you get the sitemap data (last modified, priority, images, language versions) and a check: status code, redirect chain, response time, page title, canonical URL and noindex. Turn on Find broken links and it also checks the links on every page and tells you where each broken link is.

Use it to:

• Audit a sitemap. Find URLs in your sitemap that are broken, redirect, are marked noindex or point their canonical to another page. Search engines treat those as mistakes.
• Find broken links. See every link on your site that leads to a 404, a dead domain or a server error, with the pages and link texts to fix.
• Get a full list of a site's pages in seconds, for a migration, a crawl or a content inventory.
• Check a list of URLs in bulk for status codes and redirects, without a sitemap.

Which sitemaps does it read?

• The sitemaps listed in the site's robots.txt, or the usual places when there are none: /sitemap.xml, /sitemap_index.xml, /sitemap-index.xml, /wp-sitemap.xml and /sitemap.txt.
• Sitemap indexes (sitemaps that list other sitemaps), followed all the way down.
• Gzipped sitemaps (.xml.gz), plain-text sitemaps (one URL per line) and RSS or Atom feeds used as sitemaps.
• Image, video and hreflang (language version) entries.

It works with WordPress, Shopify, Wix, Squarespace, Webflow and any other system that writes standard sitemaps.

What data do you get?

Field	Example
URL and where it was found	sitemap, your list, or a link on a page
Sitemap data	last modified, change frequency, priority, image and video count, hreflang versions
Status code	200, 301, 404, 503
Redirects	the full chain, like 307 > 308 > 200, and where it ends
Response time	in milliseconds
Page info	title, canonical URL, noindex (from the robots meta tag or the X-Robots-Tag header)
Issues	broken, soft_404, blocked, redirect, redirect_chain, noindex, canonical_elsewhere, slow, not_checked_robots_txt
For links	the pages that link to it, with the link text

What do the issues mean?

• broken: the URL doesn't work: 404, 410, a 5xx server error, a timeout or a domain that doesn't exist.
• soft_404: the server answers 200 OK, but the page says it doesn't exist ("Page not found", "Siden finnes ikke" and the like). Search engines report these as soft 404s.
• blocked: the server refused an automated visitor (401, 403 or 429). The page may work fine in a browser, so it isn't counted as broken.
• redirect / redirect_chain: the URL sends you somewhere else, once or several times. Links and sitemaps should point to the final URL.
• noindex: a sitemap URL that asks search engines not to index it.
• canonical_elsewhere: a sitemap URL whose canonical tag points to another page.
• slow: the answer took more than 5 seconds.
• not_checked_robots_txt: the site's robots.txt asks automated tools not to visit this URL, so it was not checked.

How to use it

1. Click Try for free at the top of this page.
2. Add a website (example.com) or a sitemap URL under Websites or sitemaps. Or paste URLs under URLs to check.
3. Choose what to check. For a quick list of URLs, turn off Check each URL. For a broken link report, turn on Find broken links on the pages.
4. Optional: save only URLs with issues, or only URLs that contain /blog/.
5. Click Start, then download the results as Excel, CSV or JSON, or use the API.

Input examples

Sitemap audit, saving only problems:

{
    "startUrls": ["https://www.example.com"],
    "onlyIssues": true
}

Broken link check of the blog, links within the site only:

{
    "startUrls": ["https://www.example.com/sitemap.xml"],
    "includePatterns": ["/blog/"],
    "findBrokenLinks": true,
    "checkExternalLinks": false
}

All URLs in the sitemaps, without checking them:

{
    "startUrls": ["https://www.example.com"],
    "checkUrls": false
}

Output example

A sitemap URL that redirects:

{
    "url": "https://vipps.no/nyheter",
    "source": "sitemap",
    "sitemap": "https://vipps.no/sitemap.xml",
    "lastmod": "2026-07-21T06:08:42.000Z",
    "changefreq": null,
    "priority": null,
    "imageCount": 0,
    "videoCount": 0,
    "hreflang": [],
    "checked": true,
    "statusCode": 200,
    "ok": true,
    "finalUrl": "https://vipps.no/news",
    "redirectCount": 1,
    "redirectChain": [
        { "url": "https://vipps.no/nyheter", "status": 307 },
        { "url": "https://vipps.no/news", "status": 200 }
    ],
    "responseTimeMs": 175,
    "contentType": "text/html",
    "title": "Nå kan du betale med Vipps i Spond | Vipps",
    "canonical": null,
    "noindex": false,
    "error": null,
    "issues": ["redirect"],
    "foundOn": null,
    "checkedAt": "2026-09-25T12:00:00.000Z"
}

A broken link found on a page (shortened):

{
    "url": "https://help.vippsmobilepay.com/nb-NO/categories/children-profile",
    "source": "link",
    "statusCode": 404,
    "issues": ["broken"],
    "foundOn": [
        { "page": "https://vipps.no/under-15", "anchor": "Se ofte stilte spørsmål" }
    ]
}

A RUN_SUMMARY record in the key-value store shows the sitemaps found and read, the number of URLs checked, the count of each issue, and the number of links checked and broken.

How much does it cost?

You pay $1 per 1,000 URLs checked. Server time is included.

• A site with 500 URLs: $0.50.
• A site with 20,000 URLs: $20.
• Listing the URLs in the sitemaps without checking them costs the same per URL.
• With Find broken links, every link that is checked counts as one URL too. A 200-page site whose pages link to 800 other URLs costs $1.
• URLs that robots.txt asks us not to visit are not checked and not charged.

Only save URLs with issues and Links to save only filter what you get back, so your results stay short. They don't change the price. The $5 in free monthly credit on the Apify free plan covers about 5,000 URLs a month. Set Max URLs, the URL filters or a maximum cost per run for a hard budget: the run stops before it goes over it.

FAQ

Is it safe to run on my website?

Yes. By default it sends at most 4 requests at a time to the same website, always respects a Crawl-delay in robots.txt, and visits each URL once. You can lower Max requests at a time per website to 1 for small servers.

Does it respect robots.txt?

Yes. URLs that robots.txt asks automated tools to avoid are not visited and are marked not_checked_robots_txt. It identifies itself honestly as VindR-SitemapChecker, so you can allow or block it in your robots.txt.

Why are some pages "blocked" and not "broken"?

Some websites answer automated visitors with 403 or 429, often through bot protection like Cloudflare, even though the page works in a browser. The Actor doesn't try to get around that, and marks those URLs as blocked so you don't chase problems that aren't there.

Does it find links added by JavaScript?

No. It reads the HTML the server sends, like a search engine's first pass does. Links that only appear after JavaScript runs in a browser are not found. For most websites, the navigation and content links are in the HTML.

What if a website has no sitemap?

Then you'll see a warning in the log, and the site is listed under sitesWithoutSitemap in the run summary. You can still check URLs by pasting them under URLs to check.

How big a site can it handle?

Sitemaps with hundreds of thousands of URLs are fine. Checking takes as long as the website needs to answer: with 4 requests at a time and a fast site, around 1,000 URLs a minute. Some platforms, Shopify for example, only allow a few automated requests a second. When a site answers "too many requests" (HTTP 429), the Actor waits as long as the site asks, slows down for that site and tries again, so you get real results instead of errors. On such sites, expect around 1,000 URLs in 20 minutes. Use the URL filters or Max URLs to check part of a large site.

Can I use it from my own code?

Yes. Every Apify Actor has an API. See the API tab for examples in Python, JavaScript and plain HTTP. It also works with Make, n8n, Zapier and Google Sheets, and you can schedule it to run every week.

Something missing or not working?

Open an issue on the Issues tab with the website you tried.

More tools from VindR

• Website Tech Stack Detector: what a website is built with.
• Bulk WHOIS Lookup: domain registration, expiry and SSL for many domains.
• PDF Text Extractor: the text of the PDFs a website publishes.