Broken Link & Sitemap Checker: HTTP Status, 404 & Redirects

Find broken and redirected URLs in your sitemap. Give it your site or sitemap. It reads robots.txt and the sitemap (including sitemap indexes and .xml.gz), then checks every listed URL, up to 5,000 per run. For each one it reports:

• the HTTP status;
• the redirect chain;
• a noindex header, if set;
• lastmod.

Scope: it checks the URLs listed in your sitemap. It does not crawl your pages to follow the links inside them.

Try it free first

Run it with empty input. It runs a free demo on 20 URLs of apify.com, which isn't billed, so you can see the output format.

Broken link checker for your sitemap: HTTP status codes, 404s and redirects it flags

Flag	Meaning
not_found / client_error / server_error	The URL answered 404/410, another 4xx, or 5xx
redirected / redirect_chain / too_many_redirects	The URL redirects once, more than once, or more than 5 times
https_downgrade	An https URL redirects to http
noindex_header	The response sends X-Robots-Tag: noindex
blocked_by_robots	Your robots.txt disallows the URL for this checker, so it was skipped and not billed
timeout, network_error, blocked_private_address	The URL couldn't be checked (not billed)
ok	2xx with no redirect

How to use it as a sitemap checker

1. Enter your sites (e.g. example.com) or sitemap URLs.
2. Tick I own these sites or have permission to check them. Runs without it are refused.
3. Optionally set Maximum URLs (up to 5,000) and Only output URLs with problems.
4. Schedule it weekly to catch new broken URLs.

Output: one dataset item per URL, with url, status, flags, redirects, redirectStatuses, finalUrl, lastmod and sitemap. A SUMMARY record holds the counts. Page content is never downloaded or stored.

Polite by design

• robots.txt: honoured, including Crawl-delay.
• Pacing: each site gets at most one request at a time, at least 500 ms apart.
• Request cost: HEAD requests are used where possible.
• Scope: only URLs on the same site as the sitemap are checked.
• What it never sends: cookies or credentials. It also never tries to get around anti-bot protection.
• Addresses: private and internal network addresses are refused.
• Identification: the user agent is SitemapURLHealthChecker/1.0, with a link to this page.

Site owners:

• To opt out: add User-agent: SitemapURLHealthChecker and Disallow: / to your robots.txt. The checker honours it on the next request.
• To report misuse: open an issue on this Actor's Issues tab. Reports are reviewed within one business day, and reported sites can be added to a permanent block list.

Pricing

• Pay per event: each URL that returns an HTTP status is one url-checked event.
• Not billed: skipped, blocked or failed URLs, and the demo.
• The price is shown on the Pricing tab. You can cap the maximum charge per run in Apify's run options.

Terms of use and limits

• Only check sites you own or have permission to check. This isn't a load-testing tool.
• Limits per run:
  • 5,000 URLs;
  • 50 sitemap files, nested to a depth of 3;
  • 50 MB per sitemap;
  • 15 s timeout per request;
  • a time limit you set.
• Results reflect what the server returned to this checker at run time. Some sites answer automated requests differently.

About

• Independent: made by an independent developer. Unofficial; not affiliated with or endorsed by Apify.
• Built with AI: built and maintained with the help of AI tools, under the developer's direction.