Broken Link & Redirect Checker – Bulk URL Status

What does Broken Link & Redirect Checker do?

Broken Link & Redirect Checker is a bulk URL status checker and redirect chain checker. For every URL it reports the HTTP status, every redirect hop (status, Location and time), the final URL, the response time, the content type and an honest health classification. With SEO checks on (the default) it adds the canonical URL and meta robots / X-Robots-Tag: from the HTML <head> of web pages, and from the X-Robots-Tag and Link headers of any other file, such as a PDF.

You can feed it URLs in four ways:

• a pasted list, an uploaded text/CSV file or a hosted text file
• a redirect map (old URL → expected new URL) for site migrations
• sitemaps and sitemap indexes, including .xml.gz, plain-text sitemaps and robots.txt
• a crawl of one website from a start page, to find broken links and the pages they are on

Bot protection and rate limits are reported as unverified, never as broken links. A Cloudflare challenge, a 403 from a WAF or a 429 is not a dead page, so it does not flood your audit with false positives.

It reads the HTTP status, the headers and the HTML <head>. In crawl mode it also downloads the pages of your own site (up to 2 MB each) to find their links. It never stores page text (only the link text, if you switch on Crawl mode: save the link text), and it does not render JavaScript, bypass bot protection or use proxies.

Why use Broken Link & Redirect Checker?

• SEO audit: find 404s, long redirect chains, wrong canonicals and accidental noindex.
• Site migration QA: check an old → new redirect map before and after launch: a single 301, the right destination, an indexable target.
• Scheduled link monitoring: run it weekly on your sitemap and compare datasets.
• CMS and newsletter link checks: paste the links from an export and see which are dead.
• Developer and AI-agent API: "is this URL alive, and where does it end up?" in one call.

Features:

• Honest classification: health (healthy / broken / unverified / skipped) plus a precise outcome, and the bot-protection vendor in blockedBy.
• Full redirect chain per URL, also as one line for CSV/Excel, with an https_to_http_downgrade flag.
• .xml.gz sitemaps, sitemap indexes and robots.txt Sitemap: lines. A broken child sitemap becomes its own row and does not stop the run.
• Migration verdict (pass / warn / fail with reasons) for redirect maps.
• Multiple meta robots tags merged, X-Robots-Tag per bot, canonical from HTML or the Link header, hreflang count, meta refresh.
• HEAD mode for the fastest pure status check; HEAD errors are confirmed with a GET (rate limits excepted: a 429, or a 503 with Retry-After, is not asked again).
• Polite by design: 2 parallel requests per website by default, Retry-After respected, robots.txt obeyed in crawl mode, an honest User-Agent, no proxy costs.

Apify platform features come with it: scheduling, the API, integrations (Make, Zapier, Google Sheets and others), webhooks and run monitoring.

What data does Broken Link & Redirect Checker return?

One row per unique URL. The overview view shows the key fields:

Field	Example	Meaning
url	http://apify.com/	URL checked (normalised)
health	healthy	healthy / broken / unverified / skipped
outcome	redirect_ok	precise result class (table below)
statusCode	200	final HTTP status
redirectCount	1	number of redirect hops
finalUrl	https://apify.com/	URL after all redirects
responseTimeMs	255	total time to headers over all hops
contentType	text/html	media type of the final response
canonicalUrl	https://apify.com/	canonical from HTML or the Link header
indexable	true	2xx HTML, no noindex, canonical absent or self
issues	["multiple_redirects"]	findings such as temporary_redirect, canonical_mismatch, noindex, slow_response
foundOn	https://crawlee.dev/docs	crawl mode: first page that links here

Other fields include redirectChain, redirectChainText, firstStatusCode, serverIp, metaRobots, xRobotsTag, hreflangCount, expectedUrl, migrationVerdict, blockedBy, retryAfterSeconds, errorCode, source, sitemapUrl, foundOnCount, linkType and checkedAt.

Health vs. outcome

health is the column to filter on. outcome says exactly what happened:

outcome	health	When
ok	healthy	final 2xx, no redirect
redirect_ok	healthy	final 2xx after one or more redirects
broken_not_found	broken	404 or 410
broken_client_error	broken	other 4xx (400, 405, 451 …)
broken_server_error	broken	5xx without a bot-protection signature, after a retry
dns_not_found	broken	the domain does not exist
connection_refused	broken	nothing is listening
tls_error	broken	expired, self-signed or mismatched certificate
redirect_loop	broken	the chain comes back to a URL it already visited
too_many_redirects	broken	more hops than "Maximum redirects"
redirect_error	broken	a 3xx without a valid http(s) Location
sitemap_parse_error	broken	a sitemap file returned 2xx but is not a sitemap
auth_required	unverified	401 or 407
forbidden	unverified	403 without a bot-protection signature
blocked_bot_protection	unverified	Cloudflare, AWS WAF, Akamai, Imperva, DataDome, Sucuri or Vercel challenge, or the non-standard status 999 (blockedBy: linkedin when LinkedIn sends it)
rate_limited	unverified	429 after retries, or a 503 with Retry-After ("come back later")
timeout	unverified	no answer in time (after one retry), or 408
dns_error	unverified	temporary DNS failure
connection_failed	unverified	reset or unreachable network
internal_error	unverified	a bug on our side (please report it)
invalid_url	skipped	not an http(s) URL, or no domain (a single word such as intranet)
skipped_private_address	skipped	private, loopback or reserved network address
skipped_unsupported_port	skipped	a port other than the defaults, 8000, 8008, 8080, 8443 or 8888
skipped_robots_txt	skipped	crawl mode: disallowed by robots.txt
skipped_restricted_domain_limit	skipped	more than 100 URLs on one large platform in a run

How to check URLs in bulk

1. Open the Input tab and paste your URLs into URLs to check (or upload a text/CSV file).
2. Leave Check SEO tags on to get canonical and noindex, or turn it off for a faster status-only check.
3. Click Start.
4. Download the results as CSV or Excel and filter the health column for broken.

How to validate a site-migration redirect map

1. Paste two columns from your spreadsheet into Redirect map (old URL, expected new URL), for example http://apify.com https://apify.com/.
2. Start the run and open the Migration verdict view.
3. pass = one 301/308 to the expected, indexable URL. warn = it arrives, but with extra hops, a temporary redirect, noindex or a canonical mismatch. fail = wrong destination, an error page, or no redirect at all.

How to check a sitemap for broken links

1. Add your sitemap to Sitemap URLs, for example https://docs.apify.com/sitemap.xml. A sitemap index, .xml.gz or your robots.txt work too.
2. Set Maximum URLs to check to the size you want to pay for. It caps the whole run, across all inputs.
3. Start the run. Every sitemap file also gets its own row (source: sitemap_file, free of charge).

How to find broken links on a website (crawl)

1. Enter your home page in Crawl a website from this page, for example https://crawlee.dev/.
2. Choose Maximum pages to crawl and Maximum click depth.
3. Start the run. The crawler stays on your host, obeys robots.txt and checks every link it finds. If your start page redirects to another host (for example example.de → example.com), that host is crawled too. Open the Broken links (crawl) view: foundOn and foundOnPages show where each broken link sits. If no page could be crawled, the run's status message and SUMMARY.crawlNote say why.

How much does it cost to check URLs?

The Actor uses pay-per-event pricing. You pay for each URL that got a definite answer: a 200, a redirect, a 404 or another error status, bot protection, a domain that does not exist, a refused connection or a certificate error. Timeouts, temporary DNS or connection failures and rate limits are written to the dataset but are free:

Event	Free plan	Starter	Scale	Business
URL checked	$0.0008	$0.0006	$0.0005	$0.0004
Page crawled for links (crawl mode only)	$0.001	$0.0009	$0.0008	$0.0008
Actor start (once per run)	$0.00005	$0.00005	$0.00005	$0.00005

In plain numbers:

• 1,000 URLs cost $0.80 on the Free plan, $0.60 on Starter, $0.50 on Scale and $0.40 on Business, plus $0.00005 per run.
• The $5 monthly Free-plan credit covers about 6,000 URL checks.
• A crawl of 200 pages that finds 1,500 unique URLs in total: 1,500 × $0.0008 + 200 × $0.001 = $1.40 on the Free plan, $0.76 on Business.
• Free of charge: URLs without a definite answer (timeout, dns_error, connection_failed, rate_limited), skipped URLs (invalid, private address, unsupported port, robots.txt, platform cap), duplicates, sitemap-file rows and internal errors.
• Set Maximum cost per run in the run options to cap spending. The Actor stops cleanly when the cap is reached and keeps the results so far. In crawl mode it downloads only as many pages as the cap still pays for.

Input

See the Input tab for every option. The fast example input (it is also the prefill):

{
    "urls": [
        { "url": "http://apify.com" },
        { "url": "http://docs.apify.com/sdk/js" },
        { "url": "https://apify.com/this-page-does-not-exist" },
        { "url": "https://crawlee.dev/" }
    ]
}

Redirect map (site migration):

{
    "redirectMap": "http://apify.com https://apify.com/\nhttp://docs.apify.com/sdk/js https://docs.apify.com/sdk/js/docs/overview"
}

Sitemap:

{ "sitemapUrls": ["https://docs.apify.com/sitemap.xml"], "maxUrls": 200 }

Crawl:

{ "crawlStartUrl": "https://crawlee.dev/", "maxCrawlPages": 20, "maxUrls": 300 }

HEAD vs. GET (checkSeoTags). With SEO checks on (default), each hop is a GET and only the HTML <head> of the final page is read (at most 256 KB). With SEO checks off, each hop is a HEAD request, which is faster and transfers almost nothing; a HEAD answer of 400 or above is confirmed with a GET, because some servers answer HEAD with 404/405 while the page works (headStatusCode shows the HEAD answer). Rate limits are the exception: a 429, or a 503 with Retry-After, is not confirmed, so the busy server gets no extra requests.

URL format. Each entry in URLs to check must start with http:// or https://; the Input tab and the API reject other entries. The redirect map also accepts old URLs without a scheme and adds https:// (issue scheme_added). Duplicates are merged before checking. A user:password@ part is removed before checking (credentials are never sent) and never appears in the output: inputUrl shows it as ***@, and the row gets the issue credentials_removed. A sitemap entry that is not an absolute http(s):// URL becomes an invalid_url row with the entry in inputUrl.

Text and CSV files. An uploaded or linked file (up to 10 MB) can be a plain list or a CSV export, for example from Google Sheets. Every http(s):// URL in it is checked, in file order and from any column. A comma, whitespace or a quote ends a URL, so the other columns of a CSV row are ignored; a URL that itself contains a comma must be in double quotes, as CSV requires. Semicolon-separated CSV files with a header row work too.

Maximum URLs to check (maxUrls) is one budget for the whole run. URLs from the list, the redirect map, sitemaps and the crawl all count toward it, so it also caps the "URL checked" charges, also when the platform restarts the run. Sitemap-file rows are free and not counted. In crawl mode, the crawl gets whatever the other inputs left over.

Output

You can download the dataset as JSON, CSV, Excel, HTML or XML. Real rows from the example input (run on 2026-10-13):

[
    {
        "url": "http://apify.com/",
        "inputUrl": "http://apify.com",
        "health": "healthy",
        "outcome": "redirect_ok",
        "statusCode": 200,
        "firstStatusCode": 301,
        "finalUrl": "https://apify.com/",
        "redirectCount": 1,
        "redirectChain": [
            { "url": "http://apify.com/", "statusCode": 301, "location": "https://apify.com/", "responseTimeMs": 66, "method": "GET" }
        ],
        "redirectChainText": "301 http://apify.com/ → 200 https://apify.com/",
        "responseTimeMs": 255,
        "method": "GET",
        "contentType": "text/html",
        "canonicalUrl": "https://apify.com/",
        "canonicalSource": "html",
        "canonicalMismatch": false,
        "metaRobots": "index, follow",
        "noindex": false,
        "indexable": true,
        "issues": [],
        "source": "list",
        "checkedAt": "2026-10-13T00:34:06.771Z"
    },
    {
        "url": "https://apify.com/this-page-does-not-exist",
        "inputUrl": "https://apify.com/this-page-does-not-exist",
        "health": "broken",
        "outcome": "broken_not_found",
        "statusCode": 404,
        "firstStatusCode": 404,
        "finalUrl": "https://apify.com/this-page-does-not-exist",
        "redirectCount": 0,
        "redirectChain": [],
        "redirectChainText": "404 https://apify.com/this-page-does-not-exist",
        "responseTimeMs": 55,
        "method": "GET",
        "contentType": "text/html",
        "metaRobots": "noindex, follow",
        "noindex": true,
        "indexable": false,
        "issues": [],
        "source": "list",
        "checkedAt": "2026-10-13T00:34:06.451Z"
    }
]

(Fields that were null are left out above for brevity; every row always has every field.) For a file that is not HTML, such as a PDF, xRobotsTag, noindex, nofollow and canonicalUrl come from its response headers; metaRobots, hreflangCount and indexable stay null, because they describe HTML pages. With Check SEO tags off (HEAD mode) all SEO fields are null. The 404 page carries two meta robots tags, noindex and noindex,follow, which are merged into "noindex, follow". The second example URL, http://docs.apify.com/sdk/js, ends at https://docs.apify.com/sdk/js/docs/overview after two 301s and gets the issue multiple_redirects.

Dataset views: Overview, Redirect chains, SEO tags, Migration verdict and Broken links (crawl).

The key-value store record SUMMARY holds the run totals: stoppedReason (completed, max_urls_reached, charge_limit_reached, time_limit, aborted), rows, charged (url-checked, page-crawled; 0 when chargingActive is false, because nothing was billed), byHealth, byOutcome, duplicatesMerged, invalidInputs, notCheckedDueToMaxUrls, nonHttpLinksSkipped (counts of mailto/tel/javascript links, never the addresses), pagesCrawled (pages crawled for links, saved and charged as "page crawled"; a cost limit can stop a crawl before downloaded pages are saved), sitemapFiles and rowsBeforeRestart (rows already saved when the platform restarted the run; 0 normally).

Tips

• Turn off Check SEO tags for the fastest status-only check (HEAD requests).
• Keep Parallel requests per website low (1–2) for small sites. Hosts that answer 429 drop to 1 automatically.
• Many links to one big site (for example hundreds of GitHub links found by a crawl) are checked at the per-website rate, so they dominate the run time. A crawl of 50 pages with 500 links, 313 of them on one host, took about 3.5 minutes in our test. Turn off SEO checks or raise Parallel requests per website a little if that site can take it.
• Schedule the Actor weekly on your sitemap and compare the health column with the previous dataset.
• Split lists larger than 50,000 URLs into several runs.
• Run time depends on how many URLs share one website: requests to one website start at least 0.1 s apart (at most about 10 per second) and only a few run at once, so 50,000 URLs on a single website need well over an hour. Raise the run timeout in the run options (default 60 minutes) or split the list. A list spread over many websites runs much faster. The run stops cleanly before its timeout and keeps the results so far: about 30 seconds before it, or a quarter of the run time for runs shorter than two minutes (an API call with timeout=25 still checks its URLs).
• To stop a run early and keep its results, abort it gracefully (the graceful option in Console, or gracefully=true in the API). An immediate abort during the first phase of a crawl, while pages are still being read for links, ends the run before those pages are saved: they are neither in the dataset nor charged, and there is no SUMMARY.
• Use Accept-Language and the Mobile User-Agent to test language and mobile redirects.

FAQ

Is it legal to check links with this Actor?

It sends one request per redirect hop (plus a retry after a network error or a server error, and in HEAD mode a GET to confirm an error answer) and reads the HTTP status, the headers and the SEO tags in the HTML <head>. In crawl mode it downloads the pages of your own site to find their links. It never stores page text; link text is saved only if you switch it on. Check only URLs you are allowed to audit, such as your own site, your clients' sites or links you publish. Crawl mode obeys robots.txt (including Crawl-delay) and stays on your host.

The output is technical metadata about URLs. Link text is off by default because it can contain names of people, and Drop ?query keeps tokens or e-mail addresses in query strings out of crawl results. Your results could still contain personal data if the URLs you check contain it; such data is protected by the GDPR in the EU and by other laws elsewhere. Do not process personal data without a legitimate reason, and ask a lawyer if you are unsure.

Why "unverified" instead of "broken"?

Many sites protect themselves with Cloudflare or similar services. Requests from data centers then get a 403 challenge, a 429 or a timeout, even though the page works in a browser. Those rows are unverified, with the vendor in blockedBy, so you can review them separately instead of chasing false 404s.

Why can results differ from my browser?

The check runs from an Apify data center, not from your computer. Sites may redirect by country or language, block data-center traffic, or redirect with JavaScript (not followed; a meta refresh is reported in metaRefreshUrl). Cookies set during a redirect chain are kept for that chain only.

Does it check Amazon, LinkedIn and similar links?

Yes, one status check per URL, often reported as unverified because those platforms block automated requests. At most 100 URLs per platform domain are checked in one run (the rest are skipped for free), and crawling those sites is not supported.

Can it check private or intranet URLs?

No. Private, loopback and reserved network addresses (including cloud metadata addresses) are refused for security, also when a redirect points there. Such rows are skipped_private_address and free.

Does it use proxies?

No. Requests leave from the Apify server's own IP address with an honest User-Agent. There is no CAPTCHA solving and no retry with a different identity.

What are the limits?

Up to 50,000 URLs per run, up to 5,000 crawled pages, up to 20 redirects per URL, 50 MB / 50,000 entries per sitemap file and 500 child sitemaps per run. Large runs on a single website are limited by its request rate (see Tips), not by these numbers.

How do I schedule link monitoring?

Save your input as a task, then add a schedule (for example weekly) in Apify Console. Use a webhook or an integration to send the results to Google Sheets, Slack or your own system.

Which URLs am I charged for?

For every URL that got a definite answer (healthy, broken, or blocked by bot protection) and was written to the dataset. Timeouts, temporary DNS or connection failures and rate limits are written but free, and so are skipped, invalid and duplicate URLs, sitemap-file rows and internal errors. In crawl mode, each page of your site downloaded to find links and saved is also charged as "page crawled". SUMMARY.charged shows the counts of the run.

If the platform restarts a run (rare, for example when it moves the run to another server), the rows already saved are read back from the dataset: none is written, charged or checked again, and Maximum URLs to check and your cost limit still apply to the whole run.

Where do I report a problem?

Open the Issues tab. Use the API tab to run the Actor from your own code.

Changelog

• 1.0.8 (2026-10-13): a run that the platform restarts (a migration, a reboot or a crash) reads back the rows it already saved, so no URL is written, charged or checked twice and maxUrls and the cost limit hold for the whole run. A short run timeout (for example timeout=25 from the API) no longer stops the run before it reads its input: the stop before the timeout scales with the run time. In crawl mode with a cost limit, only as many pages are downloaded as the limit pays for, the last page the limit still covers is no longer left out, and SUMMARY.pagesCrawled counts the pages saved and charged. After a graceful abort the run's status message shows the final counts. Non-HTML files (PDFs, images) get xRobotsTag, noindex and the Link-header canonical from their response headers.
• 1.0.7 (2026-10-13): an abort during the crawl now always saves every crawled page and writes SUMMARY with stoppedReason: "aborted". Crawled pages are saved in batches (faster on large crawls). Credentials in URLs are removed from the output. A page whose body stops arriving is flagged (head_truncated / page_truncated with an explanation). HEAD mode sends one request less to a server that keeps answering 5xx. Clearer messages for unreadable sitemaps and blocked ports.
• 1.0 (2026-10-13): first release. List, redirect map, sitemap and crawl modes; honest health classification; migration verdict; SEO tags. README examples were re-run on 2026-10-13.