Bulk URL Status & Redirect Chain Checker

What does it do?

Paste a list of URLs and get one clean result per URL: the final HTTP status, the full redirect chain (every hop, each marked permanent or temporary), redirect loops, the final URL, a likely soft-404 flag, the response time and the content type. It checks only the URLs you give it and never crawls. No API key, no login, no proxies.

Who is it for?

SEO teams checking redirects after a site migration, people cleaning up link lists, backlink or sitemap exports, and AI agents that need to know whether a set of URLs still works and where they end up.

Input

Field	Default	What it does
urls (required)	prefill: 10 sample URLs	One URL per line. Bare domains (example.com) are checked as https://example.com/. Duplicates are removed after normalisation (#fragments dropped).
maxRedirects	10	Redirect hops followed per URL before it counts as a loop.
method	HEAD-then-GET	HEAD-then-GET sends a light HEAD request per hop and falls back to GET when a server refuses HEAD (405, 403 or 501). GET uses GET on every hop and reads the first 16 KB of 2xx HTML pages, in memory only, so the text-based soft-404 check runs on every page.
timeoutSecs	15	Seconds to wait for each request's response headers.
maxUrls	1000	Cap on unique URLs checked. You pay per URL, so this also caps cost.
respectRobotsTxt	true	Can be turned off only for your own sites, together with ownSiteConfirmed.
ownSiteConfirmed	false	Tick only for sites you own or are authorised to check.

Example input:

{
  "urls": [
    "crawlee.dev",
    "https://crawlee.dev/docs/quick-start",
    "https://crawlee.dev/this-page-does-not-exist",
    "https://python.org/",
    "https://developer.mozilla.org/docs/Web/HTTP/Status"
  ],
  "method": "HEAD-then-GET",
  "maxRedirects": 10
}

Output

One item per unique URL. category is one of:

• ok: a 2xx answer with no redirect.
• redirect: one redirect, then a 2xx page.
• redirect-chain: two or more redirects, then a 2xx page.
• broken: a 4xx or 5xx answer, or the connection failed.
• loop: a redirect loop, or more than maxRedirects hops.
• soft-404: likely a "not found" page served with a 2xx status. This is a heuristic (see Limits).
• dns-error: the host name doesn't exist. The name is looked up before anything is requested, robots.txt included, so a dead domain is reported as dns-error (charged) and never as robots-blocked. A temporary DNS failure (the resolver didn't answer) is reported as timeout (free).
• blocked-by-site: HTTP 401, 403 or 999. The site refused the automated check (bot protection or a login wall), which says nothing about whether the page exists.
• rate-limited: HTTP 429 after retries, or not requested because the host answered 429 three times in a row.
• timeout: no answer within timeoutSecs.
• blocked-by-robots: the host's robots.txt disallows the URL, so it wasn't requested.
• blocked-private: the URL points at, or redirects to, localhost or a private network address, so it wasn't requested.
• invalid-url: the entry isn't a valid http(s) URL or domain.

Every hop in redirectChain has url, status, type (permanent for 301/308, temporary for 302/303/307, null for the final answer) and ms. Items that weren't requested or got no normal answer carry an errorCode.

Real item from a local test run on the default prefill (2 Nov 2026), with the timestamp trimmed:

{
  "input": "https://developer.mozilla.org/docs/Web/HTTP/Status",
  "url": "https://developer.mozilla.org/docs/Web/HTTP/Status",
  "category": "redirect-chain",
  "ok": true,
  "status": 200,
  "finalUrl": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status",
  "hops": 2,
  "redirectType": "mixed",
  "redirectChain": [
    { "url": "https://developer.mozilla.org/docs/Web/HTTP/Status", "status": 302, "type": "temporary", "ms": 13 },
    { "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Status", "status": 301, "type": "permanent", "ms": 69 },
    { "url": "https://developer.mozilla.org/en-US/docs/Web/HTTP/Reference/Status", "status": 200, "type": null, "ms": 41 }
  ],
  "loop": false,
  "likelySoft404": false,
  "soft404Reason": null,
  "soft404Check": "redirect-only",
  "responseTimeMs": 123,
  "contentType": "text/html",
  "method": "HEAD",
  "reason": "2 redirects (mixed)",
  "error": null,
  "errorCode": null,
  "chargeable": true
}

The run also saves an OUTPUT_SUMMARY record in the default key-value store: input count, unique URLs, duplicates removed, invalid entries, URLs checked and not checked, counts per category, events charged and when the run finished.

How it stays polite

• It requests only the URLs you list, plus their redirect hops and each host's robots.txt. It never crawls.
• About 1 request per second per host, or longer if robots.txt sets a Crawl-delay (honoured up to 30 seconds). Up to 10 different hosts are checked at the same time, and each host still gets about 1 request per second.
• It backs off on HTTP 429 and 503 and honours Retry-After (up to 30 seconds). A host that answers 429 three times in a row gets no more requests in that run.
• robots.txt is respected (RFC 9309). If a host's robots.txt returns a server error or can't be reached, the whole host is treated as off-limits. A missing robots.txt (4xx) means everything is allowed. For your own sites only, ownSiteConfirmed lets you turn robots.txt off. Redirect hops to other hosts always keep their robots.txt.
• Localhost and private network addresses are never requested, including as a redirect target.
• It identifies itself as BulkUrlStatusChecker/0.1 (+Apify Actor; polite, robots-aware). No proxies, no logins, and no attempt to get past bot protection or CAPTCHAs.
• It stores no page content. In GET mode, the first 16 KB of a 2xx HTML page is read in memory for the soft-404 check and then discarded.

How long does it take?

The default prefill (10 URLs on 8 hosts) finished in about 7 seconds in a local test run, with a peak of about 116 MB of memory. Many hosts are checked in parallel, but each host gets only about 1 request per second, so a list dominated by one host is slow: 1,000 URLs on one host take roughly 17 minutes. The default run timeout is 300 seconds, which fits about 250 URLs on a single host. Raise the run timeout or split the list for more. The run stops cleanly shortly before its timeout, keeps everything already checked, and reports the rest as not checked in OUTPUT_SUMMARY.

How much does it cost?

Pay per event: one url-checked event per unique URL with a definitive result (a final status, a redirect chain, a loop, a likely soft-404 or a DNS failure). These are free: timeouts, rate-limited URLs, blocked-by-site, robots-blocked URLs, private addresses and invalid entries. Duplicates are removed before checking, so you never pay twice for the same URL in a run. Price: $0.80 per 1,000 URLs checked ($0.0008 per url-checked event). maxUrls and your maximum charge per run both cap the total.

Use with AI agents

Designed for limited permissions, so it can be called through the Apify MCP server. Example: "Check these 200 old URLs and tell me which ones don't redirect to a working page."

Limits

• Soft-404 detection is a heuristic. It flags 2xx pages whose title, heading or short text says "not found", and deep URLs that redirect to the homepage. In the default HEAD-then-GET mode only the redirect rule runs, unless the server refused HEAD; soft404Check on each item says which checks ran. Use method: "GET" for the full check. It can be wrong in both directions, so open flagged URLs in a browser.
• A 401, 403 or 999 doesn't mean the URL is dead. Many sites block automated requests. These are blocked-by-site, free, and worth checking in a browser.
• Rate-limited and timed-out URLs aren't checked. Re-run them later or open them in a browser.
• Some servers answer HEAD differently from GET. If a result looks wrong, re-check it with method: "GET".
• responseTimeMs is measured from Apify's servers to the response headers, summed over the redirect hops. It isn't a page-speed or Core Web Vitals measurement.

FAQ

Does it follow links on the pages? No. It checks only the URLs you give it. To find broken links across a whole site, use a link checker that crawls. What's the difference between redirect and redirect-chain? One redirect hop versus two or more. redirectType shows whether the hops were permanent, temporary or mixed. Does http:// get upgraded to https://? No. A URL with http:// is checked as given, so you can see its redirect to HTTPS. Only entries without a scheme get https://.

Legal

Checks public URLs only. Stores status codes, redirect information, timings and the Content-Type header, not page content or personal data.

Changelog

See CHANGELOG.md.