Bulk URL Status & Redirect Checker

Paste a list of URLs. For each one you get the HTTP status, the full redirect chain (every hop's status and URL), the final URL, the response time, and a plain-language error when the check cannot finish.

This Actor does not crawl a site, does not read sitemaps, and does not extract links. It checks only the URLs you give it.

What you get

• Status and final URL after HTTP redirects (301, 302, 303, 307, 308, and 300 when a Location header is present).
• The chain. Each hop is the URL that was requested and the status that came back. redirectChainText is the same chain in one cell for CSV. JavaScript and <meta refresh> redirects are not followed.
• Response time for the whole chain, including retries.
• Flags: httpsDowngrade when a later hop is http after an https hop, and finalHostMatchesStart when the final host is the host you started with. A host change is not by itself an open redirect; shorteners and CDNs look the same.
• Errors for DNS failure, TLS failure, timeouts, connection failure, redirect loops, hop limits, invalid URLs, and addresses this Actor refuses to contact.

The default dataset view is a table. The Flat table view has no nested columns. Export it with ?view=flat&format=csv. A SUMMARY record in the key-value store has the totals, including how many URLs were not charged.

Quick start

1. Open Try for free or Start.
2. Paste URLs into URL list, or add them under URLs. The form starts with https://example.com/ and a max of 100.
3. Start the run. Open Output, or the CSV link, and the SUMMARY record.

Input examples

1. A short list (10 URLs maximum, about $0.015 if every URL is accepted):

{
  "urls": ["https://example.com/", "https://example.com/missing"],
  "maxUrls": 10
}

2. A pasted list, following redirects:

{
  "urlText": "https://example.com/old\nhttps://example.com/missing\n# comment lines are ignored\n",
  "maxUrls": 100,
  "method": "GET",
  "respectRobotsTxt": true,
  "maxRedirectHops": 10
}

3. HEAD only, robots off, tight limits for a list you are allowed to request:

{
  "urls": ["https://example.com/status"],
  "method": "HEAD",
  "respectRobotsTxt": false,
  "maxConcurrency": 2,
  "perHostConcurrency": 1,
  "maxRetries": 0,
  "maxResponseBytes": 0,
  "requestTimeoutSecs": 10,
  "urlTimeoutSecs": 20
}

Field	Default	What it does
urls	–	URL strings. Checked before urlText.
urlText	empty	One URL per line. Blank lines and # comments are ignored.
maxUrls	1000 (form prefill 100)	Cap on unique accepted URLs, and on the url-checked cost. Maximum 10,000.
method	GET	HEAD skips the body. Some servers answer HEAD with a different status.
maxRedirectHops	10	Redirects to follow (0–20). 0 keeps the first status.
requestTimeoutSecs	20	Timeout for one hop (3–60). Connect waits at most 10 seconds.
urlTimeoutSecs	60	Wall-clock limit for one URL, hops and retries included (5–180).
maxRetries	1	Extra tries for timeouts, connection failures, and HTTP 408/429/502/503/504. Maximum 2.
maxResponseBytes	65536	Stop reading a body after this many bytes (maximum 1 MiB). The body is not saved.
maxConcurrency	8	Parallel URL checks (1–16).
perHostConcurrency	2	Parallel checks of the same host (1–4).
respectRobotsTxt	true	Skip a URL when robots.txt disallows BulkUrlChecker.

Example output

These rows are the shape the Actor writes. They are not from a live Store run.

{
  "inputIndex": 1,
  "inputUrl": "https://start.example/old",
  "normalizedUrl": "https://start.example/old",
  "result": "ok",
  "statusCode": 200,
  "finalUrl": "https://start.example/new",
  "redirectCount": 1,
  "redirectChain": [
    {"hop": 0, "url": "https://start.example/old", "statusCode": 301, "location": "https://start.example/new", "contentType": "text/plain", "elapsedMs": 5},
    {"hop": 1, "url": "https://start.example/new", "statusCode": 200, "location": null, "contentType": "text/html", "elapsedMs": 4}
  ],
  "redirectChainText": "301 https://start.example/old -> 200 https://start.example/new",
  "responseTimeMs": 12,
  "error": null,
  "errorType": null,
  "httpsDowngrade": false,
  "finalHostMatchesStart": true,
  "blockedByRobots": false,
  "contentType": "text/html",
  "method": "GET",
  "charged": true,
  "occurrences": 1,
  "bytesRead": 8,
  "startHost": "start.example",
  "finalHost": "start.example"
}

result is one of ok (2xx), redirect (a 3xx we did not follow), client_error (4xx), server_error (5xx), unreachable (DNS, TLS, timeout, connection, loop, hop limit), blocked (robots or a refused address), or invalid (the URL was not checked).

SUMMARY includes urlsCharged, duplicatesSkipped, rejected, overMaxUrls, notCheckedBecauseChargeLimit, and stoppedByChargeLimit.

Pricing

Pay per event. Current prices:

Event	Price	When it is charged
URL checked (url-checked)	$0.0015 ($1.50 per 1,000)	Once per accepted unique URL that is checked
Actor start (apify-actor-start)	$0.00005	Once per run, per GB of memory (the default memory is one unit). Charged by the platform, not by this Actor's code

Accepted URLs	Event cost, before the start fee
100	$0.15
1,000	$1.50
10,000	$15.00

Charged once, including when the check fails: HTTP 4xx and 5xx, DNS, TLS, timeout, and connection errors, redirect loops and hop limits, a robots.txt disallow, and a host name that resolves to a non-public address. Retries, redirect hops, and the robots.txt download are part of that one charge. Dataset rows are not an extra event.

Not charged: blank lines and # comments; duplicate URLs after normalization (lowercase host, no fragment, default port removed, empty path becomes /); invalid URLs; anything other than http(s); URLs that contain a username or password; literal private, loopback, link-local, and other special-use addresses; special-use host names (localhost, .local, .internal, and similar); URLs past maxUrls; URLs not checked because the spending limit cannot fit another url-checked event ($0.0015).

Set Maximum cost per run (maxTotalChargeUsd in the API). The Actor stops before the next URL when the remaining allowance, including a free-plan cap, cannot pay for another check. The platform start fee is already counted. stoppedByChargeLimit in SUMMARY is true when that happens. This Actor does not add an extra over-limit charge.

There is no rental fee and no separate per-result dataset charge.

How a check behaves

• Requests. GET by default, with User-Agent: BulkUrlChecker/1.0 (+https://apify.com/batreek/bulk-url-status-redirect-checker). No cookies, no Authorization header, and no custom headers. .netrc and HTTP_PROXY are ignored. Accept-Encoding: identity.
• Redirects. Relative and protocol-relative Location values are resolved against the current hop. A redirect to a non-http(s) URL, a URL with credentials, or a non-public address is not requested. The chain stops and the error says why. Credentials from a Location header are removed before they are stored.
• robots.txt. On by default. For each origin, /robots.txt is fetched once (not charged, up to 3 redirects, 10 seconds, 512 KiB). Rules for BulkUrlChecker are used when present, otherwise *. Allow wins when the matching path length ties. HTTP 401 or 403 on robots.txt disallows that origin. Any other failure, including a robots.txt URL this Actor refuses to fetch, allows the check. Turn respectRobotsTxt off only for URLs you are allowed to request.
• Refused targets. No connection is made to loopback, private, link-local (including cloud metadata), CGNAT, multicast, reserved, documentation, or benchmarking ranges, or to IPv6 site-local, unique-local, 6to4, Teredo, or NAT64 addresses. Ambiguous numeric hosts (http://2130706433/, octal, hex) are interpreted and then refused when they are non-public. If DNS returns any non-public address, the name is refused. The connection uses the address from that lookup, so a later DNS answer cannot move the socket onto another address. The error does not include the resolved address.
• Bounds. Concurrency 8, per host 2, 10 redirect hops, 20 seconds per hop, 60 seconds per URL, 1 retry, 64 KiB of body. Values above the input maximum are clamped. At most 50,000 input lines are read.

Use it from the API

After publication the Actor id is batreek/bulk-url-status-redirect-checker, if that slug is free. Confirm the name in Console before you rely on it. Replace <YOUR_APIFY_TOKEN> with your own token.

curl -X POST "https://api.apify.com/v2/acts/batreek~bulk-url-status-redirect-checker/run-sync-get-dataset-items?maxTotalChargeUsd=1" \
  -H "Authorization: Bearer <YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/"], "maxUrls": 10}'

from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("batreek/bulk-url-status-redirect-checker").call(
    run_input={"urls": ["https://example.com/"], "maxUrls": 10},
    max_total_charge_usd=1,
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["statusCode"], item["finalUrl"], item["redirectChainText"])

Limitations

• No crawl, no sitemap, no link extraction, no JavaScript. Only the URLs in the input are checked, plus robots.txt for their origins when that option is on.
• No browser and no cookies. A chain that depends on a cookie, a login, or a meta refresh will not match what a browser does. The status you get is the status this client received.
• HEAD can disagree with GET. Use GET when the status has to match a browser navigation.
• Bot walls. A 401, 403, or 429 is reported as that status. The client does not pretend to be a browser.
• Bodies are discarded. Nothing past the byte cap is read, and the body is not in the dataset. This is a status check, not a downloader.
• Public internet only. Private, local, and metadata addresses are refused on purpose, including when a redirect or DNS points at them. That also means you cannot point this Actor at localhost or an address inside your own network.
• Host names that mix public and non-public DNS answers are refused, even if one answer is public.
• One charge is not refunded when a checked URL errors. Invalid and refused inputs are not charged. A spending limit stops further checks; it does not undo checks already made.
• A restarted run skips URLs already finished in this run's saved state. A crash in the moment after a charge and before that save can check the URL again; the charge uses an idempotency key so the platform does not bill it twice within three minutes.
• Not a vulnerability scanner. finalHostMatchesStart: false only means the chain left the starting host.

Tests for this Actor run locally with mocked responses. Publishing and the Store price are set in Apify Console, not by running this repository.