# Bulk URL Status & Redirect Checker (`batreek/bulk-url-status-redirect-checker`) Actor

Check a pasted list of URLs for HTTP status, the full redirect chain, the final URL, and response time. No crawl and no sitemap discovery. Refuses non-public addresses.

- **URL**: https://apify.com/batreek/bulk-url-status-redirect-checker.md
- **Developed by:** [Adam Abdou-Gustavson](https://apify.com/batreek) (community)
- **Categories:** SEO tools, Developer tools, Automation
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.50 / 1,000 url checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk URL Status & Redirect Checker

Paste a list of URLs. For each one you get the HTTP status, the full redirect chain (every hop's status and URL), the final URL, the response time, and a plain-language error when the check cannot finish.

This Actor does not crawl a site, does not read sitemaps, and does not extract links. It checks only the URLs you give it.

### What you get

- **Status and final URL** after HTTP redirects (`301`, `302`, `303`, `307`, `308`, and `300` when a `Location` header is present).
- **The chain.** Each hop is the URL that was requested and the status that came back. `redirectChainText` is the same chain in one cell for CSV. JavaScript and `<meta refresh>` redirects are not followed.
- **Response time** for the whole chain, including retries.
- **Flags:** `httpsDowngrade` when a later hop is `http` after an `https` hop, and `finalHostMatchesStart` when the final host is the host you started with. A host change is not by itself an open redirect; shorteners and CDNs look the same.
- **Errors** for DNS failure, TLS failure, timeouts, connection failure, redirect loops, hop limits, invalid URLs, and addresses this Actor refuses to contact.

The default dataset view is a table. The **Flat table** view has no nested columns. Export it with `?view=flat&format=csv`. A `SUMMARY` record in the key-value store has the totals, including how many URLs were not charged.

### Quick start

1. Open **Try for free** or **Start**.
2. Paste URLs into **URL list**, or add them under **URLs**. The form starts with `https://example.com/` and a max of 100.
3. Start the run. Open **Output**, or the CSV link, and the `SUMMARY` record.

### Input examples

**1. A short list** (10 URLs maximum, about $0.015 if every URL is accepted):

```json
{
  "urls": ["https://example.com/", "https://example.com/missing"],
  "maxUrls": 10
}
```

**2. A pasted list, following redirects:**

```json
{
  "urlText": "https://example.com/old\nhttps://example.com/missing\n# comment lines are ignored\n",
  "maxUrls": 100,
  "method": "GET",
  "respectRobotsTxt": true,
  "maxRedirectHops": 10
}
```

**3. HEAD only, robots off, tight limits** for a list you are allowed to request:

```json
{
  "urls": ["https://example.com/status"],
  "method": "HEAD",
  "respectRobotsTxt": false,
  "maxConcurrency": 2,
  "perHostConcurrency": 1,
  "maxRetries": 0,
  "maxResponseBytes": 0,
  "requestTimeoutSecs": 10,
  "urlTimeoutSecs": 20
}
```

| Field | Default | What it does |
|---|---|---|
| `urls` | – | URL strings. Checked before `urlText`. |
| `urlText` | empty | One URL per line. Blank lines and `#` comments are ignored. |
| `maxUrls` | 1000 (form prefill 100) | Cap on unique accepted URLs, and on the `url-checked` cost. Maximum 10,000. |
| `method` | `GET` | `HEAD` skips the body. Some servers answer `HEAD` with a different status. |
| `maxRedirectHops` | 10 | Redirects to follow (0–20). `0` keeps the first status. |
| `requestTimeoutSecs` | 20 | Timeout for one hop (3–60). Connect waits at most 10 seconds. |
| `urlTimeoutSecs` | 60 | Wall-clock limit for one URL, hops and retries included (5–180). |
| `maxRetries` | 1 | Extra tries for timeouts, connection failures, and HTTP 408/429/502/503/504. Maximum 2. |
| `maxResponseBytes` | 65536 | Stop reading a body after this many bytes (maximum 1 MiB). The body is not saved. |
| `maxConcurrency` | 8 | Parallel URL checks (1–16). |
| `perHostConcurrency` | 2 | Parallel checks of the same host (1–4). |
| `respectRobotsTxt` | true | Skip a URL when `robots.txt` disallows `BulkUrlChecker`. |

### Example output

These rows are the shape the Actor writes. They are not from a live Store run.

```json
{
  "inputIndex": 1,
  "inputUrl": "https://start.example/old",
  "normalizedUrl": "https://start.example/old",
  "result": "ok",
  "statusCode": 200,
  "finalUrl": "https://start.example/new",
  "redirectCount": 1,
  "redirectChain": [
    {"hop": 0, "url": "https://start.example/old", "statusCode": 301, "location": "https://start.example/new", "contentType": "text/plain", "elapsedMs": 5},
    {"hop": 1, "url": "https://start.example/new", "statusCode": 200, "location": null, "contentType": "text/html", "elapsedMs": 4}
  ],
  "redirectChainText": "301 https://start.example/old -> 200 https://start.example/new",
  "responseTimeMs": 12,
  "error": null,
  "errorType": null,
  "httpsDowngrade": false,
  "finalHostMatchesStart": true,
  "blockedByRobots": false,
  "contentType": "text/html",
  "method": "GET",
  "charged": true,
  "occurrences": 1,
  "bytesRead": 8,
  "startHost": "start.example",
  "finalHost": "start.example"
}
```

`result` is one of `ok` (2xx), `redirect` (a 3xx we did not follow), `client_error` (4xx), `server_error` (5xx), `unreachable` (DNS, TLS, timeout, connection, loop, hop limit), `blocked` (robots or a refused address), or `invalid` (the URL was not checked).

`SUMMARY` includes `urlsCharged`, `duplicatesSkipped`, `rejected`, `overMaxUrls`, `notCheckedBecauseChargeLimit`, and `stoppedByChargeLimit`.

### Pricing

Pay per event. Current prices:

| Event | Price | When it is charged |
|---|---|---|
| URL checked (`url-checked`) | **$0.0015 ($1.50 per 1,000)** | Once per accepted unique URL that is checked |
| Actor start (`apify-actor-start`) | $0.00005 | Once per run, per GB of memory (the default memory is one unit). Charged by the platform, not by this Actor's code |

| Accepted URLs | Event cost, before the start fee |
|---|---|
| 100 | $0.15 |
| 1,000 | $1.50 |
| 10,000 | $15.00 |

**Charged once, including when the check fails:** HTTP 4xx and 5xx, DNS, TLS, timeout, and connection errors, redirect loops and hop limits, a `robots.txt` disallow, and a host name that resolves to a non-public address. Retries, redirect hops, and the `robots.txt` download are part of that one charge. Dataset rows are not an extra event.

**Not charged:** blank lines and `#` comments; duplicate URLs after normalization (lowercase host, no fragment, default port removed, empty path becomes `/`); invalid URLs; anything other than http(s); URLs that contain a username or password; literal private, loopback, link-local, and other special-use addresses; special-use host names (`localhost`, `.local`, `.internal`, and similar); URLs past `maxUrls`; URLs not checked because the spending limit cannot fit another `url-checked` event ($0.0015).

Set **Maximum cost per run** (`maxTotalChargeUsd` in the API). The Actor stops before the next URL when the remaining allowance, including a free-plan cap, cannot pay for another check. The platform start fee is already counted. `stoppedByChargeLimit` in `SUMMARY` is true when that happens. This Actor does not add an extra over-limit charge.

There is no rental fee and no separate per-result dataset charge.

### How a check behaves

- **Requests.** `GET` by default, with `User-Agent: BulkUrlChecker/1.0 (+https://apify.com/batreek/bulk-url-status-redirect-checker)`. No cookies, no `Authorization` header, and no custom headers. `.netrc` and `HTTP_PROXY` are ignored. `Accept-Encoding: identity`.
- **Redirects.** Relative and protocol-relative `Location` values are resolved against the current hop. A redirect to a non-http(s) URL, a URL with credentials, or a non-public address is not requested. The chain stops and the error says why. Credentials from a `Location` header are removed before they are stored.
- **robots.txt.** On by default. For each origin, `/robots.txt` is fetched once (not charged, up to 3 redirects, 10 seconds, 512 KiB). Rules for `BulkUrlChecker` are used when present, otherwise `*`. `Allow` wins when the matching path length ties. HTTP 401 or 403 on `robots.txt` disallows that origin. Any other failure, including a `robots.txt` URL this Actor refuses to fetch, allows the check. Turn `respectRobotsTxt` off only for URLs you are allowed to request.
- **Refused targets.** No connection is made to loopback, private, link-local (including cloud metadata), CGNAT, multicast, reserved, documentation, or benchmarking ranges, or to IPv6 site-local, unique-local, 6to4, Teredo, or NAT64 addresses. Ambiguous numeric hosts (`http://2130706433/`, octal, hex) are interpreted and then refused when they are non-public. If DNS returns any non-public address, the name is refused. The connection uses the address from that lookup, so a later DNS answer cannot move the socket onto another address. The error does not include the resolved address.
- **Bounds.** Concurrency 8, per host 2, 10 redirect hops, 20 seconds per hop, 60 seconds per URL, 1 retry, 64 KiB of body. Values above the input maximum are clamped. At most 50,000 input lines are read.

### Use it from the API

After publication the Actor id is `batreek/bulk-url-status-redirect-checker`, if that slug is free. Confirm the name in Console before you rely on it. Replace `<YOUR_APIFY_TOKEN>` with your own token.

```bash
curl -X POST "https://api.apify.com/v2/acts/batreek~bulk-url-status-redirect-checker/run-sync-get-dataset-items?maxTotalChargeUsd=1" \
  -H "Authorization: Bearer <YOUR_APIFY_TOKEN>" \
  -H "Content-Type: application/json" \
  -d '{"urls": ["https://example.com/"], "maxUrls": 10}'
```

```python
from apify_client import ApifyClient

client = ApifyClient("<YOUR_APIFY_TOKEN>")
run = client.actor("batreek/bulk-url-status-redirect-checker").call(
    run_input={"urls": ["https://example.com/"], "maxUrls": 10},
    max_total_charge_usd=1,
)
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item["statusCode"], item["finalUrl"], item["redirectChainText"])
```

### Limitations

- **No crawl, no sitemap, no link extraction, no JavaScript.** Only the URLs in the input are checked, plus `robots.txt` for their origins when that option is on.
- **No browser and no cookies.** A chain that depends on a cookie, a login, or a meta refresh will not match what a browser does. The status you get is the status this client received.
- **HEAD can disagree with GET.** Use GET when the status has to match a browser navigation.
- **Bot walls.** A 401, 403, or 429 is reported as that status. The client does not pretend to be a browser.
- **Bodies are discarded.** Nothing past the byte cap is read, and the body is not in the dataset. This is a status check, not a downloader.
- **Public internet only.** Private, local, and metadata addresses are refused on purpose, including when a redirect or DNS points at them. That also means you cannot point this Actor at `localhost` or an address inside your own network.
- **Host names that mix public and non-public DNS answers are refused,** even if one answer is public.
- **One charge is not refunded** when a checked URL errors. Invalid and refused inputs are not charged. A spending limit stops further checks; it does not undo checks already made.
- **A restarted run** skips URLs already finished in this run's saved state. A crash in the moment after a charge and before that save can check the URL again; the charge uses an idempotency key so the platform does not bill it twice within three minutes.
- **Not a vulnerability scanner.** `finalHostMatchesStart: false` only means the chain left the starting host.

Tests for this Actor run locally with mocked responses. Publishing and the Store price are set in Apify Console, not by running this repository.

# Actor input Schema

## `urls` (type: `array`):

URLs to check. One per item. Duplicates are checked once. Only http and https. 1,000 checked URLs cost $1.50.

## `urlText` (type: `string`):

Optional. One URL per line, pasted from a spreadsheet or sitemap export. Blank lines and lines starting with # are ignored. Combined with URLs above; the list above is checked first.

## `maxUrls` (type: `integer`):

Maximum unique accepted URLs to check in this run. This caps the cost: 100 URLs ≈ $0.15, 1,000 ≈ $1.50, 10,000 ≈ $15.00, plus the $0.00005 start fee. Invalid URLs and duplicates are not charged and do not count toward this cap.

## `method` (type: `string`):

GET reads only the status line and a capped number of body bytes. HEAD does not download a body; some servers return a different status than GET.

## `maxRedirectHops` (type: `integer`):

How many HTTP redirects (301, 302, 303, 307, 308, and 300 with a Location) to follow. 0 records the first status and does not follow. The chain in the result includes every response. JavaScript and meta-refresh redirects are not followed.

## `requestTimeoutSecs` (type: `integer`):

Connect and read timeout for one hop, including each retry. The connect timeout is at most 10 seconds.

## `urlTimeoutSecs` (type: `integer`):

Wall-clock limit for one URL, including every redirect hop and retry. Does not include the robots.txt lookup.

## `maxRetries` (type: `integer`):

Extra attempts for timeouts, connection failures, and HTTP 408, 429, 502, 503, and 504. DNS and TLS errors are not retried. Retries are included in the one charge for that URL.

## `maxResponseBytes` (type: `integer`):

Stop reading each response after this many bytes. The body is not saved. Redirect responses are not downloaded. 0 reads nothing after the headers.

## `maxConcurrency` (type: `integer`):

How many URLs are checked at the same time. Kept modest so a pasted list is not a flood.

## `perHostConcurrency` (type: `integer`):

Parallel requests to the same host. Redirects stay under the starting host's slot.

## `respectRobotsTxt` (type: `boolean`):

Before checking a URL, fetch that origin's /robots.txt (not charged) and skip the URL when BulkUrlChecker is disallowed. 401 or 403 on robots.txt disallows the whole origin. Other robots.txt failures allow the check. Turn this off only for URLs you are allowed to request.

## Actor input object example

```json
{
  "urls": [
    "https://example.com/"
  ],
  "maxUrls": 100,
  "method": "GET",
  "maxRedirectHops": 10,
  "requestTimeoutSecs": 20,
  "urlTimeoutSecs": 60,
  "maxRetries": 1,
  "maxResponseBytes": 65536,
  "maxConcurrency": 8,
  "perHostConcurrency": 2,
  "respectRobotsTxt": true
}
```

# Actor output Schema

## `results` (type: `string`):

One row per unique URL: status, final URL, redirect chain, timing, and errors.

## `resultsCsv` (type: `string`):

The same rows as a flat CSV file, one row per URL, no nested fields.

## `summary` (type: `string`):

How many URLs were accepted, charged, rejected, or skipped because of the spending limit.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "https://example.com/"
    ],
    "maxUrls": 100
};

// Run the Actor and wait for it to finish
const run = await client.actor("batreek/bulk-url-status-redirect-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "urls": ["https://example.com/"],
    "maxUrls": 100,
}

# Run the Actor and wait for it to finish
run = client.actor("batreek/bulk-url-status-redirect-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "https://example.com/"
  ],
  "maxUrls": 100
}' |
apify call batreek/bulk-url-status-redirect-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,batreek/bulk-url-status-redirect-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/QnML4pEIrekpALhdS/builds/C2gPlEUkqENxQTHfF/openapi.json