# Bulk URL Status and Redirect Chain Checker API (`rowcraft/url-status-redirect-checker`) Actor

Check the status code, redirect chain, canonical and indexing signals of every URL in a list, with the final destination and a loop flag. robots.txt obeyed, no personal data, and you pay only for URLs checked, never for blocked or disallowed ones.

- **URL**: https://apify.com/rowcraft/url-status-redirect-checker.md
- **Developed by:** [Rowcraft](https://apify.com/rowcraft) (community)
- **Categories:** SEO tools, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $0.40 / 1,000 checked urls

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.
Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## Bulk URL Status and Redirect Chain Checker API

Check the HTTP status and the full redirect chain of every address in a list. This bulk **URL checker** asks each URL politely, follows every redirect hop by hop, and returns one row per URL: the final status code, the final destination, every hop with its type (301, 302, 303, 307, 308 or meta refresh), a redirect-loop flag, whether the URL moved to https, gained or lost `www`, gained or lost a trailing slash, how fast the server answered, the main response headers, and the canonical link, meta robots tag and title of the final page. Results come as JSON, CSV or Excel.

It is built for the jobs where a list of URLs has to be proved right or wrong: a site migration (does every old URL reach its new address in one hop?), an https rollout, a link audit from a crawl or a Search Console export, a canonical check, and a weekly watch on the URLs that matter. With **Only new rows** a scheduled run delivers, and charges, only the URLs whose status, destination or redirect chain changed since the last run.

![Input form of Bulk URL Status and Redirect Chain Checker API](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/url-status-redirect-checker-input.png)

### What you get

One row per URL:

- **The answer:** `statusCode` and `statusText` of the final response, `ok` (true for a 2xx answer), and the address that answered (`finalUrl`). A dead address is an answer too: its row has `statusCode` null and an `error` such as `dns_error`, `timeout`, `tls_error` or `connection_refused`.
- **The whole chain:** `redirectChain` lists every redirect on the way with the URL that redirected, its status code, the `location` it pointed to and the `type` (`301`, `302`, `303`, `307`, `308` or `meta-refresh`). `redirectPath` writes the same chain as one short text (`301 → 302 → 200`), so it reads well in a table and in a spreadsheet column. `redirectCount` is the length of the chain, and `redirectLoop` is true when the chain came back to an address it had already visited.
- **What the redirects did:** `httpsUpgrade` (started on http, ended on https), `wwwChange` (the host gained or lost `www.`) and `trailingSlashChange` (the path gained or lost its final slash), so a migration can be filtered with one column.
- **Speed and headers of the final response:** `responseTimeMs` (time to the first response headers), `contentType`, `contentLength`, `server`, `cacheControl`, `hsts` and `xRobotsTag`.
- **Indexing signals of the final page:** `canonical`, `canonicalMatchesFinal`, `metaRobots` and `title`, read from the first 256 KB of an HTML answer.
- **When:** `checkedAt`, in UTC.

The run also writes a `COVERAGE` record with the URLs that were not delivered and why (blocked, disallowed by robots.txt, invalid, not checked), and a `RUN_SUMMARY` with counts by status class, redirected URLs, loops and the reason the run stopped.

### Why this actor

- **The chain, not only the end.** Every hop is a row entry with its own status code, target and type. A chain from https to http and back to https (a real one, below) or a 302 where a 301 was meant is visible at once.
- **Migration monitoring built in.** **Only new rows** remembers the state of every URL it delivered: final status, final address and chain. A schedule then costs only the URLs that changed, and a webhook tells you when something did.
- **robots.txt on every host of the chain.** Before a request goes to a host, its robots.txt is read once and obeyed for that exact path, including the hosts a redirect leads to.
- **Honest about blocks.** An anti-bot page, a 403 or a 429 is reported in `COVERAGE` with the protection's name when it can be recognised, and never charged. The actor does not rotate proxies, solve CAPTCHAs or disguise itself.
- **Light on the sites it checks.** HEAD requests by default, one request per host and second, several different hosts at the same time, a clear user agent with a link to Rowcraft.
- **Rows in the order of your list.** Several URLs are checked at once, but the dataset keeps the order you gave, so it lines up with your spreadsheet.
- **Pay only for answers.** No charge for blocked, disallowed, invalid, repeated or unchanged URLs.
- **No personal data.** Links such as `mailto:` and `tel:` are skipped, e-mail addresses inside URLs and titles are masked, and the row holds no text of the page apart from its title.

### How to use it

1. Open the Actor and paste your URLs into **URLs to check**, one per line. A column copied from a spreadsheet works. An address without `http://` or `https://` is checked as `https://`.
2. Keep the defaults for a first try: HEAD requests with a GET fallback, up to 10 redirects followed, the page head read.
3. Click **Start**. The status line shows progress; the run stops cleanly at your maximum or at the charge limit you set in the run options.
4. Open the **Dataset** tab. The **Overview**, **Redirects**, **Indexing signals** and **Headers** views show the main columns. Download JSON, CSV, Excel, XML or HTML, or send the rows to Google Sheets, Slack, Make, n8n or Zapier from the Integrations tab.
5. Open the `COVERAGE` record in the key-value store to see which URLs were not delivered and why.
6. To watch a list over time, run it on a schedule with **Only new rows since the last run** turned on, and put a webhook address in **Notify this URL when the run ends**.

### Input

| Field | What it does |
|---|---|
| `urls` | URLs to check, one per line. Duplicates, invalid entries and links such as `mailto:` are skipped and never charged; a `#fragment` is dropped because browsers never send it. Up to 100,000 per run. |
| `method` | `HEAD` (default) asks for the headers only; a URL that answers 4xx or 5xx to HEAD, or drops the connection, is asked again with GET, because many servers mishandle HEAD. `GET` uses GET for every URL. |
| `followRedirects` | On by default. Off shows only the first answer of each URL; its `Location` is still listed as one entry of `redirectChain`. |
| `maxRedirects` | Default 10. A longer chain ends with `error: too_many_redirects`. |
| `readHtmlSignals` | On by default: for HTML answers the first 256 KB are read for the title, canonical, meta robots and an instant meta refresh. Off uses headers only. |
| `maxItems` | Stop after this many delivered rows. 0 = the whole list. |
| `onlyNew`, `stateName` | Deliver a URL only when its status, final address or redirect chain changed since the last run with the same state name |
| `notifyUrl`, `notifyWhenEmpty` | A webhook that receives one message when the run ends; see **Automation** below |
| `requestIntervalMs` | Pause between two requests to the same host, default 1000 ms, minimum 250 ms |

Example: four URLs, one row each.

```json
{
  "urls": ["http://github.com", "http://python.org", "https://example.com/missing-page", "http://www.wikipedia.org"]
}
```

Example: a weekly check of a redirect map after a migration, delivering only the URLs that changed, with a Slack message.

```json
{
  "urls": [
    "https://old-site.example/pricing",
    "https://old-site.example/blog/launch-post",
    "https://old-site.example/contact"
  ],
  "onlyNew": true,
  "stateName": "old-site-migration",
  "notifyUrl": "https://hooks.slack.com/services/..."
}
```

Example: a fast status pass over a long list, headers only and no redirect following.

```json
{
  "urls": ["https://shop.example/p/1001", "https://shop.example/p/1002"],
  "method": "HEAD",
  "followRedirects": false,
  "readHtmlSignals": false,
  "requestIntervalMs": 250
}
```

### Output

25 fields per row. Empty values are `null`, never missing, so CSV and Excel columns are stable.

| Field | Type | Example |
|---|---|---|
| `url` | string | `http://github.com/` |
| `finalUrl` | string | `https://github.com/` |
| `statusCode` | integer | 200 |
| `statusText` | string | OK |
| `ok` | boolean | True |
| `redirectCount` | integer | 1 |
| `redirectChain` | array | \[{"url": "http://github.com/", "statusCode": 301, "location": "http… |
| `redirectPath` | string | 301 → 302 → 200 |
| `redirectLoop` | boolean | False |
| `httpsUpgrade` | boolean | True |
| `wwwChange` | boolean | False |
| `trailingSlashChange` | boolean | False |
| `responseTimeMs` | integer | 161 |
| `contentType` | string | text/html; charset=utf-8 |
| `contentLength` | integer | 11155 |
| `server` | string | nginx |
| `cacheControl` | string | max-age=0, private, must-revalidate |
| `hsts` | string | max-age=31536000; includeSubdomains; preload |
| `xRobotsTag` | string | noindex, nofollow |
| `canonical` | string | `https://github.com/` |
| `canonicalMatchesFinal` | boolean | True |
| `metaRobots` | string | noindex, follow |
| `title` | string | Welcome to Python.org |
| `error` | string | Why no normal answer came: dns_error, timeout, tls_error, connection_refused, connection_error, request_error, redirect_loop, too_many_redirects, invalid_redirect or private_address; null when the server answered |
| `checkedAt` | string | 2026-10-14T17:34:32.601Z |

#### How to read the fields

- **`statusCode` and `finalUrl` always describe the same response:** the last one the actor received. When a redirect target cannot be reached, `finalUrl` is that target, `statusCode` is null and `error` says why; the chain still lists the hops that worked.
- **`ok`** is true only for a 2xx answer without an error. A 404, a 410 or a 500 is `ok: false` with its status code and `error: null`, because the server did answer.
- **`redirectChain`** holds one object per redirect: `url`, `statusCode`, `location` and `type`. For a meta refresh the status code is that of the page that carried the tag (200). With **Follow redirects** off the chain holds the first redirect only, and `finalUrl` stays the URL you gave.
- **`redirectPath`** is the chain as plain text: the type of every redirect in order, then the final status code, or the error when no answer came (`301 → 200`, `302 → 302 → redirect_loop`, `301 → dns_error`). A URL that did not redirect shows just its status code. With **Follow redirects** off it shows the first answer only (`301`). It is one text column, so it exports to CSV and Excel without splitting; `redirectChain` stays a list of objects for code that needs every hop.
- **`canonicalMatchesFinal`** compares the canonical URL with the final URL after the usual URL clean-up (lower-case host, default port removed, fragment dropped); a different `www`, scheme or trailing slash counts as different. It is null when the page announces no canonical. A canonical sent in the HTTP `Link` header counts as well as the tag.
- **`metaRobots`** is the content of the robots meta tag (the googlebot tag when there is no robots tag). **`xRobotsTag`** is the header with the same job. Together they show whether the final page asks not to be indexed.
- **`responseTimeMs`** is the time from the request to the first response headers, measured from the Apify data center that ran the actor. It includes the connection set-up when a new connection was needed, so use it to compare URLs of one run, not as a speed test of your site for your visitors.
- **`contentLength`** is the `Content-Length` header: null when the server sends none, and the compressed size when the answer is compressed.

#### Values of `error`

| `error` | Meaning |
|---|---|
| `dns_error` | The host name does not exist (or could not be resolved after one retry). |
| `connection_refused` | The host answered, but nothing listens on that port. |
| `timeout` | No answer within 20 seconds. |
| `tls_error` | The secure connection failed: an expired or self-signed certificate, a name mismatch, or a server that does not speak TLS on that port. |
| `connection_error` | The connection was reset or dropped (retried once). |
| `redirect_loop` | The chain came back to an address it had already visited. |
| `too_many_redirects` | The chain was longer than **Maximum redirects**. |
| `invalid_redirect` | A redirect status without a usable `Location`, or one that points to a non-web address such as `mailto:`. |
| `private_address` | A redirect led to a private or local network address, which is never requested on the platform. |
| `request_error` | Any other failure of the request itself. |

#### Example rows

Two real rows from a run (long texts shortened here, complete in the dataset):

```json
[
  {
    "url": "https://www.python.org/downloads",
    "finalUrl": "https://www.python.org/downloads/",
    "statusCode": 200,
    "statusText": "OK",
    "ok": true,
    "redirectCount": 2,
    "redirectChain": [
      {
        "url": "https://www.python.org/downloads",
        "statusCode": 301,
        "location": "http://www.python.org/downloads/",
        "type": "301"
      },
      {
        "url": "http://www.python.org/downloads/",
        "statusCode": 301,
        "location": "https://www.python.org/downloads/",
        "type": "301"
      }
    ],
    "redirectPath": "301 → 301 → 200",
    "redirectLoop": false,
    "httpsUpgrade": false,
    "wwwChange": false,
    "trailingSlashChange": true,
    "responseTimeMs": 105,
    "contentType": "text/html; charset=utf-8",
    "contentLength": 19330,
    "server": "nginx",
    "cacheControl": "max-age=604800, public",
    "hsts": "max-age=63072000; includeSubDomains; preload",
    "xRobotsTag": null,
    "canonical": null,
    "canonicalMatchesFinal": null,
    "metaRobots": null,
    "title": "Download Python | Python.org",
    "error": null,
    "checkedAt": "2026-10-14T17:43:33.053Z"
  },
  {
    "url": "https://en.wikipedia.org/wiki/main_page",
    "finalUrl": "https://en.wikipedia.org/wiki/Main_page",
    "statusCode": 200,
    "statusText": "OK",
    "ok": true,
    "redirectCount": 1,
    "redirectChain": [
      {
        "url": "https://en.wikipedia.org/wiki/main_page",
        "statusCode": 301,
        "location": "https://en.wikipedia.org/wiki/Main_page",
        "type": "301"
      }
    ],
    "redirectPath": "301 → 200",
    "redirectLoop": false,
    "httpsUpgrade": false,
    "wwwChange": false,
    "trailingSlashChange": false,
    "responseTimeMs": 163,
    "contentType": "text/html; charset=UTF-8",
    "contentLength": 51588,
    "server": "mw-web.eqiad.main-5875ccb8b6-d8kqr",
    "cacheControl": "private, s-maxage=0, max-age=0, must-revalidate, no-transform",
    "hsts": "max-age=106384710; includeSubDomains; preload",
    "xRobotsTag": null,
    "canonical": "https://en.wikipedia.org/wiki/Main_Page",
    "canonicalMatchesFinal": false,
    "metaRobots": "max-image-preview:standard",
    "title": "Wikipedia, the free encyclopedia",
    "error": null,
    "checkedAt": "2026-10-14T17:43:32.432Z"
  }
]
```

Results can be downloaded as JSON, CSV, Excel, XML or HTML from the Dataset tab, or read through the Apify API.

![Results of Bulk URL Status and Redirect Chain Checker API in the Apify dataset view](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/url-status-redirect-checker-results.png)

### Pricing

**Pay per result.** You pay for each URL checked and delivered to your dataset, nothing else: no subscription and no minimum. A row is charged whenever the check produced an answer, and a dead address or a 404 is an answer. Nothing is charged for URLs behind an anti-bot page, a 403 or a 429, URLs that robots.txt disallows, invalid or repeated entries, `mailto:` and `tel:` links, or URLs skipped by **Only new rows** because nothing changed.

| Your Apify plan | Price per 1,000 rows | Price per row |
|---|---|---|
| Free plan | **$0.60** | $0.0006 |
| Starter plan | **$0.55** | $0.00055 |
| Scale plan | **$0.50** | $0.0005 |
| Business plan and above | **$0.40** | $0.0004 |

Example: 1,000 URLs cost $0.60 on the Free plan and $0.40 on the Business plan. The Apify platform usage of a run (compute and storage) is billed by Apify separately, as for every Actor; the actor runs in 256 MB and spends most of its time waiting between requests.
A start event of $0.00005 is charged once per run (Apify default).
**Cap your spend:** set *Maximum total charge* in the run options. The actor stops cleanly when the cap is reached and you keep everything delivered until then. The run summary (`RUN_SUMMARY` in the key-value store) says why a run stopped.

### Automation: only new rows, notifications, schedules

**Only new rows since the last run.** Turn on `onlyNew` and the actor remembers, for every URL it delivered to you, a short fingerprint of its state: the final status code, the final address, the redirect chain (each hop's status and target) and the error. The memory lives in a key-value store named `rowcraft-url-status-redirect-checker-state` in your Apify account. The next run with the same `stateName` still checks every URL, but delivers and charges a URL again only when that state changed: a 301 that became a 302, a redirect target that moved, a page that started to answer 404 or went back to 200, a host that stopped resolving. Response times, headers, titles and canonicals do not count as a change. The first run delivers every URL it can check. Use a different `stateName` for each list so they keep separate memories.

**Notification when the run ends.** Put a webhook address in `notifyUrl` and the actor sends one JSON POST with the number of delivered rows, links to the dataset (JSON and CSV) and the run. It works with Slack and Discord incoming webhooks (the message is in `text` and `content`), with Make, n8n and Zapier catch hooks, and with your own endpoint. By default it is sent only when at least one row was delivered; `notifyWhenEmpty` sends it every time.

**Schedules.** Create a schedule in Apify Console (hourly, daily, weekly or cron) with `onlyNew` on. After a migration, a daily schedule over the old URLs gives you a feed of every redirect that changed, for the price of those rows only.

**Integrations.** Send the results to Make, n8n, Zapier, Google Sheets, Slack, Google Drive, Airtable or any webhook from the Integrations tab of the Actor, with no code.

**API, SDKs and AI agents.** Every run is available through the Apify API and the official JavaScript and Python clients; the Actor can also be called by AI agents through the Apify MCP server. The example input in the API tab is a working call.

### Use cases

- **Site migration:** list every old URL, run the checker before and after the switch, and filter `redirectCount` greater than 1, `statusCode` not 200, `redirectLoop` true or `error` not null. Keep the schedule on with **Only new rows** for the weeks after launch.
- **https rollout:** paste the http version of your pages and filter `httpsUpgrade` false to find the ones that never redirect, then read `hsts` to see which hosts send the header.
- **Link audit:** check the URLs exported from a crawl, a sitemap or a Search Console report, and sort by `statusCode` to get the 404s and 5xx answers and the redirect hops that can be shortened.
- **Canonical audit:** filter `canonicalMatchesFinal` false for pages whose canonical points somewhere else than the page itself, and `metaRobots` or `xRobotsTag` containing `noindex` for pages that ask not to be indexed.
- **www and trailing-slash consistency:** `wwwChange` and `trailingSlashChange` show which variants redirect and which answer on their own, so duplicate versions of a page can be removed.
- **Backlink and campaign targets:** run the landing pages of your campaigns or the targets of your backlinks and learn when one starts to redirect, answer 404 or change its destination.
- **Affiliate and partner links:** check a list of outgoing links for chains that are too long, loops and dead destinations.

Ready-made examples, each with the input filled in (open one and press Start, or save it as your own task):

- [Check http to https redirects of a list of sites](https://apify.com/rowcraft/url-status-redirect-checker/examples/http-to-https-redirects-of-popular-sites)
- [Find broken links in a list of URLs](https://apify.com/rowcraft/url-status-redirect-checker/examples/find-broken-links-in-a-url-list)
- [Canonical and redirect audit of URL variants](https://apify.com/rowcraft/url-status-redirect-checker/examples/canonical-and-redirect-audit-of-url-variants)

### What this actor does not do

- Find URLs: it checks the list you give and does not crawl a site or read a sitemap.
- Run JavaScript: a redirect made by a script, or a page that is empty until its scripts run, shows the status and head of the first answer only. An HTML `<meta http-equiv="refresh">` tag with a delay of up to 5 seconds is followed; a longer delay is a reload timer and is not.
- Judge the content: it does not detect soft 404 pages (a 200 answer whose text says "not found"), compare pages or check broken links inside a page.
- Log in, accept cookie banners, rotate proxies, solve CAPTCHAs or hide behind another identity: a 403, a 429 or an anti-bot page is reported as blocked and not charged.
- Measure page speed: `responseTimeMs` is the time to the first byte, from one data center, with no page rendering.
- Collect contacts: no e-mail addresses, phone numbers, names of people or social profiles.

### Limits and fair use

- **One request per second to a host** by default (minimum 250 ms), and up to five URLs in flight at once on different hosts. A URL takes one request for a redirect, two for an HTML page with a HEAD request (the answer and then the page head) and one more for the robots.txt of each new host. A list that sits on one host therefore runs at about one request per second at the default interval, which is one to three seconds per URL; lower the interval to 250 ms for your own sites. In our test run 1,500 URLs spread over 20 local test hosts at 250 ms took about 90 seconds, and the actor stayed under 125 MB of memory.
- **robots.txt is obeyed** for every address the chain touches, with the `rowcraft` group first and `*` after it. A URL disallowed on any host of its chain is listed as `robots_disallowed` and not charged. If a robots.txt answers with a server error the host is not read, as the standard ([RFC 9309](https://www.rfc-editor.org/rfc/rfc9309)) requires, and the URL is listed as `robots_unavailable`. Staging sites often disallow everything for all robots: if your list is on one, the result will be empty.
- **Redirects:** up to 10 by default, 30 at most. Each hop checks DNS and robots.txt of its host first.
- **Answers that count as blocked:** a 403, a 429, and the challenge pages of the common bot-protection services (Cloudflare, DataDome, Akamai, Imperva, HUMAN, AWS WAF, Sucuri, Vercel). After three such answers from one host the rest of that host's URLs are listed as blocked without being requested, and after ten blocked URLs in a row from different hosts the run stops with a warning in the run summary. A plain 404, 410, 500 or 522 is never taken for a block.
- **Time limits:** 20 seconds per request, 90 seconds per URL, one retry on a dropped connection and on 502, 503, 504 (a 429 with a `Retry-After` of up to 20 seconds is waited for once). The run stops 90 seconds before its own timeout and saves its summary, its coverage and the **Only new rows** memory.
- **Size:** up to 100,000 URLs per run, up to 2,048 characters per URL; the page head is read from the first 256 KB of an HTML answer.
- **Private addresses** (localhost, 10.x.x.x, 192.168.x.x and similar) are never requested on the platform.

### Data source and licence

The data is what each server sends to any visitor who asks for that address: the status line, the response headers, and the head of the HTML page. The actor sends a normal request after reading robots.txt, with the user agent `Rowcraft Bulk URL Status and Redirect Checker (+https://apify.com/rowcraft)`, and keeps only the fields listed in the Output section. It does not store page text, images, cookies or the values of any header other than the ones listed.

Rowcraft is independent: this actor is not provided, endorsed or authorised by the websites it checks. You decide which URLs go on the list and are responsible for having the right to check them, the way you would be for a link checker you ran yourself.

### Privacy: no personal data

Rowcraft actors are built not to collect personal data:

- The row holds the URL, the status, the chain, a few headers and four head tags. The only page text is the title, cut at 200 characters, with e-mail addresses and phone numbers masked.
- `mailto:`, `tel:`, `sms:` and similar links in your list are skipped completely: not requested, not delivered, not counted as broken and not written anywhere.
- An e-mail address inside a URL (a query such as `?email=...`) is replaced by `[email removed]` in every field, including the chain. Credentials written in front of a host (`user:password@`) are dropped and never sent.
- Cookies are neither sent nor read. An entry that is not a URL and contains spaces is not repeated in `COVERAGE`; its position in your list is.

You are responsible for using the data in line with the law that applies to you.

### Other Rowcraft actors

All Rowcraft actors read official or public sources, deliver flat rows without personal data and support only-new runs and notifications:

- [Ashby Jobs Scraper – Job Postings with Salary Ranges](https://apify.com/rowcraft/ashby-jobs-scraper) — Job postings from Ashby job boards (jobs.ashbyhq.com) by company: title, department, team, locations with country, employment and workplace type, published salary range, full text.
- [Broken Link Checker – Find 404s and Dead Links API](https://apify.com/rowcraft/broken-link-checker) — Pay per crawled page, not per link: crawl a site within robots.txt and check every link on each page for 404s, 5xx errors, DNS and TLS failures and redirects.
- [Career Site Jobs Scraper: Greenhouse, Lever, Ashby, Workday](https://apify.com/rowcraft/career-site-jobs-scraper) — Job listings from the career pages of the companies you choose on Greenhouse, Lever, Ashby, Workday, Workable, Personio, Recruitee, Teamtailor and Gem, in one run and one format: title, locations, department, salary when published, full text.
- [Chrome Web Store Scraper: Users, Ratings & Versions](https://apify.com/rowcraft/chrome-web-store-scraper) — Reads public Chrome Web Store listing pages with no login, no proxy and no personal data: users, rating, category, version, update date, permissions and company developers.
- [Domain Email and DNS Checker API: MX, SPF, DMARC, Providers](https://apify.com/rowcraft/domain-email-dns-checker) — Find which email provider, sending services and DNS host each domain uses, from public DNS records only.
- [SIRENE France Company Scraper: SIREN, NAF, Revenue, Status](https://apify.com/rowcraft/france-company-register) — French companies from the official Recherche d'entreprises API (INSEE SIRENE + INPI): SIREN and SIRET, legal form, NAF activity, head-office address, employee band, revenue and net income, labels, creation date and VAT number.
- [Gem Jobs Scraper – Job Board Listings from jobs.gem.com](https://apify.com/rowcraft/gem-jobs-scraper) — Job listings from Gem job boards (jobs.gem.com) by company: title, department, offices with country, employment type, remote, hybrid or on-site, requisition ID, publication date and full ad text, from the public Gem Job Board API.
- [Google Play Apps Scraper: App Details, Installs, Ratings](https://apify.com/rowcraft/google-play-apps-scraper) — Public Google Play app details as flat rows: installs, rating, number of ratings, price, version, last update, minimum Android, ads and in-app purchases.
- [Greenhouse Jobs Scraper – Job Listings by Company Board](https://apify.com/rowcraft/greenhouse-jobs-scraper) — Job listings from Greenhouse job boards by company: title, departments, offices, publication date, full ad text, published pay ranges.
- [Himalayas Jobs Scraper: Remote Jobs API with Salaries](https://apify.com/rowcraft/himalayas-jobs-scraper) — Remote jobs from Himalayas through its public jobs API: title, company, salary range, seniority, categories, the countries and time zones open to candidates, full description and job link.
- [KRS Poland Companies Scraper: Register Data, PKD, Status](https://apify.com/rowcraft/krs-poland-companies) — Polish companies from the official KRS register API of the Ministry of Justice: name, KRS, NIP and REGON numbers, legal form, status, address, PKD activity codes, share capital, branches and registration dates.
- [LEI Records Scraper: Legal Entity Identifier Lookup API](https://apify.com/rowcraft/lei-records-scraper) — Legal Entity Identifier (LEI) records from the Global LEI Index, read through the public GLEIF API: legal name, legal form, register number, addresses, status, renewal date and parent companies.
- [Lever Jobs Scraper – Job Postings from jobs.lever.co](https://apify.com/rowcraft/lever-jobs-scraper) — Job postings from Lever career pages (jobs.lever.co) by company: title, team, department, location, commitment, workplace type, salary range when published, description, requirements and benefits.
- [Norway Company Register Scraper: Brønnøysund Business Data](https://apify.com/rowcraft/norway-company-register-scraper) — Norwegian companies from the Central Coordinating Register for Legal Entities (Brønnøysund, NLOD open data): organisation number, legal form, industry, employees, share capital, addresses, website, VAT and bankruptcy status.
- [Personio Jobs Scraper: Job Listings, Salaries, Departments](https://apify.com/rowcraft/personio-jobs-scraper) — Job ads from any company's Personio careers feed: title, department, office, employment type, schedule, seniority, salary when published, keywords and the full description in the language you choose.
- [Platsbanken Scraper: Sweden Jobs, Employers, Org Numbers](https://apify.com/rowcraft/platsbanken-jobs-scraper) — Every job ad on Platsbanken (Arbetsförmedlingen) from the official JobTech API: headline, employer and organisation number, occupation codes, municipality, coordinates, salary type, deadline and full text.
- [Recruitee Jobs Scraper – Job Listings by Company Career Site](https://apify.com/rowcraft/recruitee-jobs-scraper) — Job listings from Recruitee career sites by company: title, department, locations with country, employment type, remote flag, published salary and full text, one request per company.
- [Robots.txt and AI Crawler Access Checker API](https://apify.com/rowcraft/robots-ai-crawler-checker) — Check which AI and search crawlers a website lets in: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, read from robots.txt by the RFC 9309 rules.
- [RSS Feed Finder and Reader – Blog and News Article API](https://apify.com/rowcraft/rss-feed-finder-reader) — Find the RSS, Atom and JSON feeds of any website and read their newest articles, from a domain name alone.
- [SEO Audit Tool: Titles, Meta Tags, Headings and Links API](https://apify.com/rowcraft/seo-audit-tool) — Audit the on-page SEO of any list of pages or websites: titles, meta descriptions, headings, canonicals, indexing, links, images and structured data, with a score and the issues found per page.
- [Shopify Products Scraper: Store Catalog Export to JSON, CSV](https://apify.com/rowcraft/shopify-products-scraper) — Products from any Shopify store's public catalog: title, vendor, type, tags, description, variants with price, compare-at price, SKU and stock, images, options, store currency.
- [Sitemap URL Extractor: Every Page with Last-Modified Dates](https://apify.com/rowcraft/sitemap-url-extractor) — Get every page a site lists in its sitemaps, with last-modified dates, from robots.txt through sitemap indexes and .gz files.
- [SSL Certificate and Security Headers Checker API](https://apify.com/rowcraft/ssl-security-headers-checker) — Check the TLS certificate and the HTTP security headers of every domain in one run, with an A+ to F grade and the exact issues to fix.
- [Store Products Scraper – Shopify, WooCommerce, VTEX](https://apify.com/rowcraft/store-products-scraper) — Export the product catalogue of any Shopify, WooCommerce or VTEX store from its public catalogue endpoint: title, brand, prices, discounts, stock, categories, images and links, one row per product, the same columns on every platform.
- [Structured Data Extractor: JSON-LD, Schema.org, Open Graph](https://apify.com/rowcraft/structured-data-extractor) — Pull the schema.org JSON-LD, Microdata, RDFa and Open Graph of any page list into flat rows: types, product price, availability, ratings, organization, breadcrumbs, with validation issues.
- [Teamtailor Jobs Scraper – Career Site Jobs by Company](https://apify.com/rowcraft/teamtailor-jobs-scraper) — Job listings from Teamtailor career sites by company: title, department, role, locations with city and country, remote or hybrid status, publication date and full ad text, from the official job feed.
- [TED Tenders Scraper: EU Public Procurement Notices API](https://apify.com/rowcraft/ted-tenders-scraper) — EU public procurement notices from TED (Tenders Electronic Daily), the official EU tenders portal, through its public Search API: title, buyer, country, CPV codes, values, deadlines, lots and document links.
- [UK Companies House Scraper: Company Search & Profiles API](https://apify.com/rowcraft/uk-companies-house-scraper) — UK companies from the official Companies House API with your free key: search by name, status, type, SIC code, date or location, or look up company numbers.
- [UK Tenders Scraper: Find a Tender + Contracts Finder API](https://apify.com/rowcraft/uk-tenders-scraper) — UK public procurement notices from Find a Tender and Contracts Finder, the official UK government portals, through their open OCDS APIs: title, buyer, region, CPV codes, values, deadlines, notice type and links.
- [VTEX Products Scraper – Public Catalog Export](https://apify.com/rowcraft/vtex-products-scraper) — Export the catalogue of any VTEX store (Auchan, Chedraui and thousands of retailers in Europe and Latin America) from its public search endpoint: title, brand, EAN, prices, list prices, stock, categories, specifications and images, one row per product.
- [Website Tech Stack Detector – Technology Lookup API](https://apify.com/rowcraft/website-tech-stack-detector) — Find the technologies behind a list of websites: CMS, e-commerce platform, analytics, tag managers, ad pixels, consent tools, chat, CDN and payments, from one polite home-page request per domain.
- [WooCommerce Products Scraper – Store API Export](https://apify.com/rowcraft/woocommerce-products-scraper) — Export every product of a WooCommerce store through its public Store API: title, SKU, prices and sale prices, stock, categories, tags, attributes, ratings, images and links, one row per product.
- [Workable Jobs Scraper: Job Listings from Career Pages](https://apify.com/rowcraft/workable-jobs-scraper) — Job listings from any company's Workable career page: title, department, location, remote or on-site, employment type, description, requirements, benefits and salary when published.
- [Workday Jobs Scraper – Jobs from myworkdayjobs.com Sites](https://apify.com/rowcraft/workday-jobs-scraper) — Job listings from Workday career sites (myworkdayjobs.com) by company: title, locations, country, posting date, time type, requisition ID and the full description.

### FAQ

**Which URLs does it accept?** Addresses with `http://` or `https://`, and bare domains or paths such as `example.com/page`, which are checked as `https://`. Links with other schemes (`mailto:`, `tel:`, `ftp:`, `javascript:`) are skipped, and so are `#fragment`-only entries.

**What is the difference between HEAD and GET here?** HEAD returns the headers without the page, so it is lighter for the sites you check. Some servers answer HEAD with a 403, 404 or 405 and GET with a 200; the actor therefore asks again with GET whenever HEAD gets a 4xx or 5xx answer or fails, and uses the GET answer. For HTML pages with **Read the page head** on, a GET follows a successful HEAD to read the title and canonical. Choose `GET` to use GET for everything.

**Why is a URL missing from the dataset?** It was blocked (an anti-bot page, a 403 or a 429), disallowed by robots.txt on its host or on a host of its chain, invalid, a duplicate, a `mailto:` or `tel:` link, or unchanged with **Only new rows**. The `COVERAGE` record and the counts in `RUN_SUMMARY` say which, and none of them is charged.

**Why is the result empty?** Every URL was disallowed, blocked, invalid or unchanged, or the list was empty. Staging servers that disallow all robots, sites behind a bot-protection service and a second run with **Only new rows** are the usual causes. `COVERAGE` gives the reason per URL, and nothing is charged for them apart from the start event.

**Is a 404 or a dead domain charged?** Yes: it is the answer you asked for. A URL whose host does not exist, refuses the connection, times out or fails the TLS handshake is delivered as a row with `statusCode` null and an `error`. Only URLs that could not be checked at all (blocked, disallowed, invalid) are free.

**Why is a 403 not a row?** A 403 or a 429 usually means the site refuses automated readers, and the actor does not try to get around that. It is listed in `COVERAGE` with its status code and not charged. A 401 (login required) is delivered as a normal row.

**What does `canonicalMatchesFinal: false` mean?** The page announces a canonical URL that is not the address it was served from, for example `/wiki/Main_Page` announced by a page served from `/wiki/Main_page`. It is a signal to look at, not always an error.

**Does it follow JavaScript redirects?** No. It follows HTTP redirects and an HTML meta refresh with a delay of up to 5 seconds. A redirect made by a script is not seen.

**Is it legal?** The actor asks each server for the address you gave it, as any visitor or link checker does, obeys robots.txt on every host, keeps to one request per second per host by default and collects headers and four head tags, not personal data or page content. Whether you may check a given site is for you to judge; use the actor only on lists you have the right to check. How you use the results is your responsibility under the law that applies to you.

**Does it collect personal data?** No personal data: no e-mail addresses, phone numbers, people's names or cookie values. See the Privacy section.

**Can I check my own site faster?** Yes: set **Delay between requests to one host** to 250 ms, which is four requests per second per host. Different hosts are checked at the same time anyway.

**How do I monitor a migration?** Run the list once with **Only new rows** and a state name, fix what the first run shows, then schedule the same input daily or weekly. Each later run costs only the URLs whose status, destination or chain changed, and `notifyUrl` sends one message when there is something to look at.

### Support and reviews

**Problems, missing fields, questions:** open an issue on the **Issues** tab of this Actor. We answer within one working day and fix confirmed bugs in the next release; the changelog lists every change.

If this Actor saved you time, a short review on its page helps other users find it and helps us keep improving it.

# Changelog

This Actor's version history is a separate document: https://apify.com/rowcraft/url-status-redirect-checker/changelog.md

# Actor input Schema

## `urls` (type: `array`):

One URL per line, for example https://www.example.com/old-page. A bare domain is checked as https://. Duplicates, invalid entries and mailto: or tel: links are skipped and never charged; #fragments are dropped.

## `method` (type: `string`):

HEAD asks for the headers only and is the lightest; if a server refuses or mishandles HEAD (4xx, 5xx, a dropped connection) the URL is asked again with GET. GET asks every URL with GET.

## `followRedirects` (type: `boolean`):

Follow 301, 302, 303, 307, 308 and instant meta refresh redirects hop by hop and record every hop. Turn off to see only the first answer of each URL (its Location is still listed in the chain).

## `maxRedirects` (type: `integer`):

A chain longer than this ends with the error too_many_redirects. A chain that comes back to a URL it already visited ends with redirect_loop.

## `readHtmlSignals` (type: `boolean`):

For HTML answers read the first 256 KB and report the title, the canonical link, the meta robots and an instant meta refresh. Turn off to use headers only: faster, with one request fewer per page.

## `maxItems` (type: `integer`):

Stop after this many rows are delivered. 0 = the whole list.

## `onlyNew` (type: `boolean`):

Remember the rows already delivered to you (in a key-value store in your Apify account) and deliver only rows that appeared since the last run with the same state key. The first run delivers everything that matches; later runs cost only the new rows. Ideal for schedules.

## `stateName` (type: `string`):

Name of the memory used by "Only new rows". Use a different name for each schedule or search (for example `stockholm-python`) so they do not share one memory.

## `notifyUrl` (type: `string`):

A webhook address that receives one JSON POST with the number of new rows and links to the dataset: a Slack or Discord incoming webhook, a Make, n8n or Zapier catch hook, or your own endpoint. The message is also in the `text` (Slack) and `content` (Discord) fields.

## `notifyWhenEmpty` (type: `boolean`):

By default the notification is sent only when at least one row was delivered.

## `requestIntervalMs` (type: `integer`):

Default 1000 ms: one request per second to the same host. Different hosts are requested at the same time. A URL takes one to three requests; robots.txt is read once per host. Minimum 250.

## Actor input object example

```json
{
  "urls": [
    "http://github.com",
    "http://python.org",
    "https://example.com/missing-page",
    "http://www.wikipedia.org"
  ],
  "method": "HEAD",
  "followRedirects": true,
  "maxRedirects": 10,
  "readHtmlSignals": true,
  "maxItems": 0,
  "onlyNew": false,
  "stateName": "default",
  "notifyWhenEmpty": false,
  "requestIntervalMs": 1000
}
```

# Actor output Schema

## `results` (type: `string`):

One row per URL with its status, redirect chain and indexing signals, in the default dataset (JSON, CSV, Excel and other formats via the dataset API).

## `runSummary` (type: `string`):

Counts by status, redirected URLs, loops, hosts with anti-bot pages, warnings and why the run stopped (RUN_SUMMARY record in the default key-value store).

## `coverage` (type: `string`):

The URLs that were not delivered, with the reason: blocked, robots_disallowed, robots_unavailable, private_address, invalid or not_checked (COVERAGE record).

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "urls": [
        "http://github.com",
        "http://python.org",
        "https://example.com/missing-page",
        "http://www.wikipedia.org"
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("rowcraft/url-status-redirect-checker").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "urls": [
        "http://github.com",
        "http://python.org",
        "https://example.com/missing-page",
        "http://www.wikipedia.org",
    ] }

# Run the Actor and wait for it to finish
run = client.actor("rowcraft/url-status-redirect-checker").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "urls": [
    "http://github.com",
    "http://python.org",
    "https://example.com/missing-page",
    "http://www.wikipedia.org"
  ]
}' |
apify call rowcraft/url-status-redirect-checker --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,rowcraft/url-status-redirect-checker"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/8VXmbDuiGxv4Y4YDf/builds/yLa8F0fJKLpZbnqvF/openapi.json