# Broken Link Checker - Bulk URL Status & Redirect Chains (`clearfetch/broken-link-checker`) Actor Check URLs in bulk or crawl a page's links: status codes, the full redirect chain hop by hop, redirect loops, https downgrades, soft 404s that answer 200 while being error pages, and certificate expiry. No proxy, no login. - **URL**: https://apify.com/clearfetch/broken-link-checker.md - **Developed by:** [Nada Hanad](https://apify.com/clearfetch) (community) - **Categories:** SEO tools, Developer tools, Automation - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.50 / 1,000 url checkeds This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Actors are written with capital "A". ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. The best way to integrate Actors is as follows. - **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use). - **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`. - **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`. - **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`. - **Any other language** — the [REST API](https://docs.apify.com/api/v2.md). For usage examples, see the [API](#api) section below. For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt). # README ## Broken Link Checker - Bulk URL Status, Redirect Chains & Soft 404s Paste a list of URLs, or give it a page and let it check every link on that page. You get the status code, the full redirect chain hop by hop, and the three failures that ordinary link checkers miss: soft 404s, expired certificates, and bot protection misreported as a dead link. **$0.50 per 1,000 URLs checked, plus $0.002 per page crawled for links.** ### Why this one Most link checkers hand you a list of status codes and leave the judgement to you. The judgement is the hard part: - **Soft 404s.** A page that answers `200 OK` while actually saying "page not found" passes every status check ever written, and quietly rots your search presence. This flags them, and does it without flagging an article *about* 404 pages: the page has to be short as well as say so. - **Bot protection is not a broken link.** Capterra, G2, LinkedIn and most review sites answer `403` to any datacenter IP. A checker that files those under "broken" produces a report nobody can act on, so 401, 403 and 429 get their own `blocked` status and are explicitly **not** counted as broken. - **Certificates a browser rejects.** HTTP clients do not verify certificates the way browsers do, so an expired certificate returns a cheerful `200` to a scraper while every real visitor hits a full-page warning. Turn on certificate checking and those come back as `insecure`, with the issuer, the dates and the reason. - **A HEAD request is an optimisation, not a verdict.** Plenty of hosts accept a `HEAD` and never answer it while serving `GET` perfectly. HEAD gets a short leash here and anything less than a clean answer falls through to a real GET, which is why this does not invent unreachable hosts. - **Every hop, not just the destination.** Redirect chains are reported hop by hop with flags for loops, `https -> http` downgrades, and chains that end up on a different host, which is how expired links turn into parked-domain spam. ### What data you get Per URL: `status` (`ok`, `redirected`, `blocked`, `broken`, `server-error`, `soft-error`, `insecure`, `unreachable`), the HTTP `statusCode`, a single `broken` boolean for filtering, `finalUrl`, every redirect hop with its own status, a chain summary (`length`, `loop`, `mixedProtocol`, `leavesHost`), the soft-404 verdict with the evidence for it, certificate issuer and days to expiry, `contentType`, and `elapsedMs`. In crawl mode each link also carries **`foundOn`** (the page it was on) and **`anchor`** (the clickable text), so a report can name the link a person actually has to go and fix. ### How to use 1. Paste your links into **URLs to check**, one per line. Or put a page into **pages whose links to check** and every link on it is pulled out and checked. 2. Optionally turn on **Check TLS certificates**, or **Report problems only** for a short report. 3. Run it. One row per URL, exportable as JSON, CSV or Excel. ### Input | Field | Type | Default | Description | |-------|------|---------|-------------| | `urls` | array | — | The links to check, one per line. Also accepts `url` and `startUrls`. | | `crawlPages` | array | — | Pages to read first; every link on them is then checked. Combinable with `urls`. | | `includeExternal` | boolean | `true` | In crawl mode, also check links pointing off the site. These rot first. | | `detectSoftErrors` | boolean | `true` | Flag pages that answer 200 while being error pages. | | `checkCertificates` | boolean | `false` | Report issuer and days to expiry; a rejected certificate marks the link `insecure`. | | `onlyProblems` | boolean | `false` | Write only what is wrong, plus a summary row of how many were checked. | | `maxRedirects` | integer | `10` | Hops to follow before giving up. Every hop is reported. | | `maxConcurrency` | integer | `10` | URLs checked in parallel. | | `timeoutSecs` | integer | `20` | Per-request timeout. One URL gets at most twice this in total. | | `proxyConfiguration` | object | off | Optional. Not needed for most sites. | ### Output example Real output. Crawling `https://apify.com/` found 90 links; with four test URLs added, all 94 were checked in 14 seconds: 76 healthy, 11 redirected, 4 blocked by bot protection, one 404, one 500, one dead host. A healthy link, with the page it was found on: ```json { "url": "https://apify.com/store", "ok": true, "status": "ok", "statusCode": 200, "broken": false, "error": null, "certificate": { "issuer": "Amazon", "subject": "*.apify.com", "validFrom": "Jul 3 00:00:00 2026 GMT", "validTo": "Jan 16 23:59:59 2027 GMT", "expiresInDays": 132, "expired": false, "authorized": true, "authorizationError": null }, "contentType": "text/html; charset=utf-8", "foundOn": "https://apify.com/", "anchor": "Start here!Ready-to-run tools for your AI agents and apps.", "external": false, "elapsedMs": 106, "checkedAt": "2026-09-06T14:02:21.076Z" } ``` A three-hop redirect chain, every hop reported: ```json { "url": "https://httpbin.org/redirect/3", "finalUrl": "https://httpbin.org/get", "ok": true, "status": "redirected", "statusCode": 200, "broken": false, "redirects": [ { "url": "https://httpbin.org/redirect/3", "status": 302, "to": "https://httpbin.org/relative-redirect/2" }, { "url": "https://httpbin.org/relative-redirect/2", "status": 302, "to": "https://httpbin.org/relative-redirect/1" }, { "url": "https://httpbin.org/relative-redirect/1", "status": 302, "to": "https://httpbin.org/get" } ], "redirectChain": { "length": 3, "loop": false, "mixedProtocol": false, "leavesHost": false }, "contentType": "application/json", "elapsedMs": 1404, "checkedAt": "2026-09-06T14:02:22.373Z" } ``` A dead link, and a `403` that is *not* one: ```json { "url": "https://httpbin.org/status/404", "ok": false, "status": "broken", "statusCode": 404, "broken": true, "elapsedMs": 1221, "checkedAt": "2026-09-06T14:02:22.189Z" } ``` ```json { "url": "https://www.capterra.com/p/150854/Apify/", "ok": false, "status": "blocked", "statusCode": 403, "broken": false, "foundOn": "https://apify.com/", "external": true, "elapsedMs": 7541, "checkedAt": "2026-09-06T13:56:41.343Z" } ``` An expired certificate, which answers `200` to a scraper and is blocked by every browser: ```json { "url": "https://expired.badssl.com/", "ok": false, "status": "insecure", "statusCode": 200, "broken": true, "certificate": { "issuer": "COMODO CA Limited", "subject": "*.badssl.com", "validTo": "Apr 12 23:59:59 2015 GMT", "expiresInDays": -4165, "expired": true, "authorized": false, "authorizationError": "CERT_HAS_EXPIRED" }, "checkedAt": "2026-09-06T14:00:11.482Z" } ``` A host that does not resolve: ```json { "url": "https://this-domain-does-not-exist-12345.com/", "ok": false, "status": "unreachable", "statusCode": null, "broken": true, "error": "getaddrinfo ENOTFOUND this-domain-does-not-exist-12345.com", "checkedAt": "2026-09-06T14:02:33.918Z" } ``` ### Pricing - **$0.0005 per URL checked** — $0.50 per 1,000. A 2,000-link site audit costs a dollar. - **$0.002 per page crawled** for links, when you use crawl mode. - Every URL checked is charged once, healthy or broken, including when `onlyProblems` hides the healthy rows: confirming a link works is the same request as finding it dead. URLs that are skipped as invalid are free. ### Use cases - **SEO audits**: 404s, redirect chains that waste crawl budget, `https` downgrades, soft 404s. - **Site migrations**: check the old URL list against the new site before and after the switch. - **Documentation and link rot**: run it on a docs page or a link roundup on a schedule and get told when an external link dies. - **Certificate monitoring**: catch a certificate about to expire on a host you link to, before visitors do. - **Affiliate and directory links**: find the ones that now point at parked domains, using the `leavesHost` flag. - **AI agents**: a tool that answers "is this link still alive, and where does it end up". ### Integrations ```bash curl -X POST "https://api.apify.com/v2/acts/clearfetch~broken-link-checker/run-sync-get-dataset-items?token=YOUR_TOKEN" \ -H "Content-Type: application/json" \ -d '{"crawlPages": ["https://example.com/"], "onlyProblems": true}' ``` ```python from apify_client import ApifyClient client = ApifyClient("YOUR_TOKEN") run = client.actor("clearfetch/broken-link-checker").call( run_input={"crawlPages": ["https://example.com/"], "checkCertificates": True} ) for link in client.dataset(run["defaultDatasetId"]).iterate_items(): if link.get("broken"): print(link["status"], link["statusCode"], link["url"], "on", link.get("foundOn")) ``` Works with the Apify integrations for n8n, Make, Zapier, Google Sheets, Slack and webhooks, with scheduled runs, and with AI agents through the Apify MCP server. ### FAQ **Do I need a proxy?** No. A proxy input exists if you want to check links that block datacenter traffic, which is what turns a `blocked` row into a real verdict. **Why is a 403 not counted as broken?** Because it usually is not. Review sites, social networks and anything behind a WAF answer 403 to automated traffic while working fine for people. They are reported as `blocked` so you can see them without them drowning the real failures. **How do you tell a soft 404 from a page about 404s?** Length. An error page is short; an article titled "How to design a good 404 page" is not. The rule needs both the wording and a body under about 1,200 characters, and the output tells you which evidence triggered it. **What about a redirect loop?** Reported, with `loop: true` on the chain summary, and the hops that formed it. **How fast is it?** 94 links off a real homepage in 14 seconds at the default concurrency of 10, certificate checks included. **Does it check images and scripts too?** No, `` links only. Asset checking is a different job. **Is this legal?** It sends the same HTTP requests a browser sends when someone clicks the link, one per URL. **Do you have the URL list already?** If you need one, `clearfetch/website-sitemap-extractor` pulls every URL a site publishes, and its output feeds straight into this. ### Changelog - **1.0.0** (2026-09) — first release: bulk checking and page crawling, hop-by-hop redirect chains with loop, downgrade and off-host flags, soft-404 detection, `blocked` as a class of its own, and certificate validity. # Actor input Schema ## `urls` (type: `array`): The links to check, one per line. Also accepts "url" and "startUrls". ## `crawlPages` (type: `array`): Pages to read first; every link on them is then checked. This is how a broken-link audit usually starts. You can use this together with the list above. ## `includeExternal` (type: `boolean`): When crawling a page, also check links pointing off the site. Off-site links are the ones that rot first. ## `detectSoftErrors` (type: `boolean`): Flag pages that answer 200 while actually being error pages. These pass every ordinary status check, which is what makes them harmful. ## `checkCertificates` (type: `boolean`): Also report the certificate issuer and how many days until it expires. A certificate a browser would reject (expired, self-signed, wrong host) marks the link "insecure" and broken, because the visitor never gets past the warning page. Adds a TLS handshake per host. ## `onlyProblems` (type: `boolean`): Write only the URLs with something wrong, plus one summary row saying how many were checked. Every URL checked is charged either way, since confirming a link is healthy is the same work as finding it broken. ## `maxRedirects` (type: `integer`): How many hops to follow before giving up. Every hop is reported, so a chain longer than this is reported as far as it got. ## `maxConcurrency` (type: `integer`): URLs checked in parallel. ## `timeoutSecs` (type: `integer`): Per-request timeout. A single URL gets at most twice this in total, however many redirects it has. ## `proxyConfiguration` (type: `object`): Optional. Not needed for most sites. ## Actor input object example ```json { "urls": [], "crawlPages": [ "https://apify.com/" ], "includeExternal": true, "detectSoftErrors": true, "checkCertificates": false, "onlyProblems": false, "maxRedirects": 10, "maxConcurrency": 10, "timeoutSecs": 20, "proxyConfiguration": { "useApifyProxy": false } } ``` # Actor output Schema ## `results` (type: `string`): One row per URL: its status classification, HTTP status code, the full redirect chain hop by hop with loop and downgrade flags, whether it is a soft 404, certificate expiry when asked for, and which page the link was found on with its anchor text. # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "urls": [], "crawlPages": [ "https://apify.com/" ] }; // Run the Actor and wait for it to finish const run = await client.actor("clearfetch/broken-link-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "urls": [], "crawlPages": ["https://apify.com/"], } # Run the Actor and wait for it to finish run = client.actor("clearfetch/broken-link-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "urls": [], "crawlPages": [ "https://apify.com/" ] }' | apify call clearfetch/broken-link-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,clearfetch/broken-link-checker" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/boAtagtha9d41PRX1/builds/GiMeKNdqfRyqbuZbK/openapi.json