# Broken Link Checker – Find 404s and Dead Links API (`rowcraft/broken-link-checker`) Actor Pay per crawled page, not per link: crawl a site within robots.txt and check every link on each page for 404s, 5xx errors, DNS and TLS failures and redirects. Anti-bot and robots.txt refusals are never called broken. No personal data. - **URL**: https://apify.com/rowcraft/broken-link-checker.md - **Developed by:** [Rowcraft](https://apify.com/rowcraft) (community) - **Categories:** SEO tools, Developer tools - **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks - **User rating**: No ratings yet ## Pricing from $0.60 / 1,000 crawled pages This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events. Since this Actor supports Apify Store discounts, the price gets lower the higher subscription plan you have. Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event ## What's an Apify Actor? An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes. In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours, and optionally produces a well-defined JSON output, datasets with results, or files in key-value store. In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server. Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md. ## How to integrate an Actor? If asked about integration, you help developers integrate Actors into their projects. You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready. Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run. For examples already wired to this Actor's own input schema, see the [API](#api) section below. Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`). # README ## Broken Link Checker: Find 404s and Dead Links API Find the **broken links** of a website before your visitors and search engines do. Give the actor a site or a page; it crawls the pages of that site, verifies every link on each page (internal and external) and returns one row per page with the links that are broken, the links that redirect, and honest counts of the ones it could not verify. The result comes as JSON, CSV or Excel, ready for a spreadsheet, a ticket queue or a weekly alert. The price is per crawled page, not per link: a page with 300 links costs the same as a page with 3, and the links themselves are free. The actor reads robots.txt on every host before it sends a single request to it, so a link to a site that disallows robots is reported as `not_checked_robots` instead of being hit anyway, and an answer that only means "I do not talk to bots" (401, 403, 429) is never reported as broken. ![Input form of Broken Link Checker – Find 404s and Dead Links API](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/broken-link-checker-input.png) ### What you get One row per page that was read and checked: - **The page:** its address after redirects, the site, the depth from your start address, the HTTP status and the page title - **Counts that add up:** `linksFound` is the number of unique web addresses on the page, `linksChecked` the ones that got a verdict (working, redirected or broken), `notCheckedCount` the ones that could not be verified, with the reasons in `notCheckedReasons`, written as text such as `blocked_antibot: 2, not_checked_robots: 1`. Working plus redirected plus broken equals `linksChecked`; adding `notCheckedCount` gives `linksFound`. - **`brokenPreview`:** the first three broken addresses with their HTTP status or state as one line of text, so a table or a spreadsheet column reads without opening anything; the whole list is in `brokenLinks`. Empty when the page has no broken link - **`brokenLinks`:** every broken link with its address, the state (`broken_4xx`, `broken_5xx`, `dns_error`, `timeout`, `tls_error`, `connection_error`, `redirect_loop`), the HTTP status or the error text, the anchor text (cut to 80 characters, without e-mail addresses and phone numbers), whether the link is `internal` or `external`, the tag it came from (`a`, `link` or `img`) and, when it redirected first, the address where the chain ended - **`redirectedLinks`:** every link that redirects to a working page: the address you link to, the address it ends on, the number of hops and the status of the first redirect (301 or 302), so you can update the old addresses - **Change tracking:** a `brokenHash` that changes when the set of broken addresses of the page changes, and the time of the check The run also writes a `COVERAGE` record with one line per site you asked for (ok, blocked, robots_disallowed, not_found, error or not_checked, with the reason and the first pages that were not read) and a `RUN_SUMMARY` with the number of addresses in every state, the external hosts and robots.txt files read, and why the run stopped. **Limits, up front.** By default 50 pages per site (up to 1,000,000), the first 2 MB of each page, the first 2,000 links of a page, at most 1,000 external websites per run (adjustable), up to 1,000 sites per run, at most 5 redirects followed per link, and one request per second to any one host. Pages that build their links with JavaScript show only the links that are in the HTML. The details are in **Limits and fair use**. ### Why this actor - **Pay per page, not per link.** The unit is the crawled page. The price does not grow with the number of links on it, and you see the cost before the run: pages to crawl times the price per page. - **robots.txt on every host.** The crawl obeys the robots.txt of your site, and the external links are verified only after the robots.txt of their own host was read and allows the address. A host that disallows is counted as not checked, not as broken. Crawl-delay is honoured up to 10 seconds. - **No false alarms from bot protection.** Many large sites answer 403 or 429 to anything that is not a browser. Those links are listed as `blocked_antibot` in the counts and never as broken. An error answer to a HEAD request is repeated with a GET before the link is reported, because some servers answer HEAD wrongly. - **Redirect chains with their end.** Redirects are followed by hand, up to five hops, and every hop is checked against robots.txt. A link that redirects to a page that is gone is reported as broken with the address where the chain ended. - **Every address once per run.** A link in the menu of 500 pages is verified once and the verdict is reused, so the load on the sites you link to stays small. - **Only what changed.** With **Only new rows** a page is delivered again only when its set of broken links changed since the last run, so a weekly schedule costs only the pages whose problems are new or fixed. - **No personal data.** Links to e-mail addresses, phone numbers and messaging apps are skipped completely, anchor texts are stripped of e-mail addresses and phone numbers, and personal profile pages are never requested. ### How to use it 1. Open the Actor and paste your site into **Websites or pages**, a domain (`example.com`) or a page address (`https://www.example.com/blog`), one per line. 2. For a first try keep **Pages to crawl per site** at 5. Every page read is one row and one charge, and the links on it are free, so a small number shows you the result for a few cents. 3. Click **Start**. The status line shows the pages read and the broken links found so far. The run stops cleanly at your page limits or at the charge limit you set in the run options. 4. Open the **Dataset** tab. The **Overview** view lists the pages with the number of broken links, the first broken addresses as text, redirects and links not checked; the **Broken links** view shows the broken links of each page. Download JSON, CSV, Excel, XML or HTML, or send the rows to Google Sheets, Slack, Make, n8n or Zapier with the Integrations tab. 5. Open the `COVERAGE` record in the key-value store to see which sites could not be read and why, and which pages were left out. 6. Raise **Pages to crawl per site** to cover the whole site. To watch a site over time, run it on a schedule with **Only new rows since the last run** turned on. Checking your own site and want it faster? Lower **Delay between requests to one host** to 200 ms: it is your server and your decision. Links to other websites are always checked in parallel across hosts, one request at a time per host. ### Input | Field | What it does | |---|---| | `startUrls` | Domains (`example.com`) or page addresses (`https://www.example.com/blog`), one per line. Several addresses of one site are crawled together as one site. Up to 1,000 sites per run. | | `maxPagesPerSite` | How many pages of each site are read, breadth first from the start address. Default 50, up to 1,000,000. Each page read is one charged row. | | `checkExternalLinks` | Also verify links to other websites. Default on. Off: only the links inside the site are verified; links to other sites are counted as `not_checked_limit`. | | `includeImages` | Also verify the address of every image on the pages. Default off: images multiply the number of requests. | | `onlyPagesWithBrokenLinks` | Deliver (and charge) only the pages where a broken link was found. Clean pages are still read and checked, but skipped. | | `includeSubdomains` | Treat `blog.example.com` and `shop.example.com` as part of `example.com`: their pages are crawled and their links count as internal. Default off. | | `maxItems` | Stop after this many delivered rows in total. 0 = no limit. | | `maxExternalHosts` | At most this many different external websites are verified in one run (default 1,000); links to further websites are counted as not checked. | | `onlyNew`, `stateName` | Deliver a page only when its set of broken links changed since the last run with the same state name | | `notifyUrl`, `notifyWhenEmpty` | A webhook that receives one message when the run ends; see **Automation** below | | `requestIntervalMs` | Pause between requests to the same host, default 1000 ms, minimum 200 ms | Example: the first five pages of one site. ```json { "startUrls": ["https://the-internet.herokuapp.com/status_codes"], "maxPagesPerSite": 5 } ``` Example: a weekly check of two sites that delivers only pages with new or changed problems, with a Slack message. ```json { "startUrls": ["https://www.example.com", "https://shop.example.com"], "maxPagesPerSite": 500, "onlyPagesWithBrokenLinks": true, "onlyNew": true, "stateName": "weekly-links", "notifyUrl": "https://hooks.slack.com/services/..." } ``` #### What counts as a link - `a href`: every link in the text of the page - `link` for `canonical`, `alternate` (language versions), `next`, `prev` and `amphtml`: the addresses search engines use - `img src` (and the lazy-loading `data-src`) only with `includeImages` Skipped completely, not counted and not requested: `mailto:`, `tel:`, `sms:`, `javascript:`, `data:` and every other address that is not http or https, links that only point to a place on the same page (`#top`), addresses that hold an e-mail address, and pages of personal profiles. The part after `#` is dropped: the actor checks that the page exists, not that the anchor on it does. Scripts, style sheets, fonts, iframes and forms are not checked. #### The states of a link | State | Meaning | Counted as | |---|---|---| | `ok` | The address answers with a success status | working | | `redirect` | The address redirects (up to 5 hops) to a page that works | redirected, listed in `redirectedLinks` | | `broken_4xx` | 404, 410 or another client error (an error answered to a HEAD request is repeated with a GET before it is reported) | broken | | `broken_5xx` | The server answers with an error and still does so on the second try | broken | | `dns_error` | The host does not exist | broken | | `timeout` | The server did not answer in 15 seconds | broken | | `tls_error` | The secure connection failed (expired or wrong certificate, no TLS) | broken | | `connection_error` | The connection was refused or reset | broken | | `redirect_loop` | More than 5 redirects, or the redirects lead back to themselves | broken | | `blocked_antibot` | 401, 403, 429 or 999, or an anti-bot page: the server refuses automated readers | not checked | | `not_checked_robots` | The robots.txt of that host disallows the address, or the robots.txt could not be read | not checked | | `not_checked_private` | The address points to a private or local network (refused on the Apify platform) | not checked | | `not_checked_limit` | External links are switched off, the limit of external hosts was reached, or the page has more than 2,000 links | not checked | ### Output 17 fields per row. Empty values are `null`, never missing, so CSV and Excel columns are stable. | Field | Type | Example | |---|---|---| | `id` | string | `https://the-internet.herokuapp.com/status_codes` | | `pageUrl` | string | `https://the-internet.herokuapp.com/status_codes` | | `site` | string | the-internet.herokuapp.com | | `depth` | integer | 0 | | `statusCode` | integer | 200 | | `pageTitle` | string | The Internet | | `linksFound` | integer | 7 | | `linksChecked` | integer | 7 | | `brokenCount` | integer | 2 | | `redirectCount` | integer | 3 | | `notCheckedCount` | integer | 0 | | `notCheckedReasons` | string | blocked_antibot: 2, not_checked_robots: 1 | | `brokenPreview` | string | The first three broken addresses as one line of text with their HTTP status or state (empty when none are broken) | | `brokenLinks` | array | Every broken link: address, state, HTTP status or error, anchor text, internal or external, tag (a, link, img) and the final address when it redirected first | | `redirectedLinks` | array | Every link that redirects: address, final address, number of hops, status of the first redirect, anchor text, internal or external, tag | | `brokenHash` | string | c86a60f74210 | | `checkedAt` | string | 2026-10-14T17:33:47.630Z | #### Example rows Two real rows from a run (long texts shortened here, complete in the dataset): ```json [ { "id": "https://the-internet.herokuapp.com/status_codes", "pageUrl": "https://the-internet.herokuapp.com/status_codes", "site": "the-internet.herokuapp.com", "depth": 0, "statusCode": 200, "pageTitle": "The Internet", "linksFound": 7, "linksChecked": 7, "brokenCount": 2, "redirectCount": 3, "notCheckedCount": 0, "notCheckedReasons": null, "brokenPreview": "404 https://the-internet.herokuapp.com/status_codes/404, 500 https://the-internet.herokuapp.com/status_codes/500", "brokenLinks": [ { "url": "https://the-internet.herokuapp.com/status_codes/404", "status": "broken_4xx", "statusCode": 404, "error": null, "anchorText": "404", "type": "internal", "tag": "a", "finalUrl": null }, { "url": "https://the-internet.herokuapp.com/status_codes/500", "status": "broken_5xx", "statusCode": 500, "error": null, "anchorText": "500", "type": "internal", "tag": "a", "finalUrl": null } ], "redirectedLinks": [ { "url": "https://github.com/tourdedave/the-internet", "finalUrl": "https://github.com/saucelabs/the-internet", "hops": 1, "redirectCode": 301, "anchorText": "Fork me on GitHub", "type": "external", "tag": "a" }, { "url": "http://www.iana.org/assignments/http-status-codes/http-status-codes.xhtml", "finalUrl": "http://www.iana.org/assignments/http-status-codes", "hops": 1, "redirectCode": 302, "anchorText": "here", "type": "external", "tag": "a" }, { "url": "http://elementalselenium.com/", "finalUrl": "https://elementalselenium.com/", "hops": 1, "redirectCode": 301, "anchorText": "Elemental Selenium", "type": "external", "tag": "a" } ], "brokenHash": "c86a60f74210", "checkedAt": "2026-10-14T17:46:54.277Z" }, { "id": "https://www.sitemaps.org/faq.html", "pageUrl": "https://www.sitemaps.org/faq.html", "site": "sitemaps.org", "depth": 1, "statusCode": 200, "pageTitle": "sitemaps.org - FAQ", "linksFound": 11, "linksChecked": 11, "brokenCount": 1, "redirectCount": 10, "notCheckedCount": 0, "notCheckedReasons": null, "brokenPreview": "dns_error http://asg.web.cmu.edu/rfc/rfc3986.php", "brokenLinks": [ { "url": "http://asg.web.cmu.edu/rfc/rfc3986.php", "status": "dns_error", "statusCode": null, "error": "the host does not exist (no DNS address)", "anchorText": "RFC-3986", "type": "external", "tag": "a", "finalUrl": null } ], "redirectedLinks": [ { "url": "http://www.sitemaps.org/", "finalUrl": "https://www.sitemaps.org/", "hops": 1, "redirectCode": 301, "anchorText": "sitemaps.org", "type": "internal", "tag": "a" }, { "url": "https://www.sitemaps.org/protocol.php", "finalUrl": "https://www.sitemaps.org/protocol.html", "hops": 1, "redirectCode": 301, "anchorText": "Protocol", "type": "internal", "tag": "a" }, { "url": "https://www.sitemaps.org/index.php", "finalUrl": "https://www.sitemaps.org/index.html", "hops": 1, "redirectCode": 301, "anchorText": "Home", "type": "internal", "tag": "a" }, "… 7 more" ], "brokenHash": "aaa5433eb63f", "checkedAt": "2026-10-14T17:47:03.353Z" } ] ``` Results can be downloaded as JSON, CSV, Excel, XML or HTML from the Dataset tab, or read through the Apify API. ![Results of Broken Link Checker – Find 404s and Dead Links API in the Apify dataset view](https://api.apify.com/v2/key-value-stores/7hbQFQdGq61v2OZ2h/records/broken-link-checker-results.png) ### Pricing The unit is the **crawled page**: one page of the site read and checked with all of its links. A page with 3 links and a page with 3,000 links cost the same, and a link that is checked on 200 pages is still verified once and still free. Cost of a run = pages delivered times the price per page; set **Pages to crawl per site** and you know the most it can cost. Sites that could not be read, pages skipped by **Only pages with broken links**, pages with an unchanged set of broken links under **Only new rows**, and pages that the charge limit stopped are not charged. **Pay per result.** You pay for each page delivered to your dataset, nothing else: no subscription, no minimum, no charge for rows your filters drop or for the requests behind them. | Your Apify plan | Price per 1,000 rows | Price per row | |---|---|---| | Free plan | **$0.90** | $0.0009 | | Starter plan | **$0.80** | $0.0008 | | Scale plan | **$0.70** | $0.0007 | | Business plan and above | **$0.60** | $0.0006 | Example: 1,000 pages cost $0.90 on the Free plan and $0.60 on the Business plan. The Apify platform usage of a run (compute, storage) is small for this actor — well under one cent per 1,000 rows — and is billed by Apify separately, as for every Actor. A start event of $0.00005 is charged once per run (Apify default). **Cap your spend:** set *Maximum total charge* in the run options. The actor stops cleanly when the cap is reached and you keep everything delivered until then. The run summary (`RUN_SUMMARY` in the key-value store) says why a run stopped. ### Automation: only new rows, notifications, schedules **Only new rows since the last run.** Turn on `onlyNew` and the actor remembers, for every page it delivered to you, the set of broken addresses it had (in a key-value store named `rowcraft-broken-link-checker-state` in your Apify account). The next run with the same `stateName` delivers a page again only when that set changed: a link broke, or a broken link was fixed or removed. A page whose problems are the same as last time is skipped and not charged. The first run delivers every page it can read. Use a different `stateName` for each site or schedule so they keep separate memories. **Only pages with broken links, together with Only new rows.** Set both: the run delivers a page only when it has broken links that are new or different since the last time. When a page becomes clean it is not delivered, but its old memory is cleared, so if it breaks the same way again later you are told again. **Notification when the run ends.** Put a webhook address in `notifyUrl` and the actor sends one JSON POST with the number of delivered rows, links to the dataset (JSON and CSV) and the run. It works with Slack and Discord incoming webhooks (the message is in `text` and `content`), with Make, n8n and Zapier catch hooks, and with your own endpoint. By default it is sent only when at least one row was delivered; `notifyWhenEmpty` sends it every time. **Schedules.** Create a schedule in Apify Console (daily, weekly, monthly or cron) with `onlyNew` on and you get a feed of the pages whose broken links changed, for the price of those pages only. **Integrations.** Send the results to Make, n8n, Zapier, Google Sheets, Slack, Google Drive, Airtable or any webhook from the Integrations tab of the Actor, with no code. **API, SDKs and AI agents.** Every run is available through the Apify API and the official JavaScript and Python clients; the Actor can also be called by AI agents through the Apify MCP server. The example input in the API tab is a working call. ### Use cases - **Site owners and webmasters:** a monthly list of the pages that link to dead addresses, with the anchor text so the link is easy to find in the editor. - **SEO and content teams:** find broken internal links, canonical and language-version links that point nowhere, and redirect chains that should be updated to the final address, before they cost crawl budget and rankings. - **Agencies:** a scheduled check of every client site with **Only pages with broken links** and a Slack message, so only the sites with new problems cost attention. - **After a migration or redesign:** crawl the new site and list every link that leads to a missing page or to a redirect, then fix the old addresses. - **Editors of large archives:** external links rot. Run the site on a schedule and replace the dead references before readers find them. - **Quality checks in a pipeline:** call the Actor from CI after a deploy and fail the build when `brokenCount` is above zero. Ready-made examples, each with the input filled in (open one and press Start, or save it as your own task): - [Check a website for broken links and redirects](https://apify.com/rowcraft/broken-link-checker/examples/check-a-website-for-broken-links) - [Find dead links on a site from the early web](https://apify.com/rowcraft/broken-link-checker/examples/dead-links-on-an-early-web-site) ### What this actor does not do - Run JavaScript: links that a script builds after the page loads are not in the HTML and are not seen. Single-page apps that render their menus in the browser show only the links they send in the first HTML. - Log in: pages behind a login are not read, and a link to such a page that answers 401 or 403 is counted as not checked. - Check anchors: a link to `page.html#section` is verified as `page.html`; the actor does not look for the section. - Check scripts, style sheets, fonts, iframes, forms or downloads inside pages; images only when you switch `includeImages` on. - Read sitemaps: the crawl follows the links of the pages, it does not read the addresses listed in a sitemap file. - Audit SEO: it does not judge titles, meta tags, headings or page speed. - Collect contacts: no e-mail addresses, phone numbers, names of people or social profiles, and no page text apart from the title and the short anchor texts. - Bypass protections: no proxies, no CAPTCHA solving, no browser disguise. A server that refuses automated readers is reported as not checked. - Fix the links or write to the site owners. ### Limits and fair use - **One request per second per host** by default (minimum 200 ms). Different hosts are served in parallel, with at most eight requests in flight at once. Links to other websites each cost one robots.txt request per host and one or two requests per address, so a page that links to hundreds of other websites takes a while: they are checked in parallel, but each host only once a second. - **Crawl-delay.** A `Crawl-delay` in the robots.txt of a host is honoured up to 10 seconds between requests to that host. - **Pages per site:** 50 by default, up to 1,000,000. The crawl is breadth first from the start address and covers only the same site: `example.com` and `www.example.com` are one site, other subdomains only with `includeSubdomains`. Addresses beyond the limit are still verified as links on the pages that were read; they just do not become rows. - **Page size:** the first 2 MB of a page are read and the rest is cut. Pages that are not HTML (PDF files, images, downloads) are checked as links, not crawled. - **Links per page:** the first 2,000 unique addresses of a page are checked; the rest are counted as `not_checked_limit` and the run summary says so. - **External websites:** at most 1,000 different external hosts per run by default; raise `maxExternalHosts` for large sites. Links beyond the limit are counted as not checked. - **Redirects:** up to five hops per link, every hop checked against robots.txt. - **Time-outs:** 15 seconds for a link, 25 seconds for a page. A host that does not answer is tried once for all the links that point to it. - **robots.txt is obeyed** for every address the actor requests, on your site and on every external host. If the robots.txt of your own start page disallows it, the site is reported as `robots_disallowed` and nothing is read. If a robots.txt answers with a server error the host is not read, as the robots.txt standard (RFC 9309) requires. - **Anti-bot pages.** If five pages of a site in a row answer with an anti-bot page the crawl of that site stops and says so in `COVERAGE`; those pages are not charged. - **Time limit.** The run stops 90 seconds before its timeout with the summary, the coverage and the **Only new rows** memory saved. A page whose links were not all checked by then is not delivered and not charged. For large sites raise the timeout, lower the delay on your own site, or split the crawl. ### Data source and licence The data is what each website sends to every visitor: the HTML of its pages and the answer line and headers of the addresses it links to. The actor requests the pages and addresses once per run, after reading robots.txt, with an honest User-Agent that names Rowcraft, and keeps only the address, the state, the status code, a short anchor text and the page title. Page text, images, headers and cookies are not stored or delivered. Rowcraft is independent: this actor is not provided, endorsed or authorised by the websites it reads or by search engines. The robots.txt handling follows RFC 9309 (`https://www.rfc-editor.org/rfc/rfc9309`). You decide which sites to crawl and answer for the list you give: crawl your own sites, or sites whose owners allow it. ### Privacy: no personal data Rowcraft actors are built not to collect personal data: - Links to e-mail addresses, phone numbers, text messages and `javascript:` are skipped completely, so they are neither requested nor counted nor listed. - Addresses that contain an e-mail address, and addresses of personal profile pages, are skipped too. - Anchor texts are cut to 80 characters and an e-mail address or phone number inside one is replaced by a marker. Page titles get the same treatment. - The actor sends no cookies, fills no forms and never logs in. You are responsible for using the data in line with the law that applies to you. ### Other Rowcraft actors All Rowcraft actors read official or public sources, deliver flat rows without personal data and support only-new runs and notifications: - [Ashby Jobs Scraper – Job Postings with Salary Ranges](https://apify.com/rowcraft/ashby-jobs-scraper) — Job postings from Ashby job boards (jobs.ashbyhq.com) by company: title, department, team, locations with country, employment and workplace type, published salary range, full text. - [Bulk Image Downloader API: Save Page Images with Sizes](https://apify.com/rowcraft/bulk-image-downloader) — Download the images of any web pages, or direct image links, into a key-value store with width, height, format, size and a download link per file. - [Career Site Jobs Scraper: Greenhouse, Lever, Ashby, Workday](https://apify.com/rowcraft/career-site-jobs-scraper) — Job listings from the career pages of the companies you choose on Greenhouse, Lever, Ashby, Workday, Workable, Personio, Recruitee, Teamtailor and Gem, in one run and one format: title, locations, department, salary when published, full text. - [Chrome Web Store Scraper: Users, Ratings & Versions](https://apify.com/rowcraft/chrome-web-store-scraper) — Reads public Chrome Web Store listing pages with no login, no proxy and no personal data: users, rating, category, version, update date, permissions and company developers. - [Domain Email and DNS Checker API: MX, SPF, DMARC, Providers](https://apify.com/rowcraft/domain-email-dns-checker) — Find which email provider, sending services and DNS host each domain uses, from public DNS records only. - [SIRENE France Company Scraper: SIREN, NAF, Revenue, Status](https://apify.com/rowcraft/france-company-register) — French companies from the official Recherche d'entreprises API (INSEE SIRENE + INPI): SIREN and SIRET, legal form, NAF activity, head-office address, employee band, revenue and net income, labels, creation date and VAT number. - [Gem Jobs Scraper – Job Board Listings from jobs.gem.com](https://apify.com/rowcraft/gem-jobs-scraper) — Job listings from Gem job boards (jobs.gem.com) by company: title, department, offices with country, employment type, remote, hybrid or on-site, requisition ID, publication date and full ad text, from the public Gem Job Board API. - [Google Play Apps Scraper: App Details, Installs, Ratings](https://apify.com/rowcraft/google-play-apps-scraper) — Public Google Play app details as flat rows: installs, rating, number of ratings, price, version, last update, minimum Android, ads and in-app purchases. - [Greenhouse Jobs Scraper – Job Listings by Company Board](https://apify.com/rowcraft/greenhouse-jobs-scraper) — Job listings from Greenhouse job boards by company: title, departments, offices, publication date, full ad text, published pay ranges. - [Himalayas Jobs Scraper: Remote Jobs API with Salaries](https://apify.com/rowcraft/himalayas-jobs-scraper) — Remote jobs from Himalayas through its public jobs API: title, company, salary range, seniority, categories, the countries and time zones open to candidates, full description and job link. - [KRS Poland Companies Scraper: Register Data, PKD, Status](https://apify.com/rowcraft/krs-poland-companies) — Polish companies from the official KRS register API of the Ministry of Justice: name, KRS, NIP and REGON numbers, legal form, status, address, PKD activity codes, share capital, branches and registration dates. - [LEI Records Scraper: Legal Entity Identifier Lookup API](https://apify.com/rowcraft/lei-records-scraper) — Legal Entity Identifier (LEI) records from the Global LEI Index, read through the public GLEIF API: legal name, legal form, register number, addresses, status, renewal date and parent companies. - [Lever Jobs Scraper – Job Postings from jobs.lever.co](https://apify.com/rowcraft/lever-jobs-scraper) — Job postings from Lever career pages (jobs.lever.co) by company: title, team, department, location, commitment, workplace type, salary range when published, description, requirements and benefits. - [Norway Company Register Scraper: Brønnøysund Business Data](https://apify.com/rowcraft/norway-company-register-scraper) — Norwegian companies from the Central Coordinating Register for Legal Entities (Brønnøysund, NLOD open data): organisation number, legal form, industry, employees, share capital, addresses, website, VAT and bankruptcy status. - [PDF Text Extractor: Text, Pages and Metadata from PDF Links](https://apify.com/rowcraft/pdf-text-extractor) — Get the text of every PDF you link, page by page, with page count, language and word count. - [Personio Jobs Scraper: Job Listings, Salaries, Departments](https://apify.com/rowcraft/personio-jobs-scraper) — Job ads from any company's Personio careers feed: title, department, office, employment type, schedule, seniority, salary when published, keywords and the full description in the language you choose. - [Platsbanken Scraper: Sweden Jobs, Employers, Org Numbers](https://apify.com/rowcraft/platsbanken-jobs-scraper) — Every job ad on Platsbanken (Arbetsförmedlingen) from the official JobTech API: headline, employer and organisation number, occupation codes, municipality, coordinates, salary type, deadline and full text. - [Recruitee Jobs Scraper – Job Listings by Company Career Site](https://apify.com/rowcraft/recruitee-jobs-scraper) — Job listings from Recruitee career sites by company: title, department, locations with country, employment type, remote flag, published salary and full text, one request per company. - [Robots.txt and AI Crawler Access Checker API](https://apify.com/rowcraft/robots-ai-crawler-checker) — Check which AI and search crawlers a website lets in: GPTBot, ClaudeBot, PerplexityBot, Google-Extended and more, read from robots.txt by the RFC 9309 rules. - [RSS Feed Finder and Reader – Blog and News Article API](https://apify.com/rowcraft/rss-feed-finder-reader) — Find the RSS, Atom and JSON feeds of any website and read their newest articles, from a domain name alone. - [SEO Audit Tool: Titles, Meta Tags, Headings and Links API](https://apify.com/rowcraft/seo-audit-tool) — Audit the on-page SEO of any list of pages or websites: titles, meta descriptions, headings, canonicals, indexing, links, images and structured data, with a score and the issues found per page. - [Shopify Products Scraper: Store Catalog Export to JSON, CSV](https://apify.com/rowcraft/shopify-products-scraper) — Products from any Shopify store's public catalog: title, vendor, type, tags, description, variants with price, compare-at price, SKU and stock, images, options, store currency. - [Sitemap URL Extractor: Every Page with Last-Modified Dates](https://apify.com/rowcraft/sitemap-url-extractor) — Get every page a site lists in its sitemaps, with last-modified dates, from robots.txt through sitemap indexes and .gz files. - [SSL Certificate and Security Headers Checker API](https://apify.com/rowcraft/ssl-security-headers-checker) — Check the TLS certificate and the HTTP security headers of every domain in one run, with an A+ to F grade and the exact issues to fix. - [Store Products Scraper – Shopify, WooCommerce, VTEX](https://apify.com/rowcraft/store-products-scraper) — Export the product catalogue of any Shopify, WooCommerce or VTEX store from its public catalogue endpoint: title, brand, prices, discounts, stock, categories, images and links, one row per product, the same columns on every platform. - [Structured Data Extractor: JSON-LD, Schema.org, Open Graph](https://apify.com/rowcraft/structured-data-extractor) — Pull the schema.org JSON-LD, Microdata, RDFa and Open Graph of any page list into flat rows: types, product price, availability, ratings, organization, breadcrumbs, with validation issues. - [Teamtailor Jobs Scraper – Career Site Jobs by Company](https://apify.com/rowcraft/teamtailor-jobs-scraper) — Job listings from Teamtailor career sites by company: title, department, role, locations with city and country, remote or hybrid status, publication date and full ad text, from the official job feed. - [TED Tenders Scraper: EU Public Procurement Notices API](https://apify.com/rowcraft/ted-tenders-scraper) — EU public procurement notices from TED (Tenders Electronic Daily), the official EU tenders portal, through its public Search API: title, buyer, country, CPV codes, values, deadlines, lots and document links. - [UK Companies House Scraper: Company Search & Profiles API](https://apify.com/rowcraft/uk-companies-house-scraper) — UK companies from the official Companies House API with your free key: search by name, status, type, SIC code, date or location, or look up company numbers. - [UK Tenders Scraper: Find a Tender + Contracts Finder API](https://apify.com/rowcraft/uk-tenders-scraper) — UK public procurement notices from Find a Tender and Contracts Finder, the official UK government portals, through their open OCDS APIs: title, buyer, region, CPV codes, values, deadlines, notice type and links. - [Bulk URL Status and Redirect Chain Checker API](https://apify.com/rowcraft/url-status-redirect-checker) — Check the status code, redirect chain, canonical and indexing signals of every URL in a list, with the final destination and a loop flag. - [VTEX Products Scraper – Public Catalog Export](https://apify.com/rowcraft/vtex-products-scraper) — Export the catalogue of any VTEX store (Auchan, Chedraui and thousands of retailers in Europe and Latin America) from its public search endpoint: title, brand, EAN, prices, list prices, stock, categories, specifications and images, one row per product. - [Website to Markdown Extractor: Clean Page Text for LLMs and RAG](https://apify.com/rowcraft/website-markdown-extractor) — Turn any list of web pages into clean Markdown and plain text for LLMs and RAG: the main content only, with headings, links, images and tables. - [Website Screenshot API: Full-Page PNG, JPEG and PDF Captures](https://apify.com/rowcraft/website-screenshot-api) — Capture full-page or viewport screenshots of any public web page as PNG, JPEG or WebP, with an optional A4 PDF. - [Website Tech Stack Detector – Technology Lookup API](https://apify.com/rowcraft/website-tech-stack-detector) — Find the technologies behind a list of websites: CMS, e-commerce platform, analytics, tag managers, ad pixels, consent tools, chat, CDN and payments, from one polite home-page request per domain. - [WooCommerce Products Scraper – Store API Export](https://apify.com/rowcraft/woocommerce-products-scraper) — Export every product of a WooCommerce store through its public Store API: title, SKU, prices and sale prices, stock, categories, tags, attributes, ratings, images and links, one row per product. - [Workable Jobs Scraper: Job Listings from Career Pages](https://apify.com/rowcraft/workable-jobs-scraper) — Job listings from any company's Workable career page: title, department, location, remote or on-site, employment type, description, requirements, benefits and salary when published. - [Workday Jobs Scraper – Jobs from myworkdayjobs.com Sites](https://apify.com/rowcraft/workday-jobs-scraper) — Job listings from Workday career sites (myworkdayjobs.com) by company: title, locations, country, posting date, time type, requisition ID and the full description. ### FAQ **What exactly does it call a broken link?** An address that answers 404, 410 or another client error, a server error that is still there on the second try, a host that does not exist, a secure connection that fails, a connection that is refused or does not answer in 15 seconds, and a redirect chain that loops or is longer than five hops. Answers that mean "I do not serve robots" (401, 403, 429, 999) and hosts whose robots.txt disallows the address are counted as not checked, because the page may be perfectly fine for a visitor. **Why does it need two requests for some links?** A HEAD request is cheap and answers most links. Some servers answer HEAD with 405, 404 or 403 but serve the same address fine with a normal request, so the actor repeats any error answer with a one-byte GET before it calls a link broken. That keeps false alarms down. **Why is the result empty?** The site could not be read, or every page was skipped. The `COVERAGE` record gives the reason per site: an anti-bot page (`blocked`), robots.txt (`robots_disallowed`), no website at that address (`not_found`), an invalid entry or a connection problem (`error`). With **Only pages with broken links** a site without broken links delivers nothing, and with **Only new rows** pages whose broken links did not change are skipped. Nothing is charged for those, apart from the start event. The `RUN_SUMMARY` shows how many pages were read and skipped. **Is it legal?** The actor reads public pages the way a browser or a search engine does, obeys robots.txt on every host, identifies itself, and collects links and statuses, not personal data. Crawling your own site is your decision; for sites of others, follow their terms. Whether and how you may use the results depends on the law that applies to you. **Does it collect personal data?** No personal data: no e-mail addresses, phone numbers or people's names. Links to e-mail addresses and phone numbers are skipped completely and never listed, and anchor texts are cleaned. See the Privacy section. **Do I pay for links?** No. You pay for crawled pages. Links, redirects and checks of other websites are included, and a link that appears on many pages is verified once. **Do I pay for pages that fail?** No. A page that answers with an error, an anti-bot page or a robots.txt refusal is not a row and is not charged. With **Only pages with broken links** clean pages are skipped and not charged either. **Why is a link not checked?** Its host disallows robots in robots.txt (`not_checked_robots`), the server refuses automated readers (`blocked_antibot`), it points to a private network (`not_checked_private`), external links are switched off or beyond your limits (`not_checked_limit`). They are counted in `notCheckedCount` with the reason in `notCheckedReasons`, so you know what was not verified. **A page with a broken link in the browser is shown as working. Why?** Some pages answer 200 for a missing page and show a message instead (a soft 404). The actor goes by the HTTP status, so it cannot tell. Pages that need JavaScript to load their content are read as the HTML the server sends. **How long does a crawl take?** At the default speed a site is read at about one request per second to its own host: every page read, and every internal address that is not read as a page, costs one request. Links to other websites run in parallel across hosts. A small site of 50 pages takes a few minutes; your own site can be checked faster by lowering the delay. **Can I check only some pages?** Yes: give the page addresses you want as start addresses and set **Pages to crawl per site** to the number of pages. The crawl starts at your address and follows the links of the same site, breadth first. **Can I call it from code or from CI?** Yes, through the Apify API, the JavaScript and Python clients and the Apify MCP server. The API tab shows a working call with the input of this Actor. ### Support and reviews **Problems, missing fields, questions:** open an issue on the **Issues** tab of this Actor. We answer within one working day and fix confirmed bugs in the next release; the changelog lists every change. If this Actor saved you time, a short review on its page helps other users find it and helps us keep improving it. # Changelog This Actor's version history is a separate document: https://apify.com/rowcraft/broken-link-checker/changelog.md # Actor input Schema ## `startUrls` (type: `array`): The sites to check, one per line: a domain (example.com) or a page address (https://www.example.com/blog). The crawl starts there and follows the links of the same site. Several addresses of one site are crawled together. ## `maxPagesPerSite` (type: `integer`): How many pages of each site are read, breadth first from the start address. Every page read is one charged row; the links on it are checked for free. Start small (5 to 20) to see the result, then raise it. ## `checkExternalLinks` (type: `boolean`): Also verify the links that lead to other websites (one request per address, robots.txt of each host read first). Switch off to check only the links inside the site. ## `includeImages` (type: `boolean`): Verify the address of every image (img tag) on the pages as well. Off by default: images multiply the number of requests. ## `onlyPagesWithBrokenLinks` (type: `boolean`): Deliver (and charge) only the pages where at least one broken link was found. Pages without broken links are still read and checked, but are skipped and not charged. ## `includeSubdomains` (type: `boolean`): Treat blog.example.com and shop.example.com as part of example.com: their pages are crawled and their links count as internal. Off: subdomains are checked as external links only. ## `maxItems` (type: `integer`): Stop after this many rows are delivered in total. 0 = no limit. ## `maxExternalHosts` (type: `integer`): A page that links to hundreds of other websites costs one robots.txt and one request for each of them. Links to websites beyond this number are counted as not checked. ## `onlyNew` (type: `boolean`): Remember the rows already delivered to you (in a key-value store in your Apify account) and deliver only rows that appeared since the last run with the same state key. The first run delivers everything that matches; later runs cost only the new rows. Ideal for schedules. ## `stateName` (type: `string`): Name of the memory used by "Only new rows". Use a different name for each schedule or search (for example `stockholm-python`) so they do not share one memory. ## `notifyUrl` (type: `string`): A webhook address that receives one JSON POST with the number of new rows and links to the dataset: a Slack or Discord incoming webhook, a Make, n8n or Zapier catch hook, or your own endpoint. The message is also in the `text` (Slack) and `content` (Discord) fields. ## `notifyWhenEmpty` (type: `boolean`): By default the notification is sent only when at least one row was delivered. ## `requestIntervalMs` (type: `integer`): Default 1000 ms (one request per second to the same host; different hosts are checked in parallel). The Crawl-delay of a robots.txt is honoured up to 10 s. Checking your own site? 200 ms is the minimum. ## Actor input object example ```json { "startUrls": [ "https://the-internet.herokuapp.com/status_codes" ], "maxPagesPerSite": 5, "checkExternalLinks": true, "includeImages": false, "onlyPagesWithBrokenLinks": false, "includeSubdomains": false, "maxItems": 0, "maxExternalHosts": 1000, "onlyNew": false, "stateName": "default", "notifyWhenEmpty": false, "requestIntervalMs": 1000 } ``` # Actor output Schema ## `results` (type: `string`): One row per page read, with its broken and redirected links, in the default dataset (JSON, CSV, Excel and other formats via the dataset API). ## `runSummary` (type: `string`): Counts, the state of every address checked, warnings and why the run stopped (RUN_SUMMARY record in the default key-value store). ## `coverage` (type: `string`): One line per site you asked for: ok, blocked, robots_disallowed, not_found, error or not_checked, with the reason and the pages not read (COVERAGE record). # API You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup. ## JavaScript example ```javascript import { ApifyClient } from 'apify-client'; // Initialize the ApifyClient with your Apify API token // Replace the '' with your token const client = new ApifyClient({ token: '', }); // Prepare Actor input const input = { "startUrls": [ "https://the-internet.herokuapp.com/status_codes" ], "maxPagesPerSite": 5 }; // Run the Actor and wait for it to finish const run = await client.actor("rowcraft/broken-link-checker").call(input); // Fetch and print Actor results from the run's dataset (if any) console.log('Results from dataset'); console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`); const { items } = await client.dataset(run.defaultDatasetId).listItems(); items.forEach((item) => { console.dir(item); }); // 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs ``` ## Python example ```python from apify_client import ApifyClient # Initialize the ApifyClient with your Apify API token # Replace '' with your token. client = ApifyClient("") # Prepare the Actor input run_input = { "startUrls": ["https://the-internet.herokuapp.com/status_codes"], "maxPagesPerSite": 5, } # Run the Actor and wait for it to finish run = client.actor("rowcraft/broken-link-checker").call(run_input=run_input) # Fetch and print Actor results from the run's dataset (if there are any) print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}") for item in client.dataset(run.default_dataset_id).iterate_items(): print(item) # 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start ``` ## CLI example ```bash echo '{ "startUrls": [ "https://the-internet.herokuapp.com/status_codes" ], "maxPagesPerSite": 5 }' | apify call rowcraft/broken-link-checker --silent --output-dataset ``` ## MCP server setup ```json { "mcpServers": { "apify": { "type": "http", "url": "https://mcp.apify.com/?tools=fetch-actor-details,rowcraft/broken-link-checker" } } } ``` The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer ` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations). ## OpenAPI specification Download the OpenAPI definition: https://api.apify.com/v2/actors/AJimreKDalAwix2qg/builds/8CwWoMBrWttSEkAEN/openapi.json