Broken Link Checker: Find Every 404 on Your Whole Site

Somewhere on your site there is a link pointing at a page that does not exist any more, and today you only find out when a visitor tells you.

This Actor takes a domain, crawls the site by following internal links, and returns one row per broken link: the address that failed, the HTTP status or the network error, the page the link sits on, and the anchor text you have to search for in your CMS.

Who runs it, and when

People who are responsible for a site and cannot click every link by hand:

• site owners and marketing teams after a redesign, a CMS migration or a bulk URL change, when old addresses stop answering;
• agencies and freelancers who take over someone else's site and need a list of what is already broken before they touch anything;
• technical writers and documentation maintainers, where a dead link in a tutorial is a support ticket;
• anyone doing a routine check, monthly or before a release, on a site whose outbound links rot on their own.

The usual moment is: you changed something, or you inherited something, and you need the list of dead links in one file instead of a browser tab per page.

What comes out, field by field

The run writes one dataset row per broken link found, plus one summary row at the end. Both are in the same dataset, and the rowType field tells them apart, so you can filter with one comparison.

Rows with rowType: "broken-link"

field	type	what it holds
rowType	string	always broken-link on a finding row
brokenUrl	string	the link target that failed
statusCode	integer or null	the HTTP status the target returned (404, 410, 500...), null when the request itself failed
error	string or null	the network level failure when there is no HTTP status: dns_error, connect_timeout, read_timeout, ssl_error, too_many_redirects
sourceUrl	string	the crawled page that contains the link
anchorText	string	the clickable text of the link, so you can find it in the page
linkType	string	internal or external
checkedAt	string	when the target was checked, ISO 8601 in UTC

One row per place the link appears: a dead address linked from five pages gives five rows, one per source page, because five pages need editing.

The row with rowType: "summary"

Every successful run writes this row, findings or no findings, so a clean site never hands you an empty dataset.

field	type	what it holds
rowType	string	always summary
startUrl	string	the URL the crawl started from, after normalisation
pagesVisited	integer	how many pages were opened and parsed
linksFound	integer	how many links were seen in total, counting repeats
urlsChecked	integer	how many distinct targets were actually requested
brokenLinks	integer	how many finding rows this run wrote
uniqueBrokenTargets	integer	how many distinct addresses those rows point at
skippedByRobots	integer	targets not requested because robots.txt disallows them
durationSeconds	number	how long the run took
chargeLimitReached	boolean	true when the run stopped early on your pay-per-event limit
chargedEvents	integer	how many url-checked events the run charged
finishedAt	string	when the run ended, ISO 8601 in UTC
message	string	the same numbers in one sentence, for a human reading the dataset

A fuller version of the same summary is also written to the key-value store under the key SUMMARY. It adds the settings the run used (maxPages, maxDepth, delaySeconds, concurrency), uniqueTargets, requestsCharged and chargeFailures.

A real finding row

Lines 94 to 102 of saidas/docs-apify-com.json in this repository, from a run against the public site https://docs.apify.com/ on 19/09/2026:

{
  "brokenUrl": "https://docs.apify.com/residential_proxy",
  "statusCode": 404,
  "error": null,
  "sourceUrl": "https://docs.apify.com/proxy",
  "anchorText": "residential proxies",
  "linkType": "internal",
  "checkedAt": "2026-09-19T22:32:12+00:00"
}

Read it as: the page /proxy has a link whose text is "residential proxies", it points at /residential_proxy, and that address answers 404. That run predates the rowType field; current builds add rowType to every row, as described above.

Input

The example below is the input this Actor is prefilled with, so you can press Start and read a real result before you point it at your own site. books.toscrape.com is a site published for exactly this kind of testing.

{
  "startUrl": "https://books.toscrape.com",
  "maxPages": 50,
  "maxDepth": 3,
  "checkExternalLinks": true
}

field	type	default	range
startUrl (required)	string	—	a bare domain (example.com, https:// is assumed) or a full URL
maxPages	integer	50	1 to 5000 pages opened by the crawler
maxDepth	integer	3	0 to 20 clicks from the start page; 0 checks only the links on the start page
requestDelaySeconds	number	0.25	0 to 60 seconds between two requests to the same host
requestTimeoutSeconds	integer	15	3 to 120 seconds before a link is reported as a timeout
concurrency	integer	8	1 to 32 targets checked in parallel; the per-host delay still applies
checkExternalLinks	boolean	true	also check links pointing to other sites; other sites are never crawled
includeSubdomains	boolean	false	treat blog.example.com as part of example.com and crawl it too

Links found on the crawled pages are all checked, even when the crawl stops at maxPages.

What this Actor does not do

• It does not fix anything. It does not edit your site, your CMS or your redirects.
• It does not run JavaScript. It reads the HTML the server returns, so a link that only appears after a script runs is not seen.
• It does not find orphan pages. It follows internal links from the start URL; a page nobody links to is never visited.
• It does not check anchors inside a page (#section), image sources, CSS or script URLs, or redirect chains. It checks link targets.
• It does not judge SEO, page speed, accessibility or content quality.
• It does not log in, fill forms, solve captchas or get past a paywall.
• It does not collect personal data. The output holds URLs, HTTP statuses and anchor text. mailto: and tel: links are skipped.
• It does not decide for you what a 403 or a 429 means. Some servers answer that to any crawler; the status is reported as it came back.
• It does not promise a number of links per minute. The pace is set by your requestDelaySeconds and by any Crawl-delay the site publishes.

Manners, robots.txt and your responsibility

• robots.txt is fetched before the crawl and always respected. There is no option to turn it off. A path disallowed for our user agent is not requested and is counted in skippedByRobots. Requests to robots.txt itself are never charged.
• If the site's robots.txt asks for a longer Crawl-delay than your requestDelaySeconds, the longer value wins.
• The Actor identifies itself on every request as LoteboBrokenLinkAuditor/0.1 (+https://apify.com/store; Apify Actor; contact via Apify Store page). That string will appear in your logs and you can write a rule for it.
• You are responsible for having the right to access the URLs you give it. Before you run it, check the terms of the site and its robots.txt, and check whether your own agreement with that site allows automated access. Point it at sites you own or are allowed to audit.

Price

Pay per event, one event, exactly as declared in .actor/actor.json:

event	price	when it is charged
url-checked	US$ 0.01	once per HTTP request made to the audited site: a page fetch, a HEAD check of a link target, or the GET fallback when HEAD is refused. robots.txt requests are never charged

Each distinct target is checked once per run, even when many pages link to it. Apify charges its own Actor start event and the platform usage of the run on top of this; those are not set by this Actor. If a run reaches your pay-per-event limit, it stops, keeps everything found so far, writes the summary row and sets chargeLimitReached to true.

About this Actor

The code, the tests and the run logs quoted here are in this repository. The Actor is written in Python and was built with the help of AI.

Example tasks

Each page below is a published example task of this Actor. It shows the input used and the fields the run returns. The same page is served as Markdown by adding .md to the URL.

• Check for broken links after a site migration: crawls the migrated site and returns every broken link with the page it sits on, so redirects missed in the move are fixed before traffic drops.
• Crawl an entire website for 404 errors: starts from the home page, follows every internal link and returns each URL that answers 404 or another error, with the page that links to it.
• Find dead outbound links on a blog or docs site: checks every external link your articles point to and returns the ones that died, with the source page.
• How do I find every broken link on my website?: crawls one domain from its home page and returns one row per broken link, with the HTTP status, the page the link sits on and the anchor text to search for in your CMS.
• How do I find dead outbound links in my old blog posts?: crawls your site and checks the links that point to other domains, so you see which external pages your posts still send readers to and which ones are gone.
• How do I check the links on one page for 404s?: checks every link on a single URL, internal and external, without crawling the rest of the site.
• How do I tell a dead link from a slow one?: separates the HTTP status from the network error, so a slow server shows up as a read timeout instead of a 404.
• How do I check for broken links without overloading my site?: one request at a time with the delay you choose, and a longer Crawl-delay in robots.txt always wins over it.