Broken Link Checker - Find 404 Errors & Redirects on a Website avatar

Broken Link Checker - Find 404 Errors & Redirects on a Website

Pricing

from $20.00 / 1,000 page analyzeds

Go to Apify Store
Broken Link Checker - Find 404 Errors & Redirects on a Website

Broken Link Checker - Find 404 Errors & Redirects on a Website

Crawl a website and check every link on it: broken links (404, 500) and redirects (301, 302) with their destination URL, HTTP status and response time. One record per page, plus a site summary listing the dead links that appear on the most pages.

Pricing

from $20.00 / 1,000 page analyzeds

Rating

5.0

(2)

Developer

My Smart Digital

My Smart Digital

Maintained by Community

Actor stats

2

Bookmarked

30

Total users

0

Monthly active users

9 days ago

Last modified

Share

Broken Link Checker — Find 404s & Redirects on Any Website

Give it a URL. It reads every link on the page — or crawls the whole site — checks each one, and tells you which links are dead (404, 500), which redirect (301, 302), and where those redirects go. A link whose server refuses to answer the check (a rate limit, a login, an anti-bot protection) is listed as not verified, with the reason — never reported dead, never charged.

No account, no API key, no browser extension. One run, one dataset.

What you get

Two kinds of records.

{
"type": "page",
"pageUrl": "https://mysmartdigital.fr",
"title": "Réservations directes pour hôtels indépendants — My Smart Digital",
"httpStatus": 200,
"linksCount": 7,
"brokenLinksCount": 0,
"redirectLinksCount": 4,
"unverifiedLinksCount": 0,
"links": [
{
"url": "https://mysmartdigital.fr/tarifs",
"text": "Formules",
"isInternal": true,
"httpStatus": 301,
"responseTime_ms": 111,
"redirectUrl": "https://mysmartdigital.fr/tarifs/"
},
{
"url": "https://mysmartdigital.fr/",
"text": "MSD.My smartdigital",
"isInternal": true,
"httpStatus": 200,
"responseTime_ms": 432
}
]
}

redirectUrl appears on 3xx links only. Redirects are reported, never followed: a 301 keeps its 301 status, shows its destination, and never counts as broken.

A link the server would not answer carries unverifiedReason instead of a verdict, and counts in unverifiedLinksCount, not in brokenLinksCount:

{
"url": "https://www.example-shop.com/collections/new",
"text": "New arrivals",
"isInternal": true,
"httpStatus": 429,
"responseTime_ms": 212,
"unverifiedReason": "The server rate-limited the check (HTTP 429)"
}

How a link is judged: it is requested with HEAD, without following redirects. When HEAD does not answer 2xx or 3xx, a GET — what a click sends, stopped as soon as the headers arrive — decides, since many servers refuse HEAD and serve the page to GET. A link is broken when that answer is an error (404, 410, 500...) or when the server cannot be reached (unknown host, connection refused, timeout). It is not verified when the answer turns the check away rather than describing the link:

  • a rate limit (429): the host is then not asked again for the rest of the page, and its other links are listed "Not requested: ...";
  • a login (401), a region block (451), LinkedIn's anti-bot answer (999);
  • an anti-bot protection that signs its answer (Cloudflare challenge, AWS WAF, DataDome): after three such answers in a row, the host is not asked again for the rest of the page either;
  • a 403 from a server that answered no other link of the page — a wall that does not sign. A 403 from a server that did serve other links of the page is that page's own, and counts as broken.

One site summary at the end of the run

{
"type": "site-summary",
"pagesCrawled": 1,
"pagesTurnedAway": 0,
"linksTotal": 7,
"brokenLinksTotal": 0,
"redirectLinksTotal": 4,
"unverifiedLinksTotal": 0,
"byStatus": { "200": 3, "301": 4 },
"byType": { "internal": 7, "external": 0 },
"topBrokenLinks": []
}

topBrokenLinks lists the 20 most frequent dead links with every page they appear on — that is the list to fix first, since one bad link in a menu or a footer breaks on every page at once. pagesCrawled counts the pages whose links were checked, which are the pages billed PAGE_ANALYZED. pagesTurnedAway counts the pages the site refused to show (see below), which are never billed. unverifiedLinksTotal counts the links listed without a verdict, never billed. byStatus counts the status each link answered ("not requested" for the links of a host not asked again, "not checked" for the links left when a page ran out of time).

Input

{
"startUrls": ["https://example.com"],
"crawlPages": true,
"maxPages": 50,
"maxConcurrency": 5,
"sameDomain": true,
"checkExternal": false,
"timeout": 10000
}
FieldTypeDefaultWhat it does
startUrlslist of URLs—Pages to start from. Required.
crawlPagesbooleanfalseOff: only the start URLs are analyzed. On: the actor follows links and analyzes up to maxPages pages.
maxPages1–100050Page ceiling for the crawl. No effect when crawlPages is off.
maxConcurrency1–205Pages processed in parallel.
sameDomainbooleantrueKeep the crawl on the start site: the host of the start URL and the host it redirects to (www.site.com landing on site.com). No effect when crawlPages is off.
checkExternalbooleanfalseAlso check links pointing to other domains. Off by default: external checks are the slow part.
timeout1000–60000 ms10000Time budget per link check.

Two things worth knowing before your first run

Links are checked, pages are crawled. Every link found on an analyzed page is hit for its status code and response time, wherever it points. What crawlPages controls is how many pages get read to collect links from — a single-page run still checks all the links on that page.

A page with too many links to check in time is still delivered. Links are checked one at a time; after 150 seconds on a page no new check starts, and the links left are listed with "Not checked: ..." in unverifiedReason — not verified, not billed. Lower timeout to get more links of such a page checked. A page that fails to load is missing from the results, and never billed.

What it costs

Pay per event, at the prices shown on this page: one PAGE_ANALYZED per analyzed page delivered, one LINK_CHECKED per link actually checked and returned with its verdict. Nothing else is charged — no start fee, and the site summary is free. A 4-page run returning 105 links, 5 of them not verified, is billed 4 pages and 100 links, no more. A link the server would not answer (see above) is listed with its unverifiedReason and is not billed.

Only web pages are billed as pages:

  • a link to an image, a PDF or any other file is checked like any link, but never opened as a page;
  • a page that answers with an HTTP error (404, 500) is listed with its status, its links are not checked, and it is not billed;
  • a page the site turns away — the challenge or block page of an anti-bot protection (Cloudflare, Akamai, AWS WAF, DataDome, Imperva) served in its place, or a 401, 403 or 429 refusal — is listed with httpStatus: 0 and turnedAwayBy naming what answered ("Cloudflare challenge", "HTTP 403"). Its links are neither checked nor followed, it is not billed, it still counts toward maxPages, and the run's status message says how many pages were turned away and by what;
  • a page reached under two addresses (http:// and https://, with or without www) is analyzed and billed once;
  • a start URL that leads to a file ends the run with a message saying so, and nothing is billed.

Limits

  • Links are checked over HTTP, not rendered in a browser: links injected by JavaScript after page load are not seen.
  • A server that blocks automated traffic does not give the real status of its links: they come back not verified, with the reason, rather than broken. Whether they work for a visitor is not known.
  • The actor does not try to get past an anti-bot protection: pages a protection answers in place of the site come back turned away, uncharged. If it is your own site, let automated traffic through for the run and start it again.
  • Only web links (http://, https://) are checked: mailto:, tel:, sms: and other app links, and same-page anchors, are skipped.
  • When a start URL redirects (www.site.com to site.com), its links are read on the page it lands on.