Crawl your own website and find every broken link (404, 5xx), redirect chain and loop, with the page and anchor text where each link sits. Same-host crawl, respects robots.txt. $1 per 1,000 links checked.
All notable changes to the Broken Link & Redirect Chain Checker Actor. Format: Keep a Changelog ;
versions follow Semantic Versioning . The Apify build version (.actor/actor.json)
tracks MAJOR.MINOR.
[0.1.1] - 2026-10-27
Security
An off-site redirect is a status check only. The body is discarded at the headers, before it is read, and nothing on that page is extracted or billed. robots.txt is checked on every hop, including a same-host path change and an outbound target. A disallowed hop is not checked (robots.txt) and is not billed, even when an earlier hop was already requested.
Ownership verification requests the exact https://<start host>/.well-known/... URL and does not follow redirects. Another host, a port, or userinfo fails the check. The token input is secret and is not written to the log or to OUTPUT.
A <base href> that is not http(s) is ignored. A stored value that is not http(s) is [invalid URL].
Query strings are never stored. A discovered link's query is sent only when Keep query strings is on. A redirect Location query is always requested, and loop detection uses that full URL.
Status checks strip authorization, proxy-authorization, cookie, x-goog-api-key, and caller-marked sensitive headers on a cross-origin redirect, including 307 and 308. The redirect cap stays 5.
Fixed
A robots.txt larger than 500 KiB is truncated and treated as unreachable, so the host is disallowed. A cut such as Allow: / is not parsed. A complete file of at most 500 KiB is parsed. A file that cannot be read disallows the host.
The per-attempt timeout starts after the host slot is acquired.
The default dataset is the charge ledger. A resumed run rebuilds the charged set from pushed rows, so a landed push is not charged again. Crawl progress is appended as deltas about every 30 seconds and on migrating, aborting, and persistState. A save snapshots array lengths before the key-value write, so entries appended during that write land in the next part. Parts are CRAWL_G<gen>_PART_<n>. A v2 head with no gen is generation 0 and still uses CRAWL_PART_<n>. Compaction writes the next generation as several parts, each at most 1 MiB of compact JSON, then switches gen and parts in one head write. The generation holds at most 64 parts (64 MiB), enough for 50,000 links and 50,000 pages. A head or part that is the wrong shape, or over those caps, fails the run before any request. A failed save is logged and does not fail the run; the dataset stays the charge ledger and OUTPUT is still written. A hard kill can re-charge at most 4 links, the pushes still in flight. An onlyNewBreaks charge is written into one ledger record for its in-flight batch before that slot is released, so it is inside that cap.
A resumed link uses the dataset row's firstSeenDepth as a floor, clamped to 0 through the hard depth cap, so a hard kill cannot put that link back at depth 0 and crawl past maxDepth. A part URL is kept only when it is an http(s) origin and path on the start host (or its www twin when Include www is on). Private addresses are not restored into the crawl. A query is not stored. Anchor text is scrubbed again on load. An outbound link that was already pushed stays in the dataset and is not crawled again.
An empty crawl head is written before the first charge. CRAWL_UNPUSHED is merged on resume even when that head was never written. A kill in the first 30 seconds re-charges at most the 4 checks still in flight. The ledger keeps only charges not yet in a committed part, and stays within 64 KiB. If that record still cannot be saved after a short retry, the run stops before another charge. A corrupt ledger, including a value that is not JSON, stops the resume before any request. Ledger URLs merged on resume stay uncommitted until a crawl part saves them, so a later kill does not bill them again and does not drop a charge that was still in flight. A user's charge limit still ends SUCCEEDED with exactly those charges. Storage errors are redacted before they are logged. 256 URLs of about 48 characters is about 12 KB, and the 64 KiB cap still bounds the record. A failed crawl-state write still publishes OUTPUT and then fails the run. A failed final save still writes OUTPUT.
The daily-test host root https://hardwoodlogic.github.io/robots.txt returns 404. The fixture file https://hardwoodlogic.github.io/broken-link-checker-fixture/robots.txt returns 200 with Allow: /.
A restarted run re-reads a charged own-host page that was not parsed, without charging it again, and keeps the same depth a fresh discovery would have used (parent + 1). OUTPUT counts every segment. resumed is true when the run continued. linksChecked stays equal to linksCharged.
With Keep query strings on, links that differ only by query collapse into one stored row.
A robots.txt 4xx, including 404, allows the host. Only a 5xx, an unreachable file, or a truncated file disallows it. An oversized robots.txt (over 500 KiB) disallows the whole host.
The per-host gap is 550 ms, so a sliding one-second window does not see three starts.
A resumed link keeps foundOn and the depth it was first seen at.
A robots.txt Disallow rule that contains a query is matched against the path and query of the URL that would be requested.
linksChecked counts successful link-checked events, the same number as linksCharged.
onlyNewBreaks without monitor mode is rejected before any request or charge.
A DNS failure (ENOTFOUND, EAI_AGAIN, and the same family) is an error row with error dns and is not billed. It is not reported as not checked (robots.txt).
Changed
Customers pay only the link-checked event, $1.00 per 1,000 links. Platform usage is included. maxPages still defaults to 100. Redirect chains still stop after 5.
Monitor mode, the robots override, and scheduled runs require the start URL on the default port. The ownership file is still checked at https://<start host>/.well-known/... with no port in the URL.
Ignore robots.txt on my site applies only to the verified start host. The www twin keeps its own robots.txt when Include www is on.
[0.1.0] - 2026-10-27
Added
First build. Crawl one host you own, status-check outbound links once, and report the redirect chain (at most 5 redirects). Monitor mode compares broken links with the previous run after the ownership file is verified.
Pay per event: link-checked at $0.001. No start fee and no apify-actor-start or apify-default-dataset-item event.
Ownership file /.well-known/broken-link-checker-verify-<token>.txt is required for monitor mode, for ignoring robots.txt on your own site, and for scheduled runs (APIFY_META_ORIGIN=SCHEDULER).
