# Changelog of Broken Link Checker – Find 404s and Dead Links API (`rowcraft/broken-link-checker`) Actor

- **URL**: https://apify.com/rowcraft/broken-link-checker/changelog.md
- **Full Actor documentation**: https://apify.com/rowcraft/broken-link-checker.md

## Changelog

### 1.0.4 (2026-10-17)

- Fix: a robots.txt served as an HTML page (catch-all page or a redirect to the home page, for example thumb.wikimedia.org) counted as an anti-bot block, so every page on that host was refused; it is now treated as a missing robots.txt, as the standard says. Real anti-bot pages are still recognised.

### 1.0.3 (2026-10-17)

- README: the Use cases section links the two public examples kept on the Store (Apify allows 100 public tasks per account; the third example stays available as a private task spec).

### 1.0.2 (2026-10-17)

- README: the list of other Rowcraft actors now includes the five website and extension tools published on 16.10.2026.
- Shared library: URL-like fields named link or guid (feed items) are no longer flagged as phone numbers; retry log line now reads "retry n/m".
- Shared library: an HTML page where JSON was expected (maintenance or block page, seen on 17.10.2026 at the Polish KRS API and at Workday) is retried and then reported with the page text instead of crashing the run on a parse error.

### 1.0.1 (2026-10-14)

- README: real screenshots of the input form and of a results table from Apify Console.

### 1.0.0 (2026-10-14)

- First release: crawls a site within robots.txt (same domain, breadth first, `maxPagesPerSite` pages) and verifies every link of every page read, internal and external. One row per page read, charged per crawled page, never per link.
- Link verdicts: `ok`, `redirect`, `broken_4xx`, `broken_5xx`, `dns_error`, `timeout`, `tls_error`, `connection_error`, `redirect_loop`; not verified and never reported as broken: `blocked_antibot` (401, 403, 429, 999 or a challenge page), `not_checked_robots`, `not_checked_private`, `not_checked_limit`.
- Method: HEAD first, repeated as a one-byte GET for any error answer (HEAD 405, 404 or 403 on a working address does not become a false alarm); redirects followed by hand up to five hops with robots.txt checked at every hop; one verdict per unique address for the whole run; robots.txt read once per origin, Crawl-delay honoured up to 10 seconds, one request per second to a host by default (minimum 200 ms), different hosts in parallel, eight requests in flight at most.
- Row: page address, site, depth, status, title, links found / checked / broken / redirected / not checked with the reasons, the broken links (state, status or error, anchor text, internal or external, tag a, link or img, final address) and the redirected links (final address, hops, first status), a fingerprint of the broken set and the check time.
- Options: `checkExternalLinks`, `includeImages`, `onlyPagesWithBrokenLinks` (clean pages are skipped and not charged), `includeSubdomains`, `maxItems`, `maxExternalHosts`.
- Standard Rowcraft automation: `onlyNew` and `stateName` (a page is delivered again only when its set of broken links changed), `notifyUrl`, `notifyWhenEmpty`, charge cap respected, `COVERAGE` per site, `RUN_SUMMARY` with the state of every address, stop with the summary saved 90 seconds before the run timeout.
- No personal data: `mailto:`, `tel:`, `sms:`, `javascript:` and addresses that hold an e-mail address or point to a personal profile are skipped completely; anchor texts and titles are cleaned of e-mail addresses and phone numbers and cut to 80 and 200 characters.