Broken Link Checker: Crawl a Site, Find 404s & Redirects
Pricing
from $0.40 / 1,000 link checkeds
`Broken link checker: crawl a site over plain HTTP, check every link once for 404s, redirect chains and timeouts. $0.40 per 1,000 links.`
url
Required
The link as first found (absolute, fragment removed, query and session parameters untouched). One row per unique URL; variants that differ only in a session ID or an ignored parameter are one row.
status
Required
ok: 2xx. redirect: reached a 2xx after one or more redirects. broken: 4xx, 5xx, DNS or connection error, redirect loop, too many redirects. timeout: no answer in 15 s. blocked (always free): the server refused us (401, 403, 429, 999), or we did not request it (robots.txt of any site including external ones, unreadable robots.txt, the per-run limit of other sites read, a slow host, social-media or Google host, blocked address). Rows with errorCode internal (an unexpected error in this Actor, status broken) are also free.
statusCode
Optional
Final HTTP status code, null when there was none.
finalUrl
Optional
Where the link ended after redirects.
redirectCount
Optional
Number of redirects followed (at most 5).
redirectChain
Optional
Every hop in order, final response last. Empty when there was no redirect.
redirectLoop
Optional
contentType
Optional
Content-Type of the final response, without parameters.
responseTimeMs
Optional
Time spent in requests for this link (rate-limit waiting excluded).
error
Optional
Plain-language reason when status is not ok or redirect.
errorCode
Optional
http-404, http-403, timeout, dns, connection-refused, connection-reset, tls, redirect-loop, too-many-redirects, bad-redirect, invalid-url, robots (disallowed by robots.txt of the site, external sites too; not requested, free), robots-unreachable (robots.txt could not be read after a retry; free), robots-host-cap (more than 500 other sites in one run; free), platform, ssrf, slow-host (the site, external ones too, asks in robots.txt for a Crawl-delay over 2 s and Allow slow hosts is off, or over the 2.8 s maximum this Actor honours: not requested, free row), network, internal (our own error, free row).
external
Optional
True when the link points to another site.
startUrl
Optional
True for the start URLs you gave.
inSitemap
Optional
Only when sitemap URLs were given: the URL is listed in the sitemap.
foundOnCount
Optional
On how many crawled pages the link was found.
foundOn
Optional
Up to 20 pages that contain the link, as known when the row was written. foundOnCount is the true number.
checkedAt
Required
