Link & Citation Verifier: Dead Links, Soft 404s, Quote Check

Give it a list of links, optionally with the quote or title each one is supposed to support. For every link you get back one flat JSON row:

• Is the page really alive? Final URL after redirects, HTTP status, and a clear status: alive, soft_404, parked, dead, blocked, paywalled, rate_limited, server_error, timeout, unreachable or invalid_url. A 200 page that says "Page not found", or a deep link that quietly redirects to the home page, is a soft 404, not "alive". A bot check is blocked, not "dead".
• Does it say what you cite? quoteFound, a 0-1 quoteMatchScore, the matching passage (quoteMatchedText) and its context. Small differences in case, accents, punctuation and spacing are ignored; paraphrases come back as partial.
• Is there a copy if it is gone? For dead links, the nearest Wayback Machine capture (archiveUrl), and whether your quote is on that copy.
• One-word answer: verdict = supported, supported_by_archive, quote_not_found, dead_link, unverifiable, ... so an agent can act without parsing anything else.

Built for AI agents, MCP clients and research pipelines that must check their sources before they answer: hallucinated or rotted citations are caught before anyone reads them.

By Tallow Lane Labs. Unofficial tool; not affiliated with, endorsed by or sponsored by the Internet Archive (Wayback Machine) or any site it checks.

Quick start

Leave the input empty and press Start: it checks a 3-link demo (two real citations and one dead link) for about $0.0105.

Typical input:

{
  "links": [
    { "url": "https://en.wikipedia.org/wiki/Link_rot",
      "quote": "is the phenomenon of hyperlinks tending over time to cease to point to their originally targeted file",
      "title": "Link rot" },
    { "url": "https://www.whitehouse.gov/ostp/ai-bill-of-rights/",
      "quote": "The Blueprint for an AI Bill of Rights is a guide for a society that protects all people from these threats",
      "citedDate": "2023-06-01", "id": "ref-7" },
    "https://example.com/some/page"
  ]
}

Only url is required. quote and title are optional. citedDate (YYYY-MM-DD) picks the Wayback capture nearest that date. id is echoed back. Plain URL strings work too, and so does the urls list.

What you get (one row per link, in input order)

{
  "inputIndex": 1,
  "id": "ref-7",
  "url": "https://www.whitehouse.gov/ostp/ai-bill-of-rights/",
  "finalUrl": "https://www.whitehouse.gov/ostp/ai-bill-of-rights/",
  "httpStatus": 404,
  "status": "dead",
  "isAlive": false,
  "statusReason": "http_404",
  "verdict": "supported_by_archive",
  "quote": "The Blueprint for an AI Bill of Rights is a guide for a society that protects all people from these threats",
  "quoteFound": null,
  "archiveFound": true,
  "archiveUrl": "https://web.archive.org/web/20230601.../https://www.whitehouse.gov/ostp/ai-bill-of-rights/",
  "archiveQuoteFound": true,
  "archiveQuoteMatchScore": 1,
  "billableEvents": ["url-checked", "archive-lookup", "quote-verified"]
}

Every row has the same 47 keys (unused ones are null), so the schema never shifts under an agent. Main fields:

Field	Meaning
status	alive, soft_404, parked, dead, blocked, paywalled, rate_limited, server_error, timeout, unreachable, invalid_url
isAlive	true (alive, paywalled) / false (dead, soft 404, parked) / null (can't tell: blocked, rate-limited, timeout, 5xx, unreachable)
statusReason	Why, e.g. http_404, dns_not_found, not_found_title, redirect_to_homepage, http_403_cloudflare_challenge, isAccessibleForFree_false
verdict	supported, supported_by_archive, quote_not_found, quote_not_found_in_archive, title_mismatch, dead_link, unverifiable, ok, invalid_url
quoteFound, quoteMatchScore, quoteMatchType	Match on the live page: exact, fuzzy (at or above the threshold, default 0.9), partial (0.6 to threshold, usually a paraphrase), none
quoteMatchedText, quoteContext	The best matching passage, and the same with about 120 characters around it
titleMatch, titleMatchScore	Expected title vs. the page's title, og:title, citation_title or first heading
pageTitle, publishedDate, modifiedDate	From the page (JSON-LD, meta and citation tags, PDF info)
contentHash	sha256 of the visible text: compare runs to see if a page changed. Page text itself is never stored
archiveUrl, archiveTimestamp, archiveQuoteFound	The Wayback capture and whether the quote is on it
retryAfterSecs	From Retry-After on 429/503
billableEvents	Which paid events this row triggered

Dataset views: Overview, Quote checks, Liveness details, Wayback copies. A run summary (counts by status and verdict, billable events, requests per domain, warnings) is saved as RUN_SUMMARY in the key-value store.

PDF links are read too (text and title), up to 20 MB.

Pricing (pay per event)

You pay only for work that actually happened:

Event	Price	When
URL checked	$0.001	Each unique URL actually requested. Invalid URLs are free; a URL repeated in the input is fetched and charged once
Quote verified	$0.003	Each link whose quote was compared with page text (live, rendered or archived copy; once per link). Not charged when there was no text to check
Browser render	$0.002	Each successful headless-browser render (only if you turn browserFallback on; off by default)
Wayback lookup	$0.001	Each archive.org lookup for a dead link (or the statuses you pick in archiveOn), whether or not a capture exists
Actor start	$0.0005	Once per run (default 1 GB memory; Apify charges this once per started GB)

Worked examples:

• Check 100 links for liveness only: about $0.1005.
• Verify 100 citations with quotes, 10 of them dead with a Wayback copy: about $0.4105.
• The blank-input demo: about $0.0105.
• maxUrls (default 500) caps a default run at about $3.50 even if every event fired on every link.

There is no charge per dataset row. Set Max cost per run in the run options to cap any run; the Actor stops cleanly when the cap is reached and says so in RUN_SUMMARY.

How it decides

• Soft 404: a 2xx page whose title or first heading says not found / 404 / doesn't exist, a short page whose text says so, a redirect to an error page, or a deep link that redirects to the site's home page.
• Parked: "this domain is for sale" pages, parking scripts, or a redirect to a domain marketplace.
• Blocked: 401, 403, 406, 451, LinkedIn's 999, and Cloudflare, PerimeterX, DataDome, Imperva, Akamai or CAPTCHA pages (even when served with 200). A blocked page is not reported as dead: isAlive is null and the verdict is unverifiable.
• Paywalled: 402, schema.org isAccessibleForFree: false, a locked or metered article:content_tier, or "subscribe to continue reading" text. If the quote is still readable, the verdict is supported.
• Rate limited: 429, or 503 with Retry-After.
• Quote match: text is normalised (case, accents, curly quotes, punctuation, whitespace). An exact match scores 1. Otherwise the best window of about the quote's length is scored by character similarity. Use ... between separate passages; each is checked and the score is length-weighted.

Fair use: what it will and won't do

• Fetches only the URLs you give (and their redirects), plus one Wayback lookup and, if found and needed, one archived copy per link. It never crawls or follows other links.
• Reports blocks, never bypasses them. No proxies or unblockers, no CAPTCHA solving, no fingerprint spoofing, no login. It identifies itself honestly as TallowLaneLabs-LinkCitationVerifier.
• Polite per-domain rate limiting: one request at a time per site with a 1 s gap by default (perDomainDelayMs), at most 5 sites in parallel (maxConcurrency), no retries on 4xx/429. archive.org always gets at least 1.5 s between requests.
• Refuses private, loopback and cloud-metadata addresses.
• Keeps only titles, dates, a hash and short matched snippets, never full page text.

Browser fallback (optional)

Some pages only show their text after JavaScript runs. Turn on browserFallback and live pages that look JavaScript-rendered, or whose static HTML lacks the quote, are rendered once in headless Chrome and checked again ($0.002 per render). The browser loads the page's own scripts and styles but not images, media or fonts. It is never used on blocked, rate-limited or paywalled pages. One page renders at a time, so 1 GB of memory is enough.

Use from AI agents (MCP)

Add the Actor to the Apify MCP server (tallowlanelabs/link-citation-verifier) and call it with links. Read verdict first. Then use quoteMatchedText as evidence, archiveUrl as a replacement link for dead sources, and status to decide whether to retry later (rate_limited, timeout, server_error).

Limits

• Soft-404 and paywall detection are heuristics. Unusual sites can be misread, and statusReason always says which rule fired.
• Sites that block automated requests come back as blocked, by design. Check those manually.
• No robots.txt check: only the exact URLs you give are requested, one at a time per site, like a link preview.
• Quote matching compares words, not meaning: a faithful paraphrase scores as partial, not supported.
• The Wayback Machine can be slow or rate-limited. A failed lookup is not charged and is noted in archiveNote.
• Pages over 5 MB of HTML or 20 MB of PDF are checked on the first part only (noted in errorMessage).

Support

Open an issue on the Issues tab or email Tallow Lane Labs at wdeornellas+tallowlanelabs@gmail.com.