# AI Answer Link Checker - Find Hallucinated URLs & Fake Quotes (`that_red_bird/ai-answer-link-verifier`) Actor

Paste an AI answer: every URL is checked to resolve and the page is tested for the sentence or quote it is cited for, with a grounding score and a Wayback copy for dead links. Lexical match, not a semantic judge. Blocked sites are reported, not guessed.

- **URL**: https://apify.com/that_red_bird/ai-answer-link-verifier.md
- **Developed by:** [mohamed alaya](https://apify.com/that_red_bird) (community)
- **Categories:** AI, Developer tools, SEO tools
- **Security:** Limited permissions
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

$2.00 / 1,000 claim checkeds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/actors/running/actors-in-store.md#pay-per-event

## What's an Apify Actor?

An Actor is a serverless cloud program that runs on the Apify platform. It has two run modes.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.

Apify vocabulary and the platform model are defined once, in the agent quickstart at https://apify.com/agents.md.

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.

Do not guess an integration path. Every one of them is in the agent quickstart at https://apify.com/agents.md: the Apify MCP server, Agent Skills with the Apify CLI, the JavaScript and Python clients, the REST API, and the account-free path for an agent with no human to sign in. It also carries the rule on stating cost before the first paid run.

For examples already wired to this Actor's own input schema, see the [API](#api) section below.

Each client library has reference documentation the quickstart does not restate: [JavaScript/TypeScript](https://docs.apify.com/api/client/js/docs.md) (`npm install apify-client`) and [Python](https://docs.apify.com/api/client/python/docs.md) (`pip install apify-client`).

# README

## AI Answer Link Checker - Find Hallucinated URLs & Fake Quotes

Paste an AI answer. This Actor finds every link in it, checks that each URL really exists, and tests whether the page contains the sentence or quote the link was cited for. You get a status and a **grounding score** per claim, and a Wayback Machine copy when a link is dead.
Built for people who build RAG and agent products, run AI content teams or fact-check chatbot output, and who have learned that "the link returns 200" does not mean "the page says that".

### Input examples

**1. A chatbot answer with links**

```json
{ "answer": "Turing \"was an English mathematician and logician\" ([source](https://en.wikipedia.org/wiki/Alan_Turing)). Productivity rose 47% in 2024 [1].\n\n[1] https://example.com/reports/2024-study" }
```

**2. Structured citations from a RAG pipeline**

```json
{ "claims": [{ "url": "https://example.com/", "quote": "This domain is for use in documentation examples without needing permission." }] }
```

**3. Links only, show only the broken ones**

```json
{ "urls": ["https://example.com/a", "https://example.org/b"], "onlyProblems": true }
```

Run with no input to see a demo on a short answer that contains a true quote, a fake quote, a dead page and an invented domain.

### How a claim is checked

1. **Pairing.** Each URL is matched to the sentence it appears in (markdown link, bare URL, or a numbered `[n]` citation) and to any quoted passage in that sentence.
2. **Link.** The URL is fetched. DNS failures, 404 and 410 are `DEAD`; a 200 page that says "not found" is `SOFT_404`; a deep link that lands on the home page is `REDIRECTED_TO_HOMEPAGE`; sites that refuse the check are `BLOCKED` (free, never guessed).
3. **Quote.** The page text is compared with the claim: exact match first, then the best one to three sentence window by overlap of content words and phrases. Numbers in the claim that are missing from the page are listed.
4. **Archive.** For links that are not live, the Wayback Machine is asked for a copy, and the copy is tested against the claim too.

### Input guide

| Field | What it does |
|---|---|
| `answer` | The text to check. Markdown links, bare URLs and `[1]` style citations are understood. |
| `claims` | Explicit `{url, quote}` pairs. |
| `urls` | Links to check with no claim attached. |
| `supportThreshold` | Overlap needed to call a claim supported (default 70). |
| `useWayback` | Look for archived copies of failing links (default on). |
| `onlyProblems` | Hide verified and working links (they are not charged). |

### Output

| Field | Meaning |
|---|---|
| `status` | `VERIFIED`, `QUOTE_WEAK`, `QUOTE_NOT_SUPPORTED`, `LINK_OK`, `LINK_OK_UNCHECKED`, `DEAD`, `SOFT_404`, `REDIRECTED_TO_HOMEPAGE`, `BLOCKED`, `UNREACHABLE` |
| `groundingScore`, `groundingLevel` | 0-100 overlap; `EXACT`, `NEAR_EXACT`, `SUPPORTED`, `WEAK` or `NOT_SUPPORTED` |
| `claim`, `claimType` | The text that was tested and whether it was a `quote` or a `sentence` |
| `bestMatchOnPage` | The closest passage on the page, so you can read it yourself |
| `missingNumbers` | Figures in the claim that the page never mentions |
| `archive` | Wayback snapshot URL and date, and whether the copy supports the claim |
| `explanation` | Plain-English verdict |

Sample row:

```json
{ "type": "claim", "url": "https://example.com/", "status": "QUOTE_NOT_SUPPORTED", "httpStatus": 200, "groundingScore": 9, "groundingLevel": "NOT_SUPPORTED",
  "claim": "this domain was used by NASA to calibrate the Voyager probes in 1977", "claimType": "quote",
  "bestMatchOnPage": "This domain is for use in documentation examples without needing permission.",
  "explanation": "The link works, but nothing on the page matches the claim (9% overlap). The quote or fact may be invented or taken from another source." }
```

### Use it with

- **A RAG or agent pipeline:** send the model's answer and citations here before showing them; drop or flag anything that is not `VERIFIED`.
- **Content QA:** run it on AI-drafted articles to catch invented sources before publication.
- **Fake Reference Checker:** companion Actor for academic references.

### FAQ

**Is this a fact-checker?** No. It checks whether the cited page contains the claim, not whether the claim is true.

**Why is a correct paraphrase marked weak?** Support is lexical overlap of words and phrases. A heavy paraphrase of a real page can score low. Read `bestMatchOnPage` and decide.

**Why BLOCKED?** Many news and publisher sites refuse requests from cloud IP ranges. Those links are reported honestly and are not charged.

### Honest limits

- Pages that build their text with JavaScript, sit behind a login, a paywall or a bot wall cannot be read.
- PDFs are not read (the link is checked, the quote is not).
- Overlap is not entailment: a page can contain the same words and mean the opposite.
- Only the first 2 MB of each page is read.

### Price

Pay per event: **$0.002 per claim checked** (`claim-checked`). An answer with 30 cited links costs **$0.06**. Blocked and unreachable sites are free.

# Actor input Schema

## `answer` (type: `string`):

Paste the chatbot or agent answer, with its links. Markdown links, bare URLs and numbered citations like \[1] with a source list are all understood. Each URL is paired with the sentence or quoted passage it is cited for. Leave all inputs empty to run a built-in demo.

## `claims` (type: `array`):

Optional. A JSON list of objects like {"url": "https://...", "quote": "the sentence the model attributed to that page"}. Use this when you already have structured citations from a RAG pipeline.

## `urls` (type: `array`):

Optional list of URLs to check for dead links, soft 404s and homepage redirects, without a claim to compare.

## `supportThreshold` (type: `integer`):

Minimum overlap between the claim and the page text for a claim to count as supported. 70 is a good default: 100 means word for word, below 45 means the page does not say it.

## `useWayback` (type: `boolean`):

For dead, blocked or soft-404 links, look up the Internet Archive and also check whether the archived copy contains the claim.

## `onlyProblems` (type: `boolean`):

Skip links that are verified or simply working. Skipped links are not charged.

## `maxLinks` (type: `integer`):

Upper bound on how many URL and claim pairs are checked in one run (hard limit 500).

## `timeoutSecs` (type: `integer`):

How long to wait for each page before it is reported unreachable.

## `concurrency` (type: `integer`):

How many pages are fetched in parallel. Requests to the same host are spaced out automatically.

## Actor input object example

```json
{
  "answer": "",
  "claims": [],
  "urls": [],
  "supportThreshold": 70,
  "useWayback": true,
  "onlyProblems": false,
  "maxLinks": 100,
  "timeoutSecs": 20,
  "concurrency": 4
}
```

# Actor output Schema

## `results` (type: `string`):

No description

## `downloadCsv` (type: `string`):

No description

## `summary` (type: `string`):

No description

## `count` (type: `string`):

No description

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "answer": "",
    "claims": [],
    "urls": [],
    "supportThreshold": 70,
    "useWayback": true,
    "onlyProblems": false,
    "maxLinks": 100,
    "timeoutSecs": 20,
    "concurrency": 4
};

// Run the Actor and wait for it to finish
const run = await client.actor("that_red_bird/ai-answer-link-verifier").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "answer": "",
    "claims": [],
    "urls": [],
    "supportThreshold": 70,
    "useWayback": True,
    "onlyProblems": False,
    "maxLinks": 100,
    "timeoutSecs": 20,
    "concurrency": 4,
}

# Run the Actor and wait for it to finish
run = client.actor("that_red_bird/ai-answer-link-verifier").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "answer": "",
  "claims": [],
  "urls": [],
  "supportThreshold": 70,
  "useWayback": true,
  "onlyProblems": false,
  "maxLinks": 100,
  "timeoutSecs": 20,
  "concurrency": 4
}' |
apify call that_red_bird/ai-answer-link-verifier --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,that_red_bird/ai-answer-link-verifier"
        }
    }
}
```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/YyovKRhjsXoyujlf0/builds/AZSnfvhWhOzhouCKE/openapi.json