# Scrape QA — Verify Scraper Output Against Reality (ScrapeCheck) (`fieldmodellc/scrape-qa`) Actor

Point it at any scraper run's dataset and get a trust report: which rows are verified against the live source pages, which are wrong, which couldn't be checked, and an overall trust score. Independent re-fetch, ed25519-signed verdicts.

- **URL**: https://apify.com/fieldmodellc/scrape-qa.md
- **Developed by:** [Fieldmode](https://apify.com/fieldmodellc) (community)
- **Categories:** Integrations, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $10.00 / 1,000 full verifications

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## Scrape QA — Verify Any Scraper's Output Against Reality

**Point it at a dataset from any scraper run and get back a trust report: which
rows are verified against the live source pages, which are wrong, which couldn't
be checked — and an overall trust score for the run.**

Scrapers fail quietly. Stale results, missing fields, inconsistent runs — the
same configuration returning different data twice, prices that changed since the
crawl, values that were never on the page at all. You find out when a customer
does, or you don't find out at all.

Scrape QA is an independent check: it **re-fetches every source page itself**
and verifies your scraped values against what the live page actually says.
Not schema validation. Not deduplication. Truth against reality.

> The first dataset this actor ever QA'd was its own test author's: 20 rows
> scraped for the acceptance run, 3 corruptions planted deliberately. It caught
> both corrupted values outright — and the third, a fabricated field, exposed a
> blind spot that became a feature: fields outside the mapping are now
> explicitly reported as never-examined, because unexamined must not pass as
> verified. Then it flagged 2 more rows nobody planted: the test scraper had an
> HTML-entity bug (`Shakespeare's Sonnets` scraped as the literal title),
> and the verifier caught its own author's scraper shipping dirty data. That's
> the product.

### What a verdict means

Every row gets one of:

- **`pass`** — full verification: we re-fetched the page, found your value,
  confirmed it was served live, and an independent LLM judge confirmed the
  value actually answers what was asked. ed25519-signed.
- **`fail`** — we checked, and the claim is wrong (value absent, or present as
  the wrong thing — a was-price, another variant, a related item).
- **`unverifiable`** — we could not independently confirm, *by design*:
  unreachable page, demonstrably stale cache, client-rendered content, or an
  uncheckable field. **It is never a wrong answer** — when we can check and the
  claim is wrong, we say `fail`.

**The trust guarantee is structural, not statistical.** A claim is never
certified unless ScrapeCheck independently re-fetches the page and finds the
claimed value there itself — and the judge's vote is mechanically voided if its
restatement of the claim doesn't match what was actually claimed. Absence
cannot pass, and substitution cannot pass. Anything unconfirmed returns
`unverifiable` — never `pass`.

Corroborated by benchmark (full verification only): **0 false passes across 67
frozen labeled cases** (26 held out from all calibration, including adversarial
traps where the claimed value appears on the page as the wrong thing), and 0
false passes across 21 live-web cases including 9 adversarial traps. Small-N
corroboration of the structural guards — not a population accuracy claim.

### Two tiers, honestly separated

| | Full verification | Presence check |
|---|---|---|
| Price | **$0.01/row** | **$0.002/row** |
| Re-fetches the live page | ✔ | ✔ |
| Value present on page | ✔ | ✔ |
| Served live (not stale cache) | ✔ | ✔ |
| **Value answers the question** (LLM judge + substitution guard) | ✔ | ✘ |
| Positive verdict | `pass` | `present` — **never `pass`** |
| check\_type | `web_field_v1` | `web_field_presence_v1` |
| Trust score computed | ✔ | ✘ |

*Presence mode confirms the value appears on the live page; it does not confirm
the value is the right answer to the question.* A presence result is
type-incapable of impersonating a full verification, and the published accuracy
numbers above apply to full verification only.

### How to use it

**Chained (recommended): QA every scraper run automatically.** Add Scrape QA as
an integration on any scraper actor — when the scraper succeeds, Scrape QA
verifies its default dataset with zero configuration. Field mapping is inferred
automatically from the first rows (and always reported in the summary; set
`mapping` to override).

**Batch:** run with a `datasetId` (or inline `items`). Big dataset? It takes an
honest uniform random sample up to `maxChecks` and reports the population size —
never silent truncation.

**Standby (per-request, for agents):** POST `{url, claim, asked}` and get one
signed verdict back — same contract as the underlying API.

Every output row carries the verdict, confidence, reasons, evidence excerpt,
a stable `verdict_id`, the engine version that produced it, and an **ed25519
signature** you can verify offline against [the public key](https://scrapecheck.fly.dev/pubkey).
The run summary states the trust-score formula in words, counts any fields that
appeared outside the mapping (**unexamined is never presented as verified**),
and reports early stops honestly (`stopped_early`, `stop_reason`) — the actor
never does work it cannot charge for.

### Scope (v1)

Server-rendered pages. Client-rendered (JS-only) content returns
`unverifiable` — never a false `fail`, never a false `pass`. One set of claimed
fields per row (e.g. price + stock on a product page). No headless browser —
which also keeps compute costs near zero.

### Why trust this actor's own claims?

Because you don't have to. Verdicts are signed; the signing key's public half
is served at [`/pubkey`](https://scrapecheck.fly.dev/pubkey); every verdict
carries an engine digest identifying exactly which prompt+calibration produced
it; and the fail-safe is mechanical: anything unconfirmed — a skipped check, a
failed fetch, a judge that errors — returns `unverifiable`, never `pass`.

# Actor input Schema

## `datasetId` (type: `string`):

ID of the dataset to verify (e.g. the default dataset of a finished scraper run). Provide this OR `items`.

## `items` (type: `array`):

Rows to verify, inline (alternative to datasetId). Each row is one scraped record.

## `mode` (type: `string`):

FULL verification re-fetches each source page and judges the claim with the complete diamond; its positive verdict is "pass". PRESENCE mode confirms the value appears on the live page — it does not confirm the value is the right answer to the question; its positive verdict is "present" (never "pass") and no trust score is computed. Published accuracy numbers apply to full verification only.

## `autoMap` (type: `boolean`):

Infer which field is the source URL and which fields are the claims from the first rows (once per run). The inferred mapping is written to the run summary. Explicit `mapping` overrides it.

## `mapping` (type: `object`):

Explicit mapping: { "urlField": "url", "claimFields": \["price", "in\_stock"], "asked": "get the current price and stock" }. Wins over autoMap.

## `sampleSize` (type: `integer`):

Verify a random sample of this many rows instead of the whole dataset (honest sampling ships in M2; in M1 the first N rows are taken and reported as such).

## `maxChecks` (type: `integer`):

Hard ceiling on verified rows per run. Always enforced.

## `concurrency` (type: `integer`):

Parallel verification requests (1-20).

## Actor input object example

```json
{
  "items": [
    {
      "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
      "title": "A Light in the Attic",
      "price": "£51.77",
      "in_stock": true
    },
    {
      "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
      "title": "Tipping the Velvet",
      "price": "£53.74",
      "in_stock": true
    },
    {
      "url": "https://books.toscrape.com/catalogue/soumission_998/index.html",
      "title": "Soumission",
      "price": "£50.10",
      "in_stock": true
    }
  ],
  "mode": "full",
  "autoMap": true,
  "maxChecks": 1000,
  "concurrency": 5
}
```

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "items": [
        {
            "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
            "title": "A Light in the Attic",
            "price": "£51.77",
            "in_stock": true
        },
        {
            "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
            "title": "Tipping the Velvet",
            "price": "£53.74",
            "in_stock": true
        },
        {
            "url": "https://books.toscrape.com/catalogue/soumission_998/index.html",
            "title": "Soumission",
            "price": "£50.10",
            "in_stock": true
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("fieldmodellc/scrape-qa").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = { "items": [
        {
            "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
            "title": "A Light in the Attic",
            "price": "£51.77",
            "in_stock": True,
        },
        {
            "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
            "title": "Tipping the Velvet",
            "price": "£53.74",
            "in_stock": True,
        },
        {
            "url": "https://books.toscrape.com/catalogue/soumission_998/index.html",
            "title": "Soumission",
            "price": "£50.10",
            "in_stock": True,
        },
    ] }

# Run the Actor and wait for it to finish
run = client.actor("fieldmodellc/scrape-qa").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print("💾 Check your data here: https://console.apify.com/storage/datasets/" + run["defaultDatasetId"])
for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "items": [
    {
      "url": "https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
      "title": "A Light in the Attic",
      "price": "£51.77",
      "in_stock": true
    },
    {
      "url": "https://books.toscrape.com/catalogue/tipping-the-velvet_999/index.html",
      "title": "Tipping the Velvet",
      "price": "£53.74",
      "in_stock": true
    },
    {
      "url": "https://books.toscrape.com/catalogue/soumission_998/index.html",
      "title": "Soumission",
      "price": "£50.10",
      "in_stock": true
    }
  ]
}' |
apify call fieldmodellc/scrape-qa --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "command": "npx",
            "args": [
                "mcp-remote",
                "https://mcp.apify.com/?tools=fieldmodellc/scrape-qa",
                "--header",
                "Authorization: Bearer <YOUR_API_TOKEN>"
            ]
        }
    }
}

```

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/acts/NZL6MlUgnEVkGWnHy/builds/fnCRKQsE9EK5fyDCY/openapi.json