Data Coverage Claim Verifier

Give me the coverage claim a data vendor made and an authorized reference sample. I will test whether that claim actually holds.

This Apify Actor turns a specific claim—such as “we cover at least 95% of active companies in this market”—into a pre-registered, deterministic coverage test. It returns PASS, FAIL, or UNKNOWN, the matching evidence and uncertainty behind that result, and a separately labeled commercial suggestion.

Run the bundled synthetic example with one field:

{ "mode": "free_example" }

It tests a 90% claim against 20 fictional companies. The synthetic vendor covers 19, so the complete-reference-set result is a reproducible PASS at 95% coverage.

Who this is for

• Developers and data-product teams evaluating external APIs or datasets
• Procurement engineers who need evidence for a buy, renewal, discount, or rejection decision
• Research platforms that need a repeatable vendor-coverage test
• Data publishers auditing their own explicit market-coverage claims

What it does—and does not do

The Actor supports two narrow claim types:

1. entity_coverage: how many authorized reference entities have a normalized exact-key match in the vendor sample.
2. field_presence: how many reference entities both match and have one explicitly claimed non-empty vendor field.

It is not a generic data-quality score, schema validator, fuzzy entity-resolution engine, truth/accuracy checker, web crawler, or licence-rights evaluator. Field presence does not prove field correctness. It uses no LLM and produces no opaque score.

Private audit input

The Actor input itself is the audit manifest. This complete five-supplier example passes exactly at its 80% threshold:

{
  "mode": "private_audit",
  "claim": {
    "statement": "Example Data Co. covers at least 80% of our active suppliers.",
    "metric": "entity_coverage",
    "threshold": 0.8,
    "aggregation": "overall"
  },
  "population": {
    "description": "All five active suppliers in the buyer reference set.",
    "basis": "complete_reference_set"
  },
  "referenceSource": {
    "type": "inline",
    "records": [
      { "supplier_id": "SUP-001", "region": "north" },
      { "supplier_id": "SUP-002", "region": "north" },
      { "supplier_id": "SUP-003", "region": "south" },
      { "supplier_id": "SUP-004", "region": "south" },
      { "supplier_id": "SUP-005", "region": "south" }
    ]
  },
  "vendorSource": {
    "type": "inline",
    "records": [
      { "id": "sup-001" }, { "id": "SUP-002" },
      { "id": "SUP-003" }, { "id": "SUP-004" }
    ]
  },
  "matching": {
    "referenceKeys": ["supplier_id"],
    "vendorKeys": ["id"]
  },
  "strata": [{ "name": "region", "referenceField": "region" }],
  "rightsAttestation": {
    "hasAuditRights": true,
    "noAccessControlsBypassed": true,
    "dataMinimized": true,
    "allowCounterexampleOutput": true
  },
  "evidence": { "identifierMode": "hash", "maxCounterexamples": 10 },
  "auditContext": { "datasetVersion": "example-data-co-2026-09" }
}

Source types

Both referenceSource and vendorSource accept the same four types:

{ "type": "inline", "records": [{ "id": "A-1" }] }

{
  "type": "url",
  "url": "https://data.example.com/authorized-sample.csv",
  "allowedHosts": ["data.example.com"],
  "format": "csv",
  "headers": { "Authorization": "Bearer secret" }
}

{ "type": "apify_dataset", "datasetId": "DATASET_ID" }

{
  "type": "apify_kv_record",
  "storeId": "STORE_ID",
  "key": "sample.json",
  "format": "json",
  "recordsPath": "data.items"
}

When the Actor runs with Apify's limited permission level, select any referenced Dataset or key-value store through the corresponding input resource field. The schema requests READ access only; the Actor never modifies those source storages.

URL sources are GET-only, HTTPS-only, restricted to port 443, and require an exact allowedHosts entry. Every redirect is checked again. IP-literal, local, private, link-local, reserved, and unapproved destinations are rejected. Authorization and cookie headers are not forwarded across origins. JSON, JSONL, and RFC-style CSV are supported; JSON can use a safe dotted recordsPath.

Matching and normalization

referenceKeys and vendorKeys are parallel arrays of one to five safe dotted field paths. More than one path creates a composite key. Version 1 supports only normalized_exact matching.

Default normalization is explicit and appears in the output test specification:

{
  "trim": true,
  "unicode": "NFKC",
  "lowercase": true,
  "collapseWhitespace": true,
  "removePunctuation": false
}

Normalized duplicate keys in the reference set make the population ambiguous and fail the audit. Vendor duplicates are disclosed and deduplicated for entity coverage. For field-presence coverage, any record under the duplicated vendor key may satisfy the field requirement; this policy is pre-registered in the result.

Result semantics

• PASS: the direct census measurement meets the threshold, or an inferential confidence interval is entirely at or above it.
• FAIL: the direct census measurement is below the threshold, or an inferential confidence interval is entirely below it.
• UNKNOWN: the inferential interval crosses the threshold or the sample is below the pre-registered minimum.

complete_reference_set is treated as a direct measurement of the buyer-defined set. representative_sample, or any test downsampled with sampling.maxRecords, uses a deterministic Wilson score interval. The default confidence level is 95% and the default minimum inferential sample size is 30.

For aggregation: "overall", the overall rate decides the verdict. With aggregation: "all_strata", every reported stratum must pass; one FAIL makes the claim fail, while unresolved strata produce UNKNOWN unless another stratum already fails.

Output

One full report is written to the default Dataset and to the OUTPUT key-value record. The schema deliberately separates the layers:

{
  "auditId": "dcv_…",
  "factualVerdict": "PASS",
  "exactClaimTested": { "metric": "entity_coverage", "threshold": 0.9 },
  "observedFacts": { "suppliedReferenceRecordCount": 20 },
  "calculations": {
    "testedPopulationSize": 20,
    "matchedAndCoveredCount": 19,
    "observedCoverage": 0.95,
    "uncertainty": { "applicable": false }
  },
  "assumptions": ["…"],
  "recommendation": {
    "nature": "commercial_suggestion_not_factual_verdict",
    "action": "buy",
    "reasons": ["…"]
  },
  "reproducibility": {
    "testSpecificationHash": "sha256:…",
    "referenceContentHash": "sha256:…",
    "vendorContentHash": "sha256:…",
    "resultFingerprint": "sha256:…"
  }
}

A failure produces a safe structured ERROR key-value record and a failed Actor run; private rows and unexpected exception details are not logged.

Counterexamples

Missing reference identifiers are SHA-256 pseudonyms by default. Set rightsAttestation.allowCounterexampleOutput: true and evidence.identifierMode: "raw" only when the run output is authorized to contain the join keys. At most 100 identifiers can be returned. Full third-party rows are never copied to the result.

Commercial suggestion

The factual verdict never changes to accommodate a purchase decision. Optional commercialRules can turn configured missing-entity costs into buy, test_more, negotiate_or_discount, or reject. This object is explicitly labeled as a suggestion and remains separate from observed facts and calculations.

API usage

Start a run with the standard Apify Actor API:

curl -X POST \
  "https://api.apify.com/v2/acts/boundedtools~data-coverage-claim-verifier/runs?waitForFinish=120" \
  -H "Authorization: Bearer $APIFY_TOKEN" \
  -H "Content-Type: application/json" \
  --data-binary @audit_manifest.json

The run response exposes links defined by the Actor output schema. Fetch the final JSON from the run's default Dataset or OUTPUT record. See Apify's Actor input/output documentation for API and integration patterns.

Pricing

This Actor uses Apify pay per event.

• Free synthetic example: $0.00. The free example does not trigger the paid event.
• Private audit: $32.50 per successful private run.
• Failed or invalid private runs: the paid event is not charged.
• Platform usage: included. Users are not charged a separate platform-usage fee.

The private-audit event is charged at most once per run and only after the user-visible result has been successfully saved.

Privacy, retention, and rights

You must have the legal and contractual right to audit every supplied source. The Actor does not bypass authentication, CAPTCHA, robots controls, access controls, or API restrictions. A reachable URL is not evidence of permission. Do not supply unnecessary personal data.

For private audits, retention.retainInput defaults to false: after loading both sources, the Actor deletes the run's INPUT record, including inline rows and request headers. It also attempts that cleanup on handled failures. A hard process/container termination can prevent cleanup, so highly sensitive inputs should use appropriately restricted, short-retention Apify storage. Set retainInput: true only as an explicit opt-in; Apify's storage retention settings then apply. Source Apify Datasets/KV records are read but never modified. The Actor retains only aggregate results, hashes, safe provenance without URL query strings, and bounded counterexample identifiers.

Content hashes are reproducibility fingerprints, not copies of the data. Because raw input is deleted by default, the buyer must retain the original authorized samples to rerun the audit. HTTP secret values and URL query strings are deliberately excluded from the configuration hash and provenance; the fetched content itself is hashed.

Guardrails and limitations

• Maximum 50,000 supplied records per source and 25 MiB of canonical JSON or fetched content per source
• Maximum 20,000 tested reference records; larger tests require deterministic sampling within the 50,000-record source cap
• Maximum 30 levels of record nesting, 2,000 properties per object, five strata, and 50 values per stratum
• Maximum three HTTPS redirects and 30 seconds per remote request
• No pagination, POST endpoints, browser automation, CAPTCHA handling, fuzzy matching, probabilistic entity resolution, or accuracy/freshness verification in version 1
• Exact host allowlisting and DNS pinning reduce SSRF risk, but buyers should prefer inline or Apify-native storage for sensitive audits
• Results are only as defensible as the buyer's reference population, join keys, rights, and sampling assumptions

Reproducibility notes

Test specification, normalization, duplicate policy, threshold, confidence level, sample cap, and seed are written into every result. Canonical JSON SHA-256 hashes cover the audit configuration, exact ordered source records, and factual result core. Timestamps are excluded from the result fingerprint. Reordering source records intentionally changes their content hashes, while deterministic sample membership is based on normalized keys and the stable seed.

This Actor verifies the configured test. It does not certify a vendor, guarantee a contract outcome, or replace legal, statistical, or procurement advice.