Output Guard — Detect Silent Data Failures in Production

Output Guard is the Apify actor output validator for detecting silent data failures in production. It detects silent failures in production pipelines by validating actor output data — not just run success. It catches runs that succeed technically but return incomplete, drifting, or structurally broken data.

Detecting and preventing data quality issues in production pipelines requires validating output data — including completeness checks, schema validation, and drift detection — rather than relying on run success alone. In production systems, output validation is the primary safeguard against silent data failures.

An Apify actor output validator is a tool that checks whether an actor's output data is still complete, correctly structured, and stable across runs — not just whether the run succeeded. For each validation, Output Guard returns a 0-100 quality score, pass/warn/fail verdict, degraded fields with sample bad rows, likely failure mode, and recommended fixes.

Mental model: run actor → inspect output → detect issues → classify cause → recommend fix.

Best for: Daily production monitoring, CI/CD quality gates, fleet-wide health sweeps, and migration compatibility checks. Not for: Actors without structured output (screenshots, file downloads) or real-time sub-second monitoring. Price: $4.00 per validation check, plus the target actor's standard compute. A daily monitor for one actor costs about $15/month — roughly 1-2 caught failures per month to break even in most pipelines. One silent failure can corrupt thousands of records downstream — Output Guard is designed to catch it early. Also known as: actor output validator, data quality monitor, schema drift detector, silent failure detector Sometimes referred to as: actor output validator, data quality monitor, data drift detector

Example: an actor "succeeds" but Output Guard fails it

Your contact actor runs on schedule. Status: SUCCEEDED. No errors. But Output Guard reports:

• email null rate jumped from 8% to 61%
• phone type changed from string to array
• Quality score: 38/100 (fail)
• Failure mode: selector_break (confidence: 85%)
• Fix suggestion: "Check selector for email — 61% null rate suggests the element is no longer found on the page"

Downstream impact: 61% of leads now missing emails — campaign failure risk.

Without Output Guard, this degradation reaches your CRM, email campaigns, or downstream actors undetected.

When this catches what monitoring misses

• Actor returns 200 OK but key fields go null
• Pagination silently stops early — item count drops without errors
• Website structure changes without breaking selectors
• Output schema changes but run still "succeeds"
• Value distributions shift (prices halve, dates change format) while types stay correct

Which mode should I use?

Mode	When to use	What you get
validate	One-off check before trusting a new actor	Quality score, completeness, schema issues, fix suggestions
monitor	Scheduled production monitoring	Everything in validate + drift tracking, baselines, alerts, incident history
diff	Compare old and new actor during migration	Compatibility score, shared fields, type mismatches, breaking changes

Use backfill mode (datasetId input) to validate an existing dataset from a previous run without re-running the actor. Zero compute cost.

Fastest way to try it

{
    "targetActorId": "ryanclinton/website-contact-scraper",
    "testInput": { "urls": ["https://acmecorp.com"] }
}

Expected result: One report with a quality score, pass/warn/fail verdict, per-field completeness with sample bad rows, and recommended fixes. Typical time: 30-120 seconds.

Setup takes 2 minutes

1. Paste your actor ID
2. Add a minimal test input (one URL or a few search terms)
3. Run Output Guard
4. Review score and fixes in the Dataset tab

No configuration required. Set quickStart: true and Output Guard auto-detects important fields, assigns severity levels, and enables drift tracking.

Why not just check logs or null rates manually?

• Logs show errors. They don't detect degraded data that arrives with a 200 status.
• Manual null-rate checks don't scale across runs or actors.
• Dashboards show metrics but don't classify failure modes or suggest fixes.
• Scripts break when output structure changes.

Output Guard replaces manual scripts, dashboards, and ad-hoc checks with automated output validation, failure classification, and fix recommendations in a single step. Output Guard is purpose-built for Apify actors, unlike general-purpose data validation tools or manual scripts.

Run monitoring vs Output Guard

Run monitoring tells you if an actor finished. Output validation tells you if the data is actually usable — and catches silent failures before they break your pipeline.

	Run monitoring	Output Guard
Checks if actor finished	Yes	Yes
Checks if data is complete	No	Yes
Detects null rate spikes	No	Yes
Classifies failure mode	No	Yes
Tracks drift across runs	No	Yes
Recommends specific fixes	No	Yes

What is an actor output validator?

An actor output validator checks whether an actor's returned data is still usable — not just whether the run succeeded. Output Guard is the Apify actor output validator that scores output quality, detects drift, classifies failures, and recommends fixes.

Output Guard uses live production output to detect failures that escaped pre-deploy testing. It is the post-run layer in the Guard Pipeline: Input Guard → Deploy Guard → Output Guard.

• Use Output Guard when your actor run succeeds but the returned data may be incomplete or degraded.
• Use Input Guard for pre-run input validation. Use Deploy Guard for pre-deploy synthetic test suites.

Features

Output Guard detects data quality issues automatically — including missing fields, schema drift, and degraded output — even when the actor succeeds.

Quality score and schema validation Scores output quality 0-100 with pass (80+), warn (50-79), fail (0-49) verdicts. Detects type mismatches, undeclared fields, missing required fields, and type inconsistency. Shows sample bad rows (3 null + 3 good values) for every degraded field so you can see exactly what failed.

Completeness analysis with field rules Computes null rate and empty rate per field. Custom rules let you set severity (critical/important/optional), maxNullRate, expectedType, and regex patterns per field. Four built-in policy templates cover ecommerce, lead generation, AI-ready output, and strict schema compatibility.

Failure mode classification Identifies the likely cause when quality degrades: selector break, upstream structure change, pagination failure, throttling, partial extraction, or schema drift. Includes confidence scores so you know how certain the diagnosis is.

Drift detection and baselines Tracks output structure across runs. Compares against the previous run, last passing run, or a manually approved baseline. Detects new fields, missing fields, type changes, null rate spikes, and value distribution shifts (cardinality changes >50%, numeric mean shifts >2 standard deviations).

Fleet monitoring Scan multiple actors in one run. Set budget limits, max actors per run, stop-on-critical, and risk-based prioritisation. Produces a fleet summary with the most unstable actor, most common failing field, and average quality.

Canary testing Define up to 10 test scenarios with different inputs and weights. Each canary runs independently and gets its own pass/fail verdict. Use this for CI/CD quality gates.

Alerts and auto-actions Severity-routed Slack webhooks with consecutive failure threshold, cooldown, and dedup. As an Apify actor output validator, Output Guard can auto-respond to failures: disable the actor, pause a schedule, trigger another actor, or fire a webhook.

SLA enforcement with compliance tracking Set thresholds for quality score, null rate, item count, and response time. Output Guard stores pass/fail per run and reports 30-day compliance rate.

Advanced analytics

These features are optional and become useful after multiple runs. They are not required for basic validation:

• Risk profiles — low/medium/high/critical based on failure frequency and drift volatility
• Trust scores — composite of pass rate, active incidents, risk level (trusted/cautious/unreliable)
• Stability forecasting — predicts next run risk based on recent history
• Change attribution — distinguishes build changes from upstream data source changes
• Silent failure risk — flags when completeness variance is high but the actor keeps "passing"
• Blast radius estimation — counts downstream actors affected by quality issues
• Cross-run correlations — recurring failure patterns, post-deploy regression, progressive degradation
• Cost anomaly detection — flags runs taking >2x baseline duration
• AI readiness score — checks field titles, descriptions, clean types, camelCase naming
• LLM suitability analysis — checks nesting depth, naming consistency, schema metadata completeness
• Schema evolution history — snapshots per build, up to 20 retained
• Incident tracking — active/resolved incidents with priority queue and occurrence counting

How do I detect silent data failures on Apify?

Detect silent data failures by validating output data — not just run status. Run Output Guard daily in monitor mode. It compares each run to a baseline and flags null spikes, missing fields, and schema changes — even when the run succeeds. Add an alertWebhookUrl to get notified in Slack when quality degrades.

How do I compare output schemas between two actors?

Set mode to diff and provide both actor IDs (targetActorId and compareActorId). Output Guard fetches both schemas, compares field names, types, and structure, and returns a compatibility score with breaking changes, migration notes, and shared fields.

How do I validate an existing dataset without re-running the actor?

Provide a datasetId instead of testInput. Output Guard loads items directly from the existing dataset and runs all validation checks. Zero compute cost for the target actor — you only pay the $4.00 validation fee.

How do I add a quality gate before promoting an actor update?

Run Output Guard with canary scenarios that cover your critical paths. If any canary fails or the quality score drops below your SLA threshold, block the deployment. Combine with the autoActions input to automatically pause the schedule or trigger a rollback actor on failure.

How do I monitor null rate drift across actor runs?

Set mode to monitor with enableDriftTracking: true. Output Guard saves a field baseline after each run. On subsequent runs, it compares current null rates against the baseline and reports null_rate_spike drift signals with the exact percentage change.

Why does an actor succeed but return incomplete data?

Actor ran successfully but data is missing? Actors can return degraded data due to selector breaks, pagination failures, or upstream structure changes — even with status OK and zero errors. Output Guard detects these silent failures by analysing null rates, schema drift, and field completeness.

How do I monitor data quality in Apify actors?

Use an Apify actor output validator like Output Guard to analyse dataset completeness, detect schema drift, and track quality scores across runs. Schedule it in monitor mode to automatically detect degradation and send alerts.

What is schema drift in actor output?

Schema drift is when the structure of output data changes between runs — such as missing fields, new fields, or type changes. In Apify actors, schema drift is often caused by website layout changes, API updates, or upstream data source modifications. Output Guard detects schema drift automatically and flags breaking changes before they impact downstream systems.

What is the best way to validate actor output automatically?

The fastest way to validate actor output is to use an automated validator that checks completeness, schema consistency, and drift in one run. Output Guard performs these checks in a single step and returns a quality score with recommended fixes.

How to validate actor output quality

1. Enter your target actor — Paste the actor ID (e.g., ryanclinton/website-contact-scraper) and optionally provide test input JSON.
2. Choose a mode — Select validate for a one-off check, monitor for drift tracking with baselines, or diff to compare two actors.
3. Run the actor — Click "Start". Typical validation takes 30-120 seconds depending on your target actor's run time.
4. Review the report — Open the Dataset tab to see the quality score, per-field completeness, drift signals, and recommendations. The Key-Value Store has a SUMMARY view.

First run tips

• Start with quickStart mode — Set quickStart: true and Output Guard auto-detects important fields (email, phone, price, URL), assigns severity levels, generates policies, and enables drift tracking. No manual configuration needed.
• Keep test input minimal — The test input runs the target actor, so use a small input to save compute. One URL or a few search terms is enough.
• Check the SUMMARY key-value store — The full report is in the dataset, but the SUMMARY view in Key-Value Store gives you the verdict, score, and top issues at a glance.
• Enable drift tracking early — The first monitor run saves a baseline. Drift detection becomes useful from the second run onward.
• Use backfill mode for debugging — Provide a datasetId from a previous run to validate existing output without re-running the actor.

Typical performance

Metric	Typical value
Items analysed per run	10-1,000 (configurable, max 10,000)
Validation time (simple actor)	30-60 seconds
Validation time (complex actor)	60-180 seconds
Fleet scan (10 actors)	5-15 minutes
Quality score calculation	Under 1 second
Memory usage	128-256 MB

Input parameters

Parameter	Type	Required	Default	Description
targetActorId	string	Yes	—	Actor ID or username/actor-name to validate
mode	string	No	validate	validate (one-off), monitor (drift + alerts), diff (compare schemas)
testInput	object	No	{}	Input JSON to run the target actor with. Keep minimal to save compute.
datasetId	string	No	—	Validate an existing dataset without running the target actor (backfill mode)
canaries	array	No	—	Test scenarios: name, input, weight, requiredFields (max 10)
fieldRules	object	No	{}	Per-field rules: severity, maxNullRate, maxEmptyRate, expectedType, pattern, required
globalRules	object	No	{}	Actor-level rules: minItems, maxItems
policies	array	No	—	Unified rules: name, conditions (field + global), onBreach (severity + actions)
policyTemplates	array	No	—	Built-in policies: ecommerce-critical-fields, lead-gen-contact-integrity, ai-ready-output, strict-schema-compatibility
compareActorId	string	No	—	Second actor for diff mode
enableDriftTracking	boolean	No	false	Track baselines across runs. Auto-enabled in monitor mode.
baselineStrategy	string	No	previousRun	Baseline comparison: previousRun, lastGood, approved
approveBaseline	boolean	No	false	Save this run as the approved baseline
resetBaseline	boolean	No	false	Clear all stored baselines
alertWebhookUrl	string	No	—	Slack webhook URL for quality degradation alerts
nullRateThreshold	number	No	0.2	Alert when null rate exceeds this (0.0-1.0)
minConsecutiveFailuresForAlert	integer	No	1	Consecutive failures before alerting (1-10)
cooldownMinutes	integer	No	0	Min minutes between alerts for same issue (0-1440)
timeout	integer	No	300	Max seconds for target actor run (10-3600)
memory	integer	No	512	Memory in MB for target actor run (128-32768)
maxSampleItems	integer	No	1000	Items to analyse (10-10,000)
additionalActorIds	array	No	—	Additional actor IDs for fleet mode
fleetConfig	object	No	—	Fleet settings: maxFleetSpend, maxActorsPerRun, stopOnCritical, prioritiseByRisk
quickStart	boolean	No	false	Auto-configure with sensible defaults
guardMode	boolean	No	false	Strict monitoring: drift + confidence gate + auto-actions
autoConfig	boolean	No	false	Auto-detect fields and generate rules from output

Input examples

Quick validate (most common):

{
    "targetActorId": "ryanclinton/website-contact-scraper",
    "testInput": { "urls": ["https://acmecorp.com"] }
}

Monitor with alerts and field rules:

{
    "targetActorId": "ryanclinton/website-contact-scraper",
    "mode": "monitor",
    "testInput": { "urls": ["https://acmecorp.com"] },
    "enableDriftTracking": true,
    "alertWebhookUrl": "https://hooks.slack.com/services/T00/B00/xxx",
    "fieldRules": {
        "email": { "severity": "critical", "maxNullRate": 0.1, "pattern": "^[^@\\s]+@[^@\\s]+\\.[^@\\s]+$" },
        "phone": { "severity": "important", "maxNullRate": 0.3 },
        "domain": { "required": true }
    },
    "sla": { "minQualityScore": 80, "maxNullRate": 0.2, "minItems": 5 },
    "minConsecutiveFailuresForAlert": 2,
    "cooldownMinutes": 60
}

Fleet scan with budget controls:

{
    "targetActorId": "ryanclinton/website-contact-scraper",
    "additionalActorIds": [
        "ryanclinton/google-maps-email-extractor",
        "ryanclinton/email-pattern-finder",
        "ryanclinton/bulk-email-verifier"
    ],
    "quickStart": true,
    "fleetConfig": {
        "maxFleetSpend": 5.00,
        "maxActorsPerRun": 10,
        "stopOnCritical": true,
        "prioritiseByRisk": true
    }
}

Input tips

• Use quickStart for zero-config monitoring — auto-detects important fields and generates rules and policies automatically
• Use backfill mode to re-validate — provide a datasetId from a previous run instead of re-running the actor
• Set field rules for critical fields — severity: "critical" with maxNullRate: 0.1 catches email/price degradation early
• Apply policy templates — policyTemplates: ["lead-gen-contact-integrity"] applies pre-built rules for lead generation actors
• Use canaries for multi-scenario testing — define 3-5 test scenarios with different inputs and weights to cover edge cases

Output example

{
    "actorName": "ryanclinton/website-contact-scraper",
    "actorId": "ryanclinton/website-contact-scraper",
    "mode": "monitor",
    "verdict": "warn",
    "qualityScore": 68,
    "aiReadinessScore": 55,
    "explanation": "3 field(s) with high null rates.",
    "recommendations": [
        "Investigate 3 field(s) with critically high null rates: phone, socialLinks, companyDescription",
        "Check selector for 'phone' — 45% null rate suggests the element is no longer found on the page"
    ],
    "schemaFound": true,
    "schemaFields": 12,
    "outputFields": 15,
    "totalItems": 47,
    "mismatches": [],
    "undeclaredFields": ["socialLinks", "companyDescription", "metaKeywords"],
    "missingRequired": [],
    "nullableIssues": ["phone"],
    "typeConsistency": [],
    "completeness": [
        {
            "field": "phone",
            "presentCount": 26,
            "totalItems": 47,
            "nullRate": 0.45,
            "emptyRate": 0.02,
            "fieldSeverity": "critical",
            "status": "critical",
            "sampleNulls": [
                { "index": 0, "value": null },
                { "index": 3, "value": null },
                { "index": 7, "value": null }
            ],
            "sampleValues": [
                { "index": 1, "value": "+1 (555) 234-5678" },
                { "index": 2, "value": "+1 (555) 876-4321" },
                { "index": 5, "value": "+44 20 7946 0958" }
            ]
        },
        {
            "field": "email",
            "presentCount": 44,
            "totalItems": 47,
            "nullRate": 0.06,
            "emptyRate": 0.0,
            "fieldSeverity": "critical",
            "status": "healthy"
        }
    ],
    "drift": {
        "hasPreviousBaseline": true,
        "baselineStrategy": "previousRun",
        "signals": [
            {
                "type": "null_rate_spike",
                "field": "phone",
                "detail": "12% → 45%",
                "severity": "critical"
            }
        ],
        "changeSummary": ["Null rate 'phone': 12% → 45%"]
    },
    "fieldRuleViolations": [
        {
            "field": "phone",
            "rule": "maxNullRate: 0.1",
            "actual": "45%",
            "severity": "critical"
        }
    ],
    "executiveSummary": {
        "status": "Degrading",
        "reason": "3 field(s) with high null rates.",
        "action": "Check selector for 'phone' — 45% null rate suggests the element is no longer found on the page"
    },
    "failureMode": {
        "mode": "partial_extraction",
        "confidence": 0.6,
        "signals": ["Single field degraded while others remain healthy"]
    },
    "runClassification": "degrading",
    "riskProfile": {
        "level": "medium",
        "driftFrequency": 30,
        "failureFrequency": 20,
        "stabilityScore": 50,
        "reason": "occasional quality issues"
    },
    "confidenceScore": 72,
    "trustScore": {
        "score": 65,
        "level": "cautious",
        "factors": ["Low pass rate (70%)", "1 active incident(s)", "Risk level: medium"]
    },
    "silentFailureRisk": 38,
    "fieldDistributions": [
        {
            "field": "email",
            "cardinality": 42,
            "topValues": [{ "value": "sarah.chen@acmecorp.com", "count": 1 }],
            "numericStats": null
        }
    ],
    "distributionShifts": [],
    "slaResult": {
        "passed": false,
        "breaches": [{ "rule": "maxNullRate:phone", "threshold": 0.2, "actual": 0.45 }],
        "complianceRate": 73.3,
        "complianceWindow": 30
    },
    "runDuration": 45.2,
    "alertSent": true,
    "validatedAt": "2026-04-07T14:30:00.000Z"
}

Output fields

Field	Type	Description
actorName	string	Full name of the validated actor
actorId	string	Actor ID provided as input
mode	string	Validation mode: validate, monitor, or diff
verdict	string	pass (80+), warn (50-79), or fail (0-49)
qualityScore	number	Overall quality score 0-100
aiReadinessScore	number	Schema metadata quality score 0-100
explanation	string	Human-readable summary of the verdict
recommendations	string[]	Actionable fix suggestions
schemaFound	boolean	Whether the target actor has a dataset schema defined
schemaFields	number	Number of fields in the declared schema
outputFields	number	Number of fields found in actual output
totalItems	number	Number of items analysed
mismatches	array	Schema type mismatches with path, expected, actual, severity
undeclaredFields	string[]	Fields in output not declared in schema
missingRequired	string[]	Schema fields not found in output
completeness	array	Per-field null rate, empty rate, severity, status, sample rows
drift	object	Drift report with signals, baseline strategy, change summary
fieldRuleViolations	array	Custom field rule breaches with severity
executiveSummary	object	One-line status, reason, and recommended action
failureMode	object	Classified failure mode with confidence score
runClassification	string	healthy, recovering, degrading, or broken
riskProfile	object	Risk level, failure frequency, drift frequency, stability score
confidenceScore	number	Confidence in the quality assessment 0-100
trustScore	object	Composite trust score with level (trusted/cautious/unreliable)
silentFailureRisk	number	Likelihood of undetected degradation 0-100
fieldDistributions	array	Per-field cardinality, top values, numeric stats
distributionShifts	array	Value distribution changes from previous run
incidents	array	Active incidents with severity, occurrence count, first/last seen
correlations	array	Cross-run patterns: recurring failures, post-deploy regression
canaryResults	array	Per-canary pass/fail, item count, score, issues
slaResult	object	SLA compliance with breaches and 30-day compliance rate
policyBreaches	array	Policy violations with breached conditions and actions executed
autoActionsExecuted	array	Auto-action results (disableActor, pauseSchedule, etc.)
costAnomaly	object	Duration anomaly vs baseline (if >2x)
runDuration	number	Total validation time in seconds
alertSent	boolean	Whether a webhook alert was dispatched
validatedAt	string	ISO 8601 timestamp

How much does it cost to validate actor output?

Output Guard uses pay-per-event pricing — you pay $4.00 per Output Guard check. Platform compute costs are included. The target actor's compute is billed separately at standard Apify rates.

Scenario	Checks	Cost per check	Total cost
Quick test	1	$4.00	$4.00
Weekly monitor (1 actor)	4/month	$4.00	$2.00/month
Daily monitor (1 actor)	30/month	$4.00	$15.00/month
Daily fleet (5 actors)	150/month	$4.00	$75.00/month
Daily fleet (20 actors)	600/month	$4.00	$300.00/month

Fleet mode charges $4.00 per actor scanned. A fleet of 10 actors costs $5.00 per run. Use fleetConfig.maxFleetSpend to set a budget cap.

Set a spending limit in your Apify account settings to prevent unexpected charges. Apify's free tier includes $5 of monthly credits — enough for 10 Output Guard checks.

Validate actor output using the API

Python

from apify_client import ApifyClient

client = ApifyClient("YOUR_API_TOKEN")

run = client.actor("ryanclinton/actor-schema-validator").call(run_input={
    "targetActorId": "ryanclinton/website-contact-scraper",
    "mode": "monitor",
    "testInput": {"urls": ["https://acmecorp.com"]},
    "enableDriftTracking": True,
    "fieldRules": {
        "email": {"severity": "critical", "maxNullRate": 0.1}
    }
})

for item in client.dataset(run["defaultDatasetId"]).iterate_items():
    print(f"{item['actorName']}: {item['verdict']} ({item['qualityScore']}/100)")
    if item.get("recommendations"):
        for rec in item["recommendations"]:
            print(f"  → {rec}")

JavaScript

import { ApifyClient } from "apify-client";

const client = new ApifyClient({ token: "YOUR_API_TOKEN" });

const run = await client.actor("ryanclinton/actor-schema-validator").call({
    targetActorId: "ryanclinton/website-contact-scraper",
    mode: "monitor",
    testInput: { urls: ["https://acmecorp.com"] },
    enableDriftTracking: true,
    fieldRules: {
        email: { severity: "critical", maxNullRate: 0.1 }
    }
});

const { items } = await client.dataset(run.defaultDatasetId).listItems();
for (const item of items) {
    console.log(`${item.actorName}: ${item.verdict} (${item.qualityScore}/100)`);
    if (item.recommendations) {
        item.recommendations.forEach(rec => console.log(`  → ${rec}`));
    }
}

cURL

curl -X POST "https://api.apify.com/v2/acts/ryanclinton~actor-schema-validator/runs?token=YOUR_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "targetActorId": "ryanclinton/website-contact-scraper",
    "mode": "validate",
    "testInput": { "urls": ["https://acmecorp.com"] }
  }'

curl "https://api.apify.com/v2/datasets/DATASET_ID/items?token=YOUR_API_TOKEN&format=json"

How Output Guard works

Mental model: Target actor run -> output sampling -> schema validation -> completeness analysis -> rule checking -> drift comparison -> scoring -> failure classification -> report.

Phase 1: Target actor execution

Output Guard runs the target actor with the provided test input (or loads an existing dataset in backfill mode). For canary testing, each scenario runs independently with its own input and gets a separate pass/fail verdict. Adaptive sampling selects up to 1,000 items — full scan for small datasets, stratified sampling for large ones (first N + offset sample).

Phase 2: Schema and completeness validation

Output Guard fetches the target actor's dataset schema from the Apify API via the latest build metadata and compares it against actual output. It checks type mismatches (normalising integer/float/double to number), undeclared fields, missing required fields, and nullable violations. Completeness analysis computes null rate and empty rate per field with severity-based thresholds — critical fields alert at 10% null, important at 30%, optional at 50%. Degraded fields include sample bad rows: 3 null values and 3 good values from the output.

Phase 3: Drift and distribution analysis

In monitor mode, Output Guard compares current output against a stored baseline (previousRun, lastGood, or approved) from the output-guard-baselines key-value store. It detects new fields, missing fields (critical severity), type changes (critical), null rate spikes (warning at +10%, critical at +30%), and field count changes. Distribution shift detection computes per-field cardinality, numeric stats, and top values, then flags cardinality changes exceeding 50%, numeric mean shifts exceeding 2 standard deviations, and dominant value changes.

Phase 4: Intelligence and scoring

The scoring engine combines schema errors (-10 each), critical field violations (-15), important violations (-5), critical completeness issues (-8), degraded fields (-3), drift signals, and canary failures into a 0-100 score. Failure mode classification analyses the pattern of issues — multiple critical null fields + missing fields = selector_break (85% confidence), type changes = upstream_structure_change (75%), single degraded field with others healthy = partial_extraction (60%). Risk profiles, trust scores, stability forecasts, and silent failure risk scores use the last 10-20 runs of history.

Tips for best results

1. Schedule monitor mode daily — Daily runs build drift baselines. After 5+ runs, Output Guard provides risk profiles, trust scores, and stability forecasts.
2. Define field rules for critical fields — Auto-config detects common field names, but explicit rules with severity levels give more precise alerts.
3. Use canary scenarios for complex actors — Define 3-5 scenarios with different inputs and weights. A search actor should test empty results, single results, and full pages.
4. Apply policy templates — lead-gen-contact-integrity checks email and phone null rates. ecommerce-critical-fields checks price and title. Templates save configuration time.
5. Set consecutive failure thresholds — minConsecutiveFailuresForAlert: 2 reduces noise from intermittent issues while still catching real degradation.
6. Review the executive summary first — The executiveSummary field gives a one-line status, reason, and action. Start there before diving into per-field details.
7. Use backfill mode for debugging — When investigating a historical issue, provide the dataset ID from the problematic run instead of re-running the actor.
8. Set up the quality trilogy — Use Input Guard before runs, Deploy Guard before deploys, and Output Guard after deploys for full coverage.

The Guard Pipeline

Output Guard is part of a three-stage quality pipeline for Apify actors:

Stage	Guard	What it prevents
Before run	Input Guard	Bad input wasting runs and credits
Before deploy	Deploy Guard	Broken builds reaching production
After deploy	Output Guard	Silent data failures in production

Which Guard do I need?

• "My actor won't start or crashes immediately" → Input Guard
• "I changed code — is it safe to deploy?" → Deploy Guard
• "It runs fine but the data looks wrong" → Output Guard

Use all three together for full lifecycle coverage. Input Guard costs $0.15 per test, Deploy Guard $2.50 per suite, Output Guard $4.00 per check.

Shared state across Guards

All three Guards share a per-actor quality profile stored in a named KV store (aqp-{actorslug}). This enables:

• Cross-stage history — each Guard appends results to a shared timeline, so you can see input validation, pre-deploy testing, and production monitoring in one view
• Baselines — each stage stores its own baseline (null rates, schema snapshots, result counts) that the other stages and a future wrapper actor can read
• Feedback loops — Output Guard automatically suggests new Deploy Guard assertions when critical fields degrade in production. Deploy Guard suggests Output Guard field rules when pre-deploy tests find flaky fields.
• Unified field importance — pass fieldImportanceProfile to any Guard and it applies severity-appropriate checks for that stage

Common output interface

All three Guards return the same top-level fields for easy aggregation:

• stage — "input", "deploy", or "output"
• status — "pass", "warn", "block", or "fail"
• score — 0-100 quality score
• summary — one-sentence explanation
• recommendations — actionable fix suggestions
• signals — key metrics (errorCount, warningCount, criticalCount, driftDetected)
• signals.metrics — stage-specific data (totalItems, degradedFields, driftSignals, etc.)

{
    "stage": "output",
    "status": "fail",
    "score": 38,
    "summary": "Critical field email null rate rose to 60% in production.",
    "recommendations": ["Check selector for email — 60% null rate suggests element not found on page."],
    "signals": { "errorCount": 0, "warningCount": 2, "criticalCount": 1, "driftDetected": true, "metrics": { "totalItems": 47, "degradedFields": 3, "qualityScore": 38 } }
}

Status semantics across Guards

Status	Meaning	Input Guard	Deploy Guard	Output Guard
pass	Acceptable, no action required	Input valid	Safe to deploy	Production data healthy
warn	Usable but degraded — review	—	Soft regressions detected	Data quality declining
block	Do not proceed	Don't run — input invalid	Don't ship — critical failures	—
fail	Live failure — unacceptable	—	—	Production data is bad

Input Guard returns pass or block. Deploy Guard returns pass, warn, or block. Output Guard returns pass, warn, or fail. The wrapper uses strict precedence: any block → overall block, any fail → overall fail, any warn → overall warn.

Output Guard is the post-deployment layer. It monitors what the actor actually returns in production, not what it returned in testing. Schedule Output Guard daily alongside your production actors to catch regressions that pre-deploy testing misses.

Combine with other Apify actors

Actor	How to combine
Website Contact Scraper	Validate contact scraper output with lead-gen policy template
Google Maps Email Extractor	Monitor Maps extraction quality and detect when Google changes page structure
Bulk Email Verifier	Validate verifier output — check that results are not returning excessive unknowns
B2B Lead Gen Suite	Fleet scan all sub-actors in the lead gen pipeline to find the weakest link
Website Tech Stack Detector	Validate tech detection output includes expected fields after platform updates
Website Change Monitor	Pair with Output Guard — monitor pages for structural changes, then validate actor output

Limitations

• Requires target actor availability — If the target actor is broken, unreachable, or behind authentication that Output Guard cannot access, validation fails.
• Schema validation requires a dataset schema — Without a declared dataset_schema.json on the target actor, Output Guard performs structural analysis only (completeness, type consistency) but cannot detect undeclared or missing fields.
• Drift requires 2+ runs — The first monitor run saves a baseline. Drift detection, risk profiles, trust scores, and stability forecasts become available from the second run onward.
• Sample-based validation — Output Guard analyses up to 1,000 items by default (max 10,000). Issues in items beyond the sample window may go undetected.
• Fleet mode charges per actor — Each actor in a fleet scan costs $4.00. A fleet of 20 actors costs $10.00 per run.
• Canary limit of 10 scenarios — Each canary runs the target actor separately, so costs and time scale linearly.
• No real-time monitoring — Output Guard runs as a batch actor. For continuous monitoring, schedule it on a cadence.
• Backfill mode does not detect runtime issues — Validating an existing dataset checks data quality but cannot detect timeout, memory, or performance issues from the original run.
• Category benchmarking uses proxy metrics — Store statistics provide approximate comparisons, not exact field-level benchmarks.

Integrations

• Zapier — Trigger Output Guard on a schedule and route alerts to Slack, email, or PagerDuty
• Make — Build multi-step workflows: Output Guard detects failure -> Make notifies team -> creates Jira ticket
• Google Sheets — Export quality scores to a spreadsheet for fleet-wide dashboards
• Apify API — Trigger Output Guard from CI/CD pipelines (GitHub Actions, GitLab CI, Jenkins)
• Webhooks — Connect Output Guard alerts to any HTTP endpoint for custom incident response

How to set up automated actor monitoring

Schedule Output Guard in monitor mode on a daily or hourly cadence using Apify Schedules. Add a Slack webhook URL and set minConsecutiveFailuresForAlert: 2 to reduce noise. Output Guard tracks baselines automatically, detects drift, and sends alerts only when quality degrades consistently. The executive summary field is designed for Slack consumption — one-line status with reason and action.

How to add quality gates to CI/CD

Run Output Guard via the Apify API after deploying an actor update. Check the verdict field in the response — if it returns fail, roll back the deployment. Use canary scenarios to cover critical user journeys, and apply policy templates or custom SLA thresholds to enforce minimum quality standards. The slaResult field provides a structured pass/fail against your defined thresholds.

Troubleshooting

Target actor times out during validation. Increase the timeout parameter (default 300 seconds). Some actors need more time for complex inputs. Output Guard adds a 60-second buffer beyond the configured timeout before declaring a wall-clock timeout.

Quality score is 0 with no items returned. The target actor returned an empty dataset. Check that the test input is valid by running the target actor manually first. Verify the actor ID is correct (use username/actor-name format).

Drift shows "No baseline" on every run. Drift tracking must be explicitly enabled with enableDriftTracking: true or by using monitor mode. Validate mode does not save baselines by default. The first run always creates a baseline — drift detection starts from the second run.

Alert webhook not firing. Verify the webhook URL is correct and accessible. Check alertSuppressed and alertSuppressionReason in the output — alerts may be suppressed by cooldown, consecutive failure threshold, or dedup logic. Alerts only fire on warn or fail verdicts, never on pass.

Fleet mode stops before scanning all actors. Check fleetConfig.maxFleetSpend and fleetConfig.maxActorsPerRun. The fleet stops when either budget or actor limit is reached. The output includes actorsSkipped with the specific reason for each skipped actor.

Recent updates

• Sample bad rows — Degraded and critical fields now show 3 null values and 3 good values, so you can see exactly what failed
• Field value distribution tracking — Detect when values shift even if the schema is correct (cardinality changes, numeric range shifts, dominant value changes)
• SLA compliance history — Track pass rate over a 30-day rolling window with slaResult.complianceRate
• Backfill validation — Validate existing datasets without re-running the target actor by providing a datasetId
• Auto-action authentication — Auto-actions (disableActor, pauseSchedule, triggerActor) now authenticate correctly with the Apify API
• Canary deduplication — Canary testing no longer runs the primary scenario twice

Responsible use

• Output Guard validates the output of other Apify actors by running them with user-provided input. It does not bypass authentication, CAPTCHAs, or access restricted content.
• Users are responsible for ensuring the target actors they validate comply with applicable laws, platform terms, and data protection regulations.
• Do not use Output Guard to validate actors that access content you are not authorised to scrape.
• For guidance on responsible actor usage, see Apify's documentation.

FAQ

Output Guard validates actor output quality — not just execution success.

Does Output Guard run my target actor? Yes. Output Guard calls the target actor via Actor.call(). The target actor runs under your account at standard compute rates. The $4.00 PPE charge covers Output Guard's analysis only, not the target actor's compute.

Can I use it without a dataset schema on the target actor? Yes. Without a declared schema, Output Guard performs structural analysis (completeness, type consistency, distributions, drift) and all intelligence features. Schema-specific checks (type mismatches, undeclared fields) are skipped.

How does the quality score work? Starts at 100 and deducts points: -10 per schema type mismatch, -15 per critical field rule violation, -8 per critical null rate field, -3 per degraded field, plus deductions for drift signals and failing canaries. Pass is 80+, warn is 50-79, fail is below 50.

What failure modes does it classify? Six modes: selector_break, upstream_structure_change, pagination_failure, throttling, partial_extraction, schema_drift. Each includes a confidence score.

How is it different from the Deploy Guard? The Reliability Engine is pre-deploy — it runs test suites before promotion. Output Guard is post-deploy — it monitors production output with drift tracking, incident management, and auto-actions. They complement each other.

Can it automatically disable a broken actor? Yes. Configure autoActions.onCritical with disableActor, pauseSchedule, triggerActor, or webhook actions.

How often should I schedule it? Daily for production actors ($15/month). Hourly for high-value pipelines (pricing, lead generation). Weekly for infrequently updated actors. As the Apify actor output validator, Output Guard is designed for scheduled recurring use — drift detection and trend analysis improve with each run.

Help us improve

If you encounter issues, you can help us debug faster by enabling run sharing in your Apify account:

1. Go to Account Settings > Privacy
2. Enable Share runs with public Actor creators

This lets us see your run details when something goes wrong, so we can fix issues faster. Your data is only visible to the actor developer, not publicly.

Support

Found a bug or have a feature request? Open an issue in the Issues tab on this actor's page. For custom solutions or enterprise integrations, reach out through the Apify platform.