Actor Test Runner — Validate Inputs, Outputs & Error Handling
Pricing
$350.00 / 1,000 test suite runs
Actor Test Runner. Available on the Apify Store with pay-per-event pricing.
Pricing
$350.00 / 1,000 test suite runs
Rating
0.0
(0)
Developer
Ryan Clinton
Maintained by CommunityActor stats
0
Bookmarked
4
Total users
1
Monthly active users
a day ago
Last modified
Categories
Share
Schema version
schemaVersion
Optional
Output-shape contract version (semver). Minor bumps are additive (new top-level fields); a major bump signals a breaking rename/removal. Chained consumers (Fleet Analytics, CI gates) branch on this to stay compatible across releases.
Agent contract
agentContract
Optional
Universal agent/MCP routing surface shared by every actor in this suite. recommendedAction (act_now | monitor | ignore) is the one identically-named field a tool-caller, webhook, or CI gate can branch on regardless of which actor produced the run. Here it mirrors decision.
Decision
decision
Optional
Machine-routable decision tag. act_now = trusted verdict, safe to deploy or halt. monitor = directional only (cold-start, low confidence, or warn verdict). ignore = no tests ran. Branch on this in Slack, PagerDuty, CI gates, and agent tool calls — never parse the prose explanation.
Release action
releaseAction
Optional
Unambiguous release action collapsing (decision, status) into one verb so a CI gate never has to reason that act_now means deploy on pass but halt on block. deploy = ship it (act_now + pass). halt = stop the release (act_now + block). review = a human should look, never auto-act (monitor). misconfigured = no tests ran / suite lint failed (ignore). Derived deterministically; decision stays the canonical Guard-fleet enum.
Release verdict trend
releaseTrend
Optional
Cross-run drift of the release verdict itself (not just fields): previousStatus, currentStatus, changed, direction (improved/regressed/unchanged), previousScore, scoreDelta, lastSeenAt. Null on the first tracked run. A regressed direction (e.g. pass → block since last run) is the highest-value alert for a scheduled deploy gate.
Release stability
releaseStability
Optional
How consistently this build has passed over its tracked history — distinct from confidence (trust in THIS run's verdict). { stablePct, runsTracked, passingRuns, note }. Null until a baseline with run history exists. Answers 'how often does this actor pass?' for a scheduled gate.
Release memory
releaseMemory
Optional
The actor's accumulated operational record — a consolidation of stored history that compounds with every run: firstSeenAt, totalRunsTracked, passingRuns, availabilityPct, regressionCount, avgDurationSec, avgResultCount, avgComputeUnits, currentConfidence, confidenceTrend, learnedCriticalFields[], learnedContract[], maintenancePressure, lastIncident, latestDecision, note. learnedCriticalFields are fields the actor has consistently returned (≥90% non-null over ≥3 runs) — their disappearance escalates to a breaking regression even without a declared fieldImportanceProfile. learnedContract[] is the per-field behaviour learned over time ({ field, presenceRate, runsSeen, classification: always/usually/optional }). maintenancePressure ({ level: low/elevated/high, reasons[] }) is a descriptive trajectory from declining confidence + rising regressions + runtime creep — not a forecast. lastIncident ({ startedAt, endedAt, runsAffected, recovered, recoveredAfterRuns }) is the most recent contiguous non-pass streak and whether the actor recovered. Null on cold-start.
Generated suite (auto mode)
generatedSuite
Optional
When preset='auto', the contract Deploy Guard discovered and ran: detectedActorType, requiredFields[], fieldTypes, minResults, maxDuration, note. Null unless auto-discovery ran. Copy it into testCases to lock the contract for future runs.
Decision reason
decisionReason
Optional
One-line plain-language justification behind the decision tag. Usable in logs, alerts, and audit trails.
decisionDrivers
Optional
Top 2-3 stable codes that drove the decision — the reasons most worth highlighting in a CI log or Slack message. Ranked subset of verdictReasonCodes + confidenceFactorCodes, ordered by impact.
Confidence level
confidenceLevel
Optional
Banded confidence: high (>=75), medium (>=50), low (<50). Pair with confidenceFactorCodes for routing logic.
Confidence breakdown
confidenceBreakdown
Optional
Sub-band decomposition: executionConfidence (how cleanly tests ran), schemaConfidence (drift severity), historyConfidence (baseline maturity), suiteDesignConfidence (suite coverage quality). Each banded high/medium/low.
confidenceFactorCodes
Optional
Stable machine-readable codes explaining the confidence score. Additive-only enum. Known codes: cold_start_cap, low_sample_size, small_history, healthy_history, drift_detected, low_suite_coverage, suite_lint_failed.
verdictReasonCodes
Optional
Stable machine-readable codes behind the pass/warn/block verdict. Additive-only enum. Known codes: VERDICT_PASS, VERDICT_WARN, VERDICT_BLOCK, CRITICAL_TEST_FAILURE, WARNING_TEST_FAILURE, BASELINE_DRIFT, COLD_START, SUITE_LINT_FAILED, NO_TESTS.
Status headline
statusHeadline
Optional
Human-readable headline combining decision + verdict + passed count + confidence. Safe to paste into dashboards, Slack, and report subject lines.
One-line summary
oneLine
Optional
Actor-name-prefixed one-liner for email subjects, Slack notifications, and agent tool-call summaries.
Run context
context
Optional
Learning maturity + baseline status. progress is cold-start (<1 prior run) / emerging (<5) / developing (<15) / mature. Reinforces that scheduled repeat runs unlock trust.
Score breakdown
scoreBreakdown
Optional
Auditable scoring: startingScore 100, deductions[] (code + points + count + reason), caps[] (e.g. cold_start_cap capping at 70), finalScore. Sum of deductions is deterministic from the same inputs.
remediation
Optional
Priority-ranked array of structured fix cards — one item per important problem with whyItMatters + suggestedFix + ownerHint. Read these top-down to fix the most impactful issues first.
Suite lint
suiteLint
Optional
Pre-execution validation of the TEST SUITE DEFINITION (not the target actor). Status: pass/warn/fail. Issues carry stable codes: NO_TESTS_SUPPLIED, SINGLE_INPUT_VARIANT, NO_DURATION_GUARD, NO_CRITICAL_CHECKS, SINGLE_TEST_BUT_CI_GATING_HINT.
Suite coverage
suiteCoverage
Optional
How much of the assertion surface the suite uses: score (0-100), assertionTypesUsed[], blindSpots[]. Guards against false confidence from a thin suite.
Drift severity
driftSeverity
Optional
Drift findings tiered into breaking / non-breaking / informational / expected. Breaking = required or critical-importance field removed or type-changed; expected = field listed in allowedDriftFields or test-level waiver.
Criticality impact
criticalityImpact
Optional
criticalFieldsHealthy (boolean), criticalFieldFailures (count), nonCriticalFieldFailures, affectedCriticalFields[]. Derived from fieldImportanceProfile.
Regression summary
regressionSummary
Optional
Compressed trend: direction (better/worse/stable), velocity (fast/moderate/slow/none), confidence (high/medium/low). Null until at least 2 prior confidence snapshots exist.
trendSignals
Optional
Compact trend signal codes — e.g. confidence_regression_moderate, flaky_tests_present, breaking_drift_detected, execution_fast_all_tests. Useful for Fleet Analytics consumption.
fleetSignals
Optional
Stable signal-code array for fleet-wide aggregation. Every entry carries { code, severity: 'critical'|'warning'|'info', scope: 'test'|'field'|'suite'|'run', actionability: 'high'|'medium'|'low', detail?, field? }. Severity is mandatory so downstream aggregators (Fleet Analytics) don't re-derive it. Code enum (additive-only): SCHEMA_DRIFT_CRITICAL (critical), SCHEMA_DRIFT_NONCRITICAL (info), TEST_FLAKY (warning), LOW_SUITE_COVERAGE (warning), CRITICAL_FIELD_FAILURE (critical), CONFIDENCE_REGRESSION (warning), RELEASE_BLOCKED (critical).
Stage
stage
Optional
Lifecycle stage this report represents. Always 'deploy' for the test runner.
Verdict
status
Optional
Raw pass/warn/block verdict before the decision layer. Use decision for automation routing; use status for display.
Confidence score
score
Optional
0-100 composite confidence. Capped at 70 during cold-start (no trusted baseline yet).
Summary
summary
Optional
Plain-language explanation of the verdict, usable in reports.
recommendations
Optional
Suggestions generated from the failure mix — what to investigate or fix next.
Signal counts
signals
Optional
errorCount / warningCount / criticalCount / driftDetected plus metrics — for dashboards and aggregators.
Actor display name
actorName
Optional
The tested actor's username/name
Actor ID
actorId
Optional
The tested actor's ID or username/name
Total tests
totalTests
Optional
Number of test cases executed
Passed
passed
Optional
Number of test cases that passed (includes known issue allowances)
Failed
failed
Optional
Number of test cases that failed unexpectedly
Known issue allowances
expectedFailures
Optional
Test cases marked expectedToFail that failed as expected
Total duration (seconds)
totalDuration
Optional
Total wall-clock time for all test cases
results
Optional
Per-test results with quality checks, forensic details, schema contracts, duration, and error classification
Release decision
releaseDecision
Optional
pass/warn/block recommendation with confidence score, critical/warning failure counts, and drift flag
Drift report
drift
Optional
Field schema drift vs previous baseline: new fields, missing fields, type changes, null-rate shifts. Null when baseline not enabled.
stability
Optional
Per-test pass-rate history and flakiness flag. Null until baseline has been built.
history
Optional
Last 20 run snapshots — status, confidence, duration, passed/failed counts. Null on cold-start.
Detected actor type
detectedActorType
Optional
Heuristic classification: scraper / contact-scraper / api-actor / ecommerce / unknown. Use to pick a richer preset.
Suggested preset
suggestedPreset
Optional
Preset name that would give richer validation for the detected actor type. Null when current preset already fits.
Tested at
testedAt
Optional
ISO 8601 timestamp of test completion
Error flag
error
Optional
True if the run itself failed
Error message
message
Optional
Error description when error is true
Failure type
failureType
Optional
Classification: invalid-input, timeout, rate-limited, blocked (403 / captcha / Cloudflare), auth-failure (401 / unauthorized), api-error, internal-error
Error timestamp
timestamp
Optional
ISO 8601 timestamp of error
