JSON Schema Contract Change Radar

Find a concrete JSON value that an earlier contract accepts and a later contract rejects. Supply two JSON Schemas, optionally add example values you want checked, and receive validated counterexamples with errors from the rejecting schema.

This is a bounded compatibility check for API inputs, MCP tool contracts, event payloads and stored JSON records. It does not call an API or execute an MCP tool. A counterexample demonstrates a specific contract difference; a search that finds none does not prove compatibility.

Choose the direction

The check always asks: does a value accepted by beforeSchema fail afterSchema? Input position determines the direction, regardless of version dates.

For request validation, compare the old accepted requests against the new request schema. For a producer's responses, you may need the reverse: put the new producer schema in beforeSchema and the old consumer schema in afterSchema. That asks whether the new producer can emit a value that the existing consumer rejects. The Actor does not infer your system's producer/consumer relationship or business impact.

Input example

This small order example is illustrative. It is not a real API change or captured request. Increasing the minimum quantity from 1 to 2 rejects the supplied { "quantity": 1 } candidate.

{
  "comparisons": [
    {
      "name": "illustrative-order-minimum",
      "beforeSchema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
          "quantity": { "type": "integer", "minimum": 1 }
        },
        "required": ["quantity"],
        "additionalProperties": false
      },
      "afterSchema": {
        "$schema": "http://json-schema.org/draft-07/schema#",
        "type": "object",
        "properties": {
          "quantity": { "type": "integer", "minimum": 2 }
        },
        "required": ["quantity"],
        "additionalProperties": false
      },
      "candidates": [{ "quantity": 1 }]
    }
  ],
  "maxCandidates": 128,
  "maxWitnesses": 5,
  "maxComparisonTimeMs": 2000,
  "maxRunTimeMs": 60000
}

Each comparison requires a unique name, beforeSchema and afterSchema. Schemas are JSON objects or boolean schemas. Optional candidates contain JSON values, not schema fragments. Useful candidates include realistic boundary cases and payloads from your own authorized test fixtures. A candidate is evidence only after both validations establish that it passes before and fails after.

Input	Default	Limit or meaning
comparisons	Required	1–10 named pairs; names must be unique and 1–80 characters
beforeSchema, afterSchema	Required per pair	At most 256 KiB of serialized JSON per schema
candidates	Optional per pair	At most 20 caller-provided JSON values, each at most 16 KiB
maxCandidates	128	1–128 candidates per comparison
maxWitnesses	5	1–5 returned witnesses per comparison
maxComparisonTimeMs	2000	100–5000 milliseconds per comparison worker
maxRunTimeMs	60000	1000–60000 milliseconds of application analysis time

The entire serialized input is limited to 1 MiB. Input structure is bounded to depth 40 and 30,000 nodes; one worker result is limited to 512 KiB. Oversized or malformed input is rejected before schema compilation. Comparisons run sequentially in isolated workers with memory and time limits. Platform startup and output/storage operations can add time beyond the application analysis budget. Limits prevent unbounded work; they do not promise that every schema within the size cap can complete.

Interpret a result

Always read status, completed, diagnostics and coverageNote together.

Status	Meaning	Completion and billing
breaking_witness	At least one value validates against the before schema and fails the after schema. Inspect the witness and validation errors.	A successfully completed comparison is billable.
unchanged	Schemas are equal after the engine's limited normalization of validation-neutral annotations and set ordering. This is not a general semantic-equivalence test.	Successfully completed and billable.
review_required with completed: true	Schemas changed, but the finite search found no validated counterexample. Manual review is needed.	Billable even though the search is inconclusive.
review_required with completed: false	Unsupported or invalid schema features prevent a supported comparison. Diagnostics explain the limitation.	Not billable.
error with completed: false	A comparison was interrupted, timed out or failed.	Not billable.

When neither schema contains a schema-context $ref, $dynamicRef or $recursiveRef, unchanged normalization ignores only schema-context title, description, default, examples, $comment, readOnly, writeOnly and deprecated. It normalizes object key order and set-like order in required, enum, type, property-name dependencies and dependentRequired. It also normalizes the order of allOf, anyOf and oneOf branches while preserving duplicates. Tuple order in items/prefixItems remains significant. Keys and array order inside literal const/enum values are preserved: a literal property named description is data, not an annotation to discard.

If either schema contains a reference, equality is deliberately more conservative: unchanged requires the original JSON to match after object-key sorting only. Annotation removal and array sorting are disabled because reference targets can depend on those locations. A changed reference-bearing pair can therefore require review even when a human considers its edits harmless.

The validation direction is not symmetric. A witness does not establish how frequently a real service uses that value, whether a deployed client fails, or whether a change is undesirable. An absent witness is not an approval to deploy. Test important real payloads and review the returned diagnostics.

JSON Schema support and limits

• JSON Schema draft-07 and draft 2020-12; an omitted $schema uses draft-07.
• References must use local #… targets within the supplied schema document and be supported by its selected dialect. The Actor does not fetch remote schemas or dereference URLs in instance data.
• Recognized formats are asserted by the configured validator, including URI validation. An unknown format produces an incomplete review result rather than being silently ignored. Format interpretation is part of this validator's contract; it may differ from consumers that treat format as an annotation.
• Unsupported dialects, mixed embedded dialects, unknown validation keywords and unsupported reference features are reported as limitations. $vocabulary, contentEncoding, contentMediaType and contentSchema are unsupported. These cases do not produce a successful compatibility claim or a billable completion.
• Candidate construction is deterministic and finite: generated structures are bounded to depth 6, 32 object properties/array items, 256 generated string characters and 2048 schema-generation steps. Supplied literal values may be larger within input limits. Seeds are considered before generated values, deduplicated and included in maxCandidates. Complex combinations, recursion, numeric/string spaces and other cases may have a real counterexample that the search does not discover.

No LLM, network scraping or remote tool execution is used to decide a finding. beforeHash and afterHash are SHA256 hashes of original schema JSON with object keys sorted and arrays/literal data retained. They are not hashes of raw uploaded-file bytes or signatures from the publisher; annotation-only changes may alter the hashes while still producing unchanged. Workers validate candidate values under both schemas, but validation cannot prove that an external service actually implements its published schema.

Outputs

The dataset contains returned comparison rows with these fields:

comparisonId, name, status, completed, direction, beforeHash, afterHash, candidateCount, witnessCount, witnesses, diagnostics, coverageNote.

Full witness evidence includes the candidate value and up to ten rejection errors per witness. Validation errors identify affected instance/schema locations using JSON Pointers. Use the JSON output when you need the complete value or diagnostic structure; the error list is bounded, not every possible validation failure.

The Output tab links to:

• The default dataset as JSON.
• RESULTS in the key-value store: full JSON comparison evidence, including witness bodies.
• OUTPUT: run summary, coverage and billing information.
• SUMMARY_CSV: a flat comparison summary for spreadsheets. It does not contain full witness bodies.

A completed comparison and a successful charge are different facts; consult the run's billing summary. Budget exhaustion or cancellation can leave comparisons unprocessed. Do not assume every supplied pair completed merely because the run produced a file.

Published-source examples

The local validation used the official Model Context Protocol schemas for 2025-03-26 and 2025-06-18, retrieved on 24 October 2026. These are three definitions from one protocol version pair, not three independent APIs.

For each side of a comparison, the input preserved the source's entire, unedited definitions map and draft-07 $schema, then selected a root with $ref: #/definitions/JSONRPCMessage, #/definitions/CallToolResult or #/definitions/Tool. This wrapping keeps each selected definition's local references resolvable without changing the source definitions. To reproduce a case, obtain the official documents from the links above and supply these wrapped schemas as beforeSchema and afterSchema, with the direction described below. The Actor accepts schema JSON, not source URLs; it does not download those documents during a comparison.

1. A JSON-RPC batch value accepted by the older JSONRPCMessage schema is rejected after batch acceptance is removed.
2. A resource_link tool result accepted by the newer CallToolResult schema is rejected by the older contract. This deliberately reverses version chronology to model a newer producer and older consumer.
3. An older Tool schema permits an undeclared numeric outputSchema property; the newer schema constrains that optional property to an object. This does not imply that a tool without outputSchema breaks.

The source definitions are real published schemas. The supplied payloads are constructed witnesses, not observed production traffic. They seed the search, so successful detection is not a claim about unseeded discovery recall. Source snapshots, retrieval dates and hashes were recorded for the local validation; this description does not promise downloadable project example files in the Store.

A local container check using the actual Apify SDK completed these three comparisons in 1.748 seconds and returned 11 witnesses, each independently validated against both schemas. The container had a 512 MB memory limit; this is not measured peak memory. This local result covers one MCP version pair and is not a cloud performance, cost or margin measurement.

The official upstream license records the repository's MIT-to-Apache-2.0 transition for specifications/code. Preserve its notices and the exact provenance when reusing the schemas. Do not describe these schema specifications as unrestricted or as blanket CC BY documentation.

Actual local output excerpt

The following selected fields are copied from the captured local RESULTS record for the third comparison. It shows the first of five returned witnesses; the other witnesses and record fields are omitted for readability. This is a local result, not a hosted-run result or observed customer payload.

{
  "name": "mcp_optional_output_schema_extension_narrowing",
  "status": "breaking_witness",
  "completed": true,
  "direction": "before_accepts_after_rejects",
  "witnessCount": 5,
  "witnesses": [
    {
      "value": {
        "name": "example",
        "inputSchema": { "type": "object" },
        "outputSchema": 7
      },
      "afterErrors": [
        {
          "instancePath": "/outputSchema",
          "schemaPath": "#/properties/outputSchema/type",
          "keyword": "type",
          "params": { "type": "object" },
          "message": "must be object"
        }
      ]
    }
  ]
}

Pricing

When pay-per-event pricing is active, the price is USD 0.01 per comparison-completed event, including platform usage. The Actor's Pricing tab is the source of truth for the active price and terms; check it before running.

A completed supported pair is one unit: a validated witness, normalized equality, or a completed finite search with no witness. Inconclusive completed work is charged. Invalid, unsupported, interrupted, timed-out or failed comparisons have completed: false and are not charged. The supported pricing configuration has no positive startup, default-dataset, empty-result or error fee. An explicit zero event budget stops comparison work before it can incur an event charge.

The paid unit covers analysis, not guaranteed discovery, a compatibility certificate or a deployment recommendation. Account/platform costs and retention still exist. Under Apify's current revenue-share rules, creator contribution before other costs is 0.8 × eligible PPE revenue − platform costs; Free-plan users do not generate creator revenue. Actual cloud margin for this Actor is unknown, and the local timing above does not establish it.

Data handling and local use

Supply only schemas and sample values you are authorized to process. Candidate values, property names, constants and validation messages can contain confidential or personal information. This tool does not promise anonymization. Apify retains input, datasets and key-value records according to your account/storage settings; deleting or limiting those records is your responsibility.

For local use, install the locked dependencies with npm ci, put the chosen input in the local Apify input record, and run npm start. The image uses Node.js 22 and the project pins Apify SDK 3.7.2. Source retrieval is not needed during comparison; package installation and Apify storage access are separate operations.