# LLM Output Guard | JSON Schema Validation for Agent Output (`broomwagon/llm-output-guard`) Actor

Check that your model's JSON matches the schema you asked for. Every failure comes back with its path, the keyword it broke, and the repair that would fix it. Shape repairs are opt-in and never invent content, so a repair can never hide a real failure.

- **URL**: https://apify.com/broomwagon/llm-output-guard.md
- **Developed by:** [Brandon Mensing](https://apify.com/broomwagon) (community)
- **Categories:** Agents, AI, Developer tools
- **Stats:** 2 total users, 1 monthly users, 100.0% runs succeeded, 0 bookmarks
- **User rating**: No ratings yet

## Pricing

from $1.00 / 1,000 record validateds

This Actor is paid per event. You are not charged for the Apify platform usage, but only a fixed price for specific events.

Learn more: https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event

## What's an Apify Actor?

Actors are web data automations that power AI and operations. They run on the Apify platform to scrape websites, process data, connect APIs, and automate workflows.
In Batch mode, an Actor accepts a well-defined JSON input, performs an action which can take anything from a few seconds to a few hours,
and optionally produces a well-defined JSON output, datasets with results, or files in key-value store.
In Standby mode, an Actor provides a web server which can be used as a website, API, or an MCP server.
Actors are written with capital "A".

## How to integrate an Actor?

If asked about integration, you help developers integrate Actors into their projects.
You adapt to their stack and deliver integrations that are safe, well-documented, and production-ready.
The best way to integrate Actors is as follows.

- **AI agents and MCP clients** — the [Apify MCP server](https://docs.apify.com/integrations/mcp.md) at `https://mcp.apify.com` (remote, streamable HTTP, OAuth on first use).
- **Agentic workflows and local Actor development** — [Agent Skills](https://apify.com/.well-known/agent-skills/index.json) with the [Apify CLI](https://docs.apify.com/cli/docs.md): `npm install -g apify-cli`, then `apify login`.
- **JavaScript/TypeScript projects** — the official [JS/TS client](https://docs.apify.com/api/client/js/docs.md): `npm install apify-client`.
- **Python projects** — the official [Python client](https://docs.apify.com/api/client/python/docs.md): `pip install apify-client`.
- **Any other language** — the [REST API](https://docs.apify.com/api/v2.md).

For usage examples, see the [API](#api) section below.

For more details, see Apify documentation as [Markdown index](https://docs.apify.com/llms.txt) and [Markdown full-text](https://docs.apify.com/llms-full.txt).

# README

## LLM Output Guard

<img align="right" src="https://broomwagon.vercel.app/icons/brand-mark.png" width="64" alt="broomwagon.">

> Check that your model's JSON is the shape you asked for, get every failure by path, and optionally repair the shape mistakes that are not really mistakes.

Structured output from a language model is right most of the time, which is the problem.
The one run in fifty where `price` comes back as `"$19.99"`, `inStock` as `"yes"`, the
whole object as a string, or a field the model invented, is the run that breaks the step
after it. This actor sits between the model and everything downstream and answers one
question per record: **does this match the schema, and if not, exactly where.**

Part of the **Broomwagon** family: deterministic tools that watch scraped and AI data
change. No second model checks the first one. The schema is the contract, and the same
record always gets the same verdict.

### What you get per record

```json
{
  "index": 1,
  "valid": false,
  "errorCount": 2,
  "errors": [
    { "path": "/price", "keyword": "type", "message": "Expected number, got string." },
    { "path": "/inStock", "keyword": "type", "message": "Expected boolean, got string." }
  ],
  "coercions": [
    { "path": "/price", "from": "$19.99", "to": 19.99, "reason": "number was quoted" },
    { "path": "/inStock", "from": "yes", "to": true, "reason": "boolean was written as text" }
  ],
  "validBeforeRepair": false,
  "wasRepaired": false,
  "data": { "...": "the payload" }
}
```

Filter on `valid` and you have a gate. Read `errors` and you know which prompt to fix.
Read `coercions` and you know whether the model was wrong or merely sloppy.

### Repairs are opt-in and never invent content

Turn on **Apply shape repairs** and the actor fixes the mistakes that are about shape,
not substance, then re-validates:

| The model sent | What it meant |
| --- | --- |
| `"9.99"`, `"$1,299.00"` | the number |
| `"yes"`, `"true"` | the boolean |
| `"{\"a\":1}"` | the object (a stringified payload) |
| `"sale"` where a list belongs | `["sale"]` |
| `"ACTIVE"` against `enum: ["active"]` | `"active"` |
| `product_name` where the schema says `productName` | the same key |

`"about twelve dollars"` stays exactly as it is, and the record stays invalid. Every
repair appears in `coercions` with its before and after, and `validBeforeRepair` records
what the verdict would have been untouched, so a repair can never hide a real failure.

### What the schema can say

Standard JSON Schema, the parts structured output actually uses: `type` (including type
arrays), `required`, `properties`, `additionalProperties`, `items`, `enum`, `const`,
`minimum`/`maximum`/`exclusiveMinimum`/`exclusiveMaximum`/`multipleOf`,
`minLength`/`maxLength`/`pattern`/`format`, `minItems`/`maxItems`/`uniqueItems`,
`anyOf`/`oneOf`/`allOf`/`not`, and `$ref` within the document.

Anything it cannot check is **named in the run report** rather than quietly ignored, so
you always know what was actually enforced.

### Try it in two minutes

Leave the prefilled schema and the three sample records in place and run. The first
record passes. The second fails on two type errors and carries the repairs that would
fix it. The third is genuinely wrong: a bad SKU, an empty name, a negative price, and a
field the schema forbids. Then set **Apply shape repairs** to true and run again: the
second record comes back valid with `wasRepaired: true`, and the third still fails,
which is the point.

### Input

- `schema` (required): the JSON Schema.
- `datasetId` | `items` | `fileUrl`: the records to check. A picked dataset wins.
- `payloadField`: name it when each record wraps the answer, e.g. `"output"`.
- `repairShape`: off by default.
- `maxErrorsPerRecord`: how many errors to keep per row (totals are always complete).

### Output

- Dataset: one verdict row per record.
- `VALIDATION`: batch report with the pass rate, the paths that fail most often, and the
  keywords they fail on. This is the record that tells you what to change in the prompt.
- `OUTPUT`: run report.

### Pricing (pay-per-event)

| Event | What you pay for | Price |
| --- | --- | --- |
| `record-validated` | Per record checked | $0.001 |
| `validation-report` | Per batch report | $0.02 |
| `apify-actor-start` | Run start, per GB of run memory | $0.005 |

**Worked example:** an agent producing 2,000 structured records a day:
2,000 × $0.001 + $0.025 = **about $2/day**, against the cost of one bad record reaching
your database. Compared with asking a second model to check the first: no tokens, no
variance, and the same answer every time.

Set a max charge on any run (`Maximum cost per run` in Console, or
`ACTOR_MAX_TOTAL_CHARGE_USD` via API) and the actor stops cleanly at your budget.

### Integrations

- **MCP / AI agents:** the intended home. Callable as a tool via the Apify MCP server, so
  an agent can validate its own structured output mid-pipeline and retry on a real error
  instead of guessing. Feed the `errors` array straight back into the retry prompt.
- **API:** `POST https://api.apify.com/v2/acts/broomwagon~llm-output-guard/runs` with
  `{"schema": {...}, "datasetId": "<the model run's dataset>", "repairShape": true}`.
- **n8n / Make:** model node → this actor → filter on `valid: true` for the happy path
  and route the rest to review. Two branches, no prompt engineering.
- **Apify Schedule:** validate a nightly generation batch and alert when the pass rate
  drops, which is usually how you find out a model or prompt changed under you.

### Roadmap

Draft 2020-12 keywords (`prefixItems`, `dependentSchemas`), schema inference from a
sample of good records, and a pass-rate trend across runs so a prompt regression shows up
as a line rather than a surprise.

### The Broomwagon family

This actor is one of nine deterministic post-processing tools from [Broomwagon](https://apify.com/broomwagon): the layer that follows your scrapers and agents, cleaning and watching what they produce. Same input, same output, every time.

- [Scraper Output Monitor](https://apify.com/broomwagon/scraper-output-monitor) — catch the day your scrape silently breaks.
- [Only New Items](https://apify.com/broomwagon/only-new-items) — deliver only records you have never delivered before.
- [Dataset Deduper](https://apify.com/broomwagon/dataset-deduper) — exact and fuzzy dedupe for any dataset, with an audit trail.
- [CSV Doctor](https://apify.com/broomwagon/csv-doctor) — repair broken CSV files, with a report of every fix.
- [PII Redactor](https://apify.com/broomwagon/pii-redactor) — strip emails, phones, SSNs, cards, and addresses, deterministically.
- [Record Linker](https://apify.com/broomwagon/record-linker) — fuzzy join two datasets that share no key.
- [Google Maps Scraper Deduper](https://apify.com/broomwagon/google-maps-deduper) — merge duplicate places across Google Maps scrapes.
- [AI Search Visibility Monitor](https://apify.com/broomwagon/ai-search-visibility-monitor) — track your brand's share of voice in AI answers.

# Actor input Schema

## `schema` (type: `object`):

The shape the output is supposed to have. Standard JSON Schema: types, required, enums, patterns, formats, bounds, arrays, $ref, anyOf/oneOf/allOf. Keywords this actor cannot check are listed in the run report rather than silently ignored.

## `datasetId` (type: `string`):

A dataset of records produced by your model or agent. Pipe it straight in; a picked dataset always wins over the sample records below.

## `items` (type: `array`):

Records provided inline as a JSON array. Ignored when a dataset is picked above.

## `fileUrl` (type: `string`):

URL of a JSON or NDJSON file of records to check.

## `payloadField` (type: `string`):

If each record wraps the model's answer in a field, e.g. "output" or "response", name it here. Leave empty to validate the whole record.

## `repairShape` (type: `boolean`):

Off by default: the actor reports the repairs it would make but changes nothing. On, it fixes shape mistakes (a quoted number, a stringified object, a single value where a list belongs, a key in the wrong case) and re-validates. Repairs never invent content, and every one is listed in the row's coercions.

## `maxErrorsPerRecord` (type: `integer`):

A record that is entirely wrong can produce hundreds of errors. The row keeps this many and reports how many were left out; the totals are always complete.

## Actor input object example

```json
{
  "schema": {
    "type": "object",
    "required": [
      "sku",
      "name",
      "price",
      "inStock"
    ],
    "additionalProperties": false,
    "properties": {
      "sku": {
        "type": "string",
        "pattern": "^[A-Z]{3}-\\d{4}$"
      },
      "name": {
        "type": "string",
        "minLength": 1
      },
      "price": {
        "type": "number",
        "minimum": 0
      },
      "inStock": {
        "type": "boolean"
      },
      "status": {
        "type": "string",
        "enum": [
          "active",
          "discontinued"
        ]
      }
    }
  },
  "items": [
    {
      "sku": "ABC-1234",
      "name": "Widget",
      "price": 9.99,
      "inStock": true,
      "status": "active"
    },
    {
      "sku": "ABC-5678",
      "name": "Gizmo",
      "price": "$19.99",
      "inStock": "yes",
      "status": "Active"
    },
    {
      "sku": "nope",
      "name": "",
      "price": -3,
      "hallucinatedField": "invented by the model"
    }
  ],
  "repairShape": false,
  "maxErrorsPerRecord": 20
}
```

# Actor output Schema

## `verdicts` (type: `string`):

One row per record: valid or not, every failing path with the reason, the repairs available, and the payload.

## `validation` (type: `string`):

Pass rate, the paths that fail most often, the keywords they fail on, and any schema keywords this actor could not check.

## `report` (type: `string`):

Records validated, passed, failed, and repaired into validity.

# API

You can run this Actor programmatically using our API. Below are code examples in JavaScript, Python, and CLI, as well as the OpenAPI specification and MCP server setup.

## JavaScript example

```javascript
import { ApifyClient } from 'apify-client';

// Initialize the ApifyClient with your Apify API token
// Replace the '<YOUR_API_TOKEN>' with your token
const client = new ApifyClient({
    token: '<YOUR_API_TOKEN>',
});

// Prepare Actor input
const input = {
    "schema": {
        "type": "object",
        "required": [
            "sku",
            "name",
            "price",
            "inStock"
        ],
        "additionalProperties": false,
        "properties": {
            "sku": {
                "type": "string",
                "pattern": "^[A-Z]{3}-\\d{4}$"
            },
            "name": {
                "type": "string",
                "minLength": 1
            },
            "price": {
                "type": "number",
                "minimum": 0
            },
            "inStock": {
                "type": "boolean"
            },
            "status": {
                "type": "string",
                "enum": [
                    "active",
                    "discontinued"
                ]
            }
        }
    },
    "items": [
        {
            "sku": "ABC-1234",
            "name": "Widget",
            "price": 9.99,
            "inStock": true,
            "status": "active"
        },
        {
            "sku": "ABC-5678",
            "name": "Gizmo",
            "price": "$19.99",
            "inStock": "yes",
            "status": "Active"
        },
        {
            "sku": "nope",
            "name": "",
            "price": -3,
            "hallucinatedField": "invented by the model"
        }
    ]
};

// Run the Actor and wait for it to finish
const run = await client.actor("broomwagon/llm-output-guard").call(input);

// Fetch and print Actor results from the run's dataset (if any)
console.log('Results from dataset');
console.log(`💾 Check your data here: https://console.apify.com/storage/datasets/${run.defaultDatasetId}`);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
items.forEach((item) => {
    console.dir(item);
});

// 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/js/docs

```

## Python example

```python
from apify_client import ApifyClient

# Initialize the ApifyClient with your Apify API token
# Replace '<YOUR_API_TOKEN>' with your token.
client = ApifyClient("<YOUR_API_TOKEN>")

# Prepare the Actor input
run_input = {
    "schema": {
        "type": "object",
        "required": [
            "sku",
            "name",
            "price",
            "inStock",
        ],
        "additionalProperties": False,
        "properties": {
            "sku": {
                "type": "string",
                "pattern": "^[A-Z]{3}-\\d{4}$",
            },
            "name": {
                "type": "string",
                "minLength": 1,
            },
            "price": {
                "type": "number",
                "minimum": 0,
            },
            "inStock": { "type": "boolean" },
            "status": {
                "type": "string",
                "enum": [
                    "active",
                    "discontinued",
                ],
            },
        },
    },
    "items": [
        {
            "sku": "ABC-1234",
            "name": "Widget",
            "price": 9.99,
            "inStock": True,
            "status": "active",
        },
        {
            "sku": "ABC-5678",
            "name": "Gizmo",
            "price": "$19.99",
            "inStock": "yes",
            "status": "Active",
        },
        {
            "sku": "nope",
            "name": "",
            "price": -3,
            "hallucinatedField": "invented by the model",
        },
    ],
}

# Run the Actor and wait for it to finish
run = client.actor("broomwagon/llm-output-guard").call(run_input=run_input)

# Fetch and print Actor results from the run's dataset (if there are any)
print(f"💾 Check your data here: https://console.apify.com/storage/datasets/{run.default_dataset_id}")
for item in client.dataset(run.default_dataset_id).iterate_items():
    print(item)

# 📚 Want to learn more 📖? Go to → https://docs.apify.com/api/client/python/docs/quick-start

```

## CLI example

```bash
echo '{
  "schema": {
    "type": "object",
    "required": [
      "sku",
      "name",
      "price",
      "inStock"
    ],
    "additionalProperties": false,
    "properties": {
      "sku": {
        "type": "string",
        "pattern": "^[A-Z]{3}-\\\\d{4}$"
      },
      "name": {
        "type": "string",
        "minLength": 1
      },
      "price": {
        "type": "number",
        "minimum": 0
      },
      "inStock": {
        "type": "boolean"
      },
      "status": {
        "type": "string",
        "enum": [
          "active",
          "discontinued"
        ]
      }
    }
  },
  "items": [
    {
      "sku": "ABC-1234",
      "name": "Widget",
      "price": 9.99,
      "inStock": true,
      "status": "active"
    },
    {
      "sku": "ABC-5678",
      "name": "Gizmo",
      "price": "$19.99",
      "inStock": "yes",
      "status": "Active"
    },
    {
      "sku": "nope",
      "name": "",
      "price": -3,
      "hallucinatedField": "invented by the model"
    }
  ]
}' |
apify call broomwagon/llm-output-guard --silent --output-dataset

```

## MCP server setup

```json
{
    "mcpServers": {
        "apify": {
            "type": "http",
            "url": "https://mcp.apify.com/?tools=fetch-actor-details,broomwagon/llm-output-guard"
        }
    }
}

```

The hosted server signs you in with OAuth on first connect, so no API token belongs in this config. Clients without OAuth support can send an `Authorization: Bearer <APIFY_API_TOKEN>` header instead, using a token from API & Integrations in Apify Console (https://console.apify.com/settings/integrations).

## OpenAPI specification

Download the OpenAPI definition: https://api.apify.com/v2/actors/9ctv1WRejVQ2f6m3W/builds/t2ob6cbbBvuvg8N5I/openapi.json