LLM Output Guard

Check that your model's JSON is the shape you asked for, get every failure by path, and optionally repair the shape mistakes that are not really mistakes.

Structured output from a language model is right most of the time, which is the problem. The one run in fifty where price comes back as "$19.99", inStock as "yes", the whole object as a string, or a field the model invented, is the run that breaks the step after it. This actor sits between the model and everything downstream and answers one question per record: does this match the schema, and if not, exactly where.

Part of the Broomwagon family: deterministic tools that watch scraped and AI data change. No second model checks the first one. The schema is the contract, and the same record always gets the same verdict.

What you get per record

{
  "index": 1,
  "valid": false,
  "errorCount": 2,
  "errors": [
    { "path": "/price", "keyword": "type", "message": "Expected number, got string." },
    { "path": "/inStock", "keyword": "type", "message": "Expected boolean, got string." }
  ],
  "coercions": [
    { "path": "/price", "from": "$19.99", "to": 19.99, "reason": "number was quoted" },
    { "path": "/inStock", "from": "yes", "to": true, "reason": "boolean was written as text" }
  ],
  "validBeforeRepair": false,
  "wasRepaired": false,
  "data": { "...": "the payload" }
}

Filter on valid and you have a gate. Read errors and you know which prompt to fix. Read coercions and you know whether the model was wrong or merely sloppy.

Repairs are opt-in and never invent content

Turn on Apply shape repairs and the actor fixes the mistakes that are about shape, not substance, then re-validates:

The model sent	What it meant
"9.99", "$1,299.00"	the number
"yes", "true"	the boolean
"{\"a\":1}"	the object (a stringified payload)
"sale" where a list belongs	["sale"]
"ACTIVE" against enum: ["active"]	"active"
product_name where the schema says productName	the same key

"about twelve dollars" stays exactly as it is, and the record stays invalid. Every repair appears in coercions with its before and after, and validBeforeRepair records what the verdict would have been untouched, so a repair can never hide a real failure.

What the schema can say

Standard JSON Schema, the parts structured output actually uses: type (including type arrays), required, properties, additionalProperties, items, enum, const, minimum/maximum/exclusiveMinimum/exclusiveMaximum/multipleOf, minLength/maxLength/pattern/format, minItems/maxItems/uniqueItems, anyOf/oneOf/allOf/not, and $ref within the document.

Anything it cannot check is named in the run report rather than quietly ignored, so you always know what was actually enforced.

Try it in two minutes

Leave the prefilled schema and the three sample records in place and run. The first record passes. The second fails on two type errors and carries the repairs that would fix it. The third is genuinely wrong: a bad SKU, an empty name, a negative price, and a field the schema forbids. Then set Apply shape repairs to true and run again: the second record comes back valid with wasRepaired: true, and the third still fails, which is the point.

Input

• schema (required): the JSON Schema.
• datasetId | items | fileUrl: the records to check. A picked dataset wins.
• payloadField: name it when each record wraps the answer, e.g. "output".
• repairShape: off by default.
• maxErrorsPerRecord: how many errors to keep per row (totals are always complete).

Output

• Dataset: one verdict row per record.
• VALIDATION: batch report with the pass rate, the paths that fail most often, and the keywords they fail on. This is the record that tells you what to change in the prompt.
• OUTPUT: run report.

Pricing (pay-per-event)

Event	What you pay for	Price
record-validated	Per record checked	$0.001
validation-report	Per batch report	$0.02
apify-actor-start	Run start, per GB of run memory	$0.005

Worked example: an agent producing 2,000 structured records a day: 2,000 × $0.001 + $0.025 = about $2/day, against the cost of one bad record reaching your database. Compared with asking a second model to check the first: no tokens, no variance, and the same answer every time.

Set a max charge on any run (Maximum cost per run in Console, or ACTOR_MAX_TOTAL_CHARGE_USD via API) and the actor stops cleanly at your budget.

Integrations

• MCP / AI agents: the intended home. Callable as a tool via the Apify MCP server, so an agent can validate its own structured output mid-pipeline and retry on a real error instead of guessing. Feed the errors array straight back into the retry prompt.
• API: POST https://api.apify.com/v2/acts/broomwagon~llm-output-guard/runs with {"schema": {...}, "datasetId": "<the model run's dataset>", "repairShape": true}.
• n8n / Make: model node → this actor → filter on valid: true for the happy path and route the rest to review. Two branches, no prompt engineering.
• Apify Schedule: validate a nightly generation batch and alert when the pass rate drops, which is usually how you find out a model or prompt changed under you.

Roadmap

Draft 2020-12 keywords (prefixItems, dependentSchemas), schema inference from a sample of good records, and a pass-rate trend across runs so a prompt regression shows up as a line rather than a surprise.

The Broomwagon family

This actor is one of nine deterministic post-processing tools from Broomwagon: the layer that follows your scrapers and agents, cleaning and watching what they produce. Same input, same output, every time.

• Scraper Output Monitor — catch the day your scrape silently breaks.
• Only New Items — deliver only records you have never delivered before.
• Dataset Deduper — exact and fuzzy dedupe for any dataset, with an audit trail.
• CSV Doctor — repair broken CSV files, with a report of every fix.
• PII Redactor — strip emails, phones, SSNs, cards, and addresses, deterministically.
• Record Linker — fuzzy join two datasets that share no key.
• Google Maps Scraper Deduper — merge duplicate places across Google Maps scrapes.
• AI Search Visibility Monitor — track your brand's share of voice in AI answers.