Criteria Checker: AI QA Gate for Emails & LLM Output avatar

Criteria Checker: AI QA Gate for Emails & LLM Output

Pricing

$0.50 / 1,000 criterion evaluateds

Go to Apify Store
Criteria Checker: AI QA Gate for Emails & LLM Output

Criteria Checker: AI QA Gate for Emails & LLM Output

Pass/fail AI QA gate for automations: check text or JSON against templates or your own criteria with calibrated probabilities. Thresholds, review band and a report. Built-in classification engine, no external API key needed. For n8n, Make and MCP.

Pricing

$0.50 / 1,000 criterion evaluateds

Rating

0.0

(0)

Developer

Ferrolark

Ferrolark

Maintained by Community

Actor stats

0

Bookmarked

2

Total users

1

Monthly active users

4 days ago

Last modified

Features

Limited permissions

Share

Criteria Checker: AI QA Gate for Content, Emails & LLM Output

A pass / fail gate for your automations. Send a text or JSON (an AI-written email, a support reply, a blog post, an LLM answer) and get a verdict per criterion and an overall pass, review or fail, with calibrated probabilities instead of a "looks good to me" from a chat model.

  • Ready templates: email-before-send, llm-answer-grounded, support-reply, blog-post-prepublish, product-description, social-post. Each bundles checks such as leftover {{placeholders}}, "as an AI language model" text, unsupported claims, tone and missing call to action.
  • Your own criteria: yes/no questions, graded scores (2-10 levels) or choices, with per-criterion thresholds, weights, required flags and minimum confidence.
  • Three outcomes, not two: answers close to the threshold become review, so uncertain items go to a person instead of slipping through. failClosed turns review into fail.
  • Decision in code: all_required or a weighted score with a minimum; output gives failedIds and reviewIds you can branch on in n8n, Make or your app.
  • Batch: up to 100 items per run (20,000 characters each), with a REPORT.md summary (pass rates per criterion and the items to look at).

Classification runs on Ferrolark's built-in calibrated engine (Jev). You do not need an external API key.

Example

{
"template": "email-before-send",
"state": "Hi {{first_name}},\n\nAs an AI language model, I've drafted this follow-up for you...",
"criteria": [{"id": "mentions_price", "instructions": "Does the email mention a price?", "required": false}]
}

Each dataset row is one checked item:

{"id": "state-1", "overall": "fail", "overallPass": false, "failedIds": ["no_placeholders", "no_ai_meta"], "reviewIds": [],
"results": [{"id": "no_placeholders", "label": "No placeholder or template text", "verdict": "fail", "reasonCode": "statement_true"}], "inputTokens": 410}

Price

  • $0.0005 per criterion evaluated (a 7-criterion template costs $0.0035 per item; 1,000 emails ≈ $3.50).
  • Not charged: invalid input or engine errors before a successful classification.
  • Classification cost is included in the event price (no separate API key billing for you).

Input

FieldDefaultWhat it does
state / referencenoneThe content, and optional source or question it must match.
itemsnoneBatch: [{"id", "state", "reference"}], up to 100.
templateemail-before-sendA criteria set, or none.
criteria / disableCriterianoneAdd or replace criteria; switch off template ones.
noulThreshold / reviewBand0.7 / 0.2Default pass probability and the review zone below it.
policy / minWeightedScoreall_required / 0.8Overall rule.
failClosedfalseReview counts as fail.
modeljev-latestPin a version when you tune thresholds.

Call it from code, n8n or an AI agent

curl -X POST "https://api.apify.com/v2/acts/ferrolark~criteria-checker/run-sync-get-dataset-items?maxTotalChargeUsd=0.05" \
-H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \
-d '{"template": "support-reply", "state": "We have re-sent your order.", "reference": "Where is my order?"}'

In n8n, put an HTTP Request node before your "send" step and an IF node on overall == "pass".

FAQ

Do I need a TypeSafe or OpenAI key?

No. Classification is built in and included in the event price.

Does it decide for me?

It decides by rules you control. The probabilities come from the classification engine; the thresholds and policy are yours. Use the review band for anything costly.

More Ferrolark tools


Built by Ferrolark.