Criteria Checker: AI QA Gate for Emails & LLM Output
Pricing
$0.50 / 1,000 criterion evaluateds
Criteria Checker: AI QA Gate for Emails & LLM Output
Pass/fail AI QA gate for automations: check text or JSON against templates or your own criteria with calibrated probabilities. Thresholds, review band and a report. Built-in classification engine, no external API key needed. For n8n, Make and MCP.
Pricing
$0.50 / 1,000 criterion evaluateds
Rating
0.0
(0)
Developer
Ferrolark
Maintained by CommunityActor stats
0
Bookmarked
2
Total users
1
Monthly active users
4 days ago
Last modified
Categories
Features
Share
Criteria Checker: AI QA Gate for Content, Emails & LLM Output
A pass / fail gate for your automations. Send a text or JSON (an AI-written email, a support reply, a blog post, an LLM answer) and get a verdict per criterion and an overall pass, review or fail, with calibrated probabilities instead of a "looks good to me" from a chat model.
- Ready templates:
email-before-send,llm-answer-grounded,support-reply,blog-post-prepublish,product-description,social-post. Each bundles checks such as leftover{{placeholders}}, "as an AI language model" text, unsupported claims, tone and missing call to action. - Your own criteria: yes/no questions, graded scores (2-10 levels) or choices, with per-criterion thresholds, weights, required flags and minimum confidence.
- Three outcomes, not two: answers close to the threshold become
review, so uncertain items go to a person instead of slipping through.failClosedturns review into fail. - Decision in code:
all_requiredor a weighted score with a minimum; output givesfailedIdsandreviewIdsyou can branch on in n8n, Make or your app. - Batch: up to 100 items per run (20,000 characters each), with a
REPORT.mdsummary (pass rates per criterion and the items to look at).
Classification runs on Ferrolark's built-in calibrated engine (Jev). You do not need an external API key.
Example
{"template": "email-before-send","state": "Hi {{first_name}},\n\nAs an AI language model, I've drafted this follow-up for you...","criteria": [{"id": "mentions_price", "instructions": "Does the email mention a price?", "required": false}]}
Each dataset row is one checked item:
{"id": "state-1", "overall": "fail", "overallPass": false, "failedIds": ["no_placeholders", "no_ai_meta"], "reviewIds": [],"results": [{"id": "no_placeholders", "label": "No placeholder or template text", "verdict": "fail", "reasonCode": "statement_true"}], "inputTokens": 410}
Price
- $0.0005 per criterion evaluated (a 7-criterion template costs $0.0035 per item; 1,000 emails ≈ $3.50).
- Not charged: invalid input or engine errors before a successful classification.
- Classification cost is included in the event price (no separate API key billing for you).
Input
| Field | Default | What it does |
|---|---|---|
state / reference | none | The content, and optional source or question it must match. |
items | none | Batch: [{"id", "state", "reference"}], up to 100. |
template | email-before-send | A criteria set, or none. |
criteria / disableCriteria | none | Add or replace criteria; switch off template ones. |
noulThreshold / reviewBand | 0.7 / 0.2 | Default pass probability and the review zone below it. |
policy / minWeightedScore | all_required / 0.8 | Overall rule. |
failClosed | false | Review counts as fail. |
model | jev-latest | Pin a version when you tune thresholds. |
Call it from code, n8n or an AI agent
curl -X POST "https://api.apify.com/v2/acts/ferrolark~criteria-checker/run-sync-get-dataset-items?maxTotalChargeUsd=0.05" \-H "Authorization: Bearer $APIFY_TOKEN" -H "Content-Type: application/json" \-d '{"template": "support-reply", "state": "We have re-sent your order.", "reference": "Where is my order?"}'
In n8n, put an HTTP Request node before your "send" step and an IF node on overall == "pass".
FAQ
Do I need a TypeSafe or OpenAI key?
No. Classification is built in and included in the event price.
Does it decide for me?
It decides by rules you control. The probabilities come from the classification engine; the thresholds and policy are yours. Use the review band for anything costly.
More Ferrolark tools
- Brand Name Availability Checker
- Domain Intelligence Report
- Support Ticket Triage
- Amazon Listing Checker
Built by Ferrolark.