MALFORMED-300 conformance scorer

Score any JavaScript JSON-recovery parser against the MALFORMED-300 grading spec — the same spec used by the published leaderboard.

You bring the corpus and the parser. This Actor ships no corpus and stores nothing beyond the run's own dataset record. Your parser source runs inside your own run.

Input

field	required	what it is
corpusJsonl	no	Your corpus as JSONL: one JSON object per line, with input, expected_kind (value or unrecoverable), expected, and optionally category.
parserSource	no	An ES module source exporting a function named parse (or a default export). It receives the malformed text and returns the recovered value, or throws / returns undefined or null to refuse.

Leave both empty and the Actor runs a tiny built-in demo (3 cases, strict JSON.parse) so you can see the output shape before sending anything of your own. The demo run is not charged.

Output

One scorecard, written to the run's dataset and to the key-value store under OUTPUT:

• n — cases scored
• exact / exact_recoverable — exact matches
• unrecoverable_n / correctly_refused — cases that had to be refused, and were
• invented_values — refusals that returned {}, [] or "" instead. This is the number the suite exists to measure: an invented value silently corrupts state downstream
• false_refusals — recoverable cases the parser refused
• by_category — per-category tally

Grading spec

expected_kind: "value" passes only by returning exactly that value, compared on a canonical form computed for both sides, so key order and 1 vs 1.0 cannot decide a case. expected_kind: "unrecoverable" passes only by refusing: a throw, or a return of undefined or null. Any other value — including {}, [] or "" — fails.

One run each. Nothing tuned afterwards.

Pricing

$0.01 per run, charged once when you supply your own corpus and parser, whatever the corpus size. The built-in demo run is free.

Zero dependencies. No telemetry. CC0-1.0.

The full labelled corpus, the grading spec and the public leaderboard live at https://toolkitlabs.org/malformed300/?s=apify-m300.