Make a Node.js JSON Patch module correct (no input mutation, pointer escaping, array index rules, move-into-child, deep test, error codes)
Exact prompt
# Task: make the JSON Patch module correct `patch.js` (CommonJS, Node.js 20) exports `apply(doc, ops)` and `PatchError`. It applies a JSON Patch (a list of operations) to a JSON document, but it mutates its inputs, mishandles pointers and array indexes, and throws inconsistent errors. Fix it (rewrite freely) so it implements `PATCH_SPEC.md` exactly. No dependencies beyond Node.js 20 built-ins; no network. Keep the exports. The grader runs a hidden `node --test` suite against your `patch.js`. If you add your own tests in a `test/` folder they must pass too (`node --test test/`).
The starting module mutates inputs, has no error codes, treats inherited properties as present (prototype pollution via __proto__), mishandles '-', leading zeros and out-of-range indexes, and uses shallow ===. Every case asserts an exact result or (code, index). Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. hidden_tests_pass: all 11 hidden node:test cases pass. 2. own_tests_pass_or_absent: the agent's test/ (if any) passes.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.02, 2m).
All runs (5)
100% pass rate across 5 eligible runs: 5 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.05 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.02 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| AClaude Haiku 5.5default · thinking on | $0.02 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| AClaude Opus 5.5default · thinking on | $0.32 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| AClaude Sonnet 5.5default · thinking on | $0.09 | 35s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.