Implement a reader/writer for a documented chunked binary container (LEB128 lengths, CRC-32, zlib chunks, critical vs ancillary)
Exact prompt
# Task: implement the CHNK container format Implement `read(buf)` and `write(...)` in `chnk.py` exactly as documented in `FORMAT.md` (including every error case and its `.offset`). Keep the `FormatError(msg, offset)` class. Python standard library only (`zlib` is fine). No network.
Every error path has an exact expected byte offset; minimal-varint and 5-byte limits, CRC over type+payload, META ordering, PNG-style critical/ancillary rule and byte-exact writer output are each tested. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. All 12 hidden unittest cases pass: basic read, no-CRC + META merge, ancillary skipping, multi-byte varints, header errors, truncation offsets, type errors, varint errors, CRC mismatches, semantic errors, byte-exact writer output, 120 random read(write()) round-trips.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.01, 1m).
All runs (4)
100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.08 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.01 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Opus 5.5default · thinking on | $0.23 | 55s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Sonnet 5.5default · thinking on | $0.08 | 25s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.