Skip to content

Decode a locksmith's typed key-code cards into cut depths, ticket numbers and dates, with check characters and cutting rules

Exact prompt

# Decode a locksmith's typed key-code cards into cut depths, ticket numbers and dates, with check characters and cutting rules Inputs: `cards.csv` (card_id, code) and `systems.csv` (system, min_depth, max_depth, max_step). Each code was typed by hand from a key-code card. The code format below is this shop's own synthetic format; every customer, key and system is invented. These rules are the shop's; where `counter-notes.txt` disagrees, the rules win. Rules: - Normalising: remove every hyphen and space, make letters upper case, then read O as 0 and both I and L as 1. The alphabet is `0123456789ABCDEFGHJKMNPQRSTVWXYZ`; character values are their positions, 0 to 31. A normalised code that is not exactly 11 characters long, or holds a character outside the alphabet, is `malformed`. - Check character: with v1 to v10 the values of the first ten characters, the 11th character's value must equal (1 × v1 + 2 × v2 + ... + 10 × v10) mod 32; otherwise the code is `bad_check`. - Payload: the first ten characters are 50 bits, most significant first, 5 bits per character. From the left: system (6 bits), the six cut depths from cut 1 to cut 6 (3 bits each), master flag (1 bit, 1 = master key), ticket number (13 bits) and cut date (12 bits, the number of days after 2020-01-01). - Cutting rules: a code whose system is not in systems.csv is `unknown_system`; otherwise it is `depth_out_of_range` if any cut depth is below min_depth or above max_depth, then `too_steep` if any two neighbouring cuts differ by more than max_step, and `ok` if none of these apply. Each card gets the first status that applies, in the order malformed, bad_check, unknown_system, depth_out_of_range, too_steep, ok. - Decoded fields: for a card whose status is `malformed` or `bad_check`, system, cuts, master, ticket and cut_date are null. For every other card they are the payload values: cuts as a string of six digits (cut 1 first), master as true or false, cut_date as YYYY-MM-DD. - Write `out/decoded.json`: `{"cards": {"<card_id>": {"status": "<status>", "system": <number or null>, "cuts": "<six digits>" or null, "master": <true/false or null>, "ticket": <number or null>, "cut_date": "<YYYY-MM-DD>" or null}, ...}, "status_counts": {"<status>": <number of cards>, ...}, "master_tickets": [<sorted ticket numbers of the ok cards with the master flag>]}`. `cards` has one entry per row in cards.csv. - Roll-ups: Statuses with no cards are included at 0. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.

Decode a synthetic locksmith's key-code cards. Each code is eleven characters in a stated 32-character alphabet (typed by hand, so letter case, hyphens, spaces and look-alike letters must be normalised by the stated rule), with a weighted check character and a 50-bit payload: system number, six cut depths, a master flag, a ticket number and a cutting date. Each card gets the first status that applies (malformed, bad check, unknown system, depth out of range, cuts too steep, ok) from the key systems table. The prompt says statuses with no cards are listed at 0. Counter notes get the look-alike, check and steepness rules wrong. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked). Why this run failed: it got every card's status and check character right and its status counts were correct, but on several cards it read the master-key flag and the ticket number from the wrong bit positions, so those values came out wrong and the cut-depth and ticket/date checks failed. It passed 3 of 5 checks.

Success criteria: statuses check_characters cut_depths tickets_and_dates status_counts

Best model for this task so far: Claude Sonnet 5.5 (effort default (no --effort flag) · thinking on) (success, $0.05, 11s).

Run this on another model

All runs (3)

67% pass rate across 3 eligible runs: 2 pass, 1 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
AClaude Opus 5.5default · thinking on$0.1217s0Fully autonomouspassTrust 80%2026-10-26
AClaude Haiku 5.5default · thinking on$0.004717s0Fully autonomouspartialTrust 80%2026-10-26
AClaude Sonnet 5.5default · thinking on$0.0511s0Fully autonomouspassTrust 80%2026-10-26

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Decode a locksmith's typed key-code cards into cut depths, ticket numbers and dates, with check characters and cutting rules

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.