Charge residents' parking permits by emission band, part year, household order and blue-badge exemption
Exact prompt
# Charge a council's residents' parking permits for the rest of the scheme year Inputs: `permits.csv` (permit_id, household, reg, co2_gkm, fuel, start_date, cancelled_on, status), `charges.json`, `blue-badges.csv` (household) and `office-note.txt`. Amounts are in pounds. `charges.json` and these rules are the record; where the office note disagrees, the rules win. Rules: - Permits with status refused are ignored. Households match ignoring letter case and surrounding spaces; report them in upper case. - A car's band is the first band in charges.json whose max_co2 is at least co2_gkm (a car exactly on a limit stays in that band; max_co2 null is the top band). Electric cars are band "electric" and free. - Diesel cars add diesel_surcharge to the band's annual price. - A permit is charged annual price x days / days_in_year, counting days from start_date to cancelled_on (or scheme_year_end), both included. - In each household, non-electric permits are ordered by start_date (ties by the permit number); every permit after the first costs additional_permit_pct percent more. - A household in blue-badges.csv pays nothing for its first non-electric permit only. - Each permit's charge is rounded half-up to 0.01 at the end. Write `out/permits.json`: `{"band": {"<permit_id>": "<band>", ...}, "charge": {"<permit_id>": <charge>, ...}, "household_totals": {"<HOUSEHOLD>": <sum>, ...}, "total": <sum of charges>}` covering every permit that is not refused. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Price a street's residents' parking permits for the rest of the scheme year. Cars sit exactly on band limits, diesel adds a surcharge, days are counted with both ends included, second permits cost more but permit numbers sort differently as text and as numbers, households are written in mixed case and spacing, and a blue badge only covers the first permit. An office note gets the band edge, day count and badge rules wrong. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: bands pro_rata household_order_and_badges totals
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0062, 29s).
All runs (6)
100% pass rate across 6 eligible runs: 6 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| GGemini 3.8 Flashmedium · thinking on | $0.30 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| GGemini 4 Argonmedium · thinking on | $0.20 | 3m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| OGPT-6.1 Soldefault · thinking off | $0.03 | 28s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| AClaude Opus 5.5default · thinking on | $0.12 | 24s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| AClaude Haiku 5.5default · thinking on | $0.0062 | 29s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| AClaude Sonnet 5.5default · thinking on | $0.06 | 14s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.