Reconcile a print shop's job tickets against its printers' meter readings and work out takings
Exact prompt
# Reconcile a print shop's job tickets against its printers' meter readings and work out takings Inputs: `printers.csv` (printer_id, kind, price_per_page), `accounts.csv` (account_id, name), `jobs.csv` (job_id, date, printer_id, account_id, pages, copies, status), `meters.csv` (printer_id, date, open_count, close_count) and `period.json` (period_start, period_end, both dates included). All printers, accounts and jobs are synthetic; prices are in dollars. These rules are the shop's; where `counter-notes.txt` disagrees, the rules win. Rules: - Meter readings: a printer-day's meter impressions are close_count minus open_count. The counter has six digits and rolls over from 999999 to 000000, so when close_count is lower than open_count, add 1000000. - Job impressions: a job's impressions are pages × copies. Jobs with status `done` or `reprint` were printed; `void` jobs were cancelled before printing and count nowhere. - Discrepancies: each printer-day may have up to 15 meter impressions that no job explains (warm-up and test pages). A printer-day is a discrepancy when its meter impressions minus the impressions of the printed jobs on that printer that day is more than 15 or less than 0. - Billing: a `done` job is billed pages × copies × its printer's price_per_page, less 10% when copies is 50 or more (a bulk job). `reprint` jobs redo the shop's own mistakes and are free; `void` jobs are not billed. Don't round until the output. - Write `out/reconciliation.json`: `{"discrepancies": [<sorted "<printer_id>@<YYYY-MM-DD>" strings>], "meter_impressions_total": <number>, "bulk_jobs": [<sorted job_ids billed with the bulk discount>], "billed_total": <dollars>, "takings_by_printer": {"<printer_id>": <dollars>, ...}, "takings_by_account": {"<account_id>": <dollars>, ...}}`. - Roll-ups: Printers in printers.csv with no billed jobs are included at 0. Accounts in accounts.csv with no billed jobs are included at 0. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Check a week of a synthetic print shop's job tickets against each printer's daily meter readings. The six-digit meters roll over, so a day can read lower at close than at open. Printed jobs (done and reprint) are compared with the meter, and up to 15 unexplained test pages per printer-day are allowed. Done jobs are billed per page with a bulk discount at 50 copies or more; reprints are free and void tickets count nowhere. Report the discrepant printer-days, total meter impressions, bulk jobs, the billed total and takings per printer and per account. The prompt says printers and accounts with no billed jobs are listed at 0. A day at exactly 15 spare impressions, a job at exactly 50 copies and an account with only a free reprint are decoys. Counter notes get the reprint, void and bulk rules wrong. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: discrepancies meter_impressions_total bulk_jobs billed_total takings_by_printer takings_by_account
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0060, 22s).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.17 | 26s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-26 | |
| AClaude Haiku 5.5default · thinking on | $0.0060 | 22s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-26 | |
| AClaude Sonnet 5.5default · thinking on | $0.06 | 31s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-26 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.