Reconcile a messy CSV ledger against bank statements
Exact prompt
You are the bookkeeper for Bramble & Oak Bakery LLC. This folder contains ledger.csv (the general ledger for the business checking account, July-September 2026; amount > 0 is a deposit, amount < 0 is a payment; the opening book balance on July 1, 2026 was 48,213.57 and agreed with the bank) and three bank statement PDFs (statement_2026-07.pdf, statement_2026-08.pdf, statement_2026-09.pdf). pdftotext is installed. Reconcile the ledger against the bank statements for the whole quarter. Ledger entries are often dated 0-3 days before the bank posts them, and ledger descriptions differ from bank descriptions; neither of those is a discrepancy. Report only real discrepancies. Write two files in this folder: 1. discrepancies.csv with header: type,ledger_entry_ids,bank_date,bank_amount,ledger_amount,explanation - type is one of: duplicate_in_ledger, amount_mismatch, sign_error, missing_in_ledger (on the bank statement but not in the ledger), missing_in_bank (in the ledger but never on any statement) - ledger_entry_ids: the ledger entry_id(s) involved, separated by ';' (empty for missing_in_ledger) - bank_date as YYYY-MM-DD (empty if not on the bank statement); amounts as signed numbers without $ or commas 2. reconciliation.json with keys: bank_closing_balance_2026_09_30, ledger_closing_balance_2026_09_30 (opening balance plus all ledger entries), sum_of_adjustments (the net amount your discrepancies add to the ledger balance to reach the bank balance), unexplained_difference (bank closing minus ledger closing minus sum_of_adjustments).
Reconciling a bakery's three-month checking-account ledger (about 1,800 rows) against three PDF bank statements. Twenty-five discrepancies are seeded (duplicates, transposed digits, sign errors and items missing on either side), plus legitimate look-alike purchases as false-positive traps. The agent must list every discrepancy and make the balances reconcile to zero. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: Finds all 25 seeded discrepancies with ≤ 2 false positives; totals match.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.02, 2m).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.34 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Sonnet 5.5default · thinking on | $0.11 | 35s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Haiku 5.5default · thinking on | $0.02 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.