Roll up quarterly store revenue from overlapping till exports with mixed units, currencies and a non-trading store
Exact prompt
# Roll up last quarter's store revenue Inputs: `stores.csv` (each store's region, currency and status), `fx.csv` (monthly USD rates), and the weekly till exports in `exports/` (one CSV per store per week). Report net revenue in USD for Q3 2026 (July to September) by region. - Net revenue = completed sales minus completed refunds. Voided transactions don't count. - Convert each transaction at the rate for the month it happened in. - Each transaction counts once, however many export files it appears in. - Only trading stores count toward regional revenue. - Round money to cents, half-up, only at the end. Write `out/revenue.json`: `{"by_region": {"<region>": <usd>, ...}, "total_usd": <usd>, "top_region": "<region>", "refunds_usd": <usd>, "duplicate_rows_dropped": <n>}`. `refunds_usd` is the total of completed refunds at trading stores, as a positive number. `duplicate_rows_dropped` counts rows skipped because their transaction was already counted (all stores). ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Combine about 120 weekly till exports from 9-12 stores into net Q3 revenue by region. Column headers quietly change the unit (currency units, cents, thousands), each export re-sends the previous weekend, refunds and voids are mixed in, rates vary by month, and one store in the list is not trading. The key is computed exactly from the generated ground truth. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked). Folder-restore note: this run's output files under work/out/ were lost in a 1:22 AM ET restore on Oct 21, 2026. The trace, step-in exchanges and output-hash list were checked against the recorded SHA-256 values and match; the hashes of the individual output files inside that list can no longer be recomputed. Measured data unchanged.
Success criteria: region_totals grand_total top_region refunds dedupe fx_by_month
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0099, 41s).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.19 | 41s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 | |
| AClaude Haiku 5.5default · thinking on | $0.0099 | 41s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 | |
| AClaude Sonnet 5.5default · thinking on | $0.07 | 17s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.