Bill a farm's veg box subscribers with inclusive pauses, a mid-month price rise and credit claims to validate
Exact prompt
# Bill a farm's veg box subscribers for November 2026 Inputs: `subscribers.csv` (sub_id, box, weekday, start_date), `pauses.csv`, `prices.csv`, `credits.csv` (claims for missing items), `delivery.json` and `newsletter.txt`. Every amount is in pounds. The data files and these rules are the record; where the newsletter disagrees with them, the rules win. Rules: - Each subscriber gets one box on every date in November 2026 that falls on their weekday and is on or after their start_date. - A pause covers every date from its from date to its to date, both included; no box is delivered on a paused date. - Each delivery is charged at the prices.csv row for that box with the latest effective_from on or before the delivery date; small boxes also pay small_box_fee per delivery. - A credit claim counts only if its date is a date that subscriber actually received a box. - Due = charges - credits. Write `out/boxes.json`: `{"deliveries": {"<sub_id>": <count>, ...}, "charges": {...}, "credits": {...}, "due": {...}, "grand_total": <sum of due>}` with every subscriber in each map (0 where nothing applies). ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Count each subscriber's November deliveries and bill them. Subscriptions start mid-month, pauses end exactly on delivery days and include both ends, only some box sizes go up in price mid-month, and some credit claims name a date with no delivery. The farm newsletter claims a flat 10% rise and an exclusive pause end. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: deliveries charges credits due_and_total
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0044, 16s).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.11 | 19s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.0044 | 16s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Sonnet 5.5default · thinking on | $0.05 | 14s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.