Skip to content

Bill a farm's veg box subscribers with inclusive pauses, a mid-month price rise and credit claims to validate

Exact prompt

# Bill a farm's veg box subscribers for November 2026 Inputs: `subscribers.csv` (sub_id, box, weekday, start_date), `pauses.csv`, `prices.csv`, `credits.csv` (claims for missing items), `delivery.json` and `newsletter.txt`. Every amount is in pounds. The data files and these rules are the record; where the newsletter disagrees with them, the rules win. Rules: - Each subscriber gets one box on every date in November 2026 that falls on their weekday and is on or after their start_date. - A pause covers every date from its from date to its to date, both included; no box is delivered on a paused date. - Each delivery is charged at the prices.csv row for that box with the latest effective_from on or before the delivery date; small boxes also pay small_box_fee per delivery. - A credit claim counts only if its date is a date that subscriber actually received a box. - Due = charges - credits. Write `out/boxes.json`: `{"deliveries": {"<sub_id>": <count>, ...}, "charges": {...}, "credits": {...}, "due": {...}, "grand_total": <sum of due>}` with every subscriber in each map (0 where nothing applies). ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.

Count each subscriber's November deliveries and bill them. Subscriptions start mid-month, pauses end exactly on delivery days and include both ends, only some box sizes go up in price mid-month, and some credit claims name a date with no delivery. The farm newsletter claims a flat 10% rise and an exclusive pause end. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: deliveries charges credits due_and_total

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0044, 16s).

Run this on another model

All runs (3)

100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
AClaude Opus 5.5default · thinking on$0.1119s0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.004416s0Fully autonomouspassTrust 80%2026-10-22
AClaude Sonnet 5.5default · thinking on$0.0514s0Fully autonomouspassTrust 80%2026-10-22

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Bill a farm's veg box subscribers with inclusive pauses, a mid-month price rise and credit claims to validate

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.