Settle flight delay and cancellation compensation across time-zone offsets, distance bands and passenger rules
Exact prompt
# Settle an airline's delay and cancellation compensation claims Inputs: `flights.csv` (flight, origin, dest, distance_km, status, sched_arr, actual_arr, cause, notice_days, reroute_arr), `passengers.csv` (pax_id, booking_ref, name, flight, type, fare_paid, checked_in), `rules.json` and `claims-desk-note.txt`. Times carry their own UTC offsets. Amounts are in euros. `rules.json` and these rules are the record; where the claims desk note disagrees, the rules win. Rules: - An operated flight's delay is the time between sched_arr and actual_arr as real instants, in whole minutes rounded down; it qualifies if the delay is at least delay_minutes. - The amount comes from the first band whose max_km is at least distance_km (max_km null is the top band). - A cancelled flight qualifies if notice_days is less than the rules' notice_days; if the passenger was rerouted and reroute_arr is no later than the band's reroute_reduce_within_min minutes after sched_arr, the amount is halved. - Nothing is due for a flight whose cause is in extraordinary_causes. - A passenger listed twice (same booking_ref and name, ignoring letter case and extra spaces, same flight) counts once, under the lower pax number. Infants with fare_paid 0 are not eligible. On an operated flight only passengers who checked in are eligible; on a cancelled flight check-in doesn't matter. - Write `out/claims.json`: `{"delay_minutes": {"<flight>": <minutes>, ...}, "compensation": {"<pax_id>": <amount>, ...}, "flight_totals": {"<flight>": <sum>, ...}, "total": <sum>}`. `delay_minutes` has every operated flight; the other maps list only amounts above 0. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Work out which passengers are owed delay or cancellation compensation and how much. Arrival times carry different UTC offsets so the clock text misleads, delays sit at exactly three hours, distances sit exactly on band limits, cancellations land either side of the 14-day notice line with reroutes that may halve the amount, some causes are extraordinary, passengers appear twice in different case and spacing, and infants and missed check-ins change eligibility. A claims-desk note gets the time, threshold, band and reroute rules wrong. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: delays bands cancellations passengers totals
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0077, 32s).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.14 | 23s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| AClaude Haiku 5.5default · thinking on | $0.0077 | 32s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 | |
| AClaude Sonnet 5.5default · thinking on | $0.07 | 27s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-23 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.