Break down a home's electricity use by appliance from one plug's power readings and flag the faulty appliance
Exact prompt
# Break down a home's electricity use by appliance from one plug's power readings and flag the faulty appliance Inputs: `power.csv` (t, watts: one reading per second from a single monitoring plug, t from 0) and `appliances.csv` (appliance_id, name, rated_w) for the appliances behind that plug. The home, the appliances and the readings are synthetic; the readings include a steady always-on load and random meter noise of a few watts. These rules are the household's; where `meter-notes.txt` disagrees, the rules win. Rules: - Events: a switch event happens at second t when |watts[t] − watts[t−1]| is 30 W or more. Its step is median(watts[t] … watts[t+8]) − median(watts[t−9] … watts[t−1]) (nine readings after and nine before, both medians of whole watts); a positive step is a switch-on and a negative step a switch-off. - Matching: an event belongs to the appliance whose rated_w R satisfies 0.75 × R ≤ |step| ≤ 1.5 × R (the ratings are spaced so that exactly one fits). Each switch-off ends that appliance's current on-period, which started at its last switch-on. - Use: an appliance's on_count is its number of switch-ons, its on_seconds the total of (switch-off t − switch-on t) over its on-periods, and its energy_wh the sum over its on-periods of the switch-on step × the period's seconds ÷ 3600. energy_wh is compared to within 0.5 Wh; on_count, on_seconds and faulty must match exactly. - Faulty: an appliance is faulty when any of its switch-on steps is more than 15% above its rated_w (step > 1.15 × rated_w). - Write `out/appliances.json`: `{"on_count": {"<appliance_id>": <n>, ...}, "on_seconds": {"<appliance_id>": <s>, ...}, "energy_wh": {"<appliance_id>": <Wh>, ...}, "faulty": [<sorted appliance_ids>]}`. - Roll-ups: Appliances in appliances.csv that never switch on are included at 0. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Split one monitoring plug's second-by-second power readings for a synthetic home into its appliances. The readings carry an always-on load and seeded meter noise; switch events, step sizes (medians of nine readings each side), the matching window around each rating and the 0.5 Wh energy tolerance are all stated. One appliance develops a fault and starts drawing well over its rating. The prompt says appliances that never switch on are listed at 0. Meter notes get the step, fault and roll-up rules wrong. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: on_count on_seconds energy_wh faulty idle_appliances
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0046, 16s).
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Opus 5.5default · thinking on | $0.13 | 19s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-25 | |
| AClaude Haiku 5.5default · thinking on | $0.0046 | 16s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-25 | |
| AClaude Sonnet 5.5default · thinking on | $0.05 | 16s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-25 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.