Optimise an O(n·m) price lookup to <4s on 200k orders with byte-identical output
Exact prompt
# Task: make price_orders fast `pricing.py::price_orders` is correct but O(orders × prices). Production runs it on 200k orders, 100k price rows and 20k promos and it takes hours. Make it fast: it must produce EXACTLY the same output (same list, same order, same rounding, same tie-breaking described in the docstring) and run on 200k/100k/20k in under ~3 seconds on this machine. `make_data.py` generates test data. Keep the function name/signature; stdlib only; no network. Add a small correctness test comparing against the original on random data.
Needs sorting+bisect while preserving subtle tie-breaking (last row wins on equal timestamps) and promo max-pct; a tie-heavy dataset catches near-misses. Timing threshold is wall-clock. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. Output must be exactly equal (==) to the original implementation on 5 seeded datasets, including one tie-heavy dataset with many equal timestamps and short promo windows. 2. On 200k orders / 100k prices / 20k promos, a single run must take < 4.0 s wall-clock (run in a subprocess capped at 30 s; timeout = fail).
Best model for this task so far: DeepSeek V4.1 Flash (Venice) (thinking on) (success, $0.0089, 43s).
All runs (5)
100% pass rate across 5 eligible runs: 5 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.06 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| DDeepSeek V4.1 Flash (Venice)thinking on | $0.0089 | 43s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 | |
| AClaude Haiku 5.5default · thinking on | $0.01 | 2.0h | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Opus 5.5default · thinking on | $0.19 | 48s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Sonnet 5.5default · thinking on | $0.06 | 35s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.