Choose the value-maximising project portfolio under a budget, an hours cap, prerequisites and exclusions
Exact prompt
# Choose next quarter's project portfolio Inputs: `projects.csv` (cost, engineering hours and estimated value of each candidate project, plus prerequisites) and `constraints.json` (budget, available hours, pairs that can't both be funded). Pick the set of projects that maximises total value. - Total cost must not exceed the budget, and total hours must not exceed the available hours. - A project can only be funded if all its prerequisites are funded too. - At most one project of each mutually exclusive pair may be funded. - Then answer one what-if: if the budget were cut by 10% (hours unchanged), what is the best total value achievable? Write `out/portfolio.json`: `{"selected": [<project ids, sorted>], "total_value": <n>, "total_cost": <n>, "total_hours": <n>, "value_if_budget_cut_10pct": <n>}`. ## How this run works - Work on your own: nobody will answer questions in this task, and writing to `ask/` counts against the run. - Write only under `out/`. Create `out/` if it is missing. Inputs are synthetic; don't fetch anything from the network. - Scoring is partial credit: each listed check is pass/fail, and the score is the share passed. Numbers are compared to within 0.005 unless the task says otherwise.
Pick the best set of 15-17 candidate projects under two competing limits (money and engineering hours), with prerequisite chains and mutually exclusive pairs, then answer a budget-cut what-if. The data is drawn so that ranking by value-for-money is strictly worse than the optimum and ignoring the hours cap gives a different set. The hidden key is the unique exact optimum. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked). Folder-restore note: this run's output files under work/out/ were lost in a 1:22 AM ET restore on Oct 21, 2026. The trace, step-in exchanges and output-hash list were checked against the recorded SHA-256 values and match; the hashes of the individual output files inside that list can no longer be recomputed. Measured data unchanged.
Success criteria: optimal_selection optimal_value cost_and_hours what_if_budget_cut
Best model for this task so far: Gemma 4 31B Instruct (Venice) (thinking on) (success, $0.0025, 2m).
All runs (6)
100% pass rate across 6 eligible runs: 6 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| QQwen3 Coder 480B Turbo (Venice)thinking on | $0.02 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-29 | |
| GGemma 4 31B Instruct (Venice)thinking on | $0.0025 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-28 | |
| DDeepSeek V4.1 Flash (Venice)thinking on | $0.0030 | 9s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-28 | |
| AClaude Opus 5.5default · thinking on | $0.10 | 16s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 | |
| AClaude Haiku 5.5default · thinking on | $0.0032 | 12s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 | |
| AClaude Sonnet 5.5default · thinking off | $0.04 | 10s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.