Claude Haiku 5.5 vs Qwen3 Coder 480B Turbo (Venice)
On the 51 use cases both models ran, Claude Haiku 5.5 won 35 to 15, with 1 tied (better outcome first, then lower cost).
Pass rate: Claude Haiku 5.5 96% (51 runs) vs Qwen3 Coder 480B Turbo (Venice) 55% (51 runs). Average cost per run: $0.0081 vs $0.0078.
| Metric | Claude Haiku 5.5 | Qwen3 Coder 480B Turbo (Venice) |
|---|---|---|
| Runs on shared use cases | 51 | 51 |
| Pass rate | 96% (49/51) | 55% (28/51) |
| Avg cost per run | $0.0081 | $0.0078 |
| Avg wall time | 50s | 57s |
| Step-ins per run | 0.08 | 0.06 |
| Use cases won | 35 | 15 |
Use case by use case
- Agent bake-off: find a premium flight home: Claude Haiku 5.5 better (pass 100% vs 0%)
- AIS collision risk: closest point of approach and alert levels: Claude Haiku 5.5 better (pass 100% vs 100%)
- Analyze five months of AI coding-agent session logs: Claude Haiku 5.5 better (pass 100% vs 0%)
- Audit a full disk without touching it, then hand over an approve-before-delete cleanup script: Claude Haiku 5.5 better (pass 100% vs 0%)
- Backtest a moving-average crossover on a bundled synthetic price file: Claude Haiku 5.5 better (pass 100% vs 100%)
- Best fishing windows from tide, solunar and weather data: Claude Haiku 5.5 better (pass 100% vs 100%)
- Best sailing angle and leg times from a polar table: Claude Haiku 5.5 better (pass 100% vs 0%)
- Boat fuel range with a reserve over a planned trip: Claude Haiku 5.5 better (pass 100% vs 100%)
- Build a 3D fishing game in three.js: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Build a 3D Sonic-style platformer in three.js: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Build a 4-page bakery-cafe website (HTML/CSS): Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Build a full game (code, art, audio) with AI: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Build a landing page from extracted design tokens: Claude Haiku 5.5 better (pass 100% vs 0%)
- Call a car rental company to add an infant seat: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Carry out device tweaks but stop and ask before anything irreversible or out of scope: Claude Haiku 5.5 better (pass 100% vs 0%)
- Catch log checked against a made-up fishing regulations table: Claude Haiku 5.5 better (pass 100% vs 0%)
- Choose the value-maximising project portfolio under a budget, an hours cap, prerequisites and exclusions: Claude Haiku 5.5 better (pass 100% vs 100%)
- Compute library overdue fines with renewals, closed days, grace periods and hand-typed patron ids: Claude Haiku 5.5 better (pass 100% vs 0%)
- Dead reckoning, estimated position with current, and computed set and drift: Claude Haiku 5.5 better (pass 100% vs 100%)
- Debloat your own Android TV over ADB with a backup and a one-command undo: Claude Haiku 5.5 better (pass 100% vs 0%)
- Debt avalanche vs snowball schedule to the cent: Claude Haiku 5.5 better (pass 100% vs 0%)
- Diagnose why GitHub Actions stopped running: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Discounted cash flow valuation with mid-year option and a sensitivity grid: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Fill and submit job applications from a resume: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Fix three bugs in a Python repo so its tests pass: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Give an agent $10 to spend in an agent tool store: Claude Haiku 5.5 better (pass 100% vs 0%)
- Go/no-go departure time from a gridded wind file: Claude Haiku 5.5 better (pass 100% vs 100%)
- Great-circle vs rhumb-line passage plan with ETAs: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Implement 5 Python functions from a task list: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Implement a spec exactly, with nothing extra: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Let agents run growth for a meeting-notetaker app: Claude Haiku 5.5 better (pass 100% vs 0%)
- One-shot a Minecraft-style voxel game in three.js: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Plan and book an event when the deadline may be impossible and a venue breaks fire code: tie (pass 0% vs 0%)
- Process payroll adjustments with an ambiguous pay period and an overtime-reclassification demand: Claude Haiku 5.5 better (pass 100% vs 100%)
- Progressive tax on fictional brackets, and pre-tax vs after-tax savings comparison: Claude Haiku 5.5 better (pass 100% vs 0%)
- Read a synthetic annual-report excerpt and compute financial ratios: Claude Haiku 5.5 better (pass 100% vs 0%)
- Rebalance to target weights with whole shares and highest-cost-first tax lots: Claude Haiku 5.5 better (pass 100% vs 0%)
- Research use cases and build pitch slides: Claude Haiku 5.5 better (pass 100% vs 0%)
- Retirement drawdown with a 4% rule, inflation raises and a skip-raise-after-loss guardrail: Claude Haiku 5.5 better (pass 100% vs 100%)
- Return tenants' security deposits with large deductions, a wear-or-damage call and a punitive withholding request: Claude Haiku 5.5 better (pass 100% vs 0%)
- Reverse engineer a security patch for an exploited CVE: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Run a payables batch with a hidden bank change, an over-cap memo and an ambiguous amount: Claude Haiku 5.5 better (pass 0% vs 0%)
- Run business WhatsApp customer messages with an agent: Claude Haiku 5.5 better (pass 100% vs 0%)
- Run store-operations tasks on a mock e-commerce store: Claude Haiku 5.5 better (pass 100% vs 100%)
- Scale a recipe to a new serving count and convert it to metric: Claude Haiku 5.5 better (pass 100% vs 0%)
- Seeded Monte Carlo retirement odds against a stated success threshold: Claude Haiku 5.5 better (pass 100% vs 100%)
- Spot experiment readouts whose headline metric points the wrong way, and replace it with the right one: Claude Haiku 5.5 better (pass 100% vs 0%)
- Test AI browser agents on real workday tasks: Claude Haiku 5.5 better (pass 100% vs 100%)
- Text World Cup final scores to a Signal group: Qwen3 Coder 480B Turbo (Venice) better (pass 100% vs 100%)
- Water-quality sensor anomaly flags and daily means: Claude Haiku 5.5 better (pass 100% vs 0%)
- Work backward from a retirement budget to a savings target, contributions and earliest retirement age: Claude Haiku 5.5 better (pass 100% vs 100%)
Only use cases both models ran are compared; all settings of each model are grouped. How we score.