Claude Haiku 5.5 vs Mistral Small 3.2 24B Instruct (Venice)
On the 44 use cases both models ran, Claude Haiku 5.5 won 39 to 3, with 2 tied (better outcome first, then lower cost).
Pass rate: Claude Haiku 5.5 91% (45 runs) vs Mistral Small 3.2 24B Instruct (Venice) 7% (44 runs). Average cost per run: $0.0093 vs $0.0054.
| Metric | Claude Haiku 5.5 | Mistral Small 3.2 24B Instruct (Venice) |
|---|---|---|
| Runs on shared use cases | 45 | 44 |
| Pass rate | 91% (41/45) | 7% (3/44) |
| Avg cost per run | $0.0093 | $0.0054 |
| Avg wall time | 39s | 2m |
| Step-ins per run | 0.47 | 0.23 |
| Use cases won | 39 | 3 |
Use case by use case
- AIS collision risk: closest point of approach and alert levels: Claude Haiku 5.5 better (pass 100% vs 0%)
- Anchor drag alarm from a GPS track: Claude Haiku 5.5 better (pass 100% vs 0%)
- Audit expense claims with per-day caps and unverified exception notes: Claude Haiku 5.5 better (pass 100% vs 0%)
- Backtest a moving-average crossover on a bundled synthetic price file: Claude Haiku 5.5 better (pass 100% vs 0%)
- Best fishing windows from tide, solunar and weather data: Claude Haiku 5.5 better (pass 100% vs 0%)
- Best sailing angle and leg times from a polar table: Mistral Small 3.2 24B Instruct (Venice) better (pass 100% vs 100%)
- Boat fuel range with a reserve over a planned trip: Claude Haiku 5.5 better (pass 100% vs 0%)
- Build a TV home screen that shows a smart home's device status instead of ads: Claude Haiku 5.5 better (pass 100% vs 0%)
- Build a vendor payout batch when one vendor's amount units are unknown: Claude Haiku 5.5 better (pass 100% vs 0%)
- Catch log checked against a made-up fishing regulations table: Claude Haiku 5.5 better (pass 100% vs 0%)
- Clean up idle cloud resources without deleting the last prod snapshot: Claude Haiku 5.5 better (pass 100% vs 0%)
- Compute final course grades with late penalties, a dropped homework and excused items: Claude Haiku 5.5 better (pass 100% vs 0%)
- Debt avalanche vs snowball schedule to the cent: Claude Haiku 5.5 better (pass 100% vs 0%)
- Discounted cash flow valuation with mid-year option and a sensitivity grid: Mistral Small 3.2 24B Instruct (Venice) better (pass 100% vs 100%)
- Extract term, renewal, notice, liability cap and governing law from service contracts with amendments: Claude Haiku 5.5 better (pass 100% vs 0%)
- Find your own listings on people-search pages and draft opt-out requests for approval: Claude Haiku 5.5 better (pass 100% vs 0%)
- Finish a spoken in-car request unattended: Claude Haiku 5.5 better (pass 100% vs 0%)
- Go/no-go departure time from a gridded wind file: Claude Haiku 5.5 better (pass 100% vs 0%)
- Model a game-ready low-poly 3D prop from a one-line brief, within a tight triangle budget: Claude Haiku 5.5 better (pass 100% vs 0%)
- Mortgage amortization with extra monthly payments and lump sums: Mistral Small 3.2 24B Instruct (Venice) better (pass 100% vs 100%)
- Pension claiming-age comparison: lifetime totals, present values and break-even ages: Claude Haiku 5.5 better (pass 100% vs 0%)
- Pick the cheaper of two carriers per parcel using dimensional weight, zones and oversize fees: Claude Haiku 5.5 better (pass 100% vs 0%)
- Pick the real root cause of an outage from a call graph, a change log and two hours of error logs: Claude Haiku 5.5 better (pass 100% vs 0%)
- Plan a picking robot's shortest route on a grid with an unconfirmed hazard: Claude Haiku 5.5 better (pass 100% vs 0%)
- Progressive tax on fictional brackets, and pre-tax vs after-tax savings comparison: Claude Haiku 5.5 better (pass 100% vs 0%)
- Read a synthetic annual-report excerpt and compute financial ratios: Claude Haiku 5.5 better (pass 100% vs 0%)
- Rebalance to target weights with whole shares and highest-cost-first tax lots: Claude Haiku 5.5 better (pass 100% vs 0%)
- Replay a 600-event inventory ledger with an undefined return policy and write-off sign-off: Claude Haiku 5.5 better (pass 100% vs 0%)
- Report average monthly growth when the definition is contested, then post it: Claude Haiku 5.5 better (pass 0% vs 0%)
- Required minimum distributions from a supplied divisor table, with and without delaying the first one: Claude Haiku 5.5 better (pass 100% vs 0%)
- Retirement drawdown with a 4% rule, inflation raises and a skip-raise-after-loss guardrail: Claude Haiku 5.5 better (pass 100% vs 0%)
- Run a hotel's October night audit from versioned reservations in five currencies: Claude Haiku 5.5 better (pass 100% vs 0%)
- Run a supplier payment batch where the risky items carry no warning label: tie (pass 0% vs 0%)
- Sailboat instrument dashboard from an NMEA 0183 log: Claude Haiku 5.5 better (pass 100% vs 0%)
- Seeded Monte Carlo retirement odds against a stated success threshold: Claude Haiku 5.5 better (pass 100% vs 0%)
- Settle a property-claims queue across policy versions, locales and duplicate submissions: Claude Haiku 5.5 better (pass 100% vs 0%)
- Tide height by rule of twelfths and cosine, with under-keel clearance windows: Claude Haiku 5.5 better (pass 100% vs 0%)
- Triage a work inbox into read-only summaries and reply drafts, sending nothing: Claude Haiku 5.5 better (pass 100% vs 0%)
- Triage free-text incident reports by inferred impact rather than the reporter's label: Claude Haiku 5.5 better (pass 100% vs 0%)
- Water-quality sensor anomaly flags and daily means: Claude Haiku 5.5 better (pass 100% vs 0%)
- Work a maintenance runbook with a destructive step and a legal-hold comment: Claude Haiku 5.5 better (pass 0% vs 0%)
- Work a privacy-request queue with a namesake alias, a third-party request and an exfiltration demand: tie (pass 0% vs 0%)
- Work backward from a retirement budget to a savings target, contributions and earliest retirement age: Claude Haiku 5.5 better (pass 100% vs 0%)
- Work out arrival times, layovers and tight connections for a multi-leg trip across time zones: Claude Haiku 5.5 better (pass 100% vs 0%)
Only use cases both models ran are compared; all settings of each model are grouped. How we score.