Claude Sonnet 5.5 vs Gemini 3.8 Flash
On the 35 use cases both models ran, Claude Sonnet 5.5 won 32 to 2, with 1 tied (better outcome first, then lower cost).
Pass rate: Claude Sonnet 5.5 91% (35 runs) vs Gemini 3.8 Flash 97% (35 runs). Average cost per run: $0.06 vs $0.35.
| Metric | Claude Sonnet 5.5 | Gemini 3.8 Flash |
|---|---|---|
| Runs on shared use cases | 35 | 35 |
| Pass rate | 91% (32/35) | 97% (34/35) |
| Avg cost per run | $0.06 | $0.35 |
| Avg wall time | 20s | 4m |
| Step-ins per run | 0.14 | 0.17 |
| Use cases won | 32 | 2 |
Use case by use case
- AIS collision risk: closest point of approach and alert levels: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Anchor drag alarm from a GPS track: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a fish tank with swimming fish and rising bubbles in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a flickering carved pumpkin on a spooky night in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a junction where cars obey a traffic light's timed sequence in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a metronome that ticks a repeating two-tempo pattern in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a row of pendulums, each swinging at its own true period, in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate a snow globe that a click shakes, with snow that settles again, in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate balls dropped onto a floor that bounce with real gravity and restitution in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate hot-air balloons drifting over hills at sunrise in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate planets orbiting the sun at their own speeds in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Animate rain running down a window over blurred city lights in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Assign a community transport scheme's volunteer driver trips when one driver's background-check status is blank: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Best fishing windows from tide, solunar and weather data: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Best sailing angle and leg times from a polar table: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Bill a hotel's group bookings across a mid-month rate rise, weekend nights, cancellations with time-zone offsets and a city-tax cap: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Boat fuel range with a reserve over a planned trip: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Catch log checked against a made-up fishing regulations table: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Charge residents' parking permits by emission band, part year, household order and blue-badge exemption: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Check and pay staff mileage claims across the April tax-year change: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Dead reckoning, estimated position with current, and computed set and drift: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Diagnose why GitHub Actions stopped running: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Draw a ticking, styled analog clock that shows the current time in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Draw a walking pixel-art character sprite animation in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Enrol children on a leisure centre's swimming lessons when one child has a medical alert on file this term: Gemini 3.8 Flash better (pass 0% vs 100%)
- Fix three bugs in a Python repo so its tests pass: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Go/no-go departure time from a gridded wind file: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Let a town's allotment plots when one tenant still has a rent-arrears row this year: tie (pass 0% vs 0%)
- Sailboat instrument dashboard from an NMEA 0183 log: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Schedule a pharmacy's home deliveries when one parcel holds a medicine missing from the delivery list: Gemini 3.8 Flash better (pass 0% vs 100%)
- Tide height by rule of twelfths and cosine, with under-keel clearance windows: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Water-quality sensor anomaly flags and daily means: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Work out a care home's fortnightly pay from shifts that span the night the clocks go back: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Work out a library's overdue fines from loans, renewals, closed days and member categories: Claude Sonnet 5.5 better (pass 100% vs 100%)
- Work out late-payment interest and compensation from part payments, credit notes and a base-rate change: Claude Sonnet 5.5 better (pass 100% vs 100%)
Only use cases both models ran are compared; all settings of each model are grouped. How we score.