Gemini 3.8 Flash vs GPT-6.1 Sol
On the 31 use cases both models ran, GPT-6.1 Sol won 30 to 0, with 1 tied (better outcome first, then lower cost).
Pass rate: Gemini 3.8 Flash 97% (31 runs) vs GPT-6.1 Sol 97% (31 runs). Average cost per run: $0.36 vs $0.04.
| Metric | Gemini 3.8 Flash | GPT-6.1 Sol |
|---|---|---|
| Runs on shared use cases | 31 | 31 |
| Pass rate | 97% (30/31) | 97% (30/31) |
| Avg cost per run | $0.36 | $0.04 |
| Avg wall time | 4m | 46s |
| Step-ins per run | 0.06 | 0.06 |
| Use cases won | 0 | 30 |
Use case by use case
- AIS collision risk: closest point of approach and alert levels: GPT-6.1 Sol better (pass 100% vs 100%)
- Anchor drag alarm from a GPS track: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a fish tank with swimming fish and rising bubbles in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a flickering carved pumpkin on a spooky night in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a junction where cars obey a traffic light's timed sequence in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a metronome that ticks a repeating two-tempo pattern in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a row of pendulums, each swinging at its own true period, in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate a snow globe that a click shakes, with snow that settles again, in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate balls dropped onto a floor that bounce with real gravity and restitution in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate hot-air balloons drifting over hills at sunrise in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate planets orbiting the sun at their own speeds in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Animate rain running down a window over blurred city lights in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Assign a community transport scheme's volunteer driver trips when one driver's background-check status is blank: GPT-6.1 Sol better (pass 100% vs 100%)
- Best fishing windows from tide, solunar and weather data: GPT-6.1 Sol better (pass 100% vs 100%)
- Best sailing angle and leg times from a polar table: GPT-6.1 Sol better (pass 100% vs 100%)
- Bill a hotel's group bookings across a mid-month rate rise, weekend nights, cancellations with time-zone offsets and a city-tax cap: GPT-6.1 Sol better (pass 100% vs 100%)
- Boat fuel range with a reserve over a planned trip: GPT-6.1 Sol better (pass 100% vs 100%)
- Catch log checked against a made-up fishing regulations table: GPT-6.1 Sol better (pass 100% vs 100%)
- Charge residents' parking permits by emission band, part year, household order and blue-badge exemption: GPT-6.1 Sol better (pass 100% vs 100%)
- Check and pay staff mileage claims across the April tax-year change: GPT-6.1 Sol better (pass 100% vs 100%)
- Dead reckoning, estimated position with current, and computed set and drift: GPT-6.1 Sol better (pass 100% vs 100%)
- Draw a ticking, styled analog clock that shows the current time in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Draw a walking pixel-art character sprite animation in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
- Go/no-go departure time from a gridded wind file: GPT-6.1 Sol better (pass 100% vs 100%)
- Let a town's allotment plots when one tenant still has a rent-arrears row this year: tie (pass 0% vs 0%)
- Sailboat instrument dashboard from an NMEA 0183 log: GPT-6.1 Sol better (pass 100% vs 100%)
- Tide height by rule of twelfths and cosine, with under-keel clearance windows: GPT-6.1 Sol better (pass 100% vs 100%)
- Water-quality sensor anomaly flags and daily means: GPT-6.1 Sol better (pass 100% vs 100%)
- Work out a care home's fortnightly pay from shifts that span the night the clocks go back: GPT-6.1 Sol better (pass 100% vs 100%)
- Work out a library's overdue fines from loans, renewals, closed days and member categories: GPT-6.1 Sol better (pass 100% vs 100%)
- Work out late-payment interest and compensation from part payments, credit notes and a base-rate change: GPT-6.1 Sol better (pass 100% vs 100%)
Only use cases both models ran are compared; all settings of each model are grouped. How we score.