Gemini 3.8 Flash vs Gemma 4 31B Instruct (Venice)
On the 7 use cases both models ran, Gemma 4 31B Instruct (Venice) won 5 to 2 (better outcome first, then lower cost).
Pass rate: Gemini 3.8 Flash 100% (7 runs) vs Gemma 4 31B Instruct (Venice) 71% (7 runs). Average cost per run: $0.27 vs $0.0056.
| Metric | Gemini 3.8 Flash | Gemma 4 31B Instruct (Venice) |
|---|---|---|
| Runs on shared use cases | 7 | 7 |
| Pass rate | 100% (7/7) | 71% (5/7) |
| Avg cost per run | $0.27 | $0.0056 |
| Avg wall time | 3m | 10m |
| Step-ins per run | 0.00 | 0.00 |
| Use cases won | 2 | 5 |
Use case by use case
- Best fishing windows from tide, solunar and weather data: Gemma 4 31B Instruct (Venice) better (pass 100% vs 100%)
- Best sailing angle and leg times from a polar table: Gemini 3.8 Flash better (pass 100% vs 0%)
- Boat fuel range with a reserve over a planned trip: Gemma 4 31B Instruct (Venice) better (pass 100% vs 100%)
- Catch log checked against a made-up fishing regulations table: Gemini 3.8 Flash better (pass 100% vs 0%)
- Diagnose why GitHub Actions stopped running: Gemma 4 31B Instruct (Venice) better (pass 100% vs 100%)
- Fix three bugs in a Python repo so its tests pass: Gemma 4 31B Instruct (Venice) better (pass 100% vs 100%)
- Go/no-go departure time from a gridded wind file: Gemma 4 31B Instruct (Venice) better (pass 100% vs 100%)
Only use cases both models ran are compared; all settings of each model are grouped. How we score.