Model rankings
Built from 106 eligible real-world runs. Under-review runs are excluded. Data as of Nov 4, 2026, 20:00 UTC. How we score
Filters & weights1 active
-MLlama 3.2 3B (local)thinking off(fewer than 5 runs)
24.4 (24–24.9)Pass rate: 0%(0/4, 4 fail)
- Cost / success
- -
- Avg time
- 11s
- Step-ins
- 0.0
4 runs · first-try 0%Compare runs →
-QQwen2.5 7B (local)thinking off(fewer than 5 runs)
49.6 (24.3–74.9)Pass rate: 25%(1/4, 1 partial, 2 fail)
- Cost / success
- $0 (local)
- Avg time
- 33s
- Step-ins
- 0.0
4 runs · first-try 25%Compare runs →
-AClaude Haiku 5.5default · thinking on + AClaude Opus 5.5default · thinking on · + scanner subagents(fewer than 5 runs)
70 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.84
- Avg time
- 3m
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-AClaude Opus 5.5max · thinking on(fewer than 5 runs)
71 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $4.83
- Avg time
- 1.7h
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-GGemini 3.8 Flash(fewer than 5 runs)
72 (69.7–74.3)Pass rate: 100%(2/2)
- Cost / success
- $0.12
- Avg time
- 3m
- Step-ins
- 0.0
2 runs · first-try 100%Compare runs →
#9AClaude Opus 5.5default · thinking on–
73.9 (69.5–78.3)Pass rate: 100%(9/9)
- Cost / success
- $0.42
- Avg time
- 10m
- Step-ins
- 0.3
9 runs · first-try 100%Compare runs →
-AClaude Opus 5.5xhigh · thinking on(fewer than 5 runs)
74 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $2.24
- Avg time
- 5.0h
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-AClaude Sonnet 5.5max · thinking on(fewer than 5 runs)
74 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $3.09
- Avg time
- 1.2h
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
- 76.1 (74.6–77.7)
Pass rate: 100%(7/7)
- Cost / success
- $0.25
- Avg time
- 57s
- Step-ins
- 0.0
7 runs · first-try 100%Compare runs →
#7QQwen3 Coder 480B Turbo (Venice)–
77 (60.1–93.9)Pass rate: 71%(5/7, 1 partial, 1 fail)
- Cost / success
- $0.0051
- Avg time
- 23s
- Step-ins
- 0.0
7 runs · first-try 71%Compare runs →
-AClaude Sonnet 5.5xhigh · thinking on(fewer than 5 runs)
79 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $1.07
- Avg time
- 14m
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-ZGLM 5.3 (Venice)(fewer than 5 runs)
79.1 (77.5–80.7)Pass rate: 100%(3/3)
- Cost / success
- $0.05
- Avg time
- 1m
- Step-ins
- 0.0
3 runs · first-try 100%Compare runs →
-ZGLM 5.3 (Venice)thinking on(fewer than 5 runs)
79.4 (78.8–80.1)Pass rate: 100%(2/2)
- Cost / success
- $0.03
- Avg time
- 42s
- Step-ins
- 0.0
2 runs · first-try 100%Compare runs →
-AClaude Opus 5.5high · thinking on(fewer than 5 runs)
82 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.60
- Avg time
- 3m
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
- 83.3 (81.9–84.6)
Pass rate: 100%(7/7)
- Cost / success
- $0.07
- Avg time
- 19s
- Step-ins
- 0.0
7 runs · first-try 100%Compare runs →
- 84.1 (76.4–91.9)
Pass rate: 86%(6/7, 1 partial)
- Cost / success
- $0.01
- Avg time
- 15s
- Step-ins
- 0.0
7 runs · first-try 86%Compare runs →
#4AClaude Sonnet 5.5default · thinking on–
84.4 (80.6–88.2)Pass rate: 100%(10/10)
- Cost / success
- $0.21
- Avg time
- 51s
- Step-ins
- 0.3
10 runs · first-try 100%Compare runs →
-AClaude Opus 5.5medium · thinking on(fewer than 5 runs)
86 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.36
- Avg time
- 2m
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-AClaude Sonnet 5.5high · thinking on(fewer than 5 runs)
87 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.22
- Avg time
- 2m
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
#3GGemma 4 31B Instruct (Venice)–
88.5 (86.4–90.5)Pass rate: 100%(7/7)
- Cost / success
- $0.0033
- Avg time
- 1m
- Step-ins
- 0.0
7 runs · first-try 100%Compare runs →
#1DDeepSeek V4.1 Flash (Venice)–
89.3 (84.9–93.7)Pass rate: 100%(7/7)
- Cost / success
- $0.0078
- Avg time
- 2m
- Step-ins
- 0.0
7 runs · first-try 100%Compare runs →
#2AClaude Haiku 5.5default · thinking on–
89.3 (86.8–91.8)Pass rate: 94%(17/18, 1 partial)
- Cost / success
- $0.02
- Avg time
- 46s
- Step-ins
- 0.1
18 runs · first-try 94%Compare runs →
-AClaude Opus 5.5low · thinking on(fewer than 5 runs)
92 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.21
- Avg time
- 52s
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-AClaude Sonnet 5.5default · thinking off(fewer than 5 runs)
92.9 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.04
- Avg time
- 10s
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
-AClaude Sonnet 5.5medium · thinking on(fewer than 5 runs)
95 (too few runs for a range)Pass rate: 100%(1/1)
- Cost / success
- $0.13
- Avg time
- 50s
- Step-ins
- 0.0
1 run · first-try 100%Compare runs →
Want a side-by-side? Click a model, tick Compare on 2–4 runs, then open the tray at the bottom. How compare works
What the columns mean
- Pass rate: share of runs that met 100% of the task's success criteria. Partial and failed runs are counted and shown, not hidden.
- Step-ins: how many times a person (or a script) had to jump in after the first prompt to correct, clarify, approve or nudge the AI. 0 means it finished on its own. Most runs here come from automated harnesses that send one prompt and never intervene, so 0 step-ins is a measured value, not a placeholder.
- Trust: how much we believe a run's numbers, from automatic checks (evidence log, recomputed cost, consistency). Automated runs with a full log and no warnings typically land at 80%.
- Trend: rank change versus 7 days ago, rebuilt from the runs that were published by then. Blank when there isn't enough history.
Head-to-head column hidden: it needs 20+ rerun pairs, and there aren't enough yet.
How rankings work
- Each run is scored 0–100 from quality, success (incl. first-try), cost-per-success and speed (percentile vs. other runs of the same use case), and autonomy, using the weights above.
- Runs are weighted by trust score and recency (half-life 90 days). The range shown is a 90% confidence interval; the whisker on each bar spans it. Models with too few runs for a real interval show no range.
- Models with fewer than 5 runs are listed but not numbered. Head-to-head reruns feed a Bradley-Terry rating, blended at 30% once there are 20+ pairs.
- Under-review runs never count. Sponsorship never affects rankings. Full details: Methodology.