GPT-6.1 Sol: real-world results
GPT-6.1 Sol passed 133 of 139 runs (96%) on 139 different use cases.
Average cost per run was $0.05, average wall time 1m, with 0.02 human step-ins per run.
Results by category
| Category | Runs | Passed |
|---|---|---|
| Creative / Animated Scene (one HTML File) | 12 | 12/12 |
| Creative / Web Animation | 6 | 6/6 |
| Finance / Retirement Planning | 4 | 4/4 |
| Coding | 2 | 2/2 |
| Data / Accounting | 2 | 2/2 |
| data/CSV Analysis | 2 | 2/2 |
| Ops/debugging | 2 | 2/2 |
| Spec-following Parser | 2 | 1/2 |
| Writing With Rubric | 2 | 2/2 |
| 3D / Game Assets | 1 | 1/1 |
| Accessibility | 1 | 1/1 |
| Algorithmic Optimisation With Perf Limit | 1 | 1/1 |
| Algorithms / Package Management | 1 | 1/1 |
| Algorithms / Routing | 1 | 1/1 |
| Allotments / Lettings | 1 | 0/1 |
| CLI Tool | 1 | 1/1 |
| Community Transport / Volunteer Drivers | 1 | 1/1 |
| Concurrency Bug | 1 | 1/1 |
| Council / Parking Permits | 1 | 1/1 |
| Creative / Animation | 1 | 1/1 |
| Creative / Data Visualisation | 1 | 1/1 |
| Data / Reliability Metrics | 1 | 1/1 |
| Data / Rules Engine | 1 | 1/1 |
| Data / Time Series | 1 | 1/1 |
| Data Cleaning / Entity Resolution | 1 | 1/1 |
| Data Structures (JavaScript) | 1 | 1/1 |
| Data Wrangling | 1 | 1/1 |
| data/SQL | 1 | 1/1 |
| Date/time Logic | 1 | 1/1 |
| Debugging | 1 | 1/1 |
| Debugging (flaky Tests / Nondeterminism) | 1 | 1/1 |
| Debugging (JavaScript) | 1 | 1/1 |
| Debugging / SQL | 1 | 1/1 |
| Device Administration (ADB) | 1 | 1/1 |
| Distributed Systems / Hashing | 1 | 1/1 |
| Doc-to-code Spec (binary Format) | 1 | 1/1 |
| Document Extraction | 1 | 1/1 |
| Document Extraction / Legal | 1 | 1/1 |
| Filesystem Audit / Cleanup Planning | 1 | 1/1 |
| Finance / Backtesting | 1 | 1/1 |
| Finance / Debt Payoff | 1 | 1/1 |
| Finance / Debugging | 1 | 1/1 |
| Finance / Expense Claims | 1 | 1/1 |
| Finance / Lending | 1 | 1/1 |
| Finance / Loans | 1 | 1/1 |
| Finance / Market Microstructure | 1 | 1/1 |
| Finance / Portfolio | 1 | 1/1 |
| Finance / Retirement Tax | 1 | 1/1 |
| Finance / Statement Analysis | 1 | 1/1 |
| Finance / Tax | 1 | 1/1 |
| Finance / Valuation | 1 | 1/1 |
| Finance Calc | 1 | 1/1 |
| Git History Investigation | 1 | 1/1 |
| Home Lab / Privacy | 1 | 1/1 |
| Hotel / Billing | 1 | 1/1 |
| Incremental Build / Cache Invalidation Bug | 1 | 1/1 |
| Knowledge Work / Extraction | 1 | 1/1 |
| Knowledge Work / Financial Analysis | 1 | 1/1 |
| Knowledge Work / Technical Writing | 1 | 1/1 |
| Library / Overdue Fines | 1 | 1/1 |
| Log Forensics (security Incident) | 1 | 1/1 |
| Long-context Needle | 1 | 1/1 |
| Marine / Algorithms | 1 | 1/1 |
| Marine / Anchoring | 1 | 1/1 |
| Marine / Collision Avoidance | 1 | 1/1 |
| Marine / Data Parsing | 1 | 1/1 |
| Marine / Environmental Monitoring | 1 | 1/1 |
| Marine / Fishing Compliance | 1 | 1/1 |
| Marine / Fishing Planning | 1 | 1/1 |
| Marine / Navigation | 1 | 1/1 |
| Marine / Passage Planning | 1 | 1/1 |
| Marine / Powerboat Planning | 1 | 1/1 |
| Marine / Sailing Instruments | 1 | 1/1 |
| Marine / Sailing Performance | 1 | 1/1 |
| Marine / Sports Scoring | 1 | 1/1 |
| Marine / Tides | 1 | 1/1 |
| Marine / Tides and Pilotage | 1 | 1/1 |
| Marine / Weather Routing | 1 | 1/1 |
| Marketing / Data Cleaning | 1 | 1/1 |
| Migration (SQL) | 1 | 1/1 |
| Migration With Backward Compatibility | 1 | 1/1 |
| Multi-file JS Refactor | 1 | 1/1 |
| Networking / Home Lab | 1 | 1/1 |
| Numeric Precision | 1 | 1/1 |
| Operations / Scheduling | 1 | 1/1 |
| Optimization/scheduling | 1 | 1/1 |
| Parsing / Evaluation | 1 | 1/1 |
| Payroll / Timesheets | 1 | 1/1 |
| Performance Optimization | 1 | 1/1 |
| Privacy / Document Drafting | 1 | 1/1 |
| Privacy / Text Processing | 1 | 1/1 |
| Productivity / Email | 1 | 1/1 |
| Refactor | 1 | 1/1 |
| Regex/parsing | 1 | 1/1 |
| Research Synthesis (offline) | 1 | 0/1 |
| Safe Autonomy / Device Administration | 1 | 1/1 |
| School Office / Trip Bookings | 1 | 1/1 |
| Security Fix | 1 | 1/1 |
| Self-hosting / Graph Planning | 1 | 0/1 |
| Shell Scripting | 1 | 0/1 |
| Small Business / Credit Control | 1 | 1/1 |
| Small Interpreter | 1 | 1/1 |
| Smart Home / Visual UI | 1 | 1/1 |
| Software / Calendar Parsing | 1 | 1/1 |
| Software / Legacy Refactor | 1 | 1/1 |
| SQL (sqlite3 Triggers) | 1 | 1/1 |
| State Machine (JavaScript) | 1 | 1/1 |
| State Machines / UI Logic | 1 | 1/1 |
| Test Writing (mutation Testing) | 1 | 0/1 |
| Text Processing / Tooling | 1 | 1/1 |
| Tool-use Planning | 1 | 1/1 |
| Tooling / Pattern Matching | 1 | 1/1 |
| Unicode / Text Normalization | 1 | 1/1 |
| Validation / Spec Implementation | 1 | 1/1 |
Head-to-head comparisons
- GPT-6.1 Sol vs Claude Haiku 5.5
- GPT-6.1 Sol vs Claude Opus 5.5
- GPT-6.1 Sol vs Claude Sonnet 5.5
- GPT-6.1 Sol vs DeepSeek V4.1 Flash (Venice)
- GPT-6.1 Sol vs Gemini 3.8 Flash
- GPT-6.1 Sol vs Gemini 4 Argon
- GPT-6.1 Sol vs Gemma 4 31B Instruct (Venice)
- GPT-6.1 Sol vs Grok 4.3 (Venice)
- GPT-6.1 Sol vs Kimi K3.1 Pro (Venice)
- GPT-6.1 Sol vs MiniMax M2.5 (Venice)
- GPT-6.1 Sol vs Mistral Small 3.2 24B Instruct (Venice)
- GPT-6.1 Sol vs Qwen3 Coder 480B Turbo (Venice)
- GPT-6.1 Sol vs GLM 4.7 (Venice)
Browse every run by GPT-6.1 Sol →
From published single-model runs only (all settings grouped). How we score.