Claude Sonnet 5.5 vs GPT-6.1 Sol

Use case by use case

• Account-takeover investigation across SSH auth, HTTP access and app audit logs in three different time bases: GPT-6.1 Sol better (pass 100% vs 100%)
• AIS collision risk: closest point of approach and alert levels: GPT-6.1 Sol better (pass 100% vs 100%)
• AIS decoder: NMEA tag blocks, multipart reassembly and bit-level fields: GPT-6.1 Sol better (pass 100% vs 100%)
• Anchor drag alarm from a GPS track: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a fish tank with swimming fish and rising bubbles in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a flickering carved pumpkin on a spooky night in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a junction where cars obey a traffic light's timed sequence in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a metronome that ticks a repeating two-tempo pattern in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a row of pendulums, each swinging at its own true period, in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate a snow globe that a click shakes, with snow that settles again, in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate balls dropped onto a floor that bounce with real gravity and restitution in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate hot-air balloons drifting over hills at sunrise in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate planets orbiting the sun at their own speeds in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animate rain running down a window over blurred city lights in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Animated fireplace in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Animated snowman in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Answer 6 multi-hop questions from ~150KB of meeting notes with corrections and decoys: GPT-6.1 Sol better (pass 100% vs 100%)
• Answer 8 business questions against a SQLite shop database: GPT-6.1 Sol better (pass 100% vs 100%)
• Answer 8 questions from 12 conflicting dated sources with correct citations: Claude Sonnet 5.5 better (pass 100% vs 0%)
• Assign a community transport scheme's volunteer driver trips when one driver's background-check status is blank: GPT-6.1 Sol better (pass 100% vs 100%)
• Audit a full disk without touching it, then hand over an approve-before-delete cleanup script: GPT-6.1 Sol better (pass 100% vs 100%)
• Backtest a moving-average crossover on a bundled synthetic price file: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Berth occupancy point and range-maximum queries at scale: GPT-6.1 Sol better (pass 100% vs 100%)
• Best fishing windows from tide, solunar and weather data: GPT-6.1 Sol better (pass 100% vs 100%)
• Best sailing angle and leg times from a polar table: GPT-6.1 Sol better (pass 100% vs 100%)
• Bill a hotel's group bookings across a mid-month rate rise, weekend nights, cancellations with time-zone offsets and a city-tax cap: GPT-6.1 Sol better (pass 100% vs 100%)
• Bisect a 42-commit local git history to the first bad commit, then fix forward with a regression test: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Boat fuel range with a reserve over a planned trip: GPT-6.1 Sol better (pass 100% vs 100%)
• Book a primary school's trip places when one trip is to a venue missing from the approved list: GPT-6.1 Sol better (pass 0% vs 100%)
• Bouncing logo screensaver in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Build a csvq CLI (select/where/sort/limit/count, csv|json|table output, exact exit codes) from a written spec: GPT-6.1 Sol better (pass 100% vs 100%)
• Build a TV home screen that shows a smart home's device status instead of ads: GPT-6.1 Sol better (pass 100% vs 100%)
• Carry out device tweaks but stop and ask before anything irreversible or out of scope: GPT-6.1 Sol better (pass 100% vs 100%)
• Catch log checked against a made-up fishing regulations table: GPT-6.1 Sol better (pass 100% vs 100%)
• Charge residents' parking permits by emission band, part year, household order and blue-badge exemption: GPT-6.1 Sol better (pass 100% vs 100%)
• Check and pay staff mileage claims across the April tax-year change: GPT-6.1 Sol better (pass 100% vs 100%)
• Classify 12 faulty sensors (frozen/°F/spike/gap/drift) with fault start hours: GPT-6.1 Sol better (pass 100% vs 100%)
• Compile downloaded DNS blocklists for a home DNS sinkhole (hosts, adblock and plain formats, IDN, allow exceptions, minimal output): GPT-6.1 Sol better (pass 100% vs 100%)
• Consistent-hash ring with weighted virtual nodes and zone-aware replica placement: Claude Sonnet 5.5 better (pass 100% vs 100%)
• De-duplicate three copy-pasted report renderers behind one row pipeline and add a Markdown renderer, byte-identical output: GPT-6.1 Sol better (pass 100% vs 100%)
• Dead reckoning, estimated position with current, and computed set and drift: GPT-6.1 Sol better (pass 100% vs 100%)
• Debloat your own Android TV over ADB with a backup and a one-command undo: GPT-6.1 Sol better (pass 100% vs 100%)
• Debt avalanche vs snowball schedule to the cent: GPT-6.1 Sol better (pass 100% vs 100%)
• Discounted cash flow valuation with mid-year option and a sensitivity grid: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Diverging stacked bar chart with exact SVG geometry: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Draw a ticking, styled analog clock that shows the current time in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Draw a walking pixel-art character sprite animation in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Earliest-arrival journey planner over a timetable (minimum change times, directed footpaths never chained, trip limit, after-midnight times): Claude Sonnet 5.5 better (pass 100% vs 100%)
• Enforce stock, totals, status-transition and audit invariants purely with SQLite triggers: GPT-6.1 Sol better (pass 100% vs 100%)
• Entity resolution on ~950 merged customer records (nicknames, swaps, aliases, phone formats) without merging households: GPT-6.1 Sol better (pass 0% vs 100%)
• Extract 8 fields from 24 invoices in 4 formats (EU numbers, credit notes, MM/DD dates): GPT-6.1 Sol better (pass 100% vs 100%)
• Final standings with tie-breaks for a club Swiss tournament (Buchholz Cut-1 with virtual opponents, Sonneborn-Berger, direct encounter): GPT-6.1 Sol better (pass 100% vs 100%)
• Find substantive changes hidden in a 'minor clean-up' contract redline: GPT-6.1 Sol better (pass 100% vs 100%)
• Find your own listings on people-search pages and draft opt-out requests for approval: GPT-6.1 Sol better (pass 100% vs 100%)
• Finish a JSON Schema subset validator (types, JSON equality, pointer-escaped paths, $ref recursion, combinators): GPT-6.1 Sol better (pass 100% vs 100%)
• Fix 14 WCAG issues in a static HTML page without losing content: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix 8 semantic bugs in a hand-rolled Node.js event emitter (re-entrancy, once/off, error events, leak warning): GPT-6.1 Sol better (pass 100% vs 100%)
• Fix a cron next-fire-time calculator (steps, names, macros, day-of-month OR day-of-week, leap days, strict validation): GPT-6.1 Sol better (pass 100% vs 100%)
• Fix a flaky test suite caused by set ordering, ignored date parameter and global random state - without touching the tests: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix a lock-ordering deadlock, a lost-update race and torn snapshots in a threaded bank ledger: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Fix a multi-currency brokerage ledger codebase with interacting planted bugs: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix a Node.js circuit breaker state machine (rolling window, half-open concurrency, timeouts, exact transition events): GPT-6.1 Sol better (pass 100% vs 100%)
• Fix a spreadsheet formula engine (precedence, error values, static cycle detection, 5,000-cell chains without recursion errors): GPT-6.1 Sol better (pass 100% vs 100%)
• Fix an async job pool (retries, deadlock, error capture) with hidden tests: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix cache invalidation in an incremental static-site builder (transitive partials, config, deletions, same-second edits): Claude Sonnet 5.5 better (pass 100% vs 100%)
• Fix catastrophic-backtracking (ReDoS) regexes in signup validators without changing what they accept: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix five behavioural bugs in an LRU cache with per-entry TTL: GPT-6.1 Sol better (pass 100% vs 100%)
• Fix keyset pagination of a SQLite feed (ties, filters before paging, exact next-cursor, rows changing between pages, cursor validation): GPT-6.1 Sol better (pass 100% vs 100%)
• Go/no-go departure time from a gridded wind file: GPT-6.1 Sol better (pass 100% vs 100%)
• Great-circle vs rhumb-line passage plan with ETAs: GPT-6.1 Sol better (pass 100% vs 100%)
• Hourly kWh from raw cumulative meter readings (mixed time zones, duplicates, counter rollover, resets, >6 h gaps, linear interpolation): GPT-6.1 Sol better (pass 100% vs 100%)
• iCalendar recurrence expansion with time-zone transitions, exceptions and overrides: GPT-6.1 Sol better (pass 100% vs 100%)
• Implement a 10-rule config-file parser (quotes, continuations, interpolation, cycles) from a written spec: Claude Sonnet 5.5 better (pass 100% vs 0%)
• Implement a reader/writer for a documented chunked binary container (LEB128 lengths, CRC-32, zlib chunks, critical vs ancillary): GPT-6.1 Sol better (pass 100% vs 100%)
• Implement npm-style semver range matching (caret/tilde/x/hyphen, prerelease rule) from a written spec: GPT-6.1 Sol better (pass 100% vs 100%)
• Let a town's allotment plots when one tenant still has a rent-arrears row this year: tie (pass 0% vs 0%)
• Life-like automaton from RLE with a Klein-bottle grid and a playable animated page: GPT-6.1 Sol better (pass 100% vs 100%)
• Lighthouse with a sweeping beam in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Limit order book matching engine with self-trade prevention, FOK and modify priority: GPT-6.1 Sol better (pass 100% vs 100%)
• Loan servicing with daily accrual, 30/360 quirks, dues crediting and a payment waterfall: GPT-6.1 Sol better (pass 100% vs 100%)
• Looping fireworks show in one self-contained HTML file: GPT-6.1 Sol better (pass 100% vs 100%)
• Make a Node.js JSON Patch module correct (no input mutation, pointer escaping, array index rules, move-into-child, deep test, error codes): GPT-6.1 Sol better (pass 100% vs 100%)
• Make a stdlib Python service pass its healthcheck (4 layered config bugs): GPT-6.1 Sol better (pass 100% vs 100%)
• Make a unified-diff patch applier correct (offset search, no-newline markers, create/delete, atomic failure): GPT-6.1 Sol better (pass 100% vs 100%)
• Make an ignore-file pattern matcher follow its spec (anchoring, directory-only, double-star, negation, parent exclusion): GPT-6.1 Sol better (pass 100% vs 100%)
• Make invoice maths exact: half-up line rounding, per-rate VAT, largest-remainder allocation, banker's FX rounding: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Migrate a 3-file Node.js module from callbacks to async/await + fix a validation bug: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Model a game-ready low-poly 3D prop from a one-line brief, within a tight triangle budget: GPT-6.1 Sol better (pass 100% vs 100%)
• Monthly cohort retention + revenue from a messy orders CSV: GPT-6.1 Sol better (pass 100% vs 100%)
• Monthly FIFO inventory report (COGS, write-offs, oversell exceptions) from a messy, shuffled stock-movement export: GPT-6.1 Sol better (pass 100% vs 100%)
• Monthly regional report from messy orders CSV + shipments JSONL + weekday-only FX table: GPT-6.1 Sol better (pass 100% vs 100%)
• Mortgage amortization with extra monthly payments and lump sums: GPT-6.1 Sol better (pass 100% vs 100%)
• Multi-window burn-rate alerts from per-minute request counts (integer thresholds, flap smoothing, page-over-ticket priority): GPT-6.1 Sol better (pass 100% vs 100%)
• Optimal single-machine job sequence (weighted tardiness with precedence): GPT-6.1 Sol better (pass 100% vs 100%)
• Optimise an O(n·m) price lookup to <4s on 200k orders with byte-identical output: GPT-6.1 Sol better (pass 100% vs 100%)
• Parse 300 transaction lines in 6 formats (EU numbers, 12h times, minor units, 'Last, First'): GPT-6.1 Sol better (pass 100% vs 100%)
• Pension claiming-age comparison: lifetime totals, present values and break-even ages: GPT-6.1 Sol better (pass 100% vs 100%)
• Plan the startup order of a self-hosted container stack (conditions, profiles, optional and disabled services, error precedence, cycles): Claude Sonnet 5.5 better (pass 100% vs 0%)
• Progressive tax on fictional brackets, and pre-tax vs after-tax savings comparison: GPT-6.1 Sol better (pass 100% vs 100%)
• Q1 budget-vs-actual variance table and CFO memo from a raw ledger (out-of-period lines, intercompany, legacy codes, reversals): Claude Sonnet 5.5 better (pass 100% vs 100%)
• Reach a goal stack configuration using a simulator CLI with minimal moves: GPT-6.1 Sol better (pass 0% vs 100%)
• Read a synthetic annual-report excerpt and compute financial ratios: GPT-6.1 Sol better (pass 100% vs 100%)
• Rebalance to target weights with whole shares and highest-cost-first tax lots: GPT-6.1 Sol better (pass 100% vs 100%)
• Reconcile bank payments against open invoices (memo references, partial payments, exact-amount and 2-3 invoice combinations, ambiguity): Claude Sonnet 5.5 better (pass 100% vs 100%)
• Refactor a legacy freight tariff into structured output while preserving its quirks for old orders: GPT-6.1 Sol better (pass 100% vs 100%)
• Refactor nested legacy pricing code + add a bulk-discount rule with exact behaviour preservation: GPT-6.1 Sol better (pass 100% vs 100%)
• Required minimum distributions from a supplied divisor table, with and without delaying the first one: GPT-6.1 Sol better (pass 100% vs 100%)
• Resolve package versions with a specified backtracking procedure (caret and OR constraints, yanked versions, numeric ordering): GPT-6.1 Sol better (pass 100% vs 100%)
• Retirement drawdown with a 4% rule, inflation raises and a skip-raise-after-loss guardrail: GPT-6.1 Sol better (pass 100% vs 100%)
• Rewrite a legalese policy in plain language without losing or inventing rules: GPT-6.1 Sol better (pass 100% vs 100%)
• Root-cause a checkout 500 spike from nginx/app/deploy logs with red herrings: GPT-6.1 Sol better (pass 100% vs 100%)
• Sailboat instrument dashboard from an NMEA 0183 log: GPT-6.1 Sol better (pass 100% vs 100%)
• Sailboat riding animated waves in one self-contained HTML file: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Sailing regatta series scoring with codes, dead heats, redress, discards and tie-breaks: GPT-6.1 Sol better (pass 100% vs 100%)
• Schedule a 40-task build DAG on 3 workers with exclusive resources: GPT-6.1 Sol better (pass 100% vs 100%)
• Scrub personal data and secrets from log lines before sharing (tokens, URL secrets, emails, IPv6/IPv4, Luhn-checked card numbers): GPT-6.1 Sol better (pass 100% vs 100%)
• Seeded Monte Carlo retirement odds against a stated success threshold: GPT-6.1 Sol better (pass 100% vs 100%)
• Ship settings format v3 that deployed v2 readers can still read (forward/backward compatibility, atomic save, idempotent migration): GPT-6.1 Sol better (pass 100% vs 100%)
• Spreadsheet-grade loan amortization for 8 loans incl. rate reset and extra payments: GPT-6.1 Sol better (pass 100% vs 100%)
• Tide height by rule of twelfths and cosine, with under-keel clearance windows: GPT-6.1 Sol better (pass 100% vs 100%)
• Tide prediction from harmonic constituents: extrema, wiggle removal, threshold crossings: GPT-6.1 Sol better (pass 100% vs 100%)
• Triage a work inbox into read-only summaries and reply drafts, sending nothing: GPT-6.1 Sol better (pass 100% vs 100%)
• Turn a meeting transcript into tracked action items (owners, relative due dates, reassignments, cancellations) plus a 120-word summary: GPT-6.1 Sol better (pass 100% vs 100%)
• Turn a minute-by-minute common-free-time scan into a sweep that handles 3,000 calendars in <3 s with identical output: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Turn weekly ad-platform exports into an exact wasted-spend report: GPT-6.1 Sol better (pass 100% vs 100%)
• Undo/redo with typing coalescing for a text buffer (word boundaries, 2-second gap, backspace vs forward-delete groups, history cap): Claude Sonnet 5.5 better (pass 100% vs 100%)
• Unicode-correct username canonicalisation (NFKC, casefold, mixed-script), terminal width, cluster-safe truncation and slugs: GPT-6.1 Sol better (pass 100% vs 100%)
• Water-quality sensor anomaly flags and daily means: GPT-6.1 Sol better (pass 100% vs 100%)
• Which tunnel peer gets this packet? Parse a VPN peer config, longest-prefix routing, ownership moves and findings: Claude Sonnet 5.5 better (pass 100% vs 100%)
• Work backward from a retirement budget to a savings target, contributions and earliest retirement age: GPT-6.1 Sol better (pass 100% vs 100%)
• Work out a care home's fortnightly pay from shifts that span the night the clocks go back: GPT-6.1 Sol better (pass 100% vs 100%)
• Work out a library's overdue fines from loans, renewals, closed days and member categories: GPT-6.1 Sol better (pass 100% vs 100%)
• Work out late-payment interest and compensation from part payments, credit notes and a base-rate change: GPT-6.1 Sol better (pass 100% vs 100%)
• Write a blameless postmortem from a timeline + chat export (automated rubric): GPT-6.1 Sol better (pass 100% vs 100%)
• Write a lexer, parser and evaluator for a small closure-based language from a 1-page spec: GPT-6.1 Sol better (pass 100% vs 100%)
• Write a pytest suite for a money module, graded by killing 14 mutants: Claude Sonnet 5.5 better (pass 100% vs 0%)
• Write a robust backup rotation script (dry-run, spaces, exit codes, safe deletion): Claude Sonnet 5.5 better (pass 100% vs 0%)
• Write an atomic, idempotent SQLite v1→v2 migration with dedupe, FK and CHECK constraints: GPT-6.1 Sol better (pass 100% vs 100%)
• Write release notes from a conventional-commit log (breaking changes, reverts, cherry-pick duplicates, semver bump): GPT-6.1 Sol better (pass 100% vs 100%)

Only use cases both models ran are compared; all settings of each model are grouped. How we score.