Claude Opus 5.5 (effort default (no --effort flag) · thinking on) on Sailing regatta series scoring with codes, dead heats, redress, discards and tie-breaks

AClaude Opus 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first trySupersededCompare

Estimated cost: $0.20
Wall time: 41s
Human step-ins: 0
Outcome: Pass

Compare with other models on this task →

Superseded. This run was rescored on 2026-10-20 and replaced by a newer run. It no longer counts toward rankings, use-case tables or the data export; it is kept here for the record.

Exact prompt

# Sailing regatta series scoring (low-point system) `input.json` (synthetic) has `entries` (`sail`), `races` (each with `finishes` in finishing order), `discards` ({"min races": discards}), and `penalty_pct`. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `results.html`. Fresh series are generated. ## Rules (N = number of entries) 1. A finish item is `{"sail": s}` or `{"sail": [s1, s2, ...]}` (a dead heat), with optional `code` (case-insensitive) and `points`. Sails not in `entries` are ignored. A boat already seen in that race is ignored (only its first item counts). 2. Codes DNC, DNS, OCS, DNF, RET, DSQ, DNE, UFD, BFD score **N + 1** and do **not** take a finishing place (boats behind move up). 3. Other boats take places 1, 2, 3, ... in item order; a dead heat of k boats takes the next k places and each scores their average (e.g. 2.5). 4. ZFP and SCP: place points + round_half_up(N * penalty_pct / 100), capped at N + 1. DPI: place points + `points`, capped at N + 1. 5. RDG with `points`: that score. RDG without `points`: the average of that boat's scores in all other races, excluding its other averaged-RDG races, rounded half-up to 1 decimal; N + 1 if there are none. 6. An entry missing from a race's items scores N + 1 with code DNC. A boat's code for a race is the upper-cased code (`""` if none). 7. Discards: d = the value for the largest key k with races sailed >= k (0 if none). Exclude the boat's d worst scores, never a DNE score; among equal scores exclude the later race first. `total` = sum of all scores; `net` = total - excluded. 8. Ranking: lower net first. Ties: (A8.1) compare each boat's non-excluded scores sorted best to worst, first difference wins; (A8.2) then compare scores in the last race, then the race before, and so on (excluded scores included). Still tied: entry order. ## Visual output: results.html A static, self-contained page (no `<script>`, `<img>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, external URLs, CSS url(http)/@import or fetches; renders headless) with `<table id="results">` and one `<tr id="row-<sail>">` per boat, **in rank order**. Layout and styling are not graded. ## answer.json `{"boats": {sail: {"points": [per race], "codes": [per race], "discarded": [0-based race indices, ascending], "total", "net", "rank"}}, "order": [sails in rank order], "discards_applied": d}` ## Scoring Hidden checks (partial credit; all must pass; numbers within 0.001): `codes_and_dnc`, `race_points_incl_ties_penalties`, `discards_and_totals`, `ranking_with_tiebreaks`, `results_html_static` (exists, static rules, ids, row order) on the given series; `fresh_run_1`, `fresh_run_2`. 30-second limit per run.

Each rule is small, but the tests are layered: one place-counting slip changes every later boat's points, discards and the final order. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. codes_and_dnc 2. race_points_incl_ties_penalties 3. discards_and_totals 4. ranking_with_tiebreaks 5. results_html_static 6. fresh_run_1 7. fresh_run_2

Setup notes: Use Case Scout batch 12 case m06-regatta-scoring (offline, synthetic fixture). Prepared with batch12/prepare.sh; hidden key/checker kept outside the agent's work dir (temp dir during the run). Prompt = TASK.md verbatim. Scored with the case's scorer: `python score.py WORKDIR KEYDIR` (python3, run from the case dir). Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. 1-min load average at scoring 0.83 (8 cores). results.html rendered offline with google-chrome headless (CDP pipe, network blocked): PNG 1280x800 at ~3 s after load; MP4 ~6 s captured at 1280x720, H.264 1280x720. Claude Code's own reported cost for claude-opus-5.5: $0.202424. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost: $0.20
Wall time: 41s
Human step-ins: 0
Turns: 4

Model usage
Model	Input tok	Output tok	Cost
AClaude Opus 5.5anthropic/claude-opus-5.5effort default (no --effort flag) · thinking on	85,528	4,538	$0.20

Reproduce this run

Use the use case prompt on the use case page.
Same setup: Claude Opus 5.5 (anthropic/claude-opus-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
It counts as a success only if it meets every success criterion: 1. codes_and_dnc 2. race_points_incl_ties_penalties 3. discards_and_totals 4. ranking_with_tiebreaks 5. results_html_static 6. fresh_run_1 7. fresh_run_2
Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Run date is not in the future : passed
Model is in the dated pricing table : passed
Cost recomputed from tokens matches : passed
Cached tokens are consistent : passed
Output throughput is plausible : passed
Autonomy matches step-ins : passed
Step-ins do not exceed turns : passed
Retries agree with first-try result : passed
Evidence log SHA-256 matches : passed
Evidence log is substantial : passed

Evidence log

[
 {
  "duration_api_ms": 39674,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.2024242,
  "usage": {
   "input_tokens": 8,
   "cache_creation_input_tokens": 12119,
   "cache_read_input_tokens": 73401,
   "output_tokens": 4538,
   "output_tokens_details": {
    "thinking_tokens": 888
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 12119,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 979,
     "cache_read_input_tokens": 22870,
     "cache_creation_input_tokens": 1395,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 1395
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-opus-5-5": {
    "inputTokens": 8,
    "outputTokens": 4538,
    "cacheReadInputTokens": 73401,
    "cacheCreationInputTokens": 12119,
    "webSearchRequests": 0,
    "costUSD": 0.2024242,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 888,
    "canonicalModel": "claude-opus-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 4,
  "subtype": "success",
  "api_error_status": null,
  "result": "I wrote `solve.py` and ran it on the given series. `python solve.py input.json answer.json` finishes almost instantly and writes `answer.json` and `results.html`. I checked several results by hand against the rules. I didn't test it on generated edge cases (odd dead heats, unknown sails, repeated items), so the two fresh-series checks are untested.\n\n**Results for the given series** (12 boats, 7 races, so 2 discards apply):\n- **Order:** GBR309, FRA91, NZL287, NZL128, USA79, NZL538, GBR719, GBR205, FRA367, GBR651, NZL851, FRA668.\n\n**Hand checks that came out right:**\n- **SCP penalty:** FRA668's SCP in R1 scores 10. The DNS ahead of it doesn't take a place, so it finished 8th, plus a penalty of 20% of 12 = 2.4, rounded to 2.\n- **Averaged RDG:** FRA91's RDG in R7 scores 2.5, the average of its other six races (15 / 6).\n- **DNE never discarded:** NZL851's DNE in R7 stays in its net, and its two worst other races are discarded instead.\n- **Tie on net:** NZL128 and USA79 both have 25.5. NZL128 ranks ahead under A8.1 because its best counting score is 1.5 against USA79's 4.\n\n**How I read the rules where they leave room:**\n- **Dead heats:** sails that aren't entries, or that already appeared in that race, are dropped from the dead heat before places are shared out. Any code on a dead-heat item applies to every boat in it.\n- **RDG:** it takes a finishing place like any other boat, and its score then replaces the place points.\n- **Unrecognised codes:** they score their finishing place.\n- **DPI without `points`:** it gets no extra penalty.\n- **Discards:** if a boat has fewer non-DNE races than the discard count, all of those races are discarded.\n- **`results.html` location:** it goes in the current directory and also next to `answer.json`, if that's a different folder.\n\n`results.html` has no scripts, images or external links. It has `<table id=\"results\">` with one `row-<sail>` row per boat in rank order, and discarded scores shown in brackets.",
  "ttft_ms": 3058,
  "type": "result",
  "duration_ms": 40100,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1125,
  "time_to_request_ms": 237,
  "first_content_frame_ms": 1126,
  "queued_turn_count": 0,
  "result_index": 0
 }
]