Skip to content

Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on Great-circle vs rhumb-line passage plan with ETAs

AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.01
Wall time
1m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

# Great-circle vs rhumb-line passage plan with ETAs `input.json` (synthetic; not for navigation): `waypoints` ([lat, lon] decimal degrees, legs may cross the 180° meridian), `speed_kn`, `depart_utc` ("YYYY-MM-DDTHH:MM:SSZ"). Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `route.svg`. Fresh inputs are used too. ## Rules (sphere, R = 3440.065 nm) - Longitude difference dlon is always wrapped into [-180°, 180°). - **Great circle**: haversine distance; initial course = atan2(sin dlon * cos lat2, cos lat1 * sin lat2 - sin lat1 * cos lat2 * cos dlon) mod 360. - **Rhumb line**: dpsi = ln(tan(pi/4 + lat2/2) / tan(pi/4 + lat1/2)); q = dlat / dpsi, or cos(lat1) when |dpsi| <= 1e-12; distance = sqrt(dlat^2 + q^2 * dlon^2) * R (radians); course = atan2(dlon, dpsi) mod 360. - The boat sails the **rhumb** legs at `speed_kn`; leg ETA = departure + cumulative rhumb distance / speed, rounded to the nearest minute (half a minute rounds up), formatted "YYYY-MM-DDTHH:MMZ". - `rhumb_extra_pct` = (total rhumb - total GC) / total GC * 100. Round distances to 2 decimals, courses to 1, the percentage to 3. ## Visual output: route.svg Static, self-contained SVG (root `<svg>`, no scripts, images, links or external assets; renders headless) with `<polyline id="route-gc">`, `<polyline id="route-rhumb">` and an element `id="waypoints"`. The drawing (projection etc.) is not graded. ## answer.json `{"legs": [{"gc_nm", "gc_initial_course", "rhumb_nm", "rhumb_course", "eta_utc"}, ...], "total_gc_nm", "total_rhumb_nm", "rhumb_extra_pct", "arrival_utc"}` ## Scoring Hidden checks (partial credit; all must pass): `great_circle_legs`, `rhumb_legs` (first four legs; distances within 0.011, courses within 0.11), `totals`, `etas` (exact strings), `route_svg_static` (route.svg exists, follows the static rules, has the ids/tags above), `fresh_run_1`, `fresh_run_2` (all keys, within 0.02). 60-second limit per run.

Antimeridian wrap on both methods, the due east/west rhumb leg where dpsi is zero, initial great-circle course, and minute-rounded UTC ETAs that drift if per-leg rounding is used. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Benchmark code for comparing AI models, not navigation or safety software.

Success criteria: 1. great_circle_legs 2. rhumb_legs 3. totals 4. etas 5. route_svg_static 6. fresh_run_1 7. fresh_run_2

Setup notes: Use Case Scout batch 10 case k09-passage-gc-vs-rhumb (offline fixture). Prepared with batch10/prepare.sh; hidden key/tests kept outside the agent's work dir. Prompt = TASK.md verbatim. Scored with the scout's scorer: `python score.py WORKDIR KEYDIR`. Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. Visual output (route.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Claude Code's own reported cost for claude-haiku-5.5: $0.013915. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.01
Wall time
1m
Human step-ins
0
Turns
9
Model usage
ModelInput tokOutput tokCost
AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on261,18013,900$0.01

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
  3. It counts as a success only if it meets every success criterion: 1. great_circle_legs 2. rhumb_legs 3. totals 4. etas 5. route_svg_static 6. fresh_run_1 7. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
[
 {
  "duration_api_ms": 59248,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.01391537,
  "usage": {
   "input_tokens": 18,
   "cache_creation_input_tokens": 22905,
   "cache_read_input_tokens": 238257,
   "output_tokens": 13900,
   "output_tokens_details": {
    "thinking_tokens": 5464
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 22905,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 636,
     "cache_read_input_tokens": 34971,
     "cache_creation_input_tokens": 473,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 473
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-haiku-5-5": {
    "inputTokens": 18,
    "outputTokens": 13900,
    "cacheReadInputTokens": 238257,
    "cacheCreationInputTokens": 22905,
    "webSearchRequests": 0,
    "costUSD": 0.01391537,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 5464,
    "canonicalModel": "claude-haiku-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 9,
  "subtype": "success",
  "api_error_status": null,
  "result": "`solve.py` is written and runs under the 60-second limit with the standard library only. It produces `answer.json` and `route.svg` in the work directory for the provided input. Arrival is `2026-12-01T04:11Z`, and the rhumb route is 0.028% longer than the great circle.\n\n**Results for `input.json`:** total great circle 2105.17 nm, total rhumb 2105.77 nm.\n\n**Checks I ran:**\n- Great-circle distances match an independent spherical law-of-cosines calculation to four decimals.\n- No ETA falls near a half-minute tie, so the rounding rule can't flip a result.\n- Edge-case input (east-west leg, zero-length leg, antimeridian crossings) runs cleanly. The SVG parses as well-formed XML and contains no script, image, or link elements.\n\n**Choices I made where the spec was open:**\n- **ETA uses unrounded cumulative distance.** I divide the unrounded rhumb distance by speed, then round the elapsed time to the nearest minute. The spec doesn't say whether to round the distance first. None of the current legs are close enough for this to matter.\n- **Totals and the percentage use unrounded sums.** Rounding happens only on output.\n- **`route.svg` goes next to `answer.json`.** When run as `python solve.py input.json answer.json` from the work directory, that is the same directory.\n- **Course normalisation.** Courses are mapped into [0, 360), and values that round up to 360 are set to 0.\n- **Drawing.** The SVG uses a Mercator projection with unwrapped longitudes, so the route stays continuous across the 180\u00b0 meridian. The spec doesn't grade the drawing.\n\nFiles are in `./usecase-runner/runs/k09-claude-haiku/work/`: `solve.py`, `answer.json`, `route.svg`. The work directory isn't a git repo, so I did not commit anything.",
  "ttft_ms": 1307,
  "type": "result",
  "duration_ms": 59860,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1202,
  "time_to_request_ms": 253,
  "first_content_frame_ms": 1202,
  "queued_turn_count": 0,
  "result_index": 0
 }
]

Get new runs and ranking changes weekly