Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on Berth occupancy point and range-maximum queries at scale

AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first tryCompare

Estimated cost: $0.0072
Wall time: 43s
Human step-ins: 0
Outcome: Pass

Compare with other models on this task →

Exact prompt

# Marina berth occupancy queries at scale `input.json` (synthetic) has `bookings` ([start, end) integer minute pairs), `points` (integer times) and `ranges` ([l, r) pairs). Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json`. One hidden input has **150,000 bookings and about 300,000 queries and must finish within 20 seconds** (other runs: 30 s). Answers must be exact. ## Rules 1. A booking with start > end is invalid: count it in `invalid_bookings` and ignore it. A booking with start == end is ignored (not invalid). Duplicate bookings count separately. 2. Occupancy at integer time t = number of valid bookings with start <= t < end. 3. `points`: occupancy at each time, in input order. 4. `ranges`: for each [l, r): if l >= r the answer is null; otherwise the maximum occupancy over all integer t with l <= t < r. In input order. 5. `peak` = maximum occupancy over all times (0 if there are no valid bookings); `peak_first_time` = the earliest time it is reached (null if peak is 0). ## answer.json `{"invalid_bookings", "peak", "peak_first_time", "points": [...], "ranges": [...]}` ## Scoring Hidden checks (partial credit; all must pass; all exact): `invalid_and_peak`, `point_queries`, `range_max_queries` on the given input; `fresh_run_1`, `fresh_run_2`; `big_150k_bookings_300k_queries_20s`.

The rules are simple and a brute-force answer is correct on small inputs, so separation comes from choosing a sweep plus range-maximum structure and getting the half-open boundaries right. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. invalid_and_peak 2. point_queries 3. range_max_queries 4. fresh_run_1 5. fresh_run_2 6. big_150k_bookings_300k_queries_20s

Setup notes: Use Case Scout batch 12 case m08-berth-occupancy-queries (offline, synthetic fixture). Prepared with batch12/prepare.sh; hidden key/checker kept outside the agent's work dir (temp dir during the run). Prompt = TASK.md verbatim. Scored with the case's scorer: `python score.py WORKDIR KEYDIR` (python3, run from the case dir). Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. 1-min load average at scoring 0.52 (8 cores). Claude Code's own reported cost for claude-haiku-5.5: $0.00724. Score: 6/6. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Metrics

Estimated cost: $0.0072
Wall time: 43s
Human step-ins: 0
Turns: 7

Model usage
Model	Input tok	Output tok	Cost
AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on	164,388	5,991	$0.0072

Reproduce this run

Use the use case prompt on the use case page.
Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
It counts as a success only if it meets every success criterion: 1. invalid_and_peak 2. point_queries 3. range_max_queries 4. fresh_run_1 5. fresh_run_2 6. big_150k_bookings_300k_queries_20s
Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Run date is not in the future : passed
Model is in the dated pricing table : passed
Cost recomputed from tokens matches : passed
Cached tokens are consistent : passed
Output throughput is plausible : passed
Autonomy matches step-ins : passed
Step-ins do not exceed turns : passed
Retries agree with first-try result : passed
Evidence log SHA-256 matches : passed
Evidence log is substantial : passed

Evidence log

[
 {
  "duration_api_ms": 31553,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.00724003,
  "usage": {
   "input_tokens": 14,
   "cache_creation_input_tokens": 13681,
   "cache_read_input_tokens": 150693,
   "output_tokens": 5991,
   "output_tokens_details": {
    "thinking_tokens": 2236
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 13681,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 453,
     "cache_read_input_tokens": 25848,
     "cache_creation_input_tokens": 372,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 372
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-haiku-5-5": {
    "inputTokens": 14,
    "outputTokens": 5991,
    "cacheReadInputTokens": 150693,
    "cacheCreationInputTokens": 13681,
    "webSearchRequests": 0,
    "costUSD": 0.00724003,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 2236,
    "canonicalModel": "claude-haiku-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 7,
  "subtype": "success",
  "api_error_status": null,
  "result": "I wrote `solve.py` in the work directory. It matches a brute-force reference on 400 random small cases and on the given `input.json`. On a 150,000-booking, 300,000-query stress input it runs in about 1.3 s, well under the 20 s limit.\n\n**Approach**\n- Occupancy only changes at booking start and end times. I build a sorted list of those breakpoints and the occupancy value on each segment between them, in one sweep.\n- Each point query is a binary search into the breakpoints.\n- Each range-max query maps to a contiguous run of segments, answered in O(1) with a sparse table.\n- `peak` is the maximum segment value. `peak_first_time` is the start of the first segment that reaches it.\n- Invalid bookings (start > end) are counted and skipped. Zero-length bookings are skipped without being counted as invalid. Duplicates are counted separately.\n- Ranges with l >= r return `null`.\n\nI also ran it on the given input, which produced `answer.json` in the same directory: 3 invalid bookings, peak 22, first reached at time 3286, with 240 point answers and 201 range answers.\n\nThe hidden checks themselves are not available here, so the 20 s and exactness results are from my own tests. The stress input is synthetic and my random generator is not the hidden one.\n\nNo git commit was made, since the repo isn't under git and you didn't ask for one.",
  "ttft_ms": 9286,
  "type": "result",
  "duration_ms": 42021,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1392,
  "time_to_request_ms": 254,
  "first_content_frame_ms": 1393,
  "queued_turn_count": 0,
  "result_index": 0
 }
]