Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on Turn a minute-by-minute common-free-time scan into a sweep that handles 3,000 calendars in <3 s with identical output
AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-17 · by @bobsiopener
- Estimated cost
- $0.0049
- Wall time
- 20s
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
# Task: make common_free fast `slots.py:common_free` is correct but far too slow for real calendars (it scans every minute). Rewrite it so that it returns exactly the same result (same list of tuples, same order) for every input, but runs fast: the grader times a call with 3,000 people x 60 busy intervals each over a horizon of 50,000,000 minutes and requires < 3 s wall-clock, and compares your output with the original on many random and edge-case inputs (touching/overlapping/empty intervals, intervals outside the window, min_len boundaries, start == end). Keep the function name and signature. Standard library only. No network.
The rewrite is short, but exact equality on edge cases (empty/inverted intervals, clipping, min_len=0, start==end, touching intervals) is where sloppy sweeps fail. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. output_identical: output equals the original implementation on 11 edge cases and 400 seeded random inputs. 2. fast_enough: one call with 3,000 people x 60 intervals over a 50,000,000-minute horizon takes < 3.0 s (subprocess capped at 30 s; timeout = fail).
Setup notes: Use Case Scout batch 5 case e06-perf-common-free-slots (offline fixture). Prepared with batch5/prepare.sh; hidden key/tests kept outside the agent's work dir. Prompt = TASK.md verbatim. Scored with the scout's scorer: `python score.py WORKDIR KEYDIR`. Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. 1-min load average at scoring: 0.86. Claude Code's own reported cost for claude-haiku-5.5: $0.004862. Score: 2/2 checks. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.
Estimated cost List-price estimate; not actually billed.
Metrics
- Estimated cost
- $0.0049
- Wall time
- 20s
- Human step-ins
- 0
- Turns
- 5
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on | 102,743 | 3,741 | $0.0049 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via
Claude Code 2.1.299 (headless -p), cloud. - It counts as a success only if it meets every success criterion: 1. output_identical: output equals the original implementation on 11 edge cases and 400 seeded random inputs. 2. fast_enough: one call with 3,000 people x 60 intervals over a 50,000,000-minute horizon takes < 3.0 s (subprocess capped at 30 s; timeout = fail).
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
[
{
"duration_api_ms": 18524,
"stop_reason": "end_turn",
"session_id": "[REDACTED]",
"total_cost_usd": 0.00486153,
"usage": {
"input_tokens": 10,
"cache_creation_input_tokens": 10330,
"cache_read_input_tokens": 92403,
"output_tokens": 3741,
"output_tokens_details": {
"thinking_tokens": 1231
},
"server_tool_use": {
"web_search_requests": 0,
"web_fetch_requests": 0
},
"service_tier": "standard",
"cache_creation": {
"ephemeral_1h_input_tokens": 10330,
"ephemeral_5m_input_tokens": 0
},
"inference_geo": "global",
"iterations": [
{
"input_tokens": 2,
"output_tokens": 553,
"cache_read_input_tokens": 21702,
"cache_creation_input_tokens": 1167,
"cache_creation": {
"ephemeral_5m_input_tokens": 0,
"ephemeral_1h_input_tokens": 1167
},
"type": "message"
}
],
"speed": "standard",
"fallback_credit": null
},
"modelUsage": {
"claude-haiku-5-5": {
"inputTokens": 10,
"outputTokens": 3741,
"cacheReadInputTokens": 92403,
"cacheCreationInputTokens": 10330,
"webSearchRequests": 0,
"costUSD": 0.00486153,
"contextWindow": 1000000,
"maxOutputTokens": 128000,
"thinkingTokens": 1231,
"canonicalModel": "claude-haiku-5-5",
"provider": "firstParty",
"costBasis": "list"
}
},
"permission_denials": [],
"terminal_reason": "completed",
"fast_mode_state": "off",
"fast_mode_disabled_reason": "sdk_opt_in_required",
"subagent_stats": {
"spawned": 0,
"requested": {
"background": 0,
"foreground": 0,
"unset": 0
},
"started_in_background": 0,
"max_depth": 0,
"spawned_by_subagents": 0,
"completed": 0,
"failed": 0,
"killed": {
"parent": 0,
"user": 0,
"system": 0
},
"refused": {
"depth_limit": 0,
"concurrency_limit": 0,
"budget": 0
},
"by_type": {}
},
"safety_stops": 0,
"is_error": false,
"num_turns": 5,
"subtype": "success",
"api_error_status": null,
"result": "I rewrote `common_free` in `slots.py` as an interval sweep. It returns the same output as the original on every input I tried, and it runs the grader-sized case in about 0.16 s against the 3 s limit.\n\n**How it works:**\n1. Drop empty intervals (`s >= e`) and clip each busy interval to `[start, end)`.\n2. Sort the clipped intervals and merge any that overlap or touch. Touching intervals leave no free minute between them, so merging them matches the original.\n3. Walk the merged busy intervals and emit each gap that is at least `min_len` long, plus a final gap up to `end`.\n\nThe function name, signature, and return type (a list of tuples) are unchanged. It uses only the standard library.\n\n**Verification:**\n- I ran the original and the new version side by side on 20,000 random inputs (including negative times, empty and out-of-window intervals, `min_len` values of -3, 0, and up to 30, and `start >= end`) plus an exhaustive sweep of single-interval cases. All 105,500 cases matched exactly, including order and tuple types.\n- Benchmark: 3,000 people \u00d7 60 intervals over a 50,000,000-minute horizon with `min_len=30` ran in 0.163 s and returned 54,701 slots.\n\nThe original is saved at `/tmp/cf_ref/slots_orig.py` and the test script at `/tmp/cf_ref/check.py`, outside the work directory. This directory is not a git repository, so I did not commit anything.",
"ttft_ms": 1210,
"type": "result",
"duration_ms": 19712,
"uuid": "[REDACTED]",
"ttft_stream_ms": 1015,
"time_to_request_ms": 211,
"first_content_frame_ms": 1016,
"queued_turn_count": 0,
"result_index": 0
}
]