Skip to content

Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on iCalendar recurrence expansion with time-zone transitions, exceptions and overrides

AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomouspartial · 0 retries
Estimated cost
$0.05
Wall time
4m
Human step-ins
0
Outcome
Partial

Compare with other models on this task →

Summary

Exact prompt

# Calendar (iCalendar subset) recurrence expansion `input.json` names `calendar.ics` and gives `timezones` (each with `initial_offset_min` and `transitions` [[UTC instant, new offset minutes], ...]; offset = local minus UTC) and a UTC window `window_start`..`window_end`. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json`. Fresh calendars are generated. Everything you need is below. ## Parsing 1. Lines end in CRLF (accept LF too). A line starting with a space or tab continues the previous line (drop the line break and that one character). Blank lines are ignored. 2. Content line: `NAME *(;PARAM=value) : value`. Names and parameter names are case-insensitive. Parameter values may be double-quoted (quotes removed; `;` and `:` inside quotes are literal). The value starts after the first `:` that is not inside quotes. 3. Only `VEVENT` components count (`events_parsed` = number of VEVENT blocks, overrides included). Nested components (e.g. `VALARM`) inside a VEVENT are skipped entirely, properties included. For repeated properties, the first wins, except that `EXDATE` and `RDATE` accumulate (comma lists allowed). 4. Text unescaping (SUMMARY): `\\` -> `\`, `\;` -> `;`, `\,` -> `,`, `\n` or `\N` -> newline. 5. Date-times: `YYYYMMDDTHHMMSSZ` is UTC; `YYYYMMDDTHHMMSS` with `TZID=` is local time in that zone (no TZID: treat as UTC); `VALUE=DATE` or an 8-digit value is an all-day date. 6. Local -> UTC: find each interval of the zone's offset table whose offset o makes local - o fall inside that interval. If several do (repeated hour), take the earliest UTC instant. If none do (skipped hour), use local - (offset in force before that transition). 7. Length: `DTEND` - `DTSTART` (both converted to UTC; dates for all-day), or `DURATION` (`P[nW][nD][T[nH][nM][nS]]`), else 0 for timed events and 1 day for all-day ones. ## Recurrence (RRULE: FREQ, INTERVAL, COUNT, UNTIL, BYDAY, BYMONTHDAY, BYMONTH) 8. Expand in **local wall time** (the DTSTART's zone) and keep DTSTART's time of day; convert each occurrence to UTC with rule 6. 9. DAILY: every INTERVAL days; BYMONTH, BYMONTHDAY and BYDAY filter. WEEKLY: weeks start Monday; every INTERVAL weeks from DTSTART's week; BYDAY lists the days (default: DTSTART's weekday); BYMONTH/BYMONTHDAY filter. MONTHLY: every INTERVAL months; days = BYMONTHDAY (negative counts from month end; days that don't exist are skipped) and/or BYDAY (`MO` = every Monday, `2TU` = second Tuesday, `-1FR` = last Friday). If both are given, take the intersection; if neither, use DTSTART's day of month (skip months without it). YEARLY: every INTERVAL years; months = BYMONTH (default DTSTART's month), with days chosen as in MONTHLY. 10. Candidates before DTSTART are dropped; DTSTART itself is an occurrence only if it matches the rule. Occurrences are generated in time order. COUNT counts generated occurrences **before** EXDATE removal. UNTIL is inclusive and compared in UTC (a date-only UNTIL on a timed event means 23:59:59 UTC that day; on all-day events compare dates). 11. Add RDATEs, then remove EXDATEs (matched as UTC instants or dates). 12. Overrides: a VEVENT with `RECURRENCE-ID` and the same UID replaces the occurrence whose UTC start equals it: it uses its own DTSTART, length and SUMMARY, and has `override: true`. `STATUS:CANCELLED` removes that occurrence. 13. An event without RRULE has one occurrence (plus RDATEs). ## Output Occurrences overlapping the window: timed events with start < window_end and end > window_start (zero-length: start in [window_start, window_end)); all-day events compare dates the same way. Each is `{"uid", "summary", "start", "end", "all_day", "override"}` with `YYYY-MM-DDTHH:MM:SSZ` (UTC) or `YYYY-MM-DD` (all-day). Sort by start (all-day dates count as 00:00:00Z), then uid. `answer.json`: `{"occurrences": [...], "count": n, "events_parsed": n}` ## Scoring Hidden checks (partial credit; all must pass; exact): `unfold_unescape_parse`, `weekly_tz_exdate_override`, `monthly_byday_bymonthday`, `all_day_and_yearly`, `daily_until_rdate_cancel`, `dst_gap_overlap_zero_length`, `full_sorted_list` on the given calendar; `fresh_run_1`, `fresh_run_2`. Per-family checks compare the occurrences whose summary starts with that family's text, in order. 30-second limit per run.

Wall-clock expansion across DST with skipped and repeated hours, COUNT before EXDATE, negative BYMONTHDAY and -1FR, inclusive UNTIL, and overrides keyed on UTC RECURRENCE-ID. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. unfold_unescape_parse 2. weekly_tz_exdate_override 3. monthly_byday_bymonthday 4. all_day_and_yearly 5. daily_until_rdate_cancel 6. dst_gap_overlap_zero_length 7. full_sorted_list 8. fresh_run_1 9. fresh_run_2

Setup notes: Use Case Scout batch 12 case m10-ics-recurrence-expansion (offline, synthetic fixture). Prepared with batch12/prepare.sh; hidden key/checker kept outside the agent's work dir (temp dir during the run). Prompt = TASK.md verbatim. Scored with the case's scorer: `python score.py WORKDIR KEYDIR` (python3, run from the case dir). Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. 1-min load average at scoring 0.13 (8 cores). Claude Code's own reported cost for claude-haiku-5.5: $0.051793. Score: 8/9. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Metrics

Estimated cost
$0.05
Wall time
4m
Human step-ins
0
Turns
8
Model usage
ModelInput tokOutput tokCost
AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on514,96962,434$0.05

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
  3. It counts as a success only if it meets every success criterion: 1. unfold_unescape_parse 2. weekly_tz_exdate_override 3. monthly_byday_bymonthday 4. all_day_and_yearly 5. daily_until_rdate_cancel 6. dst_gap_overlap_zero_length 7. full_sorted_list 8. fresh_run_1 9. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
[
 {
  "duration_api_ms": 258303,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.05179347,
  "usage": {
   "input_tokens": 16,
   "cache_creation_input_tokens": 81186,
   "cache_read_input_tokens": 433767,
   "output_tokens": 62434,
   "output_tokens_details": {
    "thinking_tokens": 52258
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 81186,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 772,
     "cache_read_input_tokens": 93512,
     "cache_creation_input_tokens": 213,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 213
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-haiku-5-5": {
    "inputTokens": 16,
    "outputTokens": 62434,
    "cacheReadInputTokens": 433767,
    "cacheCreationInputTokens": 81186,
    "webSearchRequests": 0,
    "costUSD": 0.05179347,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 52258,
    "canonicalModel": "claude-haiku-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 8,
  "subtype": "success",
  "api_error_status": null,
  "result": "I wrote `solve.py` in the work directory. It runs in about 30 ms on the given calendar and produces `answer.json` with 126 occurrences and `events_parsed` = 11. I haven't seen the hidden checks, so the scoring result is unknown.\n\nI checked the output by hand on the tricky cases in the given calendar:\n- **DST gap:** \"Gap test\" at 02:30 West on 2026-03-08 lands at 07:30Z.\n- **Repeated hour:** \"Overlap test\" at 01:30 West on 2026-11-01 takes the earlier instant, 05:30Z.\n- **Cancelled RDATE:** the Nov 11 Central instance is an RDATE that a `STATUS:CANCELLED` override removes. The Dec 1 RDATE stays.\n- **Override:** \"Briefing moved\" replaces the Mar 5 instance and shows `override: true`.\n- **Zero-length:** \"Zero length mark\" is included, and the nested VALARM text does not leak into the summary.\n- **Monthly and yearly rules:** `-1FR`, `2TU`, negative BYMONTHDAY, and skipped short months all behave as specified.\n\nI also ran unit checks on the parser and recurrence helpers, covering quoted parameters with `;` and `:`, unfolding, durations, and the monthly and yearly rule variants. All matched hand-computed values.\n\nChoices the spec didn't settle:\n- **Unmatched override:** an override whose RECURRENCE-ID matches no occurrence is inserted as its own occurrence. It is not dropped. The given calendar has no such case.\n- **Override of an EXDATE'd instance:** the override is applied after EXDATE removal, so it reinstates that instance. This follows from the step order in the spec.\n- **All-day `end`:** it is exclusive (start plus length), the usual iCalendar convention.\n- **BYMONTH on MONTHLY:** it is applied as a filter. The spec only lists it for DAILY and WEEKLY.\n- **Unknown TZID:** it is treated as UTC.\n\nBoth the given EXDATEs in \"Weekly crew briefing\" (Apr 13 and May 21) fall outside that event's schedule, so they remove nothing. That is correct under the rule, but worth knowing if the expected output looks different.",
  "ttft_ms": 1464,
  "type": "result",
  "duration_ms": 259040,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1343,
  "time_to_request_ms": 349,
  "first_content_frame_ms": 1343,
  "queued_turn_count": 0,
  "result_index": 0
 }
]

Get new runs and ranking changes weekly