Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) on Fix 14 WCAG issues in a static HTML page without losing content

AClaude Haiku 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-16 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first tryCompare

Estimated cost: $0.0081
Wall time: 37s
Human step-ins: 0
Outcome: Pass

Compare with other models on this task →

Exact prompt

# Task: fix accessibility issues in a static page `index.html` is a public library events page that fails a WCAG 2.2 AA review. Fix it in place (HTML only; you may adjust the inline CSS). Keep all existing visible text content, all links (same hrefs) and the form fields (same `name`s), and keep the page working without JavaScript where possible. Address at least: document language, landmarks (header/nav/main), skip link, image alternatives (informative vs decorative: the divider is decorative, the logo and storytime photo are not), heading hierarchy, ambiguous link text, data table semantics, form labels and grouping, the fake button, iframe title, positive tabindex, focusable content hidden from assistive tech, and the low-contrast `.muted` text (#999 on white). Write `A11Y_NOTES.md` listing each fix (one line each). No network. Note: links must have a descriptive accessible name; a link whose whole name is just "click here", "here", "more" or "read more" fails (visually-hidden extra text or aria-label inside the link counts toward the name).

Static checks: lang, landmarks, skip link, decorative vs informative alt, heading order, link text, th/scope, labels, real button, iframe title, tabindex, aria-hidden focusables, contrast ≥4.5:1, content preserved. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. html has lang starting with 'en'. 2. header, nav and main elements present. 3. First link is a skip link pointing to an existing id. 4. Logo and storytime images have alt text longer than 3 characters; the divider image has alt="". 5. Exactly one h1, first heading is h1, no heading level skips. 6. No link text 'click here'/'here'/'read more'/'more'. 7. At least 3 th cells with scope (or a thead). 8. Every input/select has a label (for/id, wrapping label, aria-label or aria-labelledby). 9. No onclick on non-button elements, and a real submit button exists. 10. Every iframe has a non-empty title. 11. No positive tabindex. 12. No focusable element inside aria-hidden="true". 13. .muted text colour has contrast >= 4.5:1 on white. 14. Original key text, all original hrefs, the form action and the 4 field names are preserved. 15. A11Y_NOTES.md has at least 10 list lines. (A table caption is reported but not required.)

Setup notes: Use Case Scout batch 4 case d06-a11y-static-html (offline fixture). Prepared with batch4/prepare.sh; hidden key/tests kept outside the agent's work dir. Prompt = TASK.md verbatim. Scored with the scout's scorer: `python score.py WORKDIR`. Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. Rescored 2026-10-16 with Scout's fixed scorer (see batch4/CHANGES-2026-10-16.md). The original score was 17/17 checks, and the model's output is unchanged. Claude Code's own reported cost for claude-haiku-5.5: $0.008128. Score: 17/17 checks. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Metrics

Estimated cost: $0.0081
Wall time: 37s
Human step-ins: 0
Turns: 7

Model usage
Model	Input tok	Output tok	Cost
AClaude Haiku 5.5anthropic/claude-haiku-5.5effort default (no --effort flag) · thinking on	164,892	7,400	$0.0081

Reproduce this run

Use the use case prompt on the use case page.
Same setup: Claude Haiku 5.5 (anthropic/claude-haiku-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
It counts as a success only if it meets every success criterion: 1. html has lang starting with 'en'. 2. header, nav and main elements present. 3. First link is a skip link pointing to an existing id. 4. Logo and storytime images have alt text longer than 3 characters; the divider image has alt="". 5. Exactly one h1, first heading is h1, no heading level skips. 6. No link text 'click here'/'here'/'read more'/'more'. 7. At least 3 th cells with scope (or a thead). 8. Every input/select has a label (for/id, wrapping label, aria-label or aria-labelledby). 9. No onclick on non-button elements, and a real submit button exists. 10. Every iframe has a non-empty title. 11. No positive tabindex. 12. No focusable element inside aria-hidden="true". 13. .muted text colour has contrast >= 4.5:1 on white. 14. Original key text, all original hrefs, the form action and the 4 field names are preserved. 15. A11Y_NOTES.md has at least 10 list lines. (A table caption is reported but not required.)
Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Run date is not in the future : passed
Model is in the dated pricing table : passed
Cost recomputed from tokens matches : passed
Cached tokens are consistent : passed
Output throughput is plausible : passed
Autonomy matches step-ins : passed
Step-ins do not exceed turns : passed
Retries agree with first-try result : passed
Evidence log SHA-256 matches : passed
Evidence log is substantial : passed

Evidence log

[
 {
  "duration_api_ms": 35953,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.0081276,
  "usage": {
   "input_tokens": 14,
   "cache_creation_input_tokens": 14618,
   "cache_read_input_tokens": 150260,
   "output_tokens": 7400,
   "output_tokens_details": {
    "thinking_tokens": 3741
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 14618,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 698,
     "cache_read_input_tokens": 26731,
     "cache_creation_input_tokens": 426,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 426
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-haiku-5-5": {
    "inputTokens": 14,
    "outputTokens": 7400,
    "cacheReadInputTokens": 150260,
    "cacheCreationInputTokens": 14618,
    "webSearchRequests": 0,
    "costUSD": 0.0081276,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 3741,
    "canonicalModel": "claude-haiku-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 7,
  "subtype": "success",
  "api_error_status": null,
  "result": "I fixed `index.html` in place and wrote `A11Y_NOTES.md` with one line per fix. I haven't run an automated checker or a screen reader; I checked the changes by reading the markup.\n\nTwo things need your attention:\n- **Storytime photo alt text is a guess.** `storytime.jpg` and `logo.png` aren't in the directory, so I couldn't see them. The photo's alt text reads \"Children and a librarian sharing a picture book at Story Time\". Please replace it with an accurate description.\n- **Two visible additions.** The select had no visible label, so I added \"Event\" as one. I also added a \"Your registration details\" legend that's visually hidden. Everything else keeps its visible text, hrefs and form `name`s.\n\nWhat changed:\n- **Structure:** `lang=\"en\"`, a skip link to `#main`, and real `<header>`, `<nav aria-label=\"Main\">`, `<main>` and `<footer>` landmarks. The accessibility and old-site links moved into the footer.\n- **Images:** the logo has alt text, the divider has `alt=\"\"`, and the photo has descriptive alt text.\n- **Headings:** the two `h3`s became `h2`, so there are no skipped levels.\n- **Links:** both \"Click here\" links keep their visible text and gain visually-hidden text that says where they go.\n- **Table:** added a hidden caption, `thead` with `th scope=\"col\"`, and row headers.\n- **Form:** every control has a `<label>`, the fields sit in a `<fieldset>`, and the name and email inputs have `autocomplete`.\n- **Button:** the `div` with `onclick` is now a real `<button type=\"submit\">`, so the form submits without JavaScript.\n- **Iframe:** has a title.\n- **Focus and hidden content:** removed the positive `tabindex=\"3\"` and the `aria-hidden` on the visible Old site link.\n- **Contrast:** `.muted` is now `#595959` (about 7:1). The button background went from `#2a6` to `#1d6f42` because white on `#2a6` was about 3:1; the new colour is about 6.2:1.",
  "ttft_ms": 1446,
  "type": "result",
  "duration_ms": 36284,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1052,
  "time_to_request_ms": 232,
  "first_content_frame_ms": 1052,
  "queued_turn_count": 0,
  "result_index": 0
 }
]