GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Fix 14 WCAG issues in a static HTML page without losing content
OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener
- Estimated cost
- $0.06
- Wall time
- 2m
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
# Task: fix accessibility issues in a static page `index.html` is a public library events page that fails a WCAG 2.2 AA review. Fix it in place (HTML only; you may adjust the inline CSS). Keep all existing visible text content, all links (same hrefs) and the form fields (same `name`s), and keep the page working without JavaScript where possible. Address at least: document language, landmarks (header/nav/main), skip link, image alternatives (informative vs decorative: the divider is decorative, the logo and storytime photo are not), heading hierarchy, ambiguous link text, data table semantics, form labels and grouping, the fake button, iframe title, positive tabindex, focusable content hidden from assistive tech, and the low-contrast `.muted` text (#999 on white). Write `A11Y_NOTES.md` listing each fix (one line each). No network. Note: links must have a descriptive accessible name; a link whose whole name is just "click here", "here", "more" or "read more" fails (visually-hidden extra text or aria-label inside the link counts toward the name).
Static checks: lang, landmarks, skip link, decorative vs informative alt, heading order, link text, th/scope, labels, real button, iframe title, tabindex, aria-hidden focusables, contrast ≥4.5:1, content preserved. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).
Success criteria: 1. html has lang starting with 'en'. 2. header, nav and main elements present. 3. First link is a skip link pointing to an existing id. 4. Logo and storytime images have alt text longer than 3 characters; the divider image has alt="". 5. Exactly one h1, first heading is h1, no heading level skips. 6. No link text 'click here'/'here'/'read more'/'more'. 7. At least 3 th cells with scope (or a thead). 8. Every input/select has a label (for/id, wrapping label, aria-label or aria-labelledby). 9. No onclick on non-button elements, and a real submit button exists. 10. Every iframe has a non-empty title. 11. No positive tabindex. 12. No focusable element inside aria-hidden="true". 13. .muted text colour has contrast >= 4.5:1 on white. 14. Original key text, all original hrefs, the form action and the 4 field names are preserved. 15. A11Y_NOTES.md has at least 10 list lines. (A table caption is reported but not required.)
Setup notes: Use Case Scout batch4 case d06-a11y-static-html (offline fixture). Prepared with batch4/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 17/17 checks. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 7 (1 failed, recovered). Single run.
Estimated cost List-price estimate; not actually billed.
Metrics
- Estimated cost
- $0.06
- Wall time
- 2m
- Human step-ins
- 0
- Turns
- 1
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on | 105,791 | 3,264 | $0.06 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via
Codex CLI 0.163.0 (codex exec --json), cloud. - It counts as a success only if it meets every success criterion: 1. html has lang starting with 'en'. 2. header, nav and main elements present. 3. First link is a skip link pointing to an existing id. 4. Logo and storytime images have alt text longer than 3 characters; the divider image has alt="". 5. Exactly one h1, first heading is h1, no heading level skips. 6. No link text 'click here'/'here'/'read more'/'more'. 7. At least 3 th cells with scope (or a thead). 8. Every input/select has a label (for/id, wrapping label, aria-label or aria-labelledby). 9. No onclick on non-button elements, and a real submit button exists. 10. Every iframe has a non-empty title. 11. No positive tabindex. 12. No focusable element inside aria-hidden="true". 13. .muted text colour has contrast >= 4.5:1 on white. 14. Original key text, all original hrefs, the form action and the 4 field names are preserved. 15. A11Y_NOTES.md has at least 10 list lines. (A table caption is reported but not required.)
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
{"type": "thread.started", "thread_id": "01a16425-332c-7790-aa65-ad14aa61bd9b"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll inspect the HTML, fix the accessibility issues in place, and check that the existing text, link destinations, and field names are preserved without using the network."}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'index.html' -g '*test*' -g 'package.json'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'index.html' -g '*test*' -g 'package.json'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/work\nindex.html\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat index.html'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat index.html'", "aggregated_output": "<!DOCTYPE html>\n<html>\n<head><meta charset=\"utf-8\"><title>Riverside Library \u2013 Events</title>\n<style>.btn{background:#2a6;color:#fff;padding:6px;cursor:pointer}.muted{color:#999}</style></head>\n<body>\n<div class=\"header\"><img src=\"logo.png\"><span class=\"title\">Riverside Library</span></div>\n<div class=\"nav\"><a href=\"/\">Home</a> | <a href=\"/events\">Events</a> | <a href=\"/contact\">Contact</a></div>\n<h1>Upcoming events</h1>\n<h3>This week</h3>\n<p>Join our <b>Story Time</b> every Tuesday at 10:00. <a href=\"/story-time\">Click here</a>.</p>\n<img src=\"storytime.jpg\">\n<img src=\"divider.png\" class=\"decorative-divider\">\n<h3>Book club</h3>\n<p class=\"muted\">Monthly book club meets the first Thursday. To read more about the current book <a href=\"/book-club\">click here</a>.</p>\n<table>\n<tr><td>Event</td><td>Day</td><td>Time</td></tr>\n<tr><td>Story Time</td><td>Tuesday</td><td>10:00</td></tr>\n<tr><td>Book club</td><td>Thursday</td><td>18:30</td></tr>\n</table>\n<h2>Register</h2>\n<form action=\"/register\" method=\"post\">\nName <input type=\"text\" name=\"name\">\nEmail <input type=\"email\" name=\"email\">\n<select name=\"event\"><option>Story Time</option><option>Book club</option></select>\n<input type=\"checkbox\" name=\"newsletter\"> Send me the newsletter\n<div class=\"btn\" onclick=\"this.closest('form').submit()\">Register</div>\n</form>\n<iframe src=\"https://maps.example/riverside\" width=\"300\" height=\"200\"></iframe>\n<div tabindex=\"3\"><a href=\"/accessibility\">Accessibility statement</a></div>\n<div aria-hidden=\"true\"><a href=\"/old-site\">Old site</a></div>\n</body>\n</html>\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la'", "aggregated_output": "total 16\ndrwxr-xr-x 6 bob bob 4096 Oct 22 11:06 .\ndrwxrwxr-x 3 bob bob 4096 Oct 22 11:05 ..\ndr-xr-xr-x 2 bob bob 40 Oct 22 11:06 .agents\ndr-xr-xr-x 2 bob bob 40 Oct 22 11:06 .aws\ndr-xr-xr-x 2 bob bob 40 Oct 22 11:06 .codex\ndr-xr-xr-x 2 bob bob 40 Oct 22 11:06 .git\n-rw-r--r-- 1 bob bob 1060 Oct 22 10:08 TASK.md\n-rw-r--r-- 1 bob bob 1559 Oct 22 10:08 index.html\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'for p in /AGENTS.md /home/AGENTS.md /home/bob/AGENTS.md /home/bob/usecase-runs/AGENTS.md /home/bob/usecase-runs/batch/AGENTS.md /home/bob/usecase-runs/batch/out/AGENTS.md /home/bob/usecase-runs/batch/out/gpt-6.1-sol/AGENTS.md /home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/AGENTS.md; do if [ -f \"$p\" ]; then cat \"$p\"; fi; done'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'for p in /AGENTS.md /home/AGENTS.md /home/bob/AGENTS.md /home/bob/usecase-runs/AGENTS.md /home/bob/usecase-runs/batch/AGENTS.md /home/bob/usecase-runs/batch/out/AGENTS.md /home/bob/usecase-runs/batch/out/gpt-6.1-sol/AGENTS.md /home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/AGENTS.md; do if [ -f \"$p\" ]; then cat \"$p\"; fi; done'", "aggregated_output": "", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'cp index.html /tmp/library-events-before.html'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'cp index.html /tmp/library-events-before.html'", "aggregated_output": "", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_6", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/work/A11Y_NOTES.md", "kind": "add"}, {"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/work/index.html", "kind": "update"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/work/A11Y_NOTES.md", "kind": "add"}, {"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/153_d06/work/index.html", "kind": "update"}], "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_7", "type": "agent_message", "text": "The fixes are in place. I\u2019m checking that the existing links and form fields are preserved and that each control and link has an accessible name. The image files weren\u2019t supplied, so I used the page context for the Story Time photo\u2019s alternative text and noted that limitation."}}
{"type": "item.started", "item": {"id": "item_8", "type": "command_execution", "command": "/bin/bash -lc \"python - <<'PY'\nfrom html.parser import HTMLParser\nfrom pathlib import Path\nclass Page(HTMLParser):\n def __init__(self, text):\n super().__init__(); self.nodes=[]; self.text=[]; self.feed(text)\n def handle_starttag(self, tag, attrs): self.nodes.append((tag,dict(attrs)))\n def handle_data(self, text): self.text.extend(text.split())\na=Page(Path('/tmp/library-events-before.html').read_text()); b=Page(Path('index.html').read_text())\nlinks=lambda p:[x['href'] for t,x in p.nodes if t=='a']\nassert links(b)==['#main']+links(a)\nfields=lambda p:[(t,x.get('type'),x['name']) for t,x in p.nodes if 'name' in x]\nassert fields(a)==fields(b)\nassert all(int(x.get('tabindex','0'))<=0 for t,x in b.nodes)\nassert all('alt' in x for t,x in b.nodes if t=='img')\nassert not any(x.get('aria-hidden')=='true' for t,x in b.nodes)\nids=[x['id'] for t,x in b.nodes if 'id' in x]\nassert len(ids)==len(set(ids))\nlabels={x['for'] for t,x in b.nodes if t=='label'}\nassert all(x['id'] in labels for t,x in b.nodes if t in ('input','select'))\nassert any(t=='button' and x.get('type')=='submit' for t,x in b.nodes)\nassert not any('onclick' in x for t,x in b.nodes)\n# Every original text token still occurs in the same order.\nit=iter(b.text)\nassert all(any(v==word for v in it) for word in a.text if not word.startswith('<')) if False else True\n# Compare body text only, excluding CSS and newly added accessible text.\nimport re\nold=Path('/tmp/library-events-before.html').read_text().split('<body>')[1]\nnew=Path('index.html').read_text().split('<body>')[1]\nold_words=re.sub('<[\"'^>]+>'\"',' ',old).split()\nnew_words=iter(re.sub('<[\"'^>]+>'\"',' ',new).split())\nassert all(any(v==word for v in new_words) for word in old_words)\nprint('PASS: original text order, link destinations, field names, labels, image alternatives, tab order, and native submission checks.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_8", "type": "command_execution", "command": "/bin/bash -lc \"python - <<'PY'\nfrom html.parser import HTMLParser\nfrom pathlib import Path\nclass Page(HTMLParser):\n def __init__(self, text):\n super().__init__(); self.nodes=[]; self.text=[]; self.feed(text)\n def handle_starttag(self, tag, attrs): self.nodes.append((tag,dict(attrs)))\n def handle_data(self, text): self.text.extend(text.split())\na=Page(Path('/tmp/library-events-before.html').read_text()); b=Page(Path('index.html').read_text())\nlinks=lambda p:[x['href'] for t,x in p.nodes if t=='a']\nassert links(b)==['#main']+links(a)\nfields=lambda p:[(t,x.get('type'),x['name']) for t,x in p.nodes if 'name' in x]\nassert fields(a)==fields(b)\nassert all(int(x.get('tabindex','0'))<=0 for t,x in b.nodes)\nassert all('alt' in x for t,x in b.nodes if t=='img')\nassert not any(x.get('aria-hidden')=='true' for t,x in b.nodes)\nids=[x['id'] for t,x in b.nodes if 'id' in x]\nassert len(ids)==len(set(ids))\nlabels={x['for'] for t,x in b.nodes if t=='label'}\nassert all(x['id'] in labels for t,x in b.nodes if t in ('input','select'))\nassert any(t=='button' and x.get('type')=='submit' for t,x in b.nodes)\nassert not any('onclick' in x for t,x in b.nodes)\n# Every original text token still occurs in the same order.\nit=iter(b.text)\nassert all(any(v==word for v in it) for word in a.text if not word.startswith('<')) if False else True\n# Compare body text only, excluding CSS and newly added accessible text.\nimport re\nold=Path('/tmp/library-events-before.html').read_text().split('<body>')[1]\nnew=Path('index.html').read_text().split('<body>')[1]\nold_words=re.sub('<[\"'^>]+>'\"',' ',old).split()\nnew_words=iter(re.sub('<[\"'^>]+>'\"',' ',new).split())\nassert all(any(v==word for v in new_words) for word in old_words)\nprint('PASS: original text order, link destinations, field names, labels, image alternatives, tab order, and native submission checks.')\nPY\"", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_9", "type": "command_execution", "command": "/bin/bash -lc \"python3 - <<'PY'\nfrom html.parser import HTMLParser\nfrom pathlib import Path\nimport re\nclass Page(HTMLParser):\n def __init__(self,path):\n super().__init__();self.nodes=[];self.feed(Path(path).read_text())\n def handle_starttag(self,t,a):self.nodes.append((t,dict(a)))\na=Page('/tmp/library-events-before.html');b=Page('index.html')\nassert [x['href'] for t,x in b.nodes if t=='a']==['#main']+[x['href'] for t,x in a.nodes if t=='a']\nassert [(t,x['name']) for t,x in a.nodes if 'name' in x]==[(t,x['name']) for t,x in b.nodes if 'name' in x]\nassert all(int(x.get('tabindex','0'))<=0 for t,x in b.nodes)\nassert all('alt' in x for t,x in b.nodes if t=='img')\nassert not any(x.get('aria-hidden')=='true' or 'onclick' in x for t,x in b.nodes)\nlabels={x['for'] for t,x in b.nodes if t=='label'}\nassert all(x['id'] in labels for t,x in b.nodes if t in ('input','select'))\nassert any(t=='button' and x.get('type')=='submit' for t,x in b.nodes)\nold=Path('/tmp/library-events-before.html').read_text().split('<body>')[1]\nnew=Path('index.html').read_text().split('<body>')[1]\nwords=iter(re.sub('<[\"'^>]+>'\"',' ',new).split())\nassert all(any(v==word for v in words) for word in re.sub('<[\"'^>]+>'\"',' ',old).split())\nprint('PASS: original visible text, hrefs, field names, labels, image alternatives, tab order, and native submission checks.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_9", "type": "command_execution", "command": "/bin/bash -lc \"python3 - <<'PY'\nfrom html.parser import HTMLParser\nfrom pathlib import Path\nimport re\nclass Page(HTMLParser):\n def __init__(self,path):\n super().__init__();self.nodes=[];self.feed(Path(path).read_text())\n def handle_starttag(self,t,a):self.nodes.append((t,dict(a)))\na=Page('/tmp/library-events-before.html');b=Page('index.html')\nassert [x['href'] for t,x in b.nodes if t=='a']==['#main']+[x['href'] for t,x in a.nodes if t=='a']\nassert [(t,x['name']) for t,x in a.nodes if 'name' in x]==[(t,x['name']) for t,x in b.nodes if 'name' in x]\nassert all(int(x.get('tabindex','0'))<=0 for t,x in b.nodes)\nassert all('alt' in x for t,x in b.nodes if t=='img')\nassert not any(x.get('aria-hidden')=='true' or 'onclick' in x for t,x in b.nodes)\nlabels={x['for'] for t,x in b.nodes if t=='label'}\nassert all(x['id'] in labels for t,x in b.nodes if t in ('input','select'))\nassert any(t=='button' and x.get('type')=='submit' for t,x in b.nodes)\nold=Path('/tmp/library-events-before.html').read_text().split('<body>')[1]\nnew=Path('index.html').read_text().split('<body>')[1]\nwords=iter(re.sub('<[\"'^>]+>'\"',' ',new).split())\nassert all(any(v==word for v in words) for word in re.sub('<[\"'^>]+>'\"',' ',old).split())\nprint('PASS: original visible text, hrefs, field names, labels, image alternatives, tab order, and native submission checks.')\nPY\"", "aggregated_output": "PASS: original visible text, hrefs, field names, labels, image alternatives, tab order, and native submission checks.\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_10", "type": "agent_message", "text": "Fixed [index.html](index.html) and documented each fix in [A11Y_NOTES.md](A11Y_NOTES.md).\n\nAll requested issues are addressed. Local checks confirmed preservation of existing visible text, hrefs, and field names. Registration now works without JavaScript.\n\nNo network used. Image assets weren\u2019t supplied, so the photo alternative uses page context."}}
{"type": "turn.completed", "usage": {"input_tokens": 105791, "cached_input_tokens": 96896, "cache_write_input_tokens": 0, "output_tokens": 3264, "reasoning_output_tokens": 69}}