Skip to content

GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Anchor drag alarm from a GPS track

OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.05
Wall time
1m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

# Anchor drag alarm from a GPS track `input.json` (synthetic, not for real anchor watch): `anchor` [lat, lon], `rode_m`, `depth_m`, `bow_height_m`, `gps_offset_m`, `margin_m`, `consecutive` N, and `track` (CSV `t,lat,lon`, t in seconds). Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `anchor.svg`. Fresh inputs are used too. ## Rules 1. Alarm radius = sqrt(rode^2 - (depth + bow_height)^2) (0 if rode <= depth + bow_height) + gps_offset + margin. 2. Track rows: skip the header. A row is **bad** (counted, skipped) if it has fewer than 3 fields, t is not an integer, lat/lon is not a finite number in range [-90, 90] / [-180, 180], or t is not greater than the last *valid* row's t. 3. Distance from the anchor (metres): R = 6371000; m = mean of the anchor and point latitudes; x = radians(dlon) * cos(m) * R (east), y = radians(dlat) * R (north); distance = hypot(x, y). A point is **outside** if distance > radius. 4. An **event** is a maximal run of consecutive valid points that are outside, with length >= N. Report `start_t` (first point), `trigger_t` (Nth point), `end_t` (last point), `max_dist_m`, and `bearing_deg` = atan2(x, y) mod 360 from the anchor to the run's last point. 5. Round the radius to 2 decimals and distances and bearings to 1. ## Visual output: anchor.svg Static, self-contained SVG (root `<svg>`, no scripts, images, links or external assets; renders headless) with `<circle id="swing-circle">`, `<polyline id="track">` and an element `id="anchor"`. The drawing itself is not graded. ## answer.json `{"radius_m", "valid_points", "bad_rows", "events": [...], "max_dist_m", "outside_points"}` ## Scoring Hidden checks (partial credit; all must pass): `swing_radius` (within 0.011), `track_cleaning` (exact), `distances`, `alarm_events` (times exact, values within 0.11), `anchor_svg_static` (anchor.svg exists, follows the static rules, has the ids/tags above), `fresh_run_1`, `fresh_run_2`. 60-second limit per run.

Catenary-free swing radius with offsets, rejecting bad and out-of-order rows against the last valid time, short excursions that must not trigger, and a real drag run with trigger time and bearing. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).

Benchmark code for comparing AI models, not navigation or safety software.

Success criteria: 1. swing_radius 2. track_cleaning 3. distances 4. alarm_events 5. anchor_svg_static 6. fresh_run_1 7. fresh_run_2

Setup notes: Use Case Scout batch10 case k07-anchor-drag-alarm (offline fixture). Prepared with batch10/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Visual output (anchor.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 6 (1 failed, recovered). Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.05
Wall time
1m
Human step-ins
0
Turns
1
Model usage
ModelInput tokOutput tokCost
OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on101,4432,102$0.05

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via Codex CLI 0.163.0 (codex exec --json), cloud.
  3. It counts as a success only if it meets every success criterion: 1. swing_radius 2. track_cleaning 3. distances 4. alarm_events 5. anchor_svg_static 6. fresh_run_1 7. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{"type": "thread.started", "thread_id": "01a163fd-977b-7f40-b809-28ee574e812c"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll check the available files, write `solve.py`, and verify it with synthetic cases.\n"}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/114_k07/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"anchor\": [\n  -23.4155114,\n  143.598916\n ],\n \"rode_m\": 30,\n \"depth_m\": 5.9,\n \"bow_height_m\": 1.2,\n \"gps_offset_m\": 4.0,\n \"margin_m\": 5.0,\n \"consecutive\": 3,\n \"track\": \"track.csv\"\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la && head -8 track.csv'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la && head -8 track.csv'", "aggregated_output": "total 32\ndrwxr-xr-x 6 bob bob 4096 Oct 22 10:22 .\ndrwxrwxr-x 3 bob bob 4096 Oct 22 10:22 ..\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:22 .agents\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:22 .aws\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:22 .codex\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:22 .git\n-rw-r--r-- 1 bob bob 2020 Oct 22 10:08 TASK.md\n-rw-r--r-- 1 bob bob 3849 Oct 22 10:08 anchor.svg\n-rw-r--r-- 1 bob bob  183 Oct 22 10:08 input.json\n-rw-r--r-- 1 bob bob 8517 Oct 22 10:08 track.csv\nt,lat,lon\n10,-23.4154985,143.5991258\n20,-23.4154431,143.5991293\n30,-23.4154293,143.5991068\n40,-23.4154629,143.5990837\n50,-23.4154318,143.5990879\n60,-23.4154115,143.5990738\n70,-23.4153861,143.5990780\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/114_k07/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/114_k07/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json && cat answer.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json && cat answer.json'", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_6", "type": "command_execution", "command": "/bin/bash -lc 'python3 solve.py input.json answer.json && cat answer.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "command_execution", "command": "/bin/bash -lc 'python3 solve.py input.json answer.json && cat answer.json'", "aggregated_output": "{\n  \"radius_m\": 38.15,\n  \"valid_points\": 290,\n  \"bad_rows\": 10,\n  \"events\": [\n    {\n      \"start_t\": 822,\n      \"trigger_t\": 852,\n      \"end_t\": 902,\n      \"max_dist_m\": 42.9,\n      \"bearing_deg\": 68.2\n    },\n    {\n      \"start_t\": 2640,\n      \"trigger_t\": 2660,\n      \"end_t\": 2967,\n      \"max_dist_m\": 50.5,\n      \"bearing_deg\": 314.9\n    },\n    {\n      \"start_t\": 2988,\n      \"trigger_t\": 3009,\n      \"end_t\": 3071,\n      \"max_dist_m\": 43.6,\n      \"bearing_deg\": 301.8\n    }\n  ],\n  \"max_dist_m\": 50.5,\n  \"outside_points\": 49\n}\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_7", "type": "command_execution", "command": "/bin/bash -lc \"python3 - <<'PY'\nimport csv, io, math\nimport xml.etree.ElementTree as ET\nimport solve\nc = dict(anchor=[0,0], rode_m=0, depth_m=1, bow_height_m=0, gps_offset_m=0, margin_m=0, consecutive=2)\ns = 't,lat,lon\\\\n1,0,0.001\\\\nbad,0,0\\\\n2,0,0.002\\\\n2,0,0\\\\n3,nan,0\\\\n3,0,0.003\\\\n4,0,0\\\\n5,0,0.001\\\\n'\nr, radius, points = solve.solve(c, csv.reader(io.StringIO(s)))\nassert r['valid_points'] == 5 and r['bad_rows'] == 3\nassert r['outside_points'] == 4\nassert r['events'] == [dict(start_t=1, trigger_t=2, end_t=3, max_dist_m=333.6, bearing_deg=90.0)]\nroot = ET.parse('anchor.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nassert root.find(f\\\"{ns}circle[@id='swing-circle']\\\") is not None\nassert root.find(f\\\"{ns}polyline[@id='track']\\\") is not None\nassert any(e.get('id') == 'anchor' for e in root.iter())\nassert not any(e.tag in {ns+x for x in ('script','image','a')} for e in root.iter())\nprint('Synthetic cleaning/event checks and SVG checks passed.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_7", "type": "command_execution", "command": "/bin/bash -lc \"python3 - <<'PY'\nimport csv, io, math\nimport xml.etree.ElementTree as ET\nimport solve\nc = dict(anchor=[0,0], rode_m=0, depth_m=1, bow_height_m=0, gps_offset_m=0, margin_m=0, consecutive=2)\ns = 't,lat,lon\\\\n1,0,0.001\\\\nbad,0,0\\\\n2,0,0.002\\\\n2,0,0\\\\n3,nan,0\\\\n3,0,0.003\\\\n4,0,0\\\\n5,0,0.001\\\\n'\nr, radius, points = solve.solve(c, csv.reader(io.StringIO(s)))\nassert r['valid_points'] == 5 and r['bad_rows'] == 3\nassert r['outside_points'] == 4\nassert r['events'] == [dict(start_t=1, trigger_t=2, end_t=3, max_dist_m=333.6, bearing_deg=90.0)]\nroot = ET.parse('anchor.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nassert root.find(f\\\"{ns}circle[@id='swing-circle']\\\") is not None\nassert root.find(f\\\"{ns}polyline[@id='track']\\\") is not None\nassert any(e.get('id') == 'anchor' for e in root.iter())\nassert not any(e.tag in {ns+x for x in ('script','image','a')} for e in root.iter())\nprint('Synthetic cleaning/event checks and SVG checks passed.')\nPY\"", "aggregated_output": "Synthetic cleaning/event checks and SVG checks passed.\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_8", "type": "agent_message", "text": "Created [solve.py](solve.py), [answer.json](answer.json), and [anchor.svg](anchor.svg). Synthetic cleaning, event, and SVG checks passed.\n\nRun with `python3 solve.py input.json answer.json` (`python` is unavailable here)."}}
{"type": "turn.completed", "usage": {"input_tokens": 101443, "cached_input_tokens": 92416, "cache_write_input_tokens": 0, "output_tokens": 2102, "reasoning_output_tokens": 39}}

Get new runs and ranking changes weekly