Skip to content

GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Dead reckoning, estimated position with current, and computed set and drift

OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.04
Wall time
1m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

# Dead reckoning, estimated position with current, and computed set and drift `input.json` (synthetic, not for navigation): `start` [lat, lon] in decimal degrees, `legs` (each `course` degrees true, `speed_kn` through the water, `minutes`), a forecast `current` (`set` = direction it flows TOWARD, degrees true; `drift_kn`), and a GPS `fix` [lat, lon] at the end of the last leg. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `track.svg`. Fresh inputs are used too. ## Rules (plane sailing, 1 minute of latitude = 1 nm) 1. A move of n nm north and e nm east from (lat, lon): dlat = n / 60; mid = lat + dlat / 2; new lat = lat + dlat; new lon = lon + e / (60 * cos(mid)). 2. **DR**: per leg, distance = speed * hours along the course. **EP**: per leg, add the current vector (drift * hours toward set) to the leg's water vector, then move once. 3. Difference from point A to point B: north = (latB - latA) * 60, east = (lonB - lonA) * 60 * cos((latA + latB) / 2). 4. **Set/drift**: vector from the final DR position to the fix; set = atan2(east, north) in degrees mod 360; drift = length / total hours. `ep_error_nm` = distance from the final EP to the fix. **CMG/SMG**: same from `start` to the fix. 5. Round positions to 5 decimals, hours to 4, angles to 1, speeds and distances to 3. `dr` and `ep` list the start and the position after every leg. ## Visual output: track.svg Static, self-contained SVG (root `<svg>`, no scripts, images, links or external assets; renders headless) with `<polyline id="dr-track">`, `<polyline id="ep-track">` and `<circle id="fix">`. The drawing itself is not graded. ## answer.json `{"dr": [[lat, lon], ...], "ep": [...], "hours", "set_deg", "drift_kn", "ep_error_nm", "cmg_deg", "smg_kn"}` ## Scoring Hidden checks (partial credit; all must pass): `dr_positions`, `ep_positions`, `set_and_drift`, `course_speed_made_good`, `track_svg_static` (track.svg exists, follows the static rules, has the ids/tags above), `fresh_run_1`, `fresh_run_2`. Positions within 2e-5 degrees, angles within 0.11, speeds and distances within 0.002. 60-second limit per run.

Mid-latitude longitude scaling per leg, current applied per leg as a vector, 'set' as the direction the current flows toward, angle wrap, and positions checked at five decimals at high latitudes. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).

Benchmark code for comparing AI models, not navigation or safety software.

Success criteria: 1. dr_positions 2. ep_positions 3. set_and_drift 4. course_speed_made_good 5. track_svg_static 6. fresh_run_1 7. fresh_run_2

Setup notes: Use Case Scout batch10 case k06-dead-reckoning-set-drift (offline fixture). Prepared with batch10/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Visual output (track.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 5 (1 failed, recovered). Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.04
Wall time
1m
Human step-ins
0
Turns
1
Model usage
ModelInput tokOutput tokCost
OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on85,4002,148$0.04

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via Codex CLI 0.163.0 (codex exec --json), cloud.
  3. It counts as a success only if it meets every success criterion: 1. dr_positions 2. ep_positions 3. set_and_drift 4. course_speed_made_good 5. track_svg_static 6. fresh_run_1 7. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{"type": "thread.started", "thread_id": "01a163fc-9fa9-77e3-bf35-7db4de4ff646"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll implement the plane-sailing calculations and a self-contained SVG, then check them with synthetic inputs.\n"}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '*test*'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '*test*'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/113_k06/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"start\": [\n  -16.7478,\n  90.7095\n ],\n \"legs\": [\n  {\n   \"course\": 98,\n   \"speed_kn\": 3.4,\n   \"minutes\": 90\n  },\n  {\n   \"course\": 31,\n   \"speed_kn\": 7.1,\n   \"minutes\": 30\n  },\n  {\n   \"course\": 258,\n   \"speed_kn\": 3.5,\n   \"minutes\": 30\n  },\n  {\n   \"course\": 157,\n   \"speed_kn\": 6.5,\n   \"minutes\": 45\n  },\n  {\n   \"course\": 197,\n   \"speed_kn\": 3.1,\n   \"minutes\": 60\n  },\n  {\n   \"course\": 68,\n   \"speed_kn\": 7.3,\n   \"minutes\": 60\n  }\n ],\n \"current\": {\n  \"set\": 239,\n  \"drift_kn\": 1.1\n },\n \"fix\": [\n  -16.8217,\n  90.8589\n ]\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc 'ls -la'", "aggregated_output": "total 16\ndrwxr-xr-x 6 bob bob 4096 Oct 22 10:21 .\ndrwxrwxr-x 3 bob bob 4096 Oct 22 10:21 ..\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:21 .agents\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:21 .aws\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:21 .codex\ndr-xr-xr-x 2 bob bob   40 Oct 22 10:21 .git\n-rw-r--r-- 1 bob bob 2165 Oct 22 10:08 TASK.md\n-rw-r--r-- 1 bob bob  521 Oct 22 10:08 input.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/113_k06/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/113_k06/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json && cat answer.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json && cat answer.json'", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_6", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport math\nimport xml.etree.ElementTree as ET\nfrom solve import solve\nbase = {'start': [0, 0], 'legs': [{'course': 0, 'speed_kn': 6, 'minutes': 60}], 'current': {'set': 90, 'drift_kn': 2}, 'fix': [0.1, 2 / (60 * math.cos(math.radians(.05)))]}\nr, dr, ep = solve(base)\nassert r['dr'] == [[0, 0], [0.1, 0.0]]\nassert r['ep'][-1] == [0.1, round(base['fix'][1], 5)]\nassert r['set_deg'] == 90 and r['drift_kn'] == 2\nassert r['ep_error_nm'] == 0\nassert r['cmg_deg'] == 18.4 and r['smg_kn'] == 6.325\nbase['start'] = [-30, -70]\nbase['legs'] = [{'course': 270, 'speed_kn': 8, 'minutes': 30}, {'course': 180, 'speed_kn': 4, 'minutes': 90}]\nbase['current']['drift_kn'] = 0\nbase['fix'] = [-30.1, -70.1]\nr, dr, ep = solve(base)\nassert dr == ep and len(dr) == 3 and r['hours'] == 2\nroot = ET.parse('track.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor tag, identifier in [('polyline', 'dr-track'), ('polyline', 'ep-track'), ('circle', 'fix')]:\n    assert root.find(f\\\"{ns}{tag}[@id='{identifier}']\\\") is not None\nassert not any(el.tag in {ns+'script', ns+'image', ns+'a'} for el in root.iter())\nprint('Calculation and SVG checks passed.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport math\nimport xml.etree.ElementTree as ET\nfrom solve import solve\nbase = {'start': [0, 0], 'legs': [{'course': 0, 'speed_kn': 6, 'minutes': 60}], 'current': {'set': 90, 'drift_kn': 2}, 'fix': [0.1, 2 / (60 * math.cos(math.radians(.05)))]}\nr, dr, ep = solve(base)\nassert r['dr'] == [[0, 0], [0.1, 0.0]]\nassert r['ep'][-1] == [0.1, round(base['fix'][1], 5)]\nassert r['set_deg'] == 90 and r['drift_kn'] == 2\nassert r['ep_error_nm'] == 0\nassert r['cmg_deg'] == 18.4 and r['smg_kn'] == 6.325\nbase['start'] = [-30, -70]\nbase['legs'] = [{'course': 270, 'speed_kn': 8, 'minutes': 30}, {'course': 180, 'speed_kn': 4, 'minutes': 90}]\nbase['current']['drift_kn'] = 0\nbase['fix'] = [-30.1, -70.1]\nr, dr, ep = solve(base)\nassert dr == ep and len(dr) == 3 and r['hours'] == 2\nroot = ET.parse('track.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor tag, identifier in [('polyline', 'dr-track'), ('polyline', 'ep-track'), ('circle', 'fix')]:\n    assert root.find(f\\\"{ns}{tag}[@id='{identifier}']\\\") is not None\nassert not any(el.tag in {ns+'script', ns+'image', ns+'a'} for el in root.iter())\nprint('Calculation and SVG checks passed.')\nPY\"", "aggregated_output": "Calculation and SVG checks passed.\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_7", "type": "agent_message", "text": "Created [solve.py](solve.py), plus [answer.json](answer.json) and [track.svg](track.svg) for the supplied input.\n\nVerified calculations and required SVG elements with synthetic checks.\n\nRun with `python3 solve.py input.json answer.json` (`python` is unavailable here)."}}
{"type": "turn.completed", "usage": {"input_tokens": 85400, "cached_input_tokens": 78336, "cache_write_input_tokens": 0, "output_tokens": 2148, "reasoning_output_tokens": 9}}

Get new runs and ranking changes weekly