GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Great-circle vs rhumb-line passage plan with ETAs
OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener
- Estimated cost
- $0.05
- Wall time
- 1m
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
# Great-circle vs rhumb-line passage plan with ETAs `input.json` (synthetic; not for navigation): `waypoints` ([lat, lon] decimal degrees, legs may cross the 180° meridian), `speed_kn`, `depart_utc` ("YYYY-MM-DDTHH:MM:SSZ"). Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `route.svg`. Fresh inputs are used too. ## Rules (sphere, R = 3440.065 nm) - Longitude difference dlon is always wrapped into [-180°, 180°). - **Great circle**: haversine distance; initial course = atan2(sin dlon * cos lat2, cos lat1 * sin lat2 - sin lat1 * cos lat2 * cos dlon) mod 360. - **Rhumb line**: dpsi = ln(tan(pi/4 + lat2/2) / tan(pi/4 + lat1/2)); q = dlat / dpsi, or cos(lat1) when |dpsi| <= 1e-12; distance = sqrt(dlat^2 + q^2 * dlon^2) * R (radians); course = atan2(dlon, dpsi) mod 360. - The boat sails the **rhumb** legs at `speed_kn`; leg ETA = departure + cumulative rhumb distance / speed, rounded to the nearest minute (half a minute rounds up), formatted "YYYY-MM-DDTHH:MMZ". - `rhumb_extra_pct` = (total rhumb - total GC) / total GC * 100. Round distances to 2 decimals, courses to 1, the percentage to 3. ## Visual output: route.svg Static, self-contained SVG (root `<svg>`, no scripts, images, links or external assets; renders headless) with `<polyline id="route-gc">`, `<polyline id="route-rhumb">` and an element `id="waypoints"`. The drawing (projection etc.) is not graded. ## answer.json `{"legs": [{"gc_nm", "gc_initial_course", "rhumb_nm", "rhumb_course", "eta_utc"}, ...], "total_gc_nm", "total_rhumb_nm", "rhumb_extra_pct", "arrival_utc"}` ## Scoring Hidden checks (partial credit; all must pass): `great_circle_legs`, `rhumb_legs` (first four legs; distances within 0.011, courses within 0.11), `totals`, `etas` (exact strings), `route_svg_static` (route.svg exists, follows the static rules, has the ids/tags above), `fresh_run_1`, `fresh_run_2` (all keys, within 0.02). 60-second limit per run.
Antimeridian wrap on both methods, the due east/west rhumb leg where dpsi is zero, initial great-circle course, and minute-rounded UTC ETAs that drift if per-leg rounding is used. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).
Benchmark code for comparing AI models, not navigation or safety software.
Success criteria: 1. great_circle_legs 2. rhumb_legs 3. totals 4. etas 5. route_svg_static 6. fresh_run_1 7. fresh_run_2
Setup notes: Use Case Scout batch10 case k09-passage-gc-vs-rhumb (offline fixture). Prepared with batch10/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Visual output (route.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 4 (1 failed, recovered). Single run.
Estimated cost List-price estimate; not actually billed.
Output
Metrics
- Estimated cost
- $0.05
- Wall time
- 1m
- Human step-ins
- 0
- Turns
- 1
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on | 84,352 | 2,633 | $0.05 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via
Codex CLI 0.163.0 (codex exec --json), cloud. - It counts as a success only if it meets every success criterion: 1. great_circle_legs 2. rhumb_legs 3. totals 4. etas 5. route_svg_static 6. fresh_run_1 7. fresh_run_2
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
{"type": "thread.started", "thread_id": "01a163ff-86ba-7da2-b4d9-7f7127dccd2c"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll write `solve.py` with the route calculations, cumulative rhumb ETAs, and a self-contained SVG, then check it with a dateline-crossing example."}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '*test*'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '*test*'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/116_k09/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"waypoints\": [\n [\n -32.1328,\n 168.9091\n ],\n [\n -35.6621,\n 170.8416\n ],\n [\n -27.8913,\n 178.3805\n ],\n [\n -27.8913,\n -167.0907\n ],\n [\n -33.2445,\n -163.3405\n ],\n [\n -34.4227,\n -161.3399\n ]\n ],\n \"speed_kn\": 8.4,\n \"depart_utc\": \"2026-11-20T17:30:00Z\"\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/116_k09/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/116_k09/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc \"python solve.py input.json answer.json && python - <<'PY'\nimport datetime as dt\nimport json\nimport math\nimport xml.etree.ElementTree as ET\nimport solve\nr = json.load(open('answer.json'))\nassert len(r['legs']) == 5\nroot = ET.parse('route.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor name in ('route-gc', 'route-rhumb'):\n assert root.find(f\\\"{ns}polyline[@id='{name}']\\\") is not None\nassert root.find(f\\\".//*[@id='waypoints']\\\") is not None\nv = solve.leg_values([0, 179], [0, -179])\nassert math.isclose(v[0], 2 * math.pi / 180 * solve.R)\nassert math.isclose(v[2], v[0]) and v[1] == v[3] == 90\nassert solve.eta(dt.datetime(2026, 1, 1, 23, 59, 30, tzinfo=dt.timezone.utc), 0) == '2026-01-02T00:00Z'\nassert solve.solve({'waypoints': [[0,0], [0,0]], 'speed_kn': 1, 'depart_utc': '2026-01-01T00:00:00Z'})['rhumb_extra_pct'] == 0\nprint('Checks passed. Arrival:', r['arrival_utc'])\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc \"python solve.py input.json answer.json && python - <<'PY'\nimport datetime as dt\nimport json\nimport math\nimport xml.etree.ElementTree as ET\nimport solve\nr = json.load(open('answer.json'))\nassert len(r['legs']) == 5\nroot = ET.parse('route.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor name in ('route-gc', 'route-rhumb'):\n assert root.find(f\\\"{ns}polyline[@id='{name}']\\\") is not None\nassert root.find(f\\\".//*[@id='waypoints']\\\") is not None\nv = solve.leg_values([0, 179], [0, -179])\nassert math.isclose(v[0], 2 * math.pi / 180 * solve.R)\nassert math.isclose(v[2], v[0]) and v[1] == v[3] == 90\nassert solve.eta(dt.datetime(2026, 1, 1, 23, 59, 30, tzinfo=dt.timezone.utc), 0) == '2026-01-02T00:00Z'\nassert solve.solve({'waypoints': [[0,0], [0,0]], 'speed_kn': 1, 'depart_utc': '2026-01-01T00:00:00Z'})['rhumb_extra_pct'] == 0\nprint('Checks passed. Arrival:', r['arrival_utc'])\nPY\"", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport datetime as dt\nimport json\nimport math\nimport xml.etree.ElementTree as ET\nimport solve\nr = json.load(open('answer.json'))\nassert len(r['legs']) == 5\nroot = ET.parse('route.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor name in ('route-gc', 'route-rhumb'):\n assert root.find(f\\\"{ns}polyline[@id='{name}']\\\") is not None\nassert root.find(\\\".//*[@id='waypoints']\\\") is not None\nv = solve.leg_values([0, 179], [0, -179])\nassert math.isclose(v[0], 2 * math.pi / 180 * solve.R)\nassert math.isclose(v[2], v[0]) and v[1] == v[3] == 90\nassert solve.eta(dt.datetime(2026, 1, 1, 23, 59, 30, tzinfo=dt.timezone.utc), 0) == '2026-01-02T00:00Z'\nassert solve.solve({'waypoints': [[0,0], [0,0]], 'speed_kn': 1, 'depart_utc': '2026-01-01T00:00:00Z'})['rhumb_extra_pct'] == 0\nprint('Checks passed. Arrival:', r['arrival_utc'])\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport datetime as dt\nimport json\nimport math\nimport xml.etree.ElementTree as ET\nimport solve\nr = json.load(open('answer.json'))\nassert len(r['legs']) == 5\nroot = ET.parse('route.svg').getroot()\nns = '{http://www.w3.org/2000/svg}'\nassert root.tag == ns + 'svg'\nfor name in ('route-gc', 'route-rhumb'):\n assert root.find(f\\\"{ns}polyline[@id='{name}']\\\") is not None\nassert root.find(\\\".//*[@id='waypoints']\\\") is not None\nv = solve.leg_values([0, 179], [0, -179])\nassert math.isclose(v[0], 2 * math.pi / 180 * solve.R)\nassert math.isclose(v[2], v[0]) and v[1] == v[3] == 90\nassert solve.eta(dt.datetime(2026, 1, 1, 23, 59, 30, tzinfo=dt.timezone.utc), 0) == '2026-01-02T00:00Z'\nassert solve.solve({'waypoints': [[0,0], [0,0]], 'speed_kn': 1, 'depart_utc': '2026-01-01T00:00:00Z'})['rhumb_extra_pct'] == 0\nprint('Checks passed. Arrival:', r['arrival_utc'])\nPY\"", "aggregated_output": "Checks passed. Arrival: 2026-12-01T04:11Z\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "agent_message", "text": "Created `solve.py`, `answer.json`, and `route.svg`.\n\nVerified dateline wrapping, ETA half-minute rounding, zero-distance handling, and required SVG elements.\n\nRun with:\n```bash\npython3 solve.py input.json answer.json\n```\n\nThis environment provides `python3`; `python` is unavailable."}}
{"type": "turn.completed", "usage": {"input_tokens": 84352, "cached_input_tokens": 77568, "cache_write_input_tokens": 0, "output_tokens": 2633, "reasoning_output_tokens": 54}}