Skip to content

GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Best fishing windows from tide, solunar and weather data

OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.05
Wall time
1m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt (this run)

# Fishing trip planner from tide, solunar and weather data `input.json` (synthetic) covers 48 hours, hour index h = 0..47 meaning the hour [h, h+1). It has `tide_m` (49 hourly heights at h = 0..48), `wind_kt` and `pressure_hpa` (48 values), `solunar_major` and `solunar_minor` (lists of [start, end] in fractional hours), `daylight` ([sunrise_hour, sunset_hour] for day 0 and day 1, in hour-of-day), and `window_hours` L. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `windows.svg`. Fresh inputs are used too. ## Hourly score - If wind > 20 kt the hour is **unfishable**: score null, ignore the rest. - +3 if any major period overlaps the hour (start < h+1 and end > h); +1.5 if any minor period overlaps (both can apply). - Tide movement d = |tide[h+1] - tide[h]|, rounded to 6 decimals: +2 if d >= 0.3, else +1 if d >= 0.15. - Wind above 10 kt (and <= 20): -1. - Pressure trend dp = pressure[h] - pressure[h-3] (0 for h < 3), rounded to 6 decimals: +1 if dp <= -1.0, -1 if dp >= +1.0. - Outside daylight: -2, where daylight means sunrise <= (h mod 24) < sunset for day h // 24. Round each score to 2 decimals. ## Windows A window starting at s covers hours s..s+L-1, all fishable; its score is the sum, rounded to 2 decimals. Sort all windows by score descending then start ascending, and greedily keep up to 3 that do not overlap any already kept one. ## answer.json `{"hourly_scores": [48 values or null], "unfishable_hours": n, "windows": [{"start", "end" (= start + L), "score"}, ...]}` ## Visual output: windows.svg Static, self-contained SVG (root `<svg>`; no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, `href`/`src` except `#...` or `data:`, CSS `url(http...)`/`@import` or fetches; renders headless) with an element `id="hourly-bars"` (the hourly scores) and one `<rect id="window-N">` for each reported window N = 1, 2, 3 in answer order. The drawing itself is not graded. ## Scoring Hidden checks (partial credit; all must pass): `hourly_scores_day1` (hours 0-23), `hourly_scores_day2` (24-47 plus unfishable count), `top_windows`, `windows_svg_static` (windows.svg exists, follows the static rules, has the ids/tags above), `fresh_run_1`, `fresh_run_2`. Tolerance 0.001. 60-second limit per run.

Overlap tests on fractional periods, threshold boundaries placed exactly at 10 and 20 kt, a 3-hour pressure trend with float rounding, daylight by day index, and greedy non-overlapping window selection with ties. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).

Benchmark code for comparing AI models, not navigation or safety software.

Success criteria: 1. hourly_scores_day1 2. hourly_scores_day2 3. top_windows 4. windows_svg_static 5. fresh_run_1 6. fresh_run_2

Setup notes: Use Case Scout batch10 case k03-fishing-window-planner (offline fixture). Prepared with batch10/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Visual output (windows.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 6/6. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 4 (1 failed, recovered). Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.05
Wall time
1m
Human step-ins
0
Turns
1
Model usage
ModelInput tokOutput tokCost
OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on88,2372,121$0.05

Reproduce this run

  1. Use the exact prompt shown above for this run (the use case page has the shared prompt).
  2. Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via Codex CLI 0.163.0 (codex exec --json), cloud.
  3. It counts as a success only if it meets every success criterion: 1. hourly_scores_day1 2. hourly_scores_day2 3. top_windows 4. windows_svg_static 5. fresh_run_1 6. fresh_run_2
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{"type": "thread.started", "thread_id": "01a163f9-d217-7db1-aa83-ac092260af60"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll check the workspace, then write `solve.py` and verify the scoring and SVG output with synthetic inputs.\n"}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '\"'!node_modules'\"'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py' -g '\"'!node_modules'\"'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/110_k03/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"window_hours\": 3,\n \"tide_m\": [\n  0.78,\n  0.48,\n  0.44,\n  0.66,\n  1.09,\n  1.63,\n  2.13,\n  2.48,\n  2.58,\n  2.41,\n  2.01,\n  1.48,\n  0.96,\n  0.58,\n  0.42,\n  0.54,\n  0.89,\n  1.4,\n  1.93,\n  2.36,\n  2.57,\n  2.51,\n  2.2,\n  1.71,\n  1.17,\n  0.72,\n  0.46,\n  0.46,\n  0.72,\n  1.18,\n  1.72,\n  2.2,\n  2.51,\n  2.57,\n  2.36,\n  1.93,\n  1.4,\n  0.89,\n  0.54,\n  0.42,\n  0.58,\n  0.97,\n  1.49,\n  2.01,\n  2.41,\n  2.58,\n  2.47,\n  2.13,\n  1.63\n ],\n \"wind_kt\": [\n  14.7,\n  15.1,\n  10.0,\n  16.0,\n  15.7,\n  14.0,\n  15.8,\n  16.8,\n  17.3,\n  17.4,\n  15.8,\n  12.4,\n  6.6,\n  3.9,\n  4.0,\n  4.0,\n  1.6,\n  1.6,\n  0,\n  2.1,\n  4.8,\n  1.9,\n  3.5,\n  6.7,\n  4.2,\n  0.6,\n  1.2,\n  0.7,\n  1.9,\n  0,\n  1.1,\n  0.8,\n  8.6,\n  24.5,\n  0,\n  1.1,\n  20.0,\n  0,\n  0,\n  0,\n  1.7,\n  1.0,\n  0,\n  1.6,\n  3.3,\n  5.1,\n  25.5,\n  5.3\n ],\n \"pressure_hpa\": [\n  1014.0,\n  1013.7,\n  1013.5,\n  1013.3,\n  1012.6,\n  1012.7,\n  1011.7,\n  1010.4,\n  1010.0,\n  1009.4,\n  1008.7,\n  1008.7,\n  1008.9,\n  1008.6,\n  1008.8,\n  1008.7,\n  1008.3,\n  1008.2,\n  1007.3,\n  1007.6,\n  1007.6,\n  1007.9,\n  1008.0,\n  1008.4,\n  1008.1,\n  1007.7,\n  1007.5,\n  1006.5,\n  1005.9,\n  1005.8,\n  1005.6,\n  1005.7,\n  1004.9,\n  1004.6,\n  1004.2,\n  1004.0,\n  1003.9,\n  1004.6,\n  1004.2,\n  1004.7,\n  1006.0,\n  1006.9,\n  1007.2,\n  1006.7,\n  1007.3,\n  1006.9,\n  1006.6,\n  1006.4\n ],\n \"solunar_major\": [\n  [\n   6.75,\n   8.75\n  ],\n  [\n   19.15,\n   21.15\n  ],\n  [\n   31.55,\n   33.55\n  ],\n  [\n   43.95,\n   45.95\n  ]\n ],\n \"solunar_minor\": [\n  [\n   12.95,\n   13.95\n  ],\n  [\n   25.35,\n   26.35\n  ],\n  [\n   37.75,\n   38.75\n  ]\n ],\n \"daylight\": [\n  [\n   6,\n   18\n  ],\n  [\n   5,\n   18\n  ]\n ]\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/110_k03/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/110_k03/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json'", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import solve\nr=json.load(open('answer.json'))\nassert len(r['hourly_scores'])==48\nassert r['unfishable_hours']==2\nroot=ET.parse('windows.svg').getroot()\nns={'s':'http://www.w3.org/2000/svg'}\nassert root.find('.//*[@id=\\\"hourly-bars\\\"]') is not None\nfor i in range(1,len(r['windows'])+1):\n    assert root.find(f'.//s:rect[@id=\\\"window-{i}\\\"]',ns) is not None\nbase=dict(tide_m=[0.3*h for h in range(49)],wind_kt=[10]*48,pressure_hpa=[1000]*48,solunar_major=[],solunar_minor=[],daylight=[[0,24],[0,24]],window_hours=3)\nx=solve(base)\nassert x['hourly_scores']==[2]*48\nassert x['windows']==[dict(start=s,end=s+3,score=6) for s in [0,3,6]]\nbase.update(solunar_major=[[1,2]],solunar_minor=[[1.5,2.5]])\nx=solve(base)\nassert x['hourly_scores'][:4]==[2,6.5,3.5,2]\nbase['wind_kt']=[21]*48\nassert solve(base)==dict(hourly_scores=[None]*48,unfishable_hours=48,windows=[])\nprint('Checks passed. Selected windows:',r['windows'])\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import solve\nr=json.load(open('answer.json'))\nassert len(r['hourly_scores'])==48\nassert r['unfishable_hours']==2\nroot=ET.parse('windows.svg').getroot()\nns={'s':'http://www.w3.org/2000/svg'}\nassert root.find('.//*[@id=\\\"hourly-bars\\\"]') is not None\nfor i in range(1,len(r['windows'])+1):\n    assert root.find(f'.//s:rect[@id=\\\"window-{i}\\\"]',ns) is not None\nbase=dict(tide_m=[0.3*h for h in range(49)],wind_kt=[10]*48,pressure_hpa=[1000]*48,solunar_major=[],solunar_minor=[],daylight=[[0,24],[0,24]],window_hours=3)\nx=solve(base)\nassert x['hourly_scores']==[2]*48\nassert x['windows']==[dict(start=s,end=s+3,score=6) for s in [0,3,6]]\nbase.update(solunar_major=[[1,2]],solunar_minor=[[1.5,2.5]])\nx=solve(base)\nassert x['hourly_scores'][:4]==[2,6.5,3.5,2]\nbase['wind_kt']=[21]*48\nassert solve(base)==dict(hourly_scores=[None]*48,unfishable_hours=48,windows=[])\nprint('Checks passed. Selected windows:',r['windows'])\nPY\"", "aggregated_output": "Checks passed. Selected windows: [{'start': 6, 'end': 9, 'score': 12.0}, {'start': 29, 'end': 32, 'score': 10.0}, {'start': 11, 'end': 14, 'score': 8.0}]\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "agent_message", "text": "Created [solve.py](/home/bob/usecase-runs/batch/out/gpt-6.1-sol/110_k03/work/solve.py), and generated `answer.json` and `windows.svg`.\n\nVerified scoring boundaries, greedy window selection, unfishable hours, and SVG structure.\n\nRun with `python3 solve.py input.json answer.json` (`python` is unavailable here)."}}
{"type": "turn.completed", "usage": {"input_tokens": 88237, "cached_input_tokens": 78592, "cache_write_input_tokens": 0, "output_tokens": 2121, "reasoning_output_tokens": 9}}

Get new runs and ranking changes weekly