Skip to content

GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking off) on Retirement drawdown with a 4% rule, inflation raises and a skip-raise-after-loss guardrail

OGPT-6.1 Soldefault · thinking off · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first try
Estimated cost
$0.05
Wall time
2m
Human step-ins
0
Outcome
Pass

Compare with other models on this task →

Summary

Exact prompt

# Retirement drawdown: the 4% rule with inflation raises and a guardrail `input.json` describes one retiree and one assumed sequence of yearly market returns and inflation rates (synthetic data). Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` reads the input and writes the answer. We re-run it on fresh inputs of the same shape, so do not hard-code anything. ## Input `start_age`, `end_age` (integers), `start_balance`, `withdrawal_rate` (e.g. 0.04), `fee_rate` (yearly fraction), `returns` and `inflation` (lists of fractions, one per year, length `end_age - start_age`). ## Rules 1. There are n = `end_age - start_age` years, k = 0 .. n-1. Year k is the year the retiree is age `start_age + k`. 2. The planned withdrawal for year 0 is W0 = `start_balance * withdrawal_rate`. For k >= 1 the planned withdrawal is last year's planned withdrawal times `(1 + inflation[k-1])`. 3. **Guardrail scenario only:** for k >= 1, if `returns[k-1] < 0` (strictly) the raise is skipped that year: the planned withdrawal stays equal to last year's planned withdrawal. Later raises apply to the frozen amount; there is no catch-up. 4. Each year, the withdrawal is taken at the start of the year: actual withdrawal w = min(planned, balance). Then balance = (balance - w) * (1 + returns[k]) * (1 - fee_rate). 5. `depletion_age` = `start_age + k` for the first year k where w < planned (the full planned amount could not be paid), otherwise `null`. `shortfall_total` = sum over all years of (planned - w). `total_withdrawn` = sum of w. 6. `path` = the balance at the end of each year (after rule 4), n values. `final_balance` = the last one. `real_final_balance` = `final_balance` divided by the product of `(1 + inflation[k])` over all n years. 7. Keep full floating-point precision while computing; round money outputs to 2 decimals only when writing. ## Output `answer.json`: `{"standard": S, "guardrail": G}` where each of S and G is an object with keys `final_balance`, `real_final_balance`, `total_withdrawn`, `shortfall_total`, `depletion_age`, `path`. "standard" ignores rule 3; "guardrail" applies it. ## Chart (required visual) Also write `chart.svg` in the working directory: a static, self-contained SVG (root element `<svg>`, no scripts, no external images, fonts, stylesheets or links) that can be rendered headless. It must contain `<polyline id="standard">` and `<polyline id="guardrail">` drawing the two `path` balance series against year. ## Scoring The `chart_svg_static` check (one more check, counted in partial credit) passes only if `chart.svg` exists after the fixture run, parses, has root `<svg>`, contains the ids above with the stated tags, and has no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, or `href`/`src` pointing anywhere except `#...` or `data:`. Its drawing is not graded. Hidden checks (partial credit = fraction passed; all must pass). Money values must match the reference within 0.011; `depletion_age` exactly; `path` must have exactly n values. - `standard_path`, `standard_summary` (the five summary keys), `guardrail_scenario`: on the given `input.json`. - `fresh_standard`, `fresh_guardrail`: on a fresh input. - `depletion_and_shortfall`: both scenarios on a fresh input where the money runs out. Each run has a 60-second limit.

Exact year-by-year balances depend on withdrawal timing, fee application order, the strict-negative guardrail with no catch-up, and how depletion and shortfall are recorded; fresh inputs include one where money runs out. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. standard_path 2. standard_summary 3. guardrail_scenario 4. fresh_standard 5. fresh_guardrail 6. depletion_and_shortfall 7. chart_svg_static

Setup notes: Batch 9 case j01-retirement-drawdown (offline fixture). Prepared with batch9/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the case scorer using absolute paths: `python score.py WORKDIR KEYDIR`. Single run, no retries. A global user-level AGENTS.md was present on the agent machine and may have been in context. The served model id is not echoed in the event log. Visual output (chart.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 7/7. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 4 (1 failed, recovered). Single run.

Estimated cost List-price estimate; not actually billed.

Output

Metrics

Estimated cost
$0.05
Wall time
2m
Human step-ins
0
Turns
1
Model usage
ModelInput tokOutput tokCost
OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking off83,3112,014$0.05

Reproduce this run

  1. Use the use case prompt on the use case page.
  2. Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking off) via Codex CLI 0.163.0 (codex exec --json), cloud.
  3. It counts as a success only if it meets every success criterion: 1. standard_path 2. standard_summary 3. guardrail_scenario 4. fresh_standard 5. fresh_guardrail 6. depletion_and_shortfall 7. chart_svg_static
  4. Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Evidence log
{"type": "thread.started", "thread_id": "01a1593f-e44b-7292-b78e-4f1e6c91ec14"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll write `solve.py` to calculate both scenarios and generate `chart.svg`, then check it against the supplied input and a depletion case."}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/j01/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"start_age\": 65,\n \"end_age\": 91,\n \"start_balance\": 1275000,\n \"withdrawal_rate\": 0.0393,\n \"fee_rate\": 0.0014,\n \"returns\": [\n  0.0744,\n  -0.0838,\n  -0.2331,\n  0.0428,\n  0.0457,\n  0.0935,\n  0.0244,\n  0.0895,\n  0.2534,\n  0.1454,\n  -0.0412,\n  0.3123,\n  -0.0709,\n  0.0895,\n  0.2531,\n  0.1087,\n  -0.1148,\n  0.114,\n  0.1936,\n  0.1639,\n  -0.0092,\n  0.1757,\n  -0.1251,\n  0.1493,\n  0.1664,\n  -0.0368\n ],\n \"inflation\": [\n  0.0371,\n  0.0125,\n  0.0295,\n  0.02,\n  0.0339,\n  0.0199,\n  0.0296,\n  0.0294,\n  0.029,\n  0.0277,\n  0.0272,\n  0.0137,\n  0.0291,\n  0.0122,\n  0.0283,\n  0.0177,\n  0.0227,\n  0.0165,\n  0.0288,\n  0.0321,\n  0.0361,\n  0.0251,\n  0.0396,\n  0.0268,\n  0.0315,\n  0.0183\n ]\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/j01/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/j01/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc \"python solve.py input.json answer.json && python - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import simulate\nr=json.load(open('answer.json'))\nassert all(len(v['path']) == 26 for v in r.values())\nroot=ET.parse('chart.svg').getroot()\nassert root.tag == '{http://www.w3.org/2000/svg}svg'\nassert {p.attrib['id'] for p in root.findall('{http://www.w3.org/2000/svg}polyline')} == {'standard', 'guardrail'}\nd=dict(start_age=65,end_age=68,start_balance=100,withdrawal_rate=.6,fee_rate=0,returns=[-.5,0,0],inflation=[.1,.1,.1])\ns=simulate(d,False)\ng=simulate(d,True)\nassert s['path'] == [20,0,0] and s['depletion_age']==66\nassert abs(s['shortfall_total']-118.6)<1e-9\nassert g['path']==[20,0,0] and g['depletion_age']==66\nassert abs(g['shortfall_total']-106)<1e-9\nprint('Fixture, SVG, and depletion checks passed.')\nprint({k:v['final_balance'] for k,v in r.items()})\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc \"python solve.py input.json answer.json && python - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import simulate\nr=json.load(open('answer.json'))\nassert all(len(v['path']) == 26 for v in r.values())\nroot=ET.parse('chart.svg').getroot()\nassert root.tag == '{http://www.w3.org/2000/svg}svg'\nassert {p.attrib['id'] for p in root.findall('{http://www.w3.org/2000/svg}polyline')} == {'standard', 'guardrail'}\nd=dict(start_age=65,end_age=68,start_balance=100,withdrawal_rate=.6,fee_rate=0,returns=[-.5,0,0],inflation=[.1,.1,.1])\ns=simulate(d,False)\ng=simulate(d,True)\nassert s['path'] == [20,0,0] and s['depletion_age']==66\nassert abs(s['shortfall_total']-118.6)<1e-9\nassert g['path']==[20,0,0] and g['depletion_age']==66\nassert abs(g['shortfall_total']-106)<1e-9\nprint('Fixture, SVG, and depletion checks passed.')\nprint({k:v['final_balance'] for k,v in r.items()})\nPY\"", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import simulate\nr=json.load(open('answer.json'))\nassert all(len(v['path']) == 26 for v in r.values())\nroot=ET.parse('chart.svg').getroot()\nassert root.tag == '{http://www.w3.org/2000/svg}svg'\nassert {p.attrib['id'] for p in root.findall('{http://www.w3.org/2000/svg}polyline')} == {'standard', 'guardrail'}\nd=dict(start_age=65,end_age=68,start_balance=100,withdrawal_rate=.6,fee_rate=0,returns=[-.5,0,0],inflation=[.1,.1,.1])\ns=simulate(d,False)\ng=simulate(d,True)\nassert s['path'] == [20,0,0] and s['depletion_age']==66\nassert abs(s['shortfall_total']-118.6)<1e-9\nassert g['path']==[20,0,0] and g['depletion_age']==66\nassert abs(g['shortfall_total']-106)<1e-9\nprint('Fixture, SVG, and depletion checks passed.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json && python3 - <<'PY'\nimport json\nimport xml.etree.ElementTree as ET\nfrom solve import simulate\nr=json.load(open('answer.json'))\nassert all(len(v['path']) == 26 for v in r.values())\nroot=ET.parse('chart.svg').getroot()\nassert root.tag == '{http://www.w3.org/2000/svg}svg'\nassert {p.attrib['id'] for p in root.findall('{http://www.w3.org/2000/svg}polyline')} == {'standard', 'guardrail'}\nd=dict(start_age=65,end_age=68,start_balance=100,withdrawal_rate=.6,fee_rate=0,returns=[-.5,0,0],inflation=[.1,.1,.1])\ns=simulate(d,False)\ng=simulate(d,True)\nassert s['path'] == [20,0,0] and s['depletion_age']==66\nassert abs(s['shortfall_total']-118.6)<1e-9\nassert g['path']==[20,0,0] and g['depletion_age']==66\nassert abs(g['shortfall_total']-106)<1e-9\nprint('Fixture, SVG, and depletion checks passed.')\nPY\"", "aggregated_output": "Fixture, SVG, and depletion checks passed.\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "agent_message", "text": "Created `solve.py`, generated `answer.json` and `chart.svg`, and verified the supplied input, required SVG polylines, and a depletion case.\n\nRun with `python3 solve.py input.json answer.json`. This environment has no `python` alias."}}
{"type": "turn.completed", "usage": {"input_tokens": 83311, "cached_input_tokens": 71040, "cache_write_input_tokens": 0, "output_tokens": 2014, "reasoning_output_tokens": 0}}

Get new runs and ranking changes weekly