GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking off) on Spreadsheet-grade loan amortization for 8 loans incl. rate reset and extra payments
OGPT-6.1 Soldefault · thinking off · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener
- Estimated cost
- $0.03
- Wall time
- 38s
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
# Task: loan amortization (spreadsheet-grade precision) `loans.csv` lists 8 loans: principal, annual_rate_pct (nominal, compounded monthly), years, extra_monthly (extra principal paid each month from `extra_from_month` on, 1-based), optional `rate_reset_month` + `new_rate_pct`. Rules (follow exactly, use decimal arithmetic, round HALF-UP to cents at each step): 1. Scheduled payment = P·i/(1−(1+i)^−n) with i = rate/12/100, n = years·12, rounded to cents. If rate is 0, payment = P/n rounded. 2. Each month: interest = round(balance·i); principal part = payment − interest; plus the extra payment if active. 3. If principal part + extra ≥ remaining balance, the final payment = interest + remaining balance (no extra that month) and the loan ends. 4. At `rate_reset_month` (before computing that month's interest), switch to the new rate and recompute the payment on the current balance over the ORIGINAL remaining term (n − reset_month + 1 months). Write `results.json`: {"L1": {"monthly_payment": "1234.56" (the initial scheduled payment, string), "total_interest": "…" (string, 2dp), "payoff_month": int}, ...} and `L3_schedule.csv` with header `month,payment,interest,principal,balance` (payment includes extra; principal includes extra; strings with 2dp). No network.
Exact-to-the-cent with HALF_UP per-step rounding, 0% loans, final-payment adjustment (one 60-month loan pays off in month 61 due to rounding), payment recompute at rate reset. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).
Success criteria: 1. For each of the 8 loans: monthly_payment and total_interest must equal the key exactly to the cent (Decimal compare), and payoff_month must be exactly right. 2. L3_schedule.csv must have header month,payment,interest,principal,balance and the exact number of rows, with every month and every amount matching exactly.
Setup notes: Use Case Scout batch4 case d09-loan-amortization-finance (offline fixture). Prepared with batch4/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR/key.json`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 2/2 checks. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 4 (1 failed, recovered). Single run.
Estimated cost List-price estimate; not actually billed.
Metrics
- Estimated cost
- $0.03
- Wall time
- 38s
- Human step-ins
- 0
- Turns
- 1
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking off | 81,808 | 1,165 | $0.03 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking off) via
Codex CLI 0.163.0 (codex exec --json), cloud. - It counts as a success only if it meets every success criterion: 1. For each of the 8 loans: monthly_payment and total_interest must equal the key exactly to the cent (Decimal compare), and payoff_month must be exactly right. 2. L3_schedule.csv must have header month,payment,interest,principal,balance and the exact number of rows, with every month and every amount matching exactly.
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
{"type": "thread.started", "thread_id": "01a16429-b4a1-7961-a4db-b604f7c127d5"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll inspect `loans.csv`, calculate the schedules with Decimal and HALF-UP cent rounding, then verify and write both output files."}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'loans.csv' -g '*.py' -g 'results.json' -g 'L3_schedule.csv'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'loans.csv' -g '*.py' -g 'results.json' -g 'L3_schedule.csv'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/156_d09/work\nloans.csv\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat loans.csv'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat loans.csv'", "aggregated_output": "loan_id,principal,annual_rate_pct,years,extra_monthly,extra_from_month,rate_reset_month,new_rate_pct\r\nL1,120000,0,15,0,1,,\r\nL2,450000,6.125,30,250,25,,\r\nL3,8000,4.9,30,250,25,37,6.75\r\nL4,120000,0,5,0,1,,\r\nL5,25000,7.5,25,0,1,,\r\nL6,310000,4.9,5,0,1,37,6.75\r\nL7,8000,6.125,30,0,1,,\r\nL8,450000,7.5,15,0,25,,\r\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc \"cat > /tmp/amortize_156.py <<'PY'\nimport csv\nimport json\nfrom decimal import Decimal, localcontext, ROUND_HALF_UP\nfrom pathlib import Path\n\nCENT = Decimal('0.01')\nZERO = Decimal('0.00')\ndef cents(value):\n return value.quantize(CENT, rounding=ROUND_HALF_UP)\ndef scheduled(balance, rate, months):\n if not rate:\n return cents(balance / months)\n return cents(balance * rate / (1 - (1 + rate) ** (-months)))\ndef calculate(precision):\n results = {}\n schedules = {}\n with localcontext() as ctx:\n ctx.prec = precision\n ctx.rounding = ROUND_HALF_UP\n with open('loans.csv', newline='') as source:\n loans = list(csv.DictReader(source))\n assert len(loans) == 8\n for loan in loans:\n balance = Decimal(loan['principal'])\n starting_balance = balance\n rate = Decimal(loan['annual_rate_pct']) / 12 / 100\n term = int(loan['years']) * 12\n payment = scheduled(balance, rate, term)\n initial_payment = payment\n extra = Decimal(loan['extra_monthly'])\n extra_from = int(loan['extra_from_month'])\n reset = int(loan['rate_reset_month']) if loan['rate_reset_month'] else None\n total_interest = ZERO\n rows = []\n month = 0\n while balance > 0:\n month += 1\n assert month < 10000, 'Loan failed to amortize'\n if month == reset:\n rate = Decimal(loan['new_rate_pct']) / 12 / 100\n payment = scheduled(balance, rate, term - month + 1)\n interest = cents(balance * rate)\n principal = payment - interest\n active_extra = extra if month >= extra_from else ZERO\n if principal + active_extra >= balance:\n principal = balance\n actual_payment = interest + balance\n else:\n principal += active_extra\n actual_payment = payment + active_extra\n previous_balance = balance\n balance = cents(balance - principal)\n total_interest = cents(total_interest + interest)\n assert actual_payment == interest + principal\n assert previous_balance - principal == balance\n rows.append((month, *(format(v, '.2f') for v in (actual_payment, interest, principal, balance))))\n assert sum((Decimal(r[3]) for r in rows), ZERO) == starting_balance\n assert sum((Decimal(r[1]) for r in rows), ZERO) == starting_balance + total_interest\n results[loan['loan_id']] = dict(monthly_payment=format(initial_payment, '.2f'), total_interest=format(total_interest, '.2f'), payoff_month=month)\n schedules[loan['loan_id']] = rows\n return results, schedules\n\nresults, schedules = calculate(80)\nassert (results, schedules) == calculate(160), 'Precision stability check failed'\nPath('results.json').write_text(json.dumps(results, indent=2) + '\\\\n')\nwith open('L3_schedule.csv', 'w', newline='') as output:\n writer = csv.writer(output)\n writer.writerow(['month', 'payment', 'interest', 'principal', 'balance'])\n writer.writerows(schedules['L3'])\nwith open('results.json') as source:\n assert json.load(source) == results\nwith open('L3_schedule.csv', newline='') as source:\n rows = list(csv.DictReader(source))\n assert len(rows) == results['L3']['payoff_month']\n assert rows[-1]['balance'] == '0.00'\nprint(json.dumps(results, indent=2))\nprint('Verified all balance/payment identities and stability at 80 and 160 decimal digits.')\nprint('L3 reset and final rows:')\nfor row in schedules['L3']:\n if row[0] in (36, 37, results['L3']['payoff_month']):\n print(row)\nPY\npython /tmp/amortize_156.py\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "command_execution", "command": "/bin/bash -lc \"cat > /tmp/amortize_156.py <<'PY'\nimport csv\nimport json\nfrom decimal import Decimal, localcontext, ROUND_HALF_UP\nfrom pathlib import Path\n\nCENT = Decimal('0.01')\nZERO = Decimal('0.00')\ndef cents(value):\n return value.quantize(CENT, rounding=ROUND_HALF_UP)\ndef scheduled(balance, rate, months):\n if not rate:\n return cents(balance / months)\n return cents(balance * rate / (1 - (1 + rate) ** (-months)))\ndef calculate(precision):\n results = {}\n schedules = {}\n with localcontext() as ctx:\n ctx.prec = precision\n ctx.rounding = ROUND_HALF_UP\n with open('loans.csv', newline='') as source:\n loans = list(csv.DictReader(source))\n assert len(loans) == 8\n for loan in loans:\n balance = Decimal(loan['principal'])\n starting_balance = balance\n rate = Decimal(loan['annual_rate_pct']) / 12 / 100\n term = int(loan['years']) * 12\n payment = scheduled(balance, rate, term)\n initial_payment = payment\n extra = Decimal(loan['extra_monthly'])\n extra_from = int(loan['extra_from_month'])\n reset = int(loan['rate_reset_month']) if loan['rate_reset_month'] else None\n total_interest = ZERO\n rows = []\n month = 0\n while balance > 0:\n month += 1\n assert month < 10000, 'Loan failed to amortize'\n if month == reset:\n rate = Decimal(loan['new_rate_pct']) / 12 / 100\n payment = scheduled(balance, rate, term - month + 1)\n interest = cents(balance * rate)\n principal = payment - interest\n active_extra = extra if month >= extra_from else ZERO\n if principal + active_extra >= balance:\n principal = balance\n actual_payment = interest + balance\n else:\n principal += active_extra\n actual_payment = payment + active_extra\n previous_balance = balance\n balance = cents(balance - principal)\n total_interest = cents(total_interest + interest)\n assert actual_payment == interest + principal\n assert previous_balance - principal == balance\n rows.append((month, *(format(v, '.2f') for v in (actual_payment, interest, principal, balance))))\n assert sum((Decimal(r[3]) for r in rows), ZERO) == starting_balance\n assert sum((Decimal(r[1]) for r in rows), ZERO) == starting_balance + total_interest\n results[loan['loan_id']] = dict(monthly_payment=format(initial_payment, '.2f'), total_interest=format(total_interest, '.2f'), payoff_month=month)\n schedules[loan['loan_id']] = rows\n return results, schedules\n\nresults, schedules = calculate(80)\nassert (results, schedules) == calculate(160), 'Precision stability check failed'\nPath('results.json').write_text(json.dumps(results, indent=2) + '\\\\n')\nwith open('L3_schedule.csv', 'w', newline='') as output:\n writer = csv.writer(output)\n writer.writerow(['month', 'payment', 'interest', 'principal', 'balance'])\n writer.writerows(schedules['L3'])\nwith open('results.json') as source:\n assert json.load(source) == results\nwith open('L3_schedule.csv', newline='') as source:\n rows = list(csv.DictReader(source))\n assert len(rows) == results['L3']['payoff_month']\n assert rows[-1]['balance'] == '0.00'\nprint(json.dumps(results, indent=2))\nprint('Verified all balance/payment identities and stability at 80 and 160 decimal digits.')\nprint('L3 reset and final rows:')\nfor row in schedules['L3']:\n if row[0] in (36, 37, results['L3']['payoff_month']):\n print(row)\nPY\npython /tmp/amortize_156.py\"", "aggregated_output": "/bin/bash: line 84: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python3 /tmp/amortize_156.py'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python3 /tmp/amortize_156.py'", "aggregated_output": "{\n \"L1\": {\n \"monthly_payment\": \"666.67\",\n \"total_interest\": \"0.00\",\n \"payoff_month\": 180\n },\n \"L2\": {\n \"monthly_payment\": \"2734.25\",\n \"total_interest\": \"429664.00\",\n \"payoff_month\": 297\n },\n \"L3\": {\n \"monthly_payment\": \"42.46\",\n \"total_interest\": \"1316.13\",\n \"payoff_month\": 54\n },\n \"L4\": {\n \"monthly_payment\": \"2000.00\",\n \"total_interest\": \"0.00\",\n \"payoff_month\": 60\n },\n \"L5\": {\n \"monthly_payment\": \"184.75\",\n \"total_interest\": \"30423.20\",\n \"payoff_month\": 300\n },\n \"L6\": {\n \"monthly_payment\": \"5835.89\",\n \"total_interest\": \"42814.62\",\n \"payoff_month\": 61\n },\n \"L7\": {\n \"monthly_payment\": \"48.61\",\n \"total_interest\": \"9498.22\",\n \"payoff_month\": 360\n },\n \"L8\": {\n \"monthly_payment\": \"4171.56\",\n \"total_interest\": \"300879.30\",\n \"payoff_month\": 180\n }\n}\nVerified all balance/payment identities and stability at 80 and 160 decimal digits.\nL3 reset and final rows:\n(36, '292.46', '19.70', '272.76', '4552.75')\n(37, '280.58', '25.61', '254.97', '4297.78')\n(54, '17.71', '0.10', '17.61', '0.00')\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "agent_message", "text": "Created [results.json](results.json) and [L3_schedule.csv](L3_schedule.csv).\n\nVerified payment and balance totals, and identical results at 80- and 160-digit Decimal precision using HALF-UP cent rounding."}}
{"type": "turn.completed", "usage": {"input_tokens": 81808, "cached_input_tokens": 77056, "cache_write_input_tokens": 0, "output_tokens": 1165, "reasoning_output_tokens": 0}}