GPT-6.1 Sol (effort default (no reasoning-effort override) · thinking on) on Loan servicing with daily accrual, 30/360 quirks, dues crediting and a payment waterfall
OGPT-6.1 Soldefault · thinking on · Codex CLI 0.163.0 (codex exec --json) · cloud · 2026-10-22 · by @bobsiopener
- Estimated cost
- $0.06
- Wall time
- 1m
- Human step-ins
- 0
- Outcome
- Pass
Compare with other models on this task →
Summary
Exact prompt
# Loan servicing: daily accrual and payment waterfall `input.json` (synthetic) gives a `loan` (`principal`, `start_date`, `annual_rate`, `day_count` "ACT/365" or "30/360", `installment`, `due_day`, `grace_days`, `late_fee`), `payments`, `rate_changes`, `fees` (each with ISO `date`), and `end_date`. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json`. Fresh inputs are generated (both day counts, with and without payoff). One early mistake changes every later number, so follow the order exactly. ## Rules 1. Use exact decimals (`decimal.Decimal` built from the JSON numbers' string form). "Round" = half up. 2. Simulate each day d from the day after `start_date` through `end_date` inclusive. Within a day, in this order: (a) rate changes dated d take effect; (b) interest accrues; (c) fees dated d are added; (d) if d is a due date, a due is created; (e) payments dated d are applied, in input order; (f) late-fee checks; (g) if d is a due date, a statement is written. 3. Accrual (only while not paid off), added to `accrued` (unrounded): - ACT/365: round(principal * rate / 365, 8 decimals) every day. - 30/360: round(principal * rate / 360 * n, 8 decimals) where n = 1 for a normal day, **0 on the 31st**, and on the last day of February n = 30 - day + 1 (Feb 28 = 3, leap Feb 29 = 2). 4. Due date each month: day `due_day`, or the month's last day if shorter. No due is created on `start_date`'s day itself or after payoff. Due amount = min(installment, principal + accrued rounded **down** to cents + outstanding fees). 5. A payment first **credits dues**: open dues dated on or before the payment date, oldest first, each up to its remaining unpaid amount. Independently, the whole payment goes through the **waterfall**: outstanding fees oldest first, then interest = min(rest, accrued rounded down to cents), then principal; anything left is `refunded`. Interest paid reduces `accrued` exactly (sub-cent fractions stay). 6. When principal reaches 0: `payoff_date` = that date, accrued is set to 0, and accrual stops. 7. Late fee: on the day due date + `grace_days`, after payments, if that due is still not fully credited, add a fee of `late_fee` (kind "late") once for that due and mark it late. 8. Statement on each due date (rule 4's days, including after payoff): `{"date", "principal", "accrued_interest" (rounded to cents), "fees_outstanding", "rate"}`. ## answer.json `{"statements": [...], "payoff_date": date or null, "final": {"principal", "accrued_interest", "fees_outstanding"}, "totals": {"interest_paid", "fees_paid", "principal_paid", "refunded", "late_fees" (count)}, "dues": [{"date", "due", "paid", "late"}, ...]}` (money as JSON numbers). ## Scoring Hidden checks (partial credit; all must pass; money within 0.001): `first_3_statements`, `all_statements`, `dues_and_late_flags`, `totals`, `final_and_payoff` on the given input; `fresh_run_1`, `fresh_run_2`, `fresh_payoff_30_360`, `fresh_payoff_act365` (whole answer) on fresh inputs. 30-second limit per run.
Eight-decimal daily accrual vs cent rounding, the 30/360 day-31 and end-of-February rules, the within-day event order, dues crediting separate from the waterfall, interest paid in whole cents only, and payoff. Original benchmark task written for usecaserank.com. Headless, offline run on a reproducible fixture. Cost shown is a list-price API-equivalent estimate (run on a ChatGPT subscription, not billed).
Success criteria: 1. first_3_statements 2. all_statements 3. dues_and_late_flags 4. totals 5. final_and_payoff 6. fresh_run_1 7. fresh_run_2 8. fresh_payoff_30_360 9. fresh_payoff_act365
Setup notes: Use Case Scout batch12 case m03-loan-servicing-waterfall (offline fixture). Prepared with batch12/prepare.sh on the shared box; only TASK.md + starting files were copied to the agent machine, hidden key/tests stayed on the box. Prompt = TASK.md verbatim on stdin. Flags: --json -s workspace-write --skip-git-repo-check --ephemeral --ignore-user-config --ignore-rules -C <workdir>. Scored on the box with the scout's scorer: `python score.py WORKDIR KEYDIR`. Single run, no retries. User-level AGENTS.md (846 bytes, sha256 83ec4fe50da8) may have been in context; --ignore-user-config does not document excluding it. Serving model not confirmed: requested via -m; the event log does not echo the serving model id. Price source is secondary (pricing.json verification='secondary', https://www.actuia.com/en/news/ai-api-pricing-in-october-2026-050-to-50-for-output/), not an official OpenAI price page. Score: 9/9. Cost is an ESTIMATE: measured tokens x pricing.json list price (ChatGPT plan, nothing billed); input_tokens include cached reads; output_tokens include reasoning tokens. Tool calls 4 (1 failed, recovered). Single run.
Estimated cost List-price estimate; not actually billed.
Metrics
- Estimated cost
- $0.06
- Wall time
- 1m
- Human step-ins
- 0
- Turns
- 1
| Model | Input tok | Output tok | Cost |
|---|---|---|---|
| OGPT-6.1 Solopenai/gpt-6.1-soleffort default (no reasoning-effort override) · thinking on | 85,274 | 2,265 | $0.06 |
Reproduce this run
- Use the use case prompt on the use case page.
- Same setup: GPT-6.1 Sol (openai/gpt-6.1-sol, effort default (no reasoning-effort override) · thinking on) via
Codex CLI 0.163.0 (codex exec --json), cloud. - It counts as a success only if it meets every success criterion: 1. first_3_statements 2. all_statements 3. dues_and_late_flags 4. totals 5. final_and_payoff 6. fresh_run_1 7. fresh_run_2 8. fresh_payoff_30_360 9. fresh_payoff_act365
- Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.
How runs are scored: Methodology.
Was this useful?
Automated checks
Trust 80% comes from these checks (how Trust % works).
- Run date is not in the future : passed
- Model is in the dated pricing table : passed
- Cost recomputed from tokens matches : passed
- Cached tokens are consistent : passed
- Output throughput is plausible : passed
- Autonomy matches step-ins : passed
- Step-ins do not exceed turns : passed
- Retries agree with first-try result : passed
- Evidence log SHA-256 matches : passed
- Evidence log is substantial : passed
Evidence log
{"type": "thread.started", "thread_id": "01a16406-4401-7c11-bcf2-3c525d2912c6"}
{"type": "turn.started"}
{"type": "item.completed", "item": {"id": "item_0", "type": "agent_message", "text": "I\u2019ll implement the daily event order and Decimal calculations, then check payment allocation and payoff behavior with small examples.\n"}}
{"type": "item.started", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_1", "type": "command_execution", "command": "/bin/bash -lc \"pwd && rg --files -g 'AGENTS.md' -g 'input.json' -g 'solve.py'\"", "aggregated_output": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/122_m03/work\ninput.json\n", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_2", "type": "command_execution", "command": "/bin/bash -lc 'cat input.json'", "aggregated_output": "{\n \"loan\": {\n \"principal\": 26400,\n \"start_date\": \"2027-03-03\",\n \"annual_rate\": 0.0941,\n \"day_count\": \"ACT/365\",\n \"installment\": 844.56,\n \"due_day\": 1,\n \"grace_days\": 15,\n \"late_fee\": 25\n },\n \"payments\": [\n {\n \"date\": \"2027-04-06\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-04-30\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-06-02\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-07-03\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-08-02\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-08-31\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-10-01\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-11-06\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-12-06\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2027-12-06\",\n \"amount\": 52.96\n },\n {\n \"date\": \"2028-01-04\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-01-31\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-03-09\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-03-09\",\n \"amount\": 34.42\n },\n {\n \"date\": \"2028-04-01\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-05-06\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-06-13\",\n \"amount\": 694.74\n },\n {\n \"date\": \"2028-07-13\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-08-26\",\n \"amount\": 1435.13\n },\n {\n \"date\": \"2028-09-01\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-10-01\",\n \"amount\": 844.56\n },\n {\n \"date\": \"2028-12-01\",\n \"amount\": 545.3\n },\n {\n \"date\": \"2029-01-01\",\n \"amount\": 1463.25\n }\n ],\n \"rate_changes\": [\n {\n \"date\": \"2027-06-25\",\n \"annual_rate\": 0.1026\n },\n {\n \"date\": \"2028-08-18\",\n \"annual_rate\": 0.0872\n }\n ],\n \"fees\": [\n {\n \"date\": \"2027-09-01\",\n \"amount\": 54.43,\n \"kind\": \"service\"\n },\n {\n \"date\": \"2027-08-13\",\n \"amount\": 21.56,\n \"kind\": \"nsf\"\n },\n {\n \"date\": \"2028-09-02\",\n \"amount\": 16.17,\n \"kind\": \"nsf\"\n }\n ],\n \"end_date\": \"2029-01-11\"\n}", "exit_code": 0, "status": "completed"}}
{"type": "item.started", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/122_m03/work/solve.py", "kind": "add"}], "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_3", "type": "file_change", "changes": [{"path": "/home/bob/usecase-runs/batch/out/gpt-6.1-sol/122_m03/work/solve.py", "kind": "add"}], "status": "completed"}}
{"type": "item.started", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json'", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_4", "type": "command_execution", "command": "/bin/bash -lc 'python solve.py input.json answer.json'", "aggregated_output": "/bin/bash: line 1: python: command not found\n", "exit_code": 127, "status": "failed"}}
{"type": "item.started", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json\npython3 - <<'PY'\nfrom solve import solve\nfrom decimal import Decimal as D\n\ndef case(day_count, start, end, principal=100, rate=0, payments=None, grace=0):\n return solve({'loan': {'principal': principal, 'annual_rate': rate, 'start_date': start, 'day_count': day_count, 'installment': 50, 'due_day': 1, 'grace_days': grace, 'late_fee': 5}, 'end_date': end, 'payments': payments or []})\nr=case('ACT/365','2028-01-31','2028-03-01', payments=[{'date':'2028-02-01','amount':110}])\nassert r['payoff_date']=='2028-02-01'\nassert r['totals']['refunded']==10\nassert len(r['dues'])==1 and not r['dues'][0]['late']\nassert len(r['statements'])==2\nfor dc, start, end, expected in [('30/360','2027-02-27','2027-02-28',D('3')),('30/360','2028-02-28','2028-02-29',D('2')),('30/360','2028-01-30','2028-01-31',D('0')),('ACT/365','2028-02-28','2028-02-29',D('0.99'))]:\n r=case(dc,start,end,principal=360,rate=1)\n assert r['final']['accrued_interest']==expected, r\nr=case('ACT/365','2028-01-31','2028-02-02',payments=[{'date':'2028-02-01','amount':20}])\nassert r['dues'][0]['paid']==20 and r['dues'][0]['late']\nassert r['final']['fees_outstanding']==5\nprint('Passed payoff, post-payoff statements, late-fee ordering, and day-count checks.')\nPY\"", "aggregated_output": "", "exit_code": null, "status": "in_progress"}}
{"type": "item.completed", "item": {"id": "item_5", "type": "command_execution", "command": "/bin/bash -lc \"python3 solve.py input.json answer.json\npython3 - <<'PY'\nfrom solve import solve\nfrom decimal import Decimal as D\n\ndef case(day_count, start, end, principal=100, rate=0, payments=None, grace=0):\n return solve({'loan': {'principal': principal, 'annual_rate': rate, 'start_date': start, 'day_count': day_count, 'installment': 50, 'due_day': 1, 'grace_days': grace, 'late_fee': 5}, 'end_date': end, 'payments': payments or []})\nr=case('ACT/365','2028-01-31','2028-03-01', payments=[{'date':'2028-02-01','amount':110}])\nassert r['payoff_date']=='2028-02-01'\nassert r['totals']['refunded']==10\nassert len(r['dues'])==1 and not r['dues'][0]['late']\nassert len(r['statements'])==2\nfor dc, start, end, expected in [('30/360','2027-02-27','2027-02-28',D('3')),('30/360','2028-02-28','2028-02-29',D('2')),('30/360','2028-01-30','2028-01-31',D('0')),('ACT/365','2028-02-28','2028-02-29',D('0.99'))]:\n r=case(dc,start,end,principal=360,rate=1)\n assert r['final']['accrued_interest']==expected, r\nr=case('ACT/365','2028-01-31','2028-02-02',payments=[{'date':'2028-02-01','amount':20}])\nassert r['dues'][0]['paid']==20 and r['dues'][0]['late']\nassert r['final']['fees_outstanding']==5\nprint('Passed payoff, post-payoff statements, late-fee ordering, and day-count checks.')\nPY\"", "aggregated_output": "Passed payoff, post-payoff statements, late-fee ordering, and day-count checks.\n", "exit_code": 0, "status": "completed"}}
{"type": "item.completed", "item": {"id": "item_6", "type": "agent_message", "text": "Created [solve.py](/home/bob/usecase-runs/batch/out/gpt-6.1-sol/122_m03/work/solve.py) and generated [answer.json](/home/bob/usecase-runs/batch/out/gpt-6.1-sol/122_m03/work/answer.json).\n\nVerified payoff, post-payoff statements, late-fee ordering, and day-count edge cases.\n\nRun with `python3 solve.py input.json answer.json` (`python` is unavailable here)."}}
{"type": "turn.completed", "usage": {"input_tokens": 85274, "cached_input_tokens": 71936, "cache_write_input_tokens": 0, "output_tokens": 2265, "reasoning_output_tokens": 78}}