Skip to content

Progressive tax on fictional brackets, and pre-tax vs after-tax savings comparison

Exact prompt

# Progressive tax on fictional brackets, and pre-tax ("traditional") vs after-tax ("Roth") savings `input.json` contains a **fictional** tax system (brackets and standard deductions per filing status), a list of incomes to tabulate, and savings scenarios. Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` computes the answers below. Use only the numbers in the input. We re-run it on fresh inputs, so do not hard-code anything. ## Rules 1. Taxable income = max(0, gross income - `standard_deduction[filing]`). 2. Brackets for a filing status are a list of `[lower_bound, rate]` sorted by bound. Rate j applies to the part of taxable income above `lower_bound_j` and up to `lower_bound_{j+1}` (the last bracket has no upper bound). Tax = sum over brackets. 3. Marginal rate = the rate of the last bracket whose lower bound is strictly below taxable income; if taxable income is 0, the first bracket's rate. 4. `tax_table`: for each `[filing, income]` in input order output `{"filing", "income", "tax", "effective_rate", "marginal_rate"}`; effective_rate = tax / income (0 when income is 0), 6 decimals. 5. Scenarios (each has `name`, `filing`, `wages`, `contribution`, `years`, `growth`, `retire_filing`, `retire_other_income`): - `tax_now_traditional` = tax on (wages - contribution); `tax_now_roth` = tax on wages; `tax_saved_now` = their difference; `marginal_rate_now` = marginal rate (rule 3) at the taxable income of (wages - contribution). - `traditional_balance` F = contribution * (1 + growth) ** years. `withdrawal_tax` = tax(retire_other_income + F) - tax(retire_other_income), both with `retire_filing`, taken as one withdrawal in a single year. `traditional_after_tax` = F - withdrawal_tax. - Same out-of-pocket cost: the after-tax account receives (contribution - tax_saved_now), so `roth_after_tax` = (contribution - tax_saved_now) * (1 + growth) ** years. - `winner` = "traditional" or "roth", whichever after-tax value is larger, or "tie" if they differ by less than 0.005. 6. Round money to 2 decimals and rates to 6 only when writing (bracket rates are written as given). ## Output `answer.json`: `{"tax_table": [...], "scenarios": [{"name", "tax_now_traditional", "tax_now_roth", "tax_saved_now", "marginal_rate_now", "traditional_balance", "withdrawal_tax", "traditional_after_tax", "roth_after_tax", "winner"}, ...]}` in input order. ## Scoring Hidden checks (partial credit = fraction passed; all must pass), money within 0.011, rates within 0.000001, winners exact: `tax_table_amounts`, `effective_and_marginal`, `scenario_tax_now`, `scenario_future_values`, `scenario_winners` on the given input; `fresh_run_1`, `fresh_run_2`: the whole answer on fresh inputs (tolerance 0.011). Each run has a 60-second limit.

Incomes placed exactly on bracket boundaries test the strict marginal-rate rule; the equal out-of-pocket comparison and the lump-withdrawal tax at retirement decide winners. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. tax_table_amounts 2. effective_and_marginal 3. scenario_tax_now 4. scenario_future_values 5. scenario_winners 6. fresh_run_1 7. fresh_run_2

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0054, 18s).

Run this on another model

All runs (12)

75% pass rate across 12 eligible runs: 9 pass, 0 partial, 3 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
?MiniMax M2.5 (Venice)thinking on$0.027m0Fully autonomouspassTrust 80%2026-10-27
ZGLM 4.7 (Venice)thinking on$0.012m0Fully autonomouspassTrust 80%2026-10-27
MMistral Small 3.2 24B Instruct (Venice)thinking on$0.002220s0Fully autonomousfailTrust 80%2026-10-26
KKimi K3.1 Pro (Venice)thinking on$0.092m0Fully autonomouspassTrust 80%2026-10-26
GGemma 4 31B Instruct (Venice)thinking on$0.00743m0Fully autonomouspassTrust 80%2026-10-24
QQwen3 Coder 480B Turbo (Venice)thinking on$0.005134s0Fully autonomousfailTrust 80%2026-10-24
DDeepSeek V4.1 Flash (Venice)thinking on$0.006728s0Fully autonomouspassTrust 80%2026-10-24
DDeepSeek V3.2 (Venice)thinking on$0.0520m0Fully autonomousfailTrust 80%2026-10-23
OGPT-6.1 Soldefault · thinking off$0.0548s0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.005418s0Fully autonomouspassTrust 80%2026-10-20
AClaude Opus 5.5default · thinking on$0.1422s0Fully autonomouspassTrust 80%2026-10-20
AClaude Sonnet 5.5default · thinking on$0.0621s0Fully autonomouspassTrust 80%2026-10-20

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Progressive tax on fictional brackets, and pre-tax vs after-tax savings comparison

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.