Skip to content

Seeded Monte Carlo retirement odds against a stated success threshold

Exact prompt

# Seeded Monte Carlo retirement odds `input.json` holds the assumptions for a Monte Carlo retirement simulation (synthetic). Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` runs the simulation exactly as specified and writes the statistics. Because the random sequence is fixed by the seed, the answer is exact. We re-run it on fresh inputs, so do not hard-code anything. ## Input `seed`, `sims`, `years` (integers), `start_balance`, `annual_spend` (first-year spending), `mean_return`, `sd_return`, `mean_inflation`, `sd_inflation`, `success_threshold` (fractions). ## Rules 1. Create one generator `rng = random.Random(seed)` and use it for everything, in this order. Run simulations s = 0 .. sims-1 one after another. 2. Each simulation starts with balance = `start_balance` and spend = `annual_spend` (as floats). For each year y = 0 .. years-1: draw `r = rng.gauss(mean_return, sd_return)`, then `i = rng.gauss(mean_inflation, sd_inflation)` (both are drawn every year, in that order, before anything else), then balance -= spend. 3. If the balance is now below 0, the simulation **fails** in year y+1: its ending balance is 0 and it stops immediately (no more draws for it). Otherwise balance *= (1 + r) and spend *= (1 + i). 4. A simulation that never fails succeeds; its ending balance is the balance after the last year. 5. `success_rate` = successes / sims (round to 6 decimals). `meets_threshold` = (unrounded success rate >= `success_threshold`). `sims_failed` = number of failures. 6. Percentiles: sort all ending balances ascending (failed ones count as 0) into E; `pX` = E[int(X/100 * (sims - 1))] for X = 10, 50, 90 (round to 2 decimals). 7. `median_failure_year`: sort the failure years ascending into F and take F[int(0.5 * (len(F) - 1))]; `null` if nothing failed. 8. `best_sim` = index of the highest ending balance (lowest index on ties). `earliest_failure_sim` = index of the simulation that failed in the earliest year (lowest index on ties); `null` if nothing failed. ## Output `answer.json`: an object with keys `success_rate`, `meets_threshold`, `sims_failed`, `p10`, `p50`, `p90`, `median_failure_year`, `best_sim`, `earliest_failure_sim`. ## Chart (required visual) Also write `chart.svg` in the working directory: a static, self-contained SVG (root element `<svg>`, no scripts, no external images, fonts, stylesheets or links) that can be rendered headless. It must contain a percentile fan: `<polyline id="p10">`, `<polyline id="p50">`, `<polyline id="p90">`, each the 10th/50th/90th percentile end-of-year balance across simulations for each year (a failed simulation counts as 0 from its failure year). ## Scoring The `chart_svg_static` check (one more check, counted in partial credit) passes only if `chart.svg` exists after the fixture run, parses, has root `<svg>`, contains the ids above with the stated tags, and has no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, or `href`/`src` pointing anywhere except `#...` or `data:`. Its drawing is not graded. Hidden checks (partial credit = fraction passed; all must pass): `success_rate_and_threshold` (rate within 0.000001, the other two exact), `percentiles` (within 0.011), `failure_stats` (exact), `best_sim` (exact) on the given input; `fresh_run_1` and `fresh_run_2`: every key on a fresh input (money within 0.011, rates within 0.011, counts and indexes exact). Each run has a 120-second limit.

With a fixed seed the success rate is exact only if the draw order (return then inflation, every year, stop on failure) and the percentile/tie rules are followed exactly; approximate simulations score near zero. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. success_rate_and_threshold 2. percentiles 3. failure_stats 4. best_sim 5. fresh_run_1 6. fresh_run_2 7. chart_svg_static

Best model for this task so far: DeepSeek V4.1 Flash (Venice) (thinking on) (success, $0.0058, 25s). Best AI model for Finance / Retirement Planning →

Latest output per model

Run this on another model

All runs (12)

83% pass rate across 12 eligible runs: 10 pass, 1 partial, 1 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
GGemini 4 Argonmedium · thinking on$0.081m0Fully autonomouspassTrust 80%2026-10-28
ZGLM 4.7 (Venice)thinking on$0.045m0Fully autonomouspassTrust 80%2026-10-27
QQwen3 Coder 480B Turbo (Venice)thinking on$0.012m0Fully autonomouspassTrust 80%2026-10-27
DDeepSeek V4.1 Flash (Venice)thinking on$0.005825s0Fully autonomouspassTrust 80%2026-10-27
KKimi K3.1 Pro (Venice)thinking on$0.195m0Fully autonomouspassTrust 80%2026-10-27
DDeepSeek V3.2 (Venice)thinking on$0.0414m0Fully autonomousfailTrust 80%2026-10-26
GGemma 4 31B Instruct (Venice)thinking on$0.0112m0Fully autonomouspassTrust 80%2026-10-24
MMistral Small 3.2 24B Instruct (Venice)thinking on$0.00141m0Fully autonomouspartialTrust 80%2026-10-23
OGPT-6.1 Soldefault · thinking off$0.041m0Fully autonomouspassTrust 80%2026-10-20
AClaude Haiku 5.5default · thinking on$0.005924s0Fully autonomouspassTrust 80%2026-10-20
AClaude Opus 5.5default · thinking on$0.1633s0Fully autonomouspassTrust 80%2026-10-20
AClaude Sonnet 5.5default · thinking off$0.0620s0Fully autonomouspassTrust 80%2026-10-20

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Seeded Monte Carlo retirement odds against a stated success threshold

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.