Skip to content

Backtest a moving-average crossover on a bundled synthetic price file

Exact prompt

# Backtest a moving-average crossover on a bundled synthetic price file `prices.csv` is a **synthetic** daily closing-price series (no real market data), and `input.json` gives the rule's parameters. Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` backtests the rule exactly as specified. This is an offline exercise: no data downloads, no broker, no trading. We re-run it on fresh files, so do not hard-code anything. ## Input `input.json`: `prices_file` (CSV path, relative to the working directory), `fast`, `slow` (window lengths, fast < slow), `cost_bps` (cost per position change, in basis points), `initial` (starting equity). The CSV has a header `date,close`; dates are ISO `YYYY-MM-DD`. ## Rules 1. Cleaning: rows can be out of order and a date can appear more than once. For a repeated date, the **last** row in the file wins. Then sort by date. Let c[0..n-1] be the closes; `days` = n. 2. SMA_w(t) = mean of c[t-w+1 .. t] (inclusive). Signal s[t] = 1 if SMA_fast(t) > SMA_slow(t) (strictly), else 0. For t < slow - 1, s[t] = 0. 3. Position: p[0] = 0 and p[t] = s[t-1] for t >= 1 (trade on the next day's close). 4. Equity starts at `initial`. For t = 1..n-1: e[t] = e[t-1] * (1 + p[t] * (c[t]/c[t-1] - 1)) * (1 - cost * |p[t] - p[t-1]|), with cost = cost_bps / 10000. The daily strategy return is e[t]/e[t-1] - 1. 5. `trades` = number of t in 1..n-1 with p[t] != p[t-1]. `exposure` = (sum of p[1..n-1]) / (n - 1). 6. `total_return` = e[n-1]/initial - 1; `cagr` = (e[n-1]/initial) ** (252 / (n - 1)) - 1; `max_drawdown` = the largest value of 1 - e[t]/max(e[0..t]) (0 or positive); `sharpe` = mean(daily returns) / population standard deviation (`statistics.pstdev`) of daily returns * sqrt(252), or 0 if the deviation is 0; `buy_hold_return` = c[n-1]/c[0] - 1. 7. Round `final_equity` (= e[n-1]) to 2 decimals and the other fractions to 6 decimals when writing. ## Output `answer.json` keys: `days`, `final_equity`, `total_return`, `trades`, `cagr`, `max_drawdown`, `sharpe`, `exposure`, `buy_hold_return`. ## Chart (required visual) Also write `chart.svg` in the working directory: a static, self-contained SVG (root element `<svg>`, no scripts, no external images, fonts, stylesheets or links) that can be rendered headless. It must contain `<polyline id="equity">` drawing strategy equity by trading day. ## Scoring The `chart_svg_static` check (one more check, counted in partial credit) passes only if `chart.svg` exists after the fixture run, parses, has root `<svg>`, contains the ids above with the stated tags, and has no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, or `href`/`src` pointing anywhere except `#...` or `data:`. Its drawing is not graded. Hidden checks (partial credit = fraction passed; all must pass): `data_cleaning` (days and buy-and-hold return), `equity_and_trades`, `risk_metrics` on the given files; `fresh_run_1`, `fresh_run_2` (all keys on freshly generated files). Tolerances: `final_equity` within 0.011, fractions within 0.00001, `days` and `trades` exact. Each run has a 60-second limit.

Duplicate dates (last wins) and swapped rows, next-day execution, strict crossover, cost per position change, population-deviation Sharpe and running-peak drawdown. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. data_cleaning 2. equity_and_trades 3. risk_metrics 4. fresh_run_1 5. fresh_run_2 6. chart_svg_static

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0047, 21s).

Latest output per model

Run this on another model

All runs (13)

69% pass rate across 13 eligible runs: 9 pass, 2 partial, 2 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
ZGLM 4.7 (Venice)thinking on$0.031m0Fully autonomouspassTrust 80%2026-10-29
GGemini 4 Argonmedium · thinking on$0.092m0Fully autonomouspassTrust 80%2026-10-28
KKimi K3.1 Pro (Venice)thinking on$0.235m0Fully autonomouspassTrust 80%2026-10-26
MMistral Small 3.2 24B Instruct (Venice)thinking on$0.008427s0Fully autonomousfailTrust 80%2026-10-25
GGemma 4 31B Instruct (Venice)thinking on$0.00694m0Fully autonomouspassTrust 80%2026-10-24
KKimi K2.5 (Venice)thinking on$0.051m0Fully autonomousfailTrust 80%2026-10-24
QQwen3 Coder 480B Turbo (Venice)thinking on$0.0225s0Fully autonomouspassTrust 80%2026-10-24
DDeepSeek V4.1 Flash (Venice)thinking on$0.009130s0Fully autonomouspartialTrust 80%2026-10-23
OGPT-6.1 Soldefault · thinking on$0.122m0Fully autonomouspassTrust 80%2026-10-22
?MiniMax M2.5 (Venice)thinking on$0.016m0Fully autonomouspartialTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.004721s0Fully autonomouspassTrust 80%2026-10-20
AClaude Opus 5.5default · thinking on$0.1637s0Fully autonomouspassTrust 80%2026-10-20
AClaude Sonnet 5.5default · thinking off$0.0618s0Fully autonomouspassTrust 80%2026-10-20

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Backtest a moving-average crossover on a bundled synthetic price file

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.