Backtest a moving-average crossover on a bundled synthetic price file
Exact prompt
# Backtest a moving-average crossover on a bundled synthetic price file `prices.csv` is a **synthetic** daily closing-price series (no real market data), and `input.json` gives the rule's parameters. Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` backtests the rule exactly as specified. This is an offline exercise: no data downloads, no broker, no trading. We re-run it on fresh files, so do not hard-code anything. ## Input `input.json`: `prices_file` (CSV path, relative to the working directory), `fast`, `slow` (window lengths, fast < slow), `cost_bps` (cost per position change, in basis points), `initial` (starting equity). The CSV has a header `date,close`; dates are ISO `YYYY-MM-DD`. ## Rules 1. Cleaning: rows can be out of order and a date can appear more than once. For a repeated date, the **last** row in the file wins. Then sort by date. Let c[0..n-1] be the closes; `days` = n. 2. SMA_w(t) = mean of c[t-w+1 .. t] (inclusive). Signal s[t] = 1 if SMA_fast(t) > SMA_slow(t) (strictly), else 0. For t < slow - 1, s[t] = 0. 3. Position: p[0] = 0 and p[t] = s[t-1] for t >= 1 (trade on the next day's close). 4. Equity starts at `initial`. For t = 1..n-1: e[t] = e[t-1] * (1 + p[t] * (c[t]/c[t-1] - 1)) * (1 - cost * |p[t] - p[t-1]|), with cost = cost_bps / 10000. The daily strategy return is e[t]/e[t-1] - 1. 5. `trades` = number of t in 1..n-1 with p[t] != p[t-1]. `exposure` = (sum of p[1..n-1]) / (n - 1). 6. `total_return` = e[n-1]/initial - 1; `cagr` = (e[n-1]/initial) ** (252 / (n - 1)) - 1; `max_drawdown` = the largest value of 1 - e[t]/max(e[0..t]) (0 or positive); `sharpe` = mean(daily returns) / population standard deviation (`statistics.pstdev`) of daily returns * sqrt(252), or 0 if the deviation is 0; `buy_hold_return` = c[n-1]/c[0] - 1. 7. Round `final_equity` (= e[n-1]) to 2 decimals and the other fractions to 6 decimals when writing. ## Output `answer.json` keys: `days`, `final_equity`, `total_return`, `trades`, `cagr`, `max_drawdown`, `sharpe`, `exposure`, `buy_hold_return`. ## Chart (required visual) Also write `chart.svg` in the working directory: a static, self-contained SVG (root element `<svg>`, no scripts, no external images, fonts, stylesheets or links) that can be rendered headless. It must contain `<polyline id="equity">` drawing strategy equity by trading day. ## Scoring The `chart_svg_static` check (one more check, counted in partial credit) passes only if `chart.svg` exists after the fixture run, parses, has root `<svg>`, contains the ids above with the stated tags, and has no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, or `href`/`src` pointing anywhere except `#...` or `data:`. Its drawing is not graded. Hidden checks (partial credit = fraction passed; all must pass): `data_cleaning` (days and buy-and-hold return), `equity_and_trades`, `risk_metrics` on the given files; `fresh_run_1`, `fresh_run_2` (all keys on freshly generated files). Tolerances: `final_equity` within 0.011, fractions within 0.00001, `days` and `trades` exact. Each run has a 60-second limit.
Duplicate dates (last wins) and swapped rows, next-day execution, strict crossover, cost per position change, population-deviation Sharpe and running-peak drawdown. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
These are synthetic test cases for comparing AI models, not financial advice.
Success criteria: 1. data_cleaning 2. equity_and_trades 3. risk_metrics 4. fresh_run_1 5. fresh_run_2 6. chart_svg_static
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0047, 21s).
Latest output per model
All runs (13)
69% pass rate across 13 eligible runs: 9 pass, 2 partial, 2 fail. Partial and failed runs are listed below with everything else.ⓘ
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.