Claude Opus 5.5 (effort default (no --effort flag) · thinking on) on Backtest a moving-average crossover on a bundled synthetic price file

AClaude Opus 5.5default · thinking on · Claude Code 2.1.299 (headless -p) · cloud · 2026-10-20 · by @bobsiopener

Trust 80%Fully autonomoussuccess · first tryCompare

Estimated cost: $0.16
Wall time: 37s
Human step-ins: 0
Outcome: Pass

Compare with other models on this task →

Exact prompt

# Backtest a moving-average crossover on a bundled synthetic price file `prices.csv` is a **synthetic** daily closing-price series (no real market data), and `input.json` gives the rule's parameters. Write `solve.py` (Python 3 standard library only) so that `python solve.py input.json answer.json` backtests the rule exactly as specified. This is an offline exercise: no data downloads, no broker, no trading. We re-run it on fresh files, so do not hard-code anything. ## Input `input.json`: `prices_file` (CSV path, relative to the working directory), `fast`, `slow` (window lengths, fast < slow), `cost_bps` (cost per position change, in basis points), `initial` (starting equity). The CSV has a header `date,close`; dates are ISO `YYYY-MM-DD`. ## Rules 1. Cleaning: rows can be out of order and a date can appear more than once. For a repeated date, the **last** row in the file wins. Then sort by date. Let c[0..n-1] be the closes; `days` = n. 2. SMA_w(t) = mean of c[t-w+1 .. t] (inclusive). Signal s[t] = 1 if SMA_fast(t) > SMA_slow(t) (strictly), else 0. For t < slow - 1, s[t] = 0. 3. Position: p[0] = 0 and p[t] = s[t-1] for t >= 1 (trade on the next day's close). 4. Equity starts at `initial`. For t = 1..n-1: e[t] = e[t-1] * (1 + p[t] * (c[t]/c[t-1] - 1)) * (1 - cost * |p[t] - p[t-1]|), with cost = cost_bps / 10000. The daily strategy return is e[t]/e[t-1] - 1. 5. `trades` = number of t in 1..n-1 with p[t] != p[t-1]. `exposure` = (sum of p[1..n-1]) / (n - 1). 6. `total_return` = e[n-1]/initial - 1; `cagr` = (e[n-1]/initial) ** (252 / (n - 1)) - 1; `max_drawdown` = the largest value of 1 - e[t]/max(e[0..t]) (0 or positive); `sharpe` = mean(daily returns) / population standard deviation (`statistics.pstdev`) of daily returns * sqrt(252), or 0 if the deviation is 0; `buy_hold_return` = c[n-1]/c[0] - 1. 7. Round `final_equity` (= e[n-1]) to 2 decimals and the other fractions to 6 decimals when writing. ## Output `answer.json` keys: `days`, `final_equity`, `total_return`, `trades`, `cagr`, `max_drawdown`, `sharpe`, `exposure`, `buy_hold_return`. ## Chart (required visual) Also write `chart.svg` in the working directory: a static, self-contained SVG (root element `<svg>`, no scripts, no external images, fonts, stylesheets or links) that can be rendered headless. It must contain `<polyline id="equity">` drawing strategy equity by trading day. ## Scoring The `chart_svg_static` check (one more check, counted in partial credit) passes only if `chart.svg` exists after the fixture run, parses, has root `<svg>`, contains the ids above with the stated tags, and has no `<script>`, `<img>`/`<image>`, `<link>`, `<iframe>`/`<object>`/`<embed>`, `on*` handlers, or `href`/`src` pointing anywhere except `#...` or `data:`. Its drawing is not graded. Hidden checks (partial credit = fraction passed; all must pass): `data_cleaning` (days and buy-and-hold return), `equity_and_trades`, `risk_metrics` on the given files; `fresh_run_1`, `fresh_run_2` (all keys on freshly generated files). Tolerances: `final_equity` within 0.011, fractions within 0.00001, `days` and `trades` exact. Each run has a 60-second limit.

Duplicate dates (last wins) and swapped rows, next-day execution, strict crossover, cost per position change, population-deviation Sharpe and running-peak drawdown. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

These are synthetic test cases for comparing AI models, not financial advice.

Success criteria: 1. data_cleaning 2. equity_and_trades 3. risk_metrics 4. fresh_run_1 5. fresh_run_2 6. chart_svg_static

Setup notes: Use Case Scout batch 9 case j11-sma-crossover-backtest (offline fixture). Prepared with batch9/prepare.sh; hidden key/tests kept outside the agent's work dir. Prompt = TASK.md verbatim. Scored with the scout's scorer: `python score.py WORKDIR KEYDIR`. Tools: Read/Write/Edit/Glob/Grep/Bash, web disabled. Single run, no retries. Visual output (chart.svg) rendered offline to PNG with google-chrome headless, network blocked, 1280x800 at ~3 s after load. Claude Code's own reported cost for claude-opus-5.5: $0.160883. Score: 6/6. Cost is an ESTIMATE: measured tokens x pricing.json list price (Claude Pro subscription, nothing billed); input_tokens include cache reads and writes. Single run.

Estimated cost List-price estimate; not actually billed.

Output

Claude Opus 5.5 (effort default (no --effort flag) · thinking on) on Backtest a moving-average crossover on a bundled synthetic price fileSuccess

Metrics

Estimated cost: $0.16
Wall time: 37s
Human step-ins: 0
Turns: 4

Model usage
Model	Input tok	Output tok	Cost
AClaude Opus 5.5anthropic/claude-opus-5.5effort default (no --effort flag) · thinking on	81,523	3,382	$0.16

Reproduce this run

Use the use case prompt on the use case page.
Same setup: Claude Opus 5.5 (anthropic/claude-opus-5.5, effort default (no --effort flag) · thinking on) via Claude Code 2.1.299 (headless -p), cloud.
It counts as a success only if it meets every success criterion: 1. data_cleaning 2. equity_and_trades 3. risk_metrics 4. fresh_run_1 5. fresh_run_2 6. chart_svg_static
Log model, tokens, wall time, human step-ins and the evidence log, then submit it as a reproduction. A different model? Submit a head-to-head rerun.

How runs are scored: Methodology.

Was this useful?

⇄ Rerun on another model

Automated checks

Trust 80% comes from these checks (how Trust % works).

Run date is not in the future : passed
Model is in the dated pricing table : passed
Cost recomputed from tokens matches : passed
Cached tokens are consistent : passed
Output throughput is plausible : passed
Autonomy matches step-ins : passed
Step-ins do not exceed turns : passed
Retries agree with first-try result : passed
Evidence log SHA-256 matches : passed
Evidence log is substantial : passed

Evidence log

[
 {
  "duration_api_ms": 35737,
  "stop_reason": "end_turn",
  "session_id": "[REDACTED]",
  "total_cost_usd": 0.160883,
  "usage": {
   "input_tokens": 8,
   "cache_creation_input_tokens": 9860,
   "cache_read_input_tokens": 71655,
   "output_tokens": 3382,
   "output_tokens_details": {
    "thinking_tokens": 230
   },
   "server_tool_use": {
    "web_search_requests": 0,
    "web_fetch_requests": 0
   },
   "service_tier": "standard",
   "cache_creation": {
    "ephemeral_1h_input_tokens": 9860,
    "ephemeral_5m_input_tokens": 0
   },
   "inference_geo": "global",
   "iterations": [
    {
     "input_tokens": 2,
     "output_tokens": 612,
     "cache_read_input_tokens": 21604,
     "cache_creation_input_tokens": 402,
     "cache_creation": {
      "ephemeral_5m_input_tokens": 0,
      "ephemeral_1h_input_tokens": 402
     },
     "type": "message"
    }
   ],
   "speed": "standard",
   "fallback_credit": null
  },
  "modelUsage": {
   "claude-opus-5-5": {
    "inputTokens": 8,
    "outputTokens": 3382,
    "cacheReadInputTokens": 71655,
    "cacheCreationInputTokens": 9860,
    "webSearchRequests": 0,
    "costUSD": 0.160883,
    "contextWindow": 1000000,
    "maxOutputTokens": 128000,
    "thinkingTokens": 230,
    "canonicalModel": "claude-opus-5-5",
    "provider": "firstParty",
    "costBasis": "list"
   }
  },
  "permission_denials": [],
  "terminal_reason": "completed",
  "fast_mode_state": "off",
  "fast_mode_disabled_reason": "sdk_opt_in_required",
  "subagent_stats": {
   "spawned": 0,
   "requested": {
    "background": 0,
    "foreground": 0,
    "unset": 0
   },
   "started_in_background": 0,
   "max_depth": 0,
   "spawned_by_subagents": 0,
   "completed": 0,
   "failed": 0,
   "killed": {
    "parent": 0,
    "user": 0,
    "system": 0
   },
   "refused": {
    "depth_limit": 0,
    "concurrency_limit": 0,
    "budget": 0
   },
   "by_type": {}
  },
  "safety_stops": 0,
  "is_error": false,
  "num_turns": 4,
  "subtype": "success",
  "api_error_status": null,
  "result": "I wrote `solve.py` and ran it on the bundled files. It produced `answer.json` and `chart.svg` with no errors. I haven't checked the numbers against an independent calculation, and the hidden tests haven't been run.\n\n**Results on the given files:**\n\n| key | value |\n|---|---|\n| days | 509 |\n| final_equity | 8589.77 |\n| total_return | -0.141023 |\n| trades | 9 |\n| cagr | -0.072635 |\n| max_drawdown | 0.294082 |\n| sharpe | -0.450763 |\n| exposure | 0.356299 |\n| buy_hold_return | -0.260528 |\n\nThe day count is right: the file has 512 data rows and 3 repeated dates, which leaves 509 days.\n\n**How it works:**\n- **Cleaning:** when a date appears more than once, the last row in the file is kept. Rows are then sorted by date. Blank rows are skipped, and stray spaces or a BOM (byte-order mark) at the start of the file don't break the read.\n- **The rule:** it follows your spec step by step. The signal compares the fast and slow averages strictly, positions lag the signal by one day, cost applies on each position change, and the Sharpe ratio uses `statistics.pstdev`. Nothing is hard-coded; all parameters come from `input.json`.\n- **Chart:** `chart.svg` is written to the working directory. It draws the equity line as `<polyline id=\"equity\">` and adds axes, the start and end dates, and a dashed line at starting equity. It has no scripts or external links. I parsed it as a check: the root is `<svg>` and the `equity` id is there.\n\nTo re-run on fresh files: `python solve.py input.json answer.json`",
  "ttft_ms": 2876,
  "type": "result",
  "duration_ms": 36180,
  "uuid": "[REDACTED]",
  "ttft_stream_ms": 1681,
  "time_to_request_ms": 251,
  "first_content_frame_ms": 1681,
  "queued_turn_count": 0,
  "result_index": 0
 }
]