AIS collision risk: closest point of approach and alert levels
Exact prompt
# AIS collision risk: CPA, TCPA and alert levels `input.json` (synthetic; not for collision avoidance): `own` ship (`lat`, `lon`, `cog` degrees true, `sog` knots), `targets` (each `mmsi`, `lat`, `lon`, `cog`, `sog`), thresholds `cpa_danger_nm`, `cpa_caution_nm`, `tcpa_max_min`. Write `solve.py` (Python 3 standard library only); `python solve.py input.json answer.json` writes `answer.json` and `radar.svg`. Fresh inputs are used too. ## Rules 1. Skip a target (list its mmsi in `skipped`, ascending) if lat, lon, cog or sog is missing/null, sog >= 102.3 (AIS "not available"), or cog >= 360 (not available). Keep the rest in input order. 2. Local flat frame around own ship in nm: x = (lon - own lon) * 60 * cos(own lat), y = (lat - own lat) * 60. Velocity of a ship = (sog * sin(cog), sog * cos(cog)) knots. 3. Relative position r = target - own, relative velocity v = v_target - v_own. If |v|^2 < 1e-9, TCPA = 0. Otherwise TCPA (hours) = max(0, -(r . v) / |v|^2). CPA = |r + v * TCPA|. 4. `range_nm` = |r|, `bearing_deg` = atan2(x, y) mod 360 (true bearing from own ship), `tcpa_min` = TCPA * 60. 5. Level: `danger` if CPA <= cpa_danger and tcpa_min <= tcpa_max; else `caution` if CPA <= cpa_caution and tcpa_min <= tcpa_max; else `none`. 6. `alerts`: mmsi of danger then caution targets, each group sorted by tcpa_min then mmsi. Round range/CPA to 3 decimals, bearing to 1, tcpa_min to 2. ## Visual output: radar.svg Static, self-contained SVG (root `<svg>`, no scripts, images, links or external assets; renders headless) with an element `id="own-ship"` and one element `id="target-<mmsi>"` for every non-skipped target. The drawing itself is not graded. ## answer.json `{"targets": [{"mmsi", "range_nm", "bearing_deg", "cpa_nm", "tcpa_min", "level"}, ...], "skipped": [...], "alerts": [...]}` ## Scoring Hidden checks (partial credit; all must pass): `filtering`, `range_bearing`, `cpa_tcpa`, `levels_and_alerts`, `radar_svg_static` (radar.svg exists, follows the static rules, has `own-ship` and every `target-<mmsi>`), `fresh_run_1`, `fresh_run_2`. Tolerances: distances 0.002 nm, bearings 0.11, minutes 0.011 (fresh runs 0.011 for all numbers). 60-second limit per run.
AIS not-available sentinels (sog 102.3, cog 360), a stationary target, diverging targets with TCPA clamped to 0, longitude scaling, near-threshold crossing targets, and a two-key alert sort. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Benchmark code for comparing AI models, not navigation or safety software.
Success criteria: 1. filtering 2. range_bearing 3. cpa_tcpa 4. levels_and_alerts 5. radar_svg_static 6. fresh_run_1 7. fresh_run_2
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0068, 27s).
Latest output per model
All runs (9)
89% pass rate across 9 eligible runs: 8 pass, 0 partial, 1 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| MMistral Small 3.2 24B Instruct (Venice)thinking on | $0.0023 | 48s | 0 | Fully autonomous | fail | Trust 80% | 2026-10-29 | |
| DDeepSeek V4.1 Flash (Venice)thinking on | $0.0073 | 23s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-27 | |
| GGemini 3.8 Flashmedium · thinking on | $0.35 | 4m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| GGemini 4 Argonmedium · thinking on | $0.20 | 4m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| OGPT-6.1 Soldefault · thinking off | $0.05 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| QQwen3 Coder 480B Turbo (Venice)thinking on | $0.02 | 49s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-21 | |
| AClaude Haiku 5.5default · thinking on | $0.0068 | 27s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 | |
| AClaude Opus 5.5default · thinking on | $0.22 | 40s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 | |
| AClaude Sonnet 5.5default · thinking off | $0.06 | 19s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.