Skip to content

Multi-window burn-rate alerts from per-minute request counts (integer thresholds, flap smoothing, page-over-ticket priority)

Exact prompt

# Multi-window burn-rate alerts from per-minute request counts `minutes.csv` has one day of per-minute request counts for a service with a **99.9% availability SLO** (error budget 0.001). Write `slo.py` (Python 3 stdlib only) that, run as `python slo.py` in this directory, reads `minutes.csv` and writes `alerts.csv` and `summary.json` following these rules. We re-run it on a freshly generated file, so do not hard-code anything. ## Input cleaning 1. The first line is a header. Each other row must have exactly 3 fields `minute,total,errors`; each field is stripped of spaces and must parse as an integer, otherwise the row is skipped. Rows with a negative value or `errors > total` are skipped. 2. If several valid rows have the same minute, the **last** valid one in the file wins. 3. Minutes run from 0 to M, where M is the largest valid minute. Minutes with no valid row have total = errors = 0. ## Burn rates 4. For minute i and window w (minutes), the window is minutes `max(0, i-w+1) .. i` (shorter at the start of the day). With T and E the sums of total and errors in it, the window is **hot at threshold X** when T > 0 and E / T >= X × 0.001. Compare exactly with integers: `E * 10000 >= (X*10) * T` (X*10 is 144 or 60). 5. A minute is a **page** minute if the 60-minute and the 5-minute windows are both hot at 14.4. It is a **ticket** candidate if the 360-minute and the 30-minute windows are both hot at 6. 6. **Flap smoothing for pages:** for every two page minutes a < b with no page minute between them and b - a equal to 2 or 3 (one or two quiet minutes between), the minutes between become page minutes too. 7. **Pages win:** ticket minutes = ticket candidates that are not page minutes (after rule 6). Then apply the same smoothing as rule 6 to ticket minutes, except a gap is only filled if none of the gap minutes is a page minute. 8. An **alert** is a maximal run of consecutive minutes of the same severity. ## Output 9. `alerts.csv`: header `severity,start,end`, one row per alert, `severity` is `page` or `ticket`, `start`/`end` are the first/last minute (inclusive). Sorted by start (page before ticket on equal start, which cannot normally happen). 10. `summary.json`: an object with `budget_remaining` = `1 - (all errors / all totals) / 0.001` over minutes 0..M rounded to 4 decimals with Python `round` (1.0 if all totals are 0; it can be negative), `page_minutes` and `ticket_minutes` (counts after rules 6-7) and `minutes` = M + 1. ## Scoring Hidden checks (partial credit = fraction passed; all must pass): `page_alerts_exact` and `ticket_alerts_exact` (rows of that severity in `alerts.csv` match exactly), `summary_exact` (all 4 values; `budget_remaining` within 0.00005), then `slo.py` is re-run on a fresh file and `fresh_page_alerts`, `fresh_ticket_alerts`, `fresh_summary` are checked the same way. Each run has a 60-second limit.

Exact alert intervals depend on input cleaning (last valid duplicate wins), short windows at the start, exact integer comparisons, page smoothing, then ticket smoothing that may not bridge page minutes; a fresh-seed rerun blocks hand-editing. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. page_alerts_exact 2. ticket_alerts_exact 3. summary_exact 4. fresh_page_alerts 5. fresh_ticket_alerts 6. fresh_summary

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0059, 26s).

Run this on another model

All runs (4)

100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
OGPT-6.1 Soldefault · thinking on$0.0344s0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.005926s0Fully autonomouspassTrust 80%2026-10-20
AClaude Opus 5.5default · thinking on$0.1328s0Fully autonomouspassTrust 80%2026-10-20
AClaude Sonnet 5.5default · thinking on$0.0616s0Fully autonomouspassTrust 80%2026-10-20

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Multi-window burn-rate alerts from per-minute request counts (integer thresholds, flap smoothing, page-over-ticket priority)

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.