Fix catastrophic-backtracking (ReDoS) regexes in signup validators without changing what they accept
Exact prompt
# Task: fix a ReDoS in the signup validators A pen-test found that crafted inputs make `validate.py` hang the API worker (catastrophic regex backtracking). Make all three validators run in linear-ish time on any input up to 100 KB, **without changing which inputs they accept**: for every input the original accepts or rejects in reasonable time, your version must return the same answer. Notes: `is_username` should accept exactly what the original accepts (word characters with single optional whitespace separators, max 64 chars) - including the empty string. Keep function names/signatures. Python stdlib only (`re` is fine; so is not using regex at all). Write `SECURITY_FIX.md` (2+ lines). No network.
Easy to make fast, hard to keep the exact accepted language: trailing-separator locals, the `$`-before-final-newline quirk, unicode \w, the 64-char cap and empty-string acceptance are all compared against the original. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. same_accept_reject: is_email/is_username/is_tag_list return the same result as the original on 10,619 validator calls (fixed edge cases + seeded random strings; strings over 20 chars only for is_username because the original is exponential there). 2. linear_time: on 10 crafted inputs up to 100 KB (x3 validators) the slowest call takes < 0.25 s (subprocess capped at 20 s). 3. notes_present: SECURITY_FIX.md has at least 2 lines.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.02, 4m).
All runs (4)
100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.05 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.02 | 4m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Opus 5.5default · thinking on | $0.27 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Sonnet 5.5default · thinking on | $0.10 | 40s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.