Write an atomic, idempotent SQLite v1→v2 migration with dedupe, FK and CHECK constraints
Exact prompt
# Task: write a safe schema migration (SQLite) `app.db` is at schema v1: `users(id, full_name, email, created 'DD/MM/YYYY')`, `orders(id, user_email, total_cents, status)`. Write `migrate.py` (stdlib sqlite3) that upgrades `app.db` IN PLACE to v2: - `users(id INTEGER PRIMARY KEY, first_name TEXT NOT NULL, last_name TEXT NOT NULL, email TEXT NOT NULL UNIQUE, created_at TEXT NOT NULL)` — trim names; first_name = first word, last_name = the rest ("Chloé de Vries" → "Chloé", "de Vries"); email lowercased+trimmed; created_at ISO `YYYY-MM-DD`. - Users whose emails differ only by case/whitespace are duplicates: keep the LOWEST id, delete the others, and re-point their orders. - `orders(id INTEGER PRIMARY KEY, user_id INTEGER NOT NULL REFERENCES users(id), total_cents INTEGER NOT NULL, status TEXT NOT NULL CHECK(status IN ('paid','refunded','pending')))` — user_id resolved via email (case-insensitive); status normalised (trim + lowercase). - Table `schema_version(version INTEGER)` containing exactly one row `2`. - Must be atomic (all-or-nothing in one transaction) and idempotent: running `python migrate.py` again on a v2 database must change nothing and exit 0. Keep order ids and kept user ids unchanged. Run it once so `app.db` is migrated. No network.
Scorer re-runs migrate.py twice on a fresh v1 db (dump hash must not change), checks exact schema, constraints, merged duplicate users and re-pointed orders. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. Scorer builds a fresh v1 database, runs migrate.py twice: both runs exit 0 and the database dump is identical after run 1 and run 2 (idempotent). 2. users schema exactly: id INTEGER, first_name TEXT NOT NULL, last_name TEXT NOT NULL, email TEXT NOT NULL, created_at TEXT NOT NULL (in that order); orders columns id, user_id, total_cents, status with the stated types; orders has a foreign key to users. 3. CHECK constraint rejects an invalid status; UNIQUE constraint rejects a duplicate email. 4. schema_version contains exactly one row with value 2. 5. User and order counts match the key (duplicates merged); orders per user and total cents per user match exactly (orders re-pointed to the kept lowest id); status counts after normalisation match. 6. Every kept user row (names split, email lowercased, ISO date) matches exactly. 7. The agent's own migrated app.db must also pass all the data/schema checks.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.01, 53s).
All runs (5)
100% pass rate across 5 eligible runs: 5 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.07 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| DDeepSeek V4.1 Flash (Venice)thinking on | $0.02 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 | |
| AClaude Haiku 5.5default · thinking on | $0.01 | 53s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Opus 5.5default · thinking on | $0.33 | 1.3h | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Sonnet 5.5default · thinking on | $0.09 | 32s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.