Fix five behavioural bugs in an LRU cache with per-entry TTL
Exact prompt
# Task: fix the TTL cache `ttlcache.py` is an LRU cache with per-entry TTL. Users report: recently-read keys get evicted, expired keys are still counted in `len()`/`in`, entries that expire exactly at their deadline are still returned, overwriting a key does not refresh its LRU position, and `purge()` crashes. Intended behaviour: - `get` on a live key marks it most-recently-used; when the cache exceeds `maxsize` the **least**-recently-used entry is evicted. - An entry is expired when `clock() >= expires_at` (an entry with ttl=10 set at t=0 is gone at t=10). Expired entries are removed when touched by `get`, and are never visible via `len()` or `in`. - `set` on an existing key replaces the value, resets its TTL and makes it most-recently-used. - `ttl` of 0 or negative means "do not store" (and removes any existing entry for that key). - `purge()` removes all expired entries and returns how many were removed. - `hits`/`misses` count `get` outcomes. Keep the public API (constructor signature, method names). Add your own tests if you like. No network.
Bugs are spread over eviction order, LRU refresh on get/overwrite, an exact expiry boundary, expired-entry visibility, zero TTL and a dict-mutation crash; hidden tests use an injected clock. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. All 11 hidden unittest cases pass against the agent's ttlcache.py (60 s timeout): LRU eviction of least-recently-used, get/overwrite refresh recency, expiry at clock() >= expires_at, expired entries invisible to len/in, overwrite resets TTL, ttl<=0 not stored, purge() returns count without crashing, hits/misses counted, constructor API kept.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0056, 21s).
All runs (4)
100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.04 | 42s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.0056 | 21s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Opus 5.5default · thinking on | $0.16 | 40s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Sonnet 5.5default · thinking on | $0.05 | 10s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.