Skip to content

Fix five behavioural bugs in an LRU cache with per-entry TTL

Exact prompt

# Task: fix the TTL cache `ttlcache.py` is an LRU cache with per-entry TTL. Users report: recently-read keys get evicted, expired keys are still counted in `len()`/`in`, entries that expire exactly at their deadline are still returned, overwriting a key does not refresh its LRU position, and `purge()` crashes. Intended behaviour: - `get` on a live key marks it most-recently-used; when the cache exceeds `maxsize` the **least**-recently-used entry is evicted. - An entry is expired when `clock() >= expires_at` (an entry with ttl=10 set at t=0 is gone at t=10). Expired entries are removed when touched by `get`, and are never visible via `len()` or `in`. - `set` on an existing key replaces the value, resets its TTL and makes it most-recently-used. - `ttl` of 0 or negative means "do not store" (and removes any existing entry for that key). - `purge()` removes all expired entries and returns how many were removed. - `hits`/`misses` count `get` outcomes. Keep the public API (constructor signature, method names). Add your own tests if you like. No network.

Bugs are spread over eviction order, LRU refresh on get/overwrite, an exact expiry boundary, expired-entry visibility, zero TTL and a dict-mutation crash; hidden tests use an injected clock. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. All 11 hidden unittest cases pass against the agent's ttlcache.py (60 s timeout): LRU eviction of least-recently-used, get/overwrite refresh recency, expiry at clock() >= expires_at, expired entries invisible to len/in, overwrite resets TTL, ttl<=0 not stored, purge() returns count without crashing, hits/misses counted, constructor API kept.

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.0056, 21s).

Run this on another model

All runs (4)

100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
OGPT-6.1 Soldefault · thinking on$0.0442s0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.005621s0Fully autonomouspassTrust 80%2026-10-17
AClaude Opus 5.5default · thinking on$0.1640s0Fully autonomouspassTrust 80%2026-10-17
AClaude Sonnet 5.5default · thinking on$0.0510s0Fully autonomouspassTrust 80%2026-10-17

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Fix five behavioural bugs in an LRU cache with per-entry TTL

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.