Make a TCP key-value server follow its wire protocol (pipelining, split reads, binary payloads, expiry, close rules)

The starting server assumes one command per recv and one client at a time; a correct fix needs a per-connection buffer, length-prefixed payload framing that tolerates CRLF inside payloads, per-client threads with a shared locked store, lazy expiry and exact error/close semantics. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. basic_commands: PING/SET/GET/DEL/INCR exact replies incl. empty payload, case-insensitive commands, case-sensitive keys, negative INCR. 2. error_replies: unknown command, wrong args, bad length and non-integer INCR give the exact errors and keep the connection open. 3. pipelining: 601 pipelined commands in one write get 601 in-order replies. 4. split_reads_binary: a SET with a payload containing CRLF/NUL/0xFF sent one byte at a time round-trips exactly. 5. concurrent_clients: a client stalled mid-payload does not block others; 20 clients x 50 INCR on one key gives 1000. 6. expiry: EXPIRE semantics incl. fractional seconds, :0 for missing keys, SET clearing expiry. 7. connection_closing_rules: line too long, bad payload terminator, payload > 1 MiB and QUIT each send the right reply and close the connection. 8. survives_bad_clients: after junk bytes and clients disconnecting mid-command, the server still answers PING.

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.03, 2m).

Run this on another model

All runs (3)

100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ