Skip to content

Build a csvq CLI (select/where/sort/limit/count, csv|json|table output, exact exit codes) from a written spec

Exact prompt

# Task: build the `csvq` command-line tool Write `csvq.py` (Python 3 standard library only, run as `python3 csvq.py ...`). It reads CSV (RFC 4180 quoting, UTF-8, header row required) from a file argument or from stdin when the file is `-` or omitted, and writes to stdout. `people.csv` is a sample. Options (may appear in any order): - `--select COLS` comma-separated column names to output, in that order (default: all, original order). - `--where EXPR` repeatable; all must match. EXPR is `col OP value` with OP one of `=`, `!=`, `<`, `<=`, `>`, `>=`, `~` (case-insensitive substring). If both the cell and value parse as numbers, compare numerically, otherwise as strings. An empty cell never matches `<`,`<=`,`>`,`>=`. - `--sort COL` ascending; `--sort -COL` descending. Numeric if every non-empty value in the column is a number, else string. Empty cells always sort last (both directions). Sort is stable. - `--limit N` output at most N rows (after filtering and sorting). - `--format csv|json|table` (default csv). csv: header + rows, quoted as needed by Python's csv module, `\n` line endings. json: a JSON array of objects (keys in output column order, all values strings), `ensure_ascii=False`, followed by a newline. table: columns left-aligned, padded to the widest cell (header included) with two spaces between columns, no trailing spaces, a dashes line under the header the width of each column. - `--count` print only the number of matching rows (after --where, ignoring --limit) and a newline. - `-h/--help` usage, exit 0. Errors go to stderr with a one-line message and a non-zero exit: unknown column (in any option) exits 2; bad usage (unknown option, bad --where, non-integer/negative --limit, bad --format) exits 2; unreadable input file exits 1. Nothing is printed to stdout on error. No network.

Byte-exact output: RFC 4180 quoting, numeric-vs-string comparison, empty cells sorting last in both directions, stable descending sort, `--sort -COL` argument parsing, table padding without trailing spaces, and distinct exit codes with clean stdout on error. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).

Success criteria: 1. 21 hidden invocations (alternating file argument and stdin) produce byte-identical stdout and exit code 0, or for error cases the exact exit code 2 with empty stdout and a stderr message. 2. A missing input file exits 1 with empty stdout and a stderr message. 3. -h exits 0 and its usage text mentions --where. 4. Reading stdin via '-' works.

Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.02, 1m).

Run this on another model

All runs (4)

100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ

Runs for this use case
ModelCostTimeStep-insⓘAutonomyOutcomeTrustⓘDateCompare
OGPT-6.1 Soldefault · thinking on$0.061m0Fully autonomouspassTrust 80%2026-10-22
AClaude Haiku 5.5default · thinking on$0.021m0Fully autonomouspassTrust 80%2026-10-17
AClaude Opus 5.5default · thinking on$0.2255s0Fully autonomouspassTrust 80%2026-10-17
AClaude Sonnet 5.5default · thinking on$0.0930s0Fully autonomouspassTrust 80%2026-10-17

Embed this result

A live badge for this use case. It links back here and updates as new runs are published.

UseCaseRank badge for Build a csvq CLI (select/where/sort/limit/count, csv|json|table output, exact exit codes) from a written spec

Get new runs and ranking changes weekly

Which models got better or cheaper on real tasks like this one.