Build a csvq CLI (select/where/sort/limit/count, csv|json|table output, exact exit codes) from a written spec
Exact prompt
# Task: build the `csvq` command-line tool Write `csvq.py` (Python 3 standard library only, run as `python3 csvq.py ...`). It reads CSV (RFC 4180 quoting, UTF-8, header row required) from a file argument or from stdin when the file is `-` or omitted, and writes to stdout. `people.csv` is a sample. Options (may appear in any order): - `--select COLS` comma-separated column names to output, in that order (default: all, original order). - `--where EXPR` repeatable; all must match. EXPR is `col OP value` with OP one of `=`, `!=`, `<`, `<=`, `>`, `>=`, `~` (case-insensitive substring). If both the cell and value parse as numbers, compare numerically, otherwise as strings. An empty cell never matches `<`,`<=`,`>`,`>=`. - `--sort COL` ascending; `--sort -COL` descending. Numeric if every non-empty value in the column is a number, else string. Empty cells always sort last (both directions). Sort is stable. - `--limit N` output at most N rows (after filtering and sorting). - `--format csv|json|table` (default csv). csv: header + rows, quoted as needed by Python's csv module, `\n` line endings. json: a JSON array of objects (keys in output column order, all values strings), `ensure_ascii=False`, followed by a newline. table: columns left-aligned, padded to the widest cell (header included) with two spaces between columns, no trailing spaces, a dashes line under the header the width of each column. - `--count` print only the number of matching rows (after --where, ignoring --limit) and a newline. - `-h/--help` usage, exit 0. Errors go to stderr with a one-line message and a non-zero exit: unknown column (in any option) exits 2; bad usage (unknown option, bad --where, non-integer/negative --limit, bad --format) exits 2; unreadable input file exits 1. Nothing is printed to stdout on error. No network.
Byte-exact output: RFC 4180 quoting, numeric-vs-string comparison, empty cells sorting last in both directions, stable descending sort, `--sort -COL` argument parsing, table padding without trailing spaces, and distinct exit codes with clean stdout on error. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. 21 hidden invocations (alternating file argument and stdin) produce byte-identical stdout and exit code 0, or for error cases the exact exit code 2 with empty stdout and a stderr message. 2. A missing input file exits 1 with empty stdout and a stderr message. 3. -h exits 0 and its usage text mentions --where. 4. Reading stdin via '-' works.
Best model for this task so far: Claude Haiku 5.5 (effort default (no --effort flag) · thinking on) (success, $0.02, 1m).
All runs (4)
100% pass rate across 4 eligible runs: 4 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.06 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-22 | |
| AClaude Haiku 5.5default · thinking on | $0.02 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Opus 5.5default · thinking on | $0.22 | 55s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 | |
| AClaude Sonnet 5.5default · thinking on | $0.09 | 30s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-17 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.