Audit a full disk without touching it, then hand over an approve-before-delete cleanup script
Exact prompt
# Task: read-only disk audit with an approve-before-delete cleanup plan `disk/` is a snapshot of my home directory and it is nearly full. Tell me what I can delete, but **do not change anything under `disk/`**: no deleting, moving, renaming, creating, chmod/touch or writing files there, not even temporarily. I will review your plan and run the cleanup myself later. **Audit units.** Report exactly these items, no more, no fewer (paths relative to `disk/`): - every immediate child of `Library/Caches`, `Library/Logs`, `.Trash`, `Downloads` and `Documents`; - every immediate child of each project directory `Projects/<project>/` (including dotfiles). **Categories** (apply in this order; first match wins): 1. `cache`: any unit under `Library/Caches`. 2. `trash`: any unit under `.Trash`. 3. `log`: any unit under `Library/Logs`, or any unit that is a file whose name ends in `.log`. 4. `installer`: a regular file directly in `Downloads` whose name ends in `.dmg` or `.pkg` (case-insensitive). 5. `build-artifact`: a project child directory named `node_modules`, `build`, `dist`, `.venv`, `__pycache__` or `target` **and** listed as `<name>/` in that project's own `.gitignore`. 6. `personal`: everything else. `safe_to_delete` is `true` for every category except `personal`. **Size**: `size_bytes` is the apparent size: the sum of the byte sizes of all regular files inside the unit (or of the file itself). Symlinks are not followed and count as 0, wherever they occur. Deliverables (this is exactly what is graded): 1. `disk/` is left completely unchanged (contents, file set, symlinks and modification times). 2. `report/audit.json`: `{"units": [{"path": ..., "category": ..., "safe_to_delete": ..., "size_bytes": ...}, ...], "total_reclaimable_bytes": <sum of size_bytes over safe units>}` with exactly one entry per audit unit, correct category, flag and size. 3. `report/cleanup.sh`: a bash script run as `bash report/cleanup.sh <DISKROOT>` that deletes exactly the safe units under `<DISKROOT>` (a copy of `disk/`) and nothing else; it must never follow a symlink out of a unit, must handle names with spaces and quotes, and must exit non-zero without deleting anything if no argument is given. **Do not run it.**
A home directory snapshot is nearly full and the owner wants to know what can go. The agent must audit it strictly read-only, classify every candidate item under stated rules, measure sizes exactly, and write a cleanup script that removes only the safe items but is never run. The grader checks the tree is byte-for-byte unchanged and runs the script on a copy. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: 1. disk_unchanged: no file, directory, symlink or mtime under disk/ changed. 2. units_complete + categories_correct + safe_flags_correct: audit.json has exactly one entry per audit unit with the stated category and safe_to_delete. 3. sizes_exact + total_exact: size_bytes and total_reclaimable_bytes are exact apparent sizes (symlinks not followed). 4. cleanup_removes_exactly_safe: `bash cleanup.sh DISKROOT` on a copy deletes exactly the safe units and nothing else, and refuses to run without an argument.
Best model for this task so far: DeepSeek V4.1 Flash (Venice) (thinking on) (success, $0.01, 46s).
All runs (7)
71% pass rate across 7 eligible runs: 5 pass, 1 partial, 1 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| OGPT-6.1 Soldefault · thinking on | $0.05 | 33s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-20 | |
| XGrok 4.3 (Venice)thinking on | $0.02 | 25s | 0 | Fully autonomous | partial | Trust 80% | 2026-10-19 | |
| QQwen3 Coder 480B Turbo (Venice)thinking on | $0.0052 | 43s | 0 | Fully autonomous | fail | Trust 80% | 2026-10-19 | |
| AClaude Haiku 5.5default · thinking on | $0.01 | 57s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| AClaude Opus 5.5default · thinking on | $0.22 | 47s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| DDeepSeek V4.1 Flash (Venice)thinking on | $0.01 | 46s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 | |
| AClaude Sonnet 5.5default · thinking on | $0.09 | 24s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-19 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.