Find every deprecated API call in a large repo and write a migration checklist
Exact prompt
Audit this repository (the MongoDB Node.js driver, version 1.4.0, about 57k lines of JavaScript) for every call site of these deprecated Node.js APIs: 1. The Buffer constructor, called either as `new Buffer(...)` or as a plain function `Buffer(...)` (DEP0005). 2. `require('domain')` (DEP0032). Cover every .js file in the repository (lib, test, examples, dev, docs, and so on). Count only real code, not occurrences inside comments or string literals. Write two files in the repository root: - findings.csv with header: file,line,api,replacement (file = path relative to the repo root; line = 1-based line where the call starts; api = "new Buffer()", "Buffer()" or "require('domain')"; replacement = the suggested modern API, e.g. Buffer.alloc / Buffer.from / Buffer.allocUnsafe, or an alternative to domain) - MIGRATION.md: a migration checklist grouped by directory, with counts and any call sites that need special care. Do not modify any source files.
An audit of a real 57k-line JavaScript codebase (MongoDB Node.js driver 1.4.0) for every deprecated Buffer-constructor and domain call, producing a CSV of call sites and a migration checklist. Ground truth comes from an AST scan, and ten look-alike matches inside comments act as traps. It compares one large model working alone, the same large model as a planner delegating the scanning to smaller, cheaper subagents, and a mid-size model working alone. Original benchmark task written for usecaserank.com (not based on a specific public post). Headless, offline run on a reproducible fixture. Each run's notes say how its cost was measured (estimates are marked).
Success criteria: Recall vs. a grep-derived ground-truth list ≥ 95%; false positives ≤ 5%.
Best model for this task so far: Claude Sonnet 5.5 (effort default (no --effort flag) · thinking on) (success, $0.22, 1m). Best AI model for Software & DevOps →
All runs (3)
100% pass rate across 3 eligible runs: 3 pass, 0 partial, 0 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| AClaude Sonnet 5.5default · thinking on | $0.22 | 1m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Opus 5.5default · thinking on · + scanner subagents + AClaude Haiku 5.5default · thinking on | $0.84 | 3m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| AClaude Opus 5.5default · thinking on | $0.75 | 3m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.