Llama 3.2 3B (local) vs Qwen2.5 7B (local)

On the 15 use cases both models ran, Qwen2.5 7B (local) won 2 to 0, with 13 tied (better outcome first, then lower cost).

Pass rate: Llama 3.2 3B (local) 0% (15 runs) vs Qwen2.5 7B (local) 7% (15 runs). Average cost per run: $0 (local) vs $0 (local).

Use case by use case

Audit expense claims with per-day caps and unverified exception notes: tie (pass 0% vs 0%)
Build a 4-page bakery-cafe website (HTML/CSS): Qwen2.5 7B (local) better (pass 0% vs 0%)
Build a landing page from extracted design tokens: Qwen2.5 7B (local) better (pass 0% vs 100%)
Build a vendor payout batch when one vendor's amount units are unknown: tie (pass 0% vs 0%)
Carry out a customer data-erasure request without erasing a namesake: tie (pass 0% vs 0%)
Diagnose why GitHub Actions stopped running: tie (pass 0% vs 0%)
Finish a spoken in-car request unattended: tie (pass 0% vs 0%)
Implement 5 Python functions from a task list: tie (pass 0% vs 0%)
Plan a picking robot's shortest route on a grid with an unconfirmed hazard: tie (pass 0% vs 0%)
Rename a data field across a codebase with hidden coupling and a partner-facing export: tie (pass 0% vs 0%)
Render per-environment deploy configs when the payment credential is missing: tie (pass 0% vs 0%)
Replay a 600-event inventory ledger with an undefined return policy and write-off sign-off: tie (pass 0% vs 0%)
Report weekly active users when 'active' and 'week' are still undefined: tie (pass 0% vs 0%)
Reverse engineer a security patch for an exploited CVE: tie (pass 0% vs 0%)
Schedule a cross-time-zone meeting in a DST-mismatch week and send invites only after sign-off: tie (pass 0% vs 0%)

Only use cases both models ran are compared; all settings of each model are grouped. How we score.