Diagnose why GitHub Actions stopped running
Exact prompt
Our GitHub Actions CI stopped running (no runs at all, no errors). The repo is in ./repo (see repo/NOTES.md and repo/.github/workflows/ci.yml). Find the root cause and fix the workflow file in place so CI runs again for pushes and PRs to the default branch. Then write DIAGNOSIS.md in the current directory with the root cause in 3-5 sentences. Do not use the network.
Task prompt given to the agent (headless, offline, synthetic fixture inputs): Our GitHub Actions CI stopped running (no runs at all, no errors). The repo is in ./repo (see repo/NOTES.md and repo/.github/workflows/ci.yml). Find the root cause and fix the workflow file in place so CI runs again for pushes and PRs to the default branch. Then write DIAGNOSIS.md in the current directory with the root cause in 3-5 sentences. Do not use the network. Inspired by https://x.com/kunchenguid/status/2098814897354137710 (unverified; not checked with the author). This is an independent re-run on a reproducible synthetic task; it does not report or reproduce the original post's numbers. Cost shown is a list-price API-equivalent estimate (run on a Claude Pro subscription, not billed).
Success criteria: All automated rubric checks pass: push trigger includes trunk (or unfiltered); pull_request trigger includes trunk (or unfiltered); push paths filter does not exclude src/ changes; DIAGNOSIS.md names branch rename main->trunk as root cause; workflow is valid YAML with jobs.test.
Best model for this task so far: Gemma 4 31B Instruct (Venice) (success, $0.0010, 27s). Best AI model for Software & DevOps →
All runs (11)
82% pass rate across 11 eligible runs: 9 pass, 0 partial, 2 fail. Partial and failed runs are listed below with everything else.ⓘ
| Model | Cost | Time | Step-insⓘ | Autonomy | Outcome | Trustⓘ | Date | Compare |
|---|---|---|---|---|---|---|---|---|
| MLlama 3.2 3B (local)thinking off | $0 (local) | 9s | 0 | Fully autonomous | fail | Trust 80% | 2026-10-20 | |
| QQwen2.5 7B (local)thinking off | $0 (local) | 22s | 0 | Fully autonomous | fail | Trust 80% | 2026-10-20 | |
| AClaude Haiku 5.5default · thinking on | $0.0041 | 15s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| ZGLM 5.3 (Venice) | $0.06 | 3m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| GGemini 3.8 Flash | $0.13 | 4m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-16 | |
| XGrok 4.3 (Venice) | $0.0057 | 7s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-15 | |
| QQwen3 Coder 480B Turbo (Venice) | $0.0018 | 9s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-15 | |
| DDeepSeek V4.1 Flash (Venice) | $0.0025 | 2m | 0 | Fully autonomous | pass | Trust 80% | 2026-10-14 | |
| GGemma 4 31B Instruct (Venice) | $0.0010 | 27s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-14 | |
| AClaude Opus 5.5 | $0.13 | 22s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-14 | |
| AClaude Sonnet 5.5 | $0.04 | 11s | 0 | Fully autonomous | pass | Trust 80% | 2026-10-14 |
Embed this result
A live badge for this use case. It links back here and updates as new runs are published.
Get new runs and ranking changes weekly
Which models got better or cheaper on real tasks like this one.