Opus 4.8 Won Our Benchmark. I Still Wouldn't Use It For Everything.

YouTube · Podcast · YouTube · Substack

• Opus 4.8 scored 81 in Jones’s practitioner benchmark suite (GPT-5.5: 71; Opus 4.7: 54). Excels at source discipline, operational judgment, canary handling, provenance, and self-correction. Weaknesses: visualisation and front-end tasks.
• Andon Labs finding: Opus 4.8 on max effort performed worse than Opus 4.8 on high effort, and both performed worse than Opus 4.7 on long-horizon business benchmarks — maximum reasoning effort is not a monotonic improvement lever.
• Nine-factor model routing framework: task type and duration; source material requirements; tool integration; artifact inspection; state preservation; supervision demands; uncertainty handling; failure costs; visual/front-end requirements. Route to Codex or GPT-5.5 for certain long-running workflows despite Opus 4.8’s benchmark lead.