Opus 4.8 Effort Controls — Counter-Intuitive Benchmark Finding #
- Opus 4.8 scored 81 in my benchmark. I still wouldn’t default to it. (Nate’s Newsletter / Substack, 2026-06-03) — Nate B. Jones benchmarked Opus 4.8 at 81 (vs. GPT-5.5 at 71, Opus 4.7 at 54) on a practitioner workflow suite. Key finding from Andon Labs: Opus 4.8 on max effort performed worse than Opus 4.8 on high effort, and both performed worse than Opus 4.7 on long-horizon business benchmarks. The effort controls introduced in Opus 4.8 are not a monotonic “more = better” dial — there is an optimal effort level per task type, beyond which additional reasoning effort degrades output. Nine decision factors Jones uses for model routing: task type and duration; source material requirements; tool integration availability; artifact inspection; state preservation; supervision demands; uncertainty handling; failure costs; visual/front-end requirements.
- Claude Code Changelog (Anthropic, 2026-06-03–04) — Recent UX changes:
claude agents --json now includes waitingFor field showing what a session is blocked on (e.g. a pending permission prompt) — first machine-readable signal for the state of waiting subagents. Workflow keyword trigger: new /config setting to prevent the word “workflow” in a prompt from launching a dynamic workflow inadvertently. Grep-then-edit now a first-class workflow: viewing a file with single-file grep no longer requires a separate Read before Edit.
Cross-links #
- Vibe Coding Approaches The nine-factor model routing framework (Jones) is the practitioner operationalisation of the Dynamic Workflows governance question (#16 in the 2026-06-02 review) — who decides which model runs which subagent, and based on what criteria?
- Claude Integrations The
waitingFor JSON field in claude agents --json is directly useful for the Managed Agents Memory + Outcomes use cases where headless agents may block on permissions mid-run.
- Emerging pattern: The counter-intuitive max-effort degradation finding (Andon Labs) suggests effort controls require calibration per task class rather than being set globally. The assumption “higher effort = better output” is false for long-horizon business tasks. This is a practical workflow architecture implication that will take time to diffuse into practitioner guidance.
- Quality signal: Jones’s benchmark (Opus 4.8 at 81, GPT-5.5 at 71) is a practitioner-grade comparison with named methodology — not a vendor leaderboard. The Andon Labs corroboration (max > high effort = worse) provides independent confirmation of the calibration finding. Both are worth anchoring future model comparisons against.