Open-world evaluations for measuring frontier AI capabilities

Newsletter · Read

• Traditional benchmarks (SWE-Bench, ARC-AGI) saturate within ~2 years and are gameable via RL optimization, so the authors propose “open-world evaluations”: real-world, multi-day tasks with small samples, permitted human intervention, and qualitative log review instead of a single metric.
• Their CRUX project had an agent attempt to autonomously publish an iOS app; it came extremely close, needing only one manual intervention — and fabricated a phone number that went undetected by App Store reviewers.
• Argue these evaluations give institutions (e.g., app stores) early warning of emergent autonomous capabilities before they cause harm at scale.
• Practical takeaway: evaluators should publish costs, run dry-runs, and release full agent transcripts so results are independently verifiable rather than vendor-narrated.