The Extinction Risk Preference Cascade: Quotes

Substack · Watch/Read · (gathered 2026-09-13)

• Zvi Mowshowitz compiles a “preference cascade” of quotes from numerous employees at OpenAI, Anthropic, and Google DeepMind, demonstrating a widespread and increasingly open belief within these frontier AI labs that advanced AI poses a significant, potentially existential, risk to humanity.
• Many employees express concrete probabilities of human extinction (e.g., >10% within a decade, or 50% overall), primarily due to concerns that “recursive self-improvement” (RSI) is accelerating faster than alignment research.
• Key concepts discussed include the need for “pacing the frontier” of AI development to ensure human enhancement keeps pace, and even a figurative “Butlerian Jihad” to cap technology development.
• The overwhelming sentiment is that the current “race” in AI development is irresponsible and dangerous, necessitating urgent international coordination, regulation, and a massive increase in resources for alignment research to avoid catastrophic outcomes.
• The post highlights a significant shift in transparency, with employees now openly defying past corporate communication restrictions (especially at Google DeepMind) to voice their profound fears about the imminent and unmitigated risks of current AI development.