One resignation turned the embers of AI fear into a wildfire

Newsletter/Podcast · Listen

• Nathan Lambert argues that a recent AI researcher’s resignation, citing safety risks, unexpectedly ignited a “wildfire” of public fear around AI existential risk, largely because “fear sells” and the public “ground had dried out.” He views this as “very bad for the AI ecosystem” due to increased polarization and a shift towards extreme views.
• He critiques the focus on “mass extinction” as being on “very poor footing,” advocating instead for attention to tangible risks like cyberattacks and bio-risks. He also introduces his concept of “Lossy self-improvement” as an alternative to “Recursive Self-Improvement (RSI),” arguing AI progress is jagged and human bottlenecks persist.
• Lambert points to a “religious energy” and “out of touch” perspective within some frontier labs (especially Anthropic), suggesting their internal culture distorts their understanding of technical progress and risk, and that their forecasting of future AI capabilities may be flawed.
• He warns that frontier AI labs are not taking practical safety seriously enough, as evidenced by incidents like the OpenAI-HuggingFace hack, and financial pressures will likely prevent sustained caution. He emphasizes the need for transparency, accountability, and the continued development of open models for cyber hardening and adapting to future AI risks.