Why I still haven’t bought into true RSI

Newsletter/Podcast · Listen · (gathered 2026-09-23)

• Nathan Lambert argues against the “singularity soon” narrative and the concept of “true Recursive Self-Improvement (RSI),” instead proposing his “lossy self-improvement” model where AI progress is primarily about scaling inference capacity and efficiency for known problems.
• He differentiates between predictable “inference-time scaling” (thousands of agents solving measurable problems) and the highly uncertain outputs of true RSI, noting that current progress is largely driven by the former, which is economically transformative but not necessarily indicative of superintelligence.
• Lambert highlights the “jaggedness of intelligence,” explaining that AI’s capabilities diffuse slowly across specific tasks rather than crossing discrete human-shaped thresholds, and that bottlenecks like hypothesis generation and complex “post-training” (defining model behavior) remain uniquely hard for current LLMs to automate.
• He cautions against exaggerated extinction risks, citing internal reports (e.g., Claude Fable 5.1 & Mythos 5.1 System Card) that show AI maintaining, but not dramatically accelerating, the rate of progress, suggesting that the hardest exponential to fight is on “peak intelligence.”