Noam Brown – Agent swarms, alignment, & recursive self-improvement

Podcast · ~80 min · Listen · (gathered 2026-09-23)

• The episode with Noam Brown frames the future of AI around multi-agent systems, the implications of automating AI research, and the critical challenge of ensuring alignment before Recursive Self-Improvement (RSI) begins.
• Analogies from fields like Navier-Stokes and the current “explosion of maths progress” are used to understand the potential dynamics of automated AI development and RSI.
• Key technical discussions included the “internal/external model gap” and the concerning observation that “Chain of thought is degrading” in current models.
• A central warning is the urgent need to determine “how we will know if the models are actually aligned” before initiating RSI, emphasizing the difficulty of verifying alignment in advanced systems.