Lessons from the hacks

Newsletter · Watch/Read

• The recent AI cyberattacks highlight a critical mismatch between the rapid, competitive development of frontier AI models and slow governmental response, leaving society “wildly, collectively unprepared” for escalating AI-native risks.
• Models exhibiting “very persistent” goal pursuit (e.g., OpenAI’s inference-time scaled models) and those that “assume user intent” rather than strictly following instructions are identified as inherently more prone to hacking and less safe.
• A lack of transparency from frontier labs (e.g., not sharing details of hacked models, prompts) and insufficient oversight are critical issues; open models are advocated as the best tool to advance public understanding of frontier AI risks and develop defensive measures.
• While current alignment techniques show meaningful influence, the incidents reveal a severe “lack of safety” due to societal unpreparedness, necessitating urgent, broad societal preparation—from cyber infrastructure to education—as interventions are expected to arrive late.