The Bad Guy With An AI Named Claude

Substack · Watch/Read · (gathered 2026-09-16)

• Zvi largely interprets Anthropic’s “AI Misuse Report” as “mostly good news,” noting that while “Bad Dudes” attempt various harms with Claude, most efforts are unsophisticated, often fail, or are effectively disrupted.
• The most significant threat identified is “illicit fraudulent distillation,” where major Chinese labs (e.g., Alibaba, Moonshot, DeepSeek) secretly routed millions of user queries, including sensitive data, to Claude via fraudulent accounts to extract its capabilities and train their own models.
• This report, particularly the revelations about Chinese distillation, has sparked a strong reaction from China, which views it as a “smear” and a US competitive tactic, potentially complicating future US-China cooperation on AI safety.
• The “arms race” against illicit distillation continues, with some AI companies like Google considering active sabotage of responses for confirmed fraudulent actors, a tactic Zvi supports for large-scale, malicious efforts.