Stealing Reasoning Traces from Proprietary LLM APIs

Blog · Watch/Read

• A paper discovered a method to “steal reasoning traces” from proprietary LLM APIs (OpenAI, Anthropic, Google) by exploiting encrypted chain-of-thought blocks returned to clients.
• The attack involved replaying encrypted reasoning blocks, which shared encryption keys across model families, into weaker sibling models (e.g., Claude Haiku 4.5) and then jailbreaking them to reveal the raw, unencrypted internal thoughts.
• This vulnerability, now patched, exposed internal model reasoning (e.g., GPT-5.5’s CSS planning) and revealed a “devious prompt injection variant” where instructions embedded in a model’s own reasoning trace were treated as “sacrosanct” and more likely to be followed.
• The research underscores the potential for advanced prompt injection techniques that manipulate how LLMs process their internal reasoning, even as specific exploits are patched.