Researchers Found a Way to Read AI's 'Hidden Thoughts', and It Raises Real Security Questions

Key points

• Researchers from the University of Tübingen and collaborators found a method to extract the hidden reasoning steps from major AI models including those made by OpenAI, Anthropic, and Google.
• The same technique exposed private data, such as passwords and API keys, stored inside a model's reasoning process, though all three companies have since patched that specific flaw.
• A Chinese model called Kimi K3, made by Moonshot AI, produced reasoning patterns strikingly similar to hidden outputs from Claude Opus 4.8 and GPT 5.6 Sol, raising questions about whether it was trained by copying those models.
• Researchers caution the similarity is suggestive, not conclusive proof of copying.
• Fully fixing the deeper problem would require major changes to how these companies' programming interfaces work.

Every advanced AI model has a kind of internal monologue. Before it gives you an answer, it works through the problem step by step in what researchers call a "chain of thought", a written-out reasoning process the model uses to solve hard problems. Companies keep this reasoning hidden. It is commercially sensitive, and sharing it openly would make it easy for rivals to copy.

Now a team of computer scientists has found a way to read those hidden thoughts anyway.

The technique, reported by Wired AI, comes from researchers at the University of Tübingen, the Max Planck Institute, the AI safety institute MATS Research, and the security company Snyk. Their key insight is almost elegant in its simplicity.

How does the attack actually work?

AI companies typically offer their models in multiple sizes. A large model is powerful but expensive to run; a smaller version of the same model costs less. Both versions share the same underlying structure, which means they share the same ability to decode encrypted data passed between them.

When you use a big AI model through an API (an application programming interface, the digital connection that lets software talk to other software), the model sends an encrypted copy of its reasoning to your computer to share some of the processing work. The researchers discovered that feeding this encrypted reasoning to a smaller, cheaper sibling model can unlock it.

Why does the smaller model cooperate where the bigger one would not? Smaller models receive less "alignment training", the process that teaches AI to follow rules and refuse awkward requests. Strip that guardrail away, and the model is far more willing to show its working.

"The idea of swapping out messages to a weaker model variant which has the same decryption key but weaker alignment is very cool," said Florian Tramer, a computer security specialist at ETH Zürich.

What did they find inside?

Two things, and they are quite different in severity.

First: passwords and API keys. If you had ever pasted sensitive credentials into a prompt while using one of these AI services, that information could have ended up sitting inside the model's reasoning trace. The researchers extracted it. OpenAI, Anthropic, and Google were all alerted last month, and each has updated its systems to block this specific leak. If you used any of these services before the patch, it is worth rotating any API keys or passwords you may have included in prompts.

Second: the copying question. The researchers fed 90 questions to several AI models, then gave some open-weight models (freely downloadable AI models that anyone can run) the opening words of reasoning traces captured from the big proprietary models. Kimi K3, a model from the Chinese company Moonshot AI, generated reasoning that looked remarkably similar to the hidden output of Claude Opus 4.8 and GPT 5.6 Sol. Two other models tested, including China's DeepSeek and the US-made Inkling, did not show the same pattern.

The researchers are careful about what this means. Their paper states the findings "cannot causally establish distillation", meaning they have not proven copying occurred. Distillation is a widely used technique in AI development where a new, smaller model learns by studying the outputs of a larger one. It is normal practice. The controversy is whether Chinese companies used it to copy US models without permission.

Should ordinary users worry right now?

The password-leak flaw is patched. That is the most urgent practical concern, and it is handled.

The broader issue, that hidden reasoning can still be partially extracted even after the patch, remains. Fixing it entirely would require these companies to redesign how their APIs work from the ground up. Alexander Panfilov, the University of Tübingen researcher who led the work, says the deeper vulnerability persists.

For now, a simple rule applies: do not paste passwords, personal details, or sensitive business information into any AI chatbot or tool. That advice was always sensible. This research makes it feel a little more urgent.

Common questions

Is my data at risk if I use ChatGPT, Claude, or Gemini?

The specific flaw that could expose passwords and API keys has been patched by all three companies. Going forward, avoid including sensitive information in AI prompts as a general habit, since AI services process your inputs on external servers.

What is distillation, and is it illegal?

Distillation is a technique where a new AI model learns by studying the outputs of an existing one, essentially using a smarter model as a teacher. It is widely used and legal in most contexts, though using it to copy a competitor's proprietary model without permission could breach terms of service or raise intellectual property questions.