Apple Researchers Find a Hidden Glitch That Can Quietly Corrupt AI-Generated Images
A small number of rogue data points, called outlier tokens, can throw off the systems that create AI images. New research explains what they are and how to fix them.
Key points
- Apple ML Research identified a flaw called outlier tokens inside AI image-generation systems built on a design known as Diffusion Transformers.
- These outlier tokens appear in both the part of the system that reads images and the part that creates new ones.
- The problem is sharpest in the middle layers of the model, the processing stages between input and final output.
- Left unchecked, outlier tokens pull the model's attention to the wrong places, reducing image quality.
Picture a classroom where one loud student keeps interrupting every lesson. That student never says anything useful, yet the rest of the class still turns to look. Something very similar happens inside the AI systems that generate images from text prompts.
Researchers at Apple ML Research have now mapped the problem in detail. The troublemakers are called outlier tokens. A token, in this context, is a small chunk of information the AI processes, roughly like a word in a sentence but for images. An outlier token carries an unusually large numerical value, making it appear far more important than it actually is.
What exactly goes wrong?
Outlier tokens steal attention. Processing power gets devoted to them even though they carry little real information about the image being built.
The research focuses on a class of model called Diffusion Transformers, or DiTs. These power many of today's best AI image generators and work in two main stages. First, an encoder (the model's reading glasses) converts a real image into a compact mathematical description. Then a denoiser (the creative engine) gradually builds a new image from noise, guided by that description.
Apple's team found that trouble arrives at both stages. Pretrained encoders can hand distorted representations to the denoiser before it even starts work, and the denoiser itself can develop its own outlier tokens as it runs, especially deep in its middle layers. We covered the broader question of why Apple's image models hallucinate in our 3 August story, and this new finding adds a structural explanation to that picture.
Does this affect images people actually use?
Potentially yes, though the paper describes a research finding, not a flaw in a shipping product.
When attention drifts toward meaningless tokens, the final image can wander from what the user asked for. Details get dropped. Consistency suffers. Complex prompts give outlier tokens more opportunity to cause problems. The researchers also studied suppression techniques, and their work points toward practical fixes that could be built into future models.
For anyone using AI image tools today, no action is needed. This is foundational work, the kind engineers draw on when designing the next generation of tools.
What happens next?
No product changes or public model updates have been announced.
What makes this work matter is that it carries a known problem from an older class of model into newer territory. Researchers had spotted outlier tokens in Vision Transformers (AI systems trained to recognise, rather than generate, images) before, but their role in generative systems was largely unstudied until now. Studying the disease carefully is the necessary first step before curing it. If you'd like background on how these generators work at the architecture level, our plain-English guide from 23 July is a good place to start.
The pattern here is worth watching: Apple's research arm keeps publishing work that identifies structural limits in image AI rather than simply shipping new capabilities. That's either admirable scientific honesty or a sign these problems are harder to fix than they look. Probably both.
Common questions
Will this affect AI image tools I already use?
Not directly or immediately. This is foundational research, not a product update. The findings will likely inform how future image-generation models are designed and trained.
What is a Diffusion Transformer in plain terms?
It's the type of AI architecture, or internal structure, used by many modern image-generation tools. Starting with random noise, it gradually shapes that noise into a coherent image, guided by your text prompt.
