Why AI Image Models Still Make Things Up, and What Apple's Researchers Are Doing About It

Key points

• Multimodal large language models, AI systems that process both images and text, frequently "hallucinate" by generating descriptions that contradict what is actually in a picture.
• Preference alignment, a training method that teaches a model to prefer accurate answers over plausible-sounding ones, is well-established for text-only AI but far less studied for image-understanding models.
• Apple ML Research published a comprehensive study examining how alignment techniques affect image-language models, covering what works, what does not, and why.
• The findings matter beyond research labs: these models power features like image captioning, visual search and accessibility tools that millions of people use every day.

Tell an AI to describe a photo and it will usually get most things right. But sometimes it confidently adds objects, people or text that are simply not in the picture. Researchers call this hallucination, and it is a known, stubborn problem with a class of AI systems called multimodal large language models, or MLLMs, which are AI models trained to understand both images and written language at the same time.

For a text-only chatbot, hallucination means stating a wrong fact. For an image model, it can mean something more disorienting: describing a red car as blue, inventing a sign on a shop wall, or claiming a person is smiling when they are not. The model is not lying, it just has no reliable anchor to the image in front of it.

Why does this keep happening?

Text-only AI models have benefited enormously from a training step called preference alignment. The short version: after the initial training, researchers show the model pairs of answers and teach it to prefer the more accurate, helpful one. Over thousands of examples, the model learns to self-correct.

The trouble is that this correction step has been studied far less for models that handle images. A model trained mostly on text develops strong instincts about what sentences should sound like. When you add images, those instincts do not automatically transfer to checking visual facts.

What did the Apple study find?

The research, published by Apple ML Research and first surfaced in a broader preprint review, set out to map the problem systematically. The team examined how different preference alignment strategies change the behaviour of image-language models, testing which approaches reduce hallucination and which ones fall short.

A key insight the study surfaces is that alignment for MLLMs must account for a second dimension of accuracy: not just "is this sentence factually correct?" but "does this sentence match what is in the image?" Those are different questions, and training methods built for text alone do not automatically handle the visual check.

The study is positioned as a resource for the wider research community, mapping out what is understood, what remains open and where the most promising directions lie.

What does this mean for ordinary users?

If you use an app that reads out image descriptions, tags photos automatically or answers questions about pictures you upload, you are relying on this technology. Hallucinations in those contexts are more than a curiosity: a wrong description in an accessibility tool can genuinely mislead a visually impaired user.

Better alignment training is one of the clearest paths toward models that stay honest about what they actually see. Progress here does not require users to do anything differently. It happens inside the training process, long before a product reaches your phone.

Common questions

Is hallucination the same problem in image models as in text chatbots?

Not quite. Text chatbots hallucinate by stating wrong facts. Image models can do that too, but they can also contradict the visual evidence right in front of them, describing things that are not there or missing things that clearly are.

Does this study change any products I can use right now?

No. This is foundational research, not a product announcement. Its value is in giving other researchers a clearer map of the problem so that future models, across many companies, can be built with stronger alignment training from the start.