AI Still Can't Learn From a Few Pictures the Way You Can
Apple researchers tested leading AI vision models on a simple human skill: spot what a group of images have in common, then apply that idea to a new picture. The models largely failed.
Key points
- Apple ML Research published a study in 2025 showing that today's best vision-language AI models struggle to identify a shared concept across a small set of example images.
- The researchers created a benchmark called VICIS (Visual Concept Inference from Sets) to measure this specific gap in AI capability.
- Every model tested performed poorly on the VICIS task, pointing to a real blind spot in current AI design.
- The failure matters because real-world AI tools increasingly ask models to learn from a handful of visual examples, not just written instructions.
Here's a skill every five-year-old has. Show a child three pictures of dogs and one picture of a cat, and they'll tell you the cat doesn't belong. Show three pictures of birthday parties and then a photo of a beach bonfire, and they'll guess the bonfire could be a celebration too.
Current AI vision models can't do this reliably. That's the finding from Apple ML Research, whose team designed a test called VICIS, short for Visual Concept Inference from Sets, to measure exactly this gap. We first covered Apple ML Research's work on AI perception limits on 17 July 2026.
A vision-language model, the technology that lets an AI both look at images and read or write text, can follow detailed written instructions with impressive accuracy. Ask one to describe a photo or count objects, and it usually delivers. Strip away the words and ask it to figure out a rule purely from a handful of pictures, though, and performance drops sharply.
Why should ordinary people care about this?
This gap matters because many practical AI tools already rely on learning from examples. A shop owner who wants an AI to sort product photos by style, or a teacher who wants it to flag a certain type of diagram, is implicitly asking the model to do exactly what VICIS tests. If the model can't do it, those tools will produce unreliable results without any obvious warning.
The VICIS test works as follows. The model receives a small context set: a handful of images that all share some concept, maybe a colour scheme, a mood, or a relationship between objects. It also receives one new query image. Its job is to generate or select images that keep the concept from the context set while also fitting the query. Simple in theory. Hard in practice.
Every model the Apple team tested fell short.
Two things are worth keeping in mind. First, this is a research paper, not a product announcement. The models that failed are the same ones powering tools millions of people use today, so the finding is real, but it doesn't mean those tools are useless. It means one specific capability is weaker than we might assume.
Second, survivorship bias is worth naming. When you hear about an AI tool that learned a new style from a few pictures and nailed it, that story made the news because it worked. The thousand times it produced confident nonsense did not.
Honest takeaway: If you use an AI image tool and feed it a few example photos expecting it to get the idea, test its output carefully before trusting it. Check whether it truly followed the concept or just copied surface details like colour. Your own eyes are still the better judge here.
