AI Still Can't Learn From a Few Pictures the Way You Can
Apple researchers tested leading AI vision models on a simple human skill: spot what a group of images have in common, then apply that idea to a new picture. The models largely failed.
Key points
- Apple ML Research published a study in 2025 showing that today's best vision-language AI models struggle to identify a shared concept across a small set of example images.
- The researchers created a benchmark called VICIS (Visual Concept Inference from Sets) to measure this specific gap in AI capability.
- Every state-of-the-art model tested performed poorly on the VICIS task, suggesting a meaningful blind spot in current AI design.
- The failure matters because real-world AI tools increasingly ask models to learn from a handful of visual examples, not just written instructions.
Here is a skill every five-year-old has. Show them three pictures of dogs and one picture of a cat, and they will tell you the cat does not belong. Show them three pictures of birthday parties and then a photo of a beach bonfire, and they will guess the bonfire could be a celebration too.
Current AI vision models cannot do this reliably. That is the finding from Apple ML Research, whose team designed a test called VICIS, short for Visual Concept Inference from Sets, to measure exactly this gap.
A vision-language model, the technology that lets an AI both look at images and read or write text, can follow detailed written instructions with impressive accuracy. Ask one to describe a photo, count objects, or read a label, and it usually delivers. But strip away the words and ask it to figure out a rule purely from a handful of pictures? Performance drops sharply.
Why should ordinary people care about this?
This gap matters because many practical AI tools already rely on learning from examples. A shop owner who wants an AI to sort product photos by style, or a teacher who wants it to flag a certain type of diagram, is implicitly asking the model to do exactly what VICIS tests. If the model cannot do it, those tools will produce unreliable results without any obvious warning.
The VICIS test works like this. The model receives a small "context set", a handful of images that all share some concept, maybe a colour scheme, a mood, a shape, or a relationship between objects. It also receives one new "query" image. Its job is to generate, or select, images that keep the concept from the context set while also fitting the query. Simple in theory. Turns out to be hard in practice.
Every model the Apple team tested fell short.
Two things are worth keeping in mind here. First, this is a research paper, not a product announcement. The models that failed are the same ones powering tools millions of people use today, so the finding is real, but it does not mean those tools are useless. It means one specific capability is weaker than we might assume.
Second, survivorship bias is worth naming. When you hear about an AI tool that learned a new style from a few pictures and nailed it, that story made the news because it worked. The thousand times it produced confident nonsense did not.
Honest takeaway: If you use an AI image tool and feed it a few example photos expecting it to "get the idea", test its output carefully before trusting it. Show it examples, then check whether it truly followed the concept or just copied surface details like colour. Your own eyes are still the better judge here.
