AI Learns to Think in Pictures Internally, Cutting Video Analysis Time Without Losing Accuracy

Key points

• Apple ML Research published a new AI training framework called Internalized Visual Thinking (IVT) for reasoning about video.
• IVT removes the need to generate intermediate images during analysis, which was a major speed bottleneck in previous approaches.
• The method trains models to do visual reasoning silently, inside the model itself, using only text as the final output.
• This approach targets "proactive" video reasoning, meaning the AI anticipates what happens next rather than just describing what it sees.

What is this, in plain language?

Think of how you watch a football replay and mentally picture where the ball will go next. You don't draw a diagram; you reason visually in your head. IVT teaches an AI to do the same thing.

Previous systems used a technique called Visual Chain-of-Thought, a process where the AI reasons step by step and actually generates intermediate pictures along the way, almost like a student drawing rough sketches before writing an answer. Those in-between images are helpful, but producing them is slow and expensive, especially when working through a video frame by frame.

Apple ML Research describes IVT as a post-training framework, meaning it's applied after a model is already built, like a finishing course. It teaches the model to internalize that visual scratchpad work during training, so by the time it's deployed, it skips generating those images entirely and reasons straight to a text answer.

Why does video make this particularly tricky?

Video is harder than a single image for one simple reason: time. An AI analysing a video clip needs to understand not just what's in each frame, but how objects move and what's likely to happen next.

That last part, predicting what comes next, is what researchers call proactive reasoning. It's the difference between an AI that says "a player is running" and one that says "that player is about to receive the ball in the bottom-left corner." Sports analysis and self-driving vehicles both need exactly this kind of forward-thinking, as does our earlier finding that today's AI video tools lose track of when things happen in long recordings.

The slowdown caused by generating intermediate images during that process isn't a minor inconvenience. It's the kind of delay that makes real-time applications impractical.

What does this mean for everyday technology?

Faster, smarter video AI matters in places ordinary people will notice. Gaming NPCs could read a scene and react to player movement far more convincingly. Broadcast tools and coaching software could offer near-instant tactical breakdowns. In any app that watches and understands a video clip, waiting times should drop.

IVT doesn't change what the AI knows. It changes how efficiently it thinks, which is the part that determines whether a technology is usable in the real world or stuck in a research lab. Apple ML Research has form here: we covered the team's work on finding hidden patterns behind AI object-detection mistakes back in July.

The detail available so far focuses on the training method rather than a specific product launch. Wider adoption will depend on how easily other teams can apply the approach to their own models.

Common questions

Does this replace existing video AI tools?

Not immediately. IVT is a training technique that other teams would need to apply to their own models, so it's a building block rather than a finished product you can use today.

Is this only useful for video games and sport?

No. Any system that needs to understand video in real time, including security cameras and video calling features, could benefit from faster and more accurate visual reasoning.