Researchers Build a Test to See If AI Can Actually Summarise a 16-Minute Video

Key points

• Apple ML Research introduced LVSum, a benchmark for testing how well AI summarises long videos, in a 2024 research paper.
• LVSum contains 72 videos across 13 subject areas, with an average length of 16 minutes per video.
• Each video carries up to 10 human-written summaries that include specific timestamps, so the AI must say not just what happened but when.
• Current multimodal large language models, the AI systems that can process video and text together, struggle to keep events in the right order over long recordings.

Imagine asking a colleague to recap a 16-minute meeting recording. You want the decisions, yes, but you also want to know when each key moment landed, so you can scrub straight to it. That basic skill turns out to be surprisingly hard for AI, and it's the gap LVSum was built to measure.

The benchmark, described in a paper from Apple ML Research, runs AI systems through a structured test. Researchers gathered 72 videos spread across 13 subject areas, from cooking demonstrations to legal proceedings. Each runs about 16 minutes. Human annotators then wrote up to 10 summaries per video, each pegged to specific timestamps in the footage.

The timestamps are the whole point. A summary that says "the speaker discussed the contract" is nearly useless if you need to find that moment quickly. LVSum asks whether an AI can do better, producing what researchers call temporally grounded summaries: text that tells you both the content and exactly when it occurs.

The models under test are multimodal large language models, AI systems trained to process video and text together rather than text alone. They're the technology behind tools that can watch a clip and describe it. Existing systems already handle short clips reasonably well. Long video is a different problem.

Over 16 minutes, a model must hold a great deal in memory and keep the sequence of events straight. The research found that maintaining what the paper calls "temporal fidelity", an accurate sense of what came before what, is where current systems break down most visibly.

Apple ML Research has been busy on the evaluation front: we covered their virtual-user framework for testing AI assistants on 14 July 2026, and LVSum reads as part of the same instinct: build the measuring stick before claiming the win.

What does this mean for ordinary users?

If you use an AI tool to recap a long meeting recording or a documentary, treat the output as a rough guide. The AI may get the facts broadly right but place them in the wrong order or miss time cues entirely. Spot-check against the original before sharing any AI-generated summary with colleagues.

LVSum doesn't fix the problem. It's a measuring stick, a standardised way for researchers to compare how different AI systems perform on the same videos. Benchmarks like this tend to drive improvement: once there's an agreed test, labs can see clearly where their models fall short and aim at a specific target.

The dataset spans 13 domains deliberately, because an AI that only recaps cooking videos well isn't much practical use. That variety forces models to generalise.

The paper stops short of naming which commercial products were evaluated. The full scoring breakdown sits in the complete research text.