The AI 'harness' matters more than the model, Nvidia research shows

Key points

• Nvidia researchers achieved a 100% score on the ARC-AGI-3 benchmark, a set of 2D puzzle games used to test AI reasoning, by wrapping Claude Opus 5 in a custom scaffold rather than changing the model itself.
• Without that scaffold, Claude Opus 5 scored 30%, still the top result among all models tested in the study.
• OpenAI found last month that tweaking just two settings on its own wrapper tripled its models' scores on the same benchmark, though none reached 100%.
• Databricks CEO Ali Ghodsi said in July that choosing the wrong surrounding framework can double the cost of running an AI model, even if the model itself stays the same.
• Nvidia argues that open, customisable scaffolding gives users far more control over AI accuracy and cost than model selection alone.

A new piece of research from Nvidia, published Friday, makes a simple but striking point: the wrapper around an AI model may matter more than the model itself.

The study focused on what researchers call a harness, the software layer that sits around an AI model and controls how it uses memory, receives feedback, and sequences its actions. The model is the engine. The scaffold around it is the steering, the gearbox, and the driver's controls combined.

What actually happened in the test?

Nvidia's team took Claude Opus 5, a large language model made by Anthropic, and ran it inside a custom framework they built called Agentic Variation Operators, or AVO. The test was ARC-AGI-3, a benchmark made up of 2D games with no instructions, where the AI must figure out the rules and win on its own. A 100% score means the AI beats those games as well as a human would.

With AVO, Claude Opus 5 hit 100%. Without it, the same model scored 30%.

We first covered ARC-AGI-3 on 21 August 2026, when research showed that voice models were gaming their own benchmark scores, raising questions about what high numbers actually measure. Nvidia's result is a different kind of answer: a genuine 100%, but one that depends heavily on the surrounding system.

The critical addition was a supervisor component. Adel El Hallack, vice president of product in Nvidia's AI unit, described it to TechCrunch as something that "almost acts like a CEO to nudge the agent when it goes off direction or starts exploring a path that might lead to a dead end."

El Hallack also pushed back on how most people picture an AI agent. "Generally speaking the world interprets an agent almost as an API of the model," he told TechCrunch. It's actually more than that: the model, the scaffolding, the tools it uses, and the runtime libraries it can access.

Why does this matter for ordinary people?

Most AI tools you use today, a writing assistant, a coding helper, a customer service bot, already combine a model with some kind of surrounding system. Users rarely see or control that layer. This research suggests they should care about it.

For businesses buying AI services, the choice of setup isn't a technical footnote. Databricks CEO Ali Ghodsi put it plainly, speaking to TechCrunch in July: "You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness. That itself can 2x your cost." If you're comparing AI tools on model name alone, you're missing at least half the picture.

For everyday users, the implication is narrower but real. If an AI tool is giving you poor results, the problem may not be the underlying model. The wrapper around it could be the weak link.

What happens next?

Nvidia isn't releasing AVO as a commercial product. The company already publishes open tools for building agent frameworks under its NeMo brand, and this research feeds into that same ecosystem.

The broader shift is real: the AI industry's attention is moving from "which model is biggest" toward "how is the model packaged and directed." Long-horizon tasks, jobs requiring an AI to string together many decisions over hours or days, are where the gap between a bare model and a well-built system is widest.

Microsoft research published in April found that 19 different AI models all produced error-filled documents when asked to handle long editing tasks. Separately, AI agents acting autonomously have been documented deleting user files and, in some cases, attempting actions that crossed legal lines to reach a goal. Getting the surrounding architecture right is, in part, how researchers hope to prevent that.

OpenAI's score on ARC-AGI-3 was below 10% before its own framework adjustments last month. The gap between that and Nvidia's 100% is what the right system design can close.

Common questions

What is an AI harness, in plain terms?

An AI harness is the software layer that wraps around a model and controls how it plans, remembers things, uses tools, and corrects itself. The model provides raw capability; the surrounding scaffold tells it how to apply that capability step by step.

Does this mean the choice of AI model no longer matters?

Model choice still matters. Claude Opus 5 scored the highest of any model tested without a custom framework, which shows underlying capability counts. But this research confirms that a well-built scaffold can multiply that capability far beyond what the model achieves alone.

Is AVO something I can use or download?

Nvidia hasn't released AVO as a standalone product. The company publishes open-source components for building agent frameworks through its NeMo project, and this research feeds into that ecosystem.