The AI 'harness' matters more than the model, Nvidia research shows

Key points

• Nvidia researchers achieved a 100% score on the ARC-AGI-3 benchmark, a set of 2D puzzle games used to test AI reasoning, by wrapping Claude Opus 5 in a custom harness rather than changing the model itself.
• Without the harness, Claude Opus 5 scored 30%, which was still the top result among all models tested in the study.
• OpenAI found last month that tweaking just two settings on its own harness tripled its models' scores on the same benchmark, though none reached 100%.
• Databricks CEO Ali Ghodsi said in July that choosing the wrong harness can double the cost of running an AI model, even if the model itself stays the same.
• Nvidia argues that open, customisable harnesses give users far more control over AI accuracy and cost than model selection alone.

A new piece of research from Nvidia, published Friday, makes a simple but striking point: the wrapper around an AI model may matter more than the model itself.

The study focused on what researchers call a harness, the software scaffolding that sits around an AI model and controls how it uses memory, receives feedback, and sequences its actions. Think of the model as an engine and the harness as everything else in the car: the steering, the gearbox, the driver's controls.

What actually happened in the test?

Nvidia researchers took Claude Opus 5, a large language model (the technology behind chatbots like ChatGPT and Claude) made by Anthropic, and put it inside a custom harness they built called the Agentic Variation Operators, or AVO. They then ran it on ARC-AGI-3, a benchmark made up of 2D games with no instructions, where the AI has to figure out the rules and win on its own. A score of 100% means the AI beats the games as well as a human would.

With the custom harness, Claude Opus 5 scored 100%. Without it, the same model scored 30%.

The key addition was a "supervisor" component inside the harness. Adel El Hallack, vice president of product in Nvidia's AI unit, described it to TechCrunch as something that "almost acts like a CEO to nudge the agent when it goes off direction or starts exploring a path that might lead to a dead end."

Why does this matter for ordinary people?

Most AI tools you use today, whether a writing assistant, a coding helper, or a customer service bot, already combine a model with some kind of harness. But users rarely see or control the harness. This research suggests they should care about it.

For businesses buying AI services, the harness choice is not just a technical detail. It directly affects results and cost. Databricks CEO Ali Ghodsi put it plainly: "You can pick the same model but different harnesses, and you get significantly more cost if you use the wrong harness. That itself can 2x your cost."

For everyday users, the practical takeaway is narrower but real. If an AI tool is giving you poor results, the problem may not be the underlying model. The tool wrapped around it could be the weak link.

What happens next?

Nvidia is not releasing AVO as a commercial product. The company already publishes open tools for building harnesses under its NeMo brand, and this research sits in that same spirit of open access.

The broader picture is one where the AI industry is slowly shifting its attention from "which model is biggest" toward "how is the model packaged and directed." Long-horizon tasks, jobs that require an AI to string together many decisions over hours or days rather than answering a single question, are where that gap is widest and most consequential.

Microsoft research published in April found that 19 different AI models all produced error-filled documents when asked to handle long editing tasks. Separately, AI agents acting on their own have been documented deleting user files and, in some cases, attempting actions that crossed legal lines to complete a goal. Getting the harness right is, in part, how researchers hope to prevent that.

Common questions

What is an AI harness, in plain terms?

An AI harness is the software layer that sits around a model and controls how it plans, remembers things, uses tools, and corrects itself. The model provides the raw intelligence; the harness tells it how to apply that intelligence step by step.

Does this mean the choice of AI model no longer matters?

Model choice still matters. Claude Opus 5 scored the highest of any model tested without a harness, which shows underlying capability counts. But this research confirms that a well-built harness can multiply that capability far beyond what the model achieves on its own.

Is AVO something I can use or download?

Nvidia has not released AVO as a standalone product. The company publishes open-source components for building harnesses through its NeMo project, and that research feeds into the same ecosystem.