Nvidia's New Retrieval Models Cut AI Agent Costs by 14 Percent

Key points

• Nvidia's Nemotron 3.5 Embed 8B ranked first among dense retrieval models on the ViDoRe v3.1 leaderboard as of 20 October 2026, scoring 61.17 nDCG@10.
• Adding Nemotron 3.5 Rerank to an agent pipeline cut inference token costs by approximately 14 percent in Nvidia's own tests.
• The five new models cover embedding (converting documents into searchable form), reranking (sorting results by true relevance) and document parsing, and all carry open, commercial licences.
• Nvidia scored 72.6 on MIRACL Vision, a multilingual image-retrieval benchmark covering 18 languages, beating the nearest rival by 4.3 points.
• AI2Day has tracked Nvidia infrastructure news closely this month, including a separate $250 million city-AI commitment in Liverpool.

Every time an AI agent, software that can carry out multi-step tasks on its own, fails to find the right document on its first attempt, it searches again. And again. Each extra search costs money because it burns through inference tokens, the computational units that measure how much work a language model does. At enterprise scale, with thousands of queries running every hour, those wasted tokens add up fast.

Nvidia's answer is the expanded Nemotron 3.5 Retriever collection, announced on 20 October 2026 via a detailed post on Hugging Face. The collection adds five models built to give agents better evidence on the first try.

What do these models actually do?

Think of the new tools as a three-part filing system. The embedding models convert documents into a form a computer can search quickly. The reranker then takes the search results and re-sorts them, pushing the most genuinely useful documents to the top. The parser, called Nemotron 3.5 Parse at 631 million parameters, pulls structured information out of raw documents before any of that happens.

The flagship, Nemotron-3.5-Embed-8B-BF16, handles both text and images. It can search across charts, tables, infographics and schematics, not just plain prose. That matters because real business documents, think quarterly reports or supplier contracts, mix all of these formats together.

For teams with tighter hardware budgets, Nemotron-3.5-Embed-1B-BF16 distils most of the larger model's ability into a smaller package. A third variant, Nemotron-3.5-Embed-1B-NVFP4, applies quantization, a technique that shrinks a model's memory footprint by using lower-precision numbers, for high-throughput deployments.

Does the cost saving hold up in practice?

Yes, according to Nvidia's internal benchmarks, though independent verification is still pending. In tests across three standard retrieval benchmarks, ViDoRe v3, BRIGHT, and BrowseComp-Plus, adding the Nemotron 3.5 Rerank 8B model reduced the total tokens an agent consumed per task by roughly 14 percent when paired with the 8B embedder, and 13 percent with the 1B version. Retrieval accuracy improved in both cases.

The reranker also beat a rival model from Qwen on the ViDoRe v3.1 reranking leaderboard, scoring 60.49 against Qwen-3-VL-Reranker-8B's 59.17. The smaller 1B embedding model retained 95.4 percent of the 8B model's benchmark score, which is a meaningful result for organisations that cannot afford the larger model's hardware requirements.

For multilingual work, text retrieval was evaluated across 34 languages, and visual retrieval was tested on MIRACL Vision across 18 languages.

The practical picture is straightforward. A retailer's AI agent asked to flag products heading for a stockout must combine demand charts, inventory tables and supplier delivery schedules. With weaker retrieval, it loops through documents repeatedly before it can answer. With a stronger reranker, the most relevant evidence surfaces first, the agent reasons over it once, and the bill for that query shrinks.

AI2Day has tracked infrastructure spending as a rising share of the AI story this month; our recent coverage noted that infrastructure quietly tripled in our editorial mix even as model announcements grabbed the headlines. Nvidia's move here fits that pattern: the company is not announcing a new large language model but rather the plumbing that makes existing ones cheaper to run.

All five models carry open, commercial licences, which means a startup and a Fortune 500 company face the same barrier to entry: none.

Common questions

Who benefits most from these models right now?

Large organisations running many AI agents simultaneously, such as financial institutions, logistics companies or healthcare systems, will see the biggest cost reductions because they generate the most wasted tokens today.

Do ordinary users need to do anything?

Not directly. These are tools for developers and IT teams. If you use an AI-powered search or assistant at work, your employer's technology team would need to integrate these models, after which queries would simply cost less and return better results.

Are these models free to use commercially?

Yes. All five models in the Nemotron 3.5 collection ship with open, commercially permissive licences, meaning businesses can build products with them without paying Nvidia a licensing fee.