Nvidia's New Retrieval Models Cut the Cost of Running AI Agents
Five new open models help AI agents find better answers faster, reducing the wasted computing that makes enterprise AI expensive. Here is what is actually in them and what it costs you if you ignore them.
Key points
- Nvidia's Nemotron Retriever collection now includes five new models, available on Hugging Face, that directly cut the token waste driving up enterprise AI agent costs.
- The flagship embedding and reranking models both run at 8 billion parameters; two smaller 1-billion-parameter versions target edge and cost-sensitive deployments.
- A document-extraction model called Nemotron 3.5 Parse handles pulling usable text from messy real-world files.
- All five models are commercially available as open weights, meaning companies can run them on their own servers without paying per-query fees.
The problem Nvidia is trying to solve is unglamorous but expensive. An AI agent, software that carries out multi-step tasks on its own such as researching a supplier or drafting a contract, can only be as good as the information it pulls in before answering. When that information is weak, the agent loops: it burns through inference tokens, the units of computing that cloud providers charge for, before settling on a response. At enterprise scale, those loops add up fast.
Nvidia's answer is the updated Nemotron Retriever collection. The five new models handle three distinct jobs. Embedding models convert documents and queries into a mathematical form that lets a computer measure relevance; the new Nemotron 3.5 Embed comes in an 8-billion-parameter version for maximum accuracy and two 1-billion-parameter versions for cheaper, faster deployments. Reranking models then take an initial set of retrieved documents and re-sort them by how genuinely useful each one is; Nemotron 3.5 Rerank 8B does that job. Nemotron 3.5 Parse extracts clean, structured text from the PDFs and scanned files that most businesses actually store their knowledge in.
All five models are open weights. A company can download them, run them on its own servers, and pay nothing per query, which contrasts sharply with paying a cloud provider every time an agent retrieves a document.
What has changed since Nvidia's last retrieval release?
Our earlier story on the Nemotron 3.5 collection reported a 14 percent cost reduction from the previous generation. Today's models add two things that release lacked: multimodal retrieval, meaning the system can handle images alongside text, and dedicated document parsing for the messy files real enterprises actually use.
Nvidia also announced this retrieval expansion as part of a broader wave that includes robotics models, a protein-design tool developed with the University of Washington, and models for quantum computing calibration. The retrieval piece is the part most relevant to businesses spending real money on AI agents today.
Should I trust the performance claims?
Nvidia calls these models best-in-class, and benchmark numbers can be curated to flatter. The 14 percent figure from the prior release was Nvidia's own measurement, not an independent audit. Results will vary by document type and how well your data is structured. The only reliable test is running the models against your own documents.
These are real, downloadable models with a clear commercial use case. That's more than vaporware. But every headline about AI cost savings deserves a raised eyebrow until your own bills confirm it.
If your team's already running AI agents and paying per-query retrieval costs to a cloud provider, downloading the 1-billion-parameter versions and benchmarking them against your own document set costs nothing but an afternoon.
