Nvidia's $20 Billion Groq Bet Goes Live: What It Means for AI Speed

Key points

• Nvidia confirmed its Groq 3 LPX rack entered full production in March 2026, following its $20 billion acquisition of Groq's assets in December 2024.
• Each LPX rack holds 256 Groq 3 chips and delivers 3,400 tokens per second, a measure of how fast an AI model generates text.
• The racks will first go live at Nebius, a cloud provider, alongside Nvidia's Vera central processors and Rubin graphics processors.
• Nvidia CEO Jensen Huang has projected $1 trillion in combined sales for Blackwell and Vera Rubin chips through 2027.
• Rivals Advanced Micro Devices and Cerebras are chasing the same goal: faster, cheaper AI responses at scale.

That spinning circle you sometimes see when an AI chatbot pauses mid-thought has a name: latency. Cutting it is the hottest race in AI hardware right now, and Nvidia just moved to the front.

Nvidia announced Monday that its Groq 3 LPX rack, a large server unit holding 256 specialised chips, is now in full production and will go online later this year. CNBC Tech reported the details from an Nvidia briefing.

What is this chip actually for?

Think of it as a speed specialist. Standard GPUs are flexible workhorses: they train AI models and run them. Groq chips do one thing extremely well, which is make already-trained models respond faster.

Specifically, they speed up the "decode" phase, the part where a model generates each word of its reply. The Groq 3 chip keeps 500 megabytes of fast memory called SRAM directly on the chip itself, cutting out the usual trip to slower external memory where most delay creeps in.

Nvidia says its Groq 3 LPX rack hits 3,400 tokens per second (tokens are roughly word-chunks, so this translates to extremely fast text output), based on a benchmark from research firm Artificial Analysis. Groq chips are made by Samsung, while Nvidia's standard GPUs come from TSMC, the world's largest contract chipmaker.

Why does this matter to ordinary users?

Faster responses make AI agents, software that handles multi-step tasks like booking travel or writing code on your behalf, feel snappy rather than sluggish. Coding assistants in particular need this kind of speed.

For cloud companies it also opens a pricing opportunity. Nvidia senior director Dion Harris told reporters that faster chips unlock premium service tiers for customers who can't afford to wait. We reported in August that Nvidia was already planning to raise AI server prices by more than 15% for major customers, so those premium tiers may land on top of an already rising bill.

The racks will be deployed at Nebius, sitting alongside Nvidia's Vera and Rubin systems.

How does it stack up against the competition?

OpenAI's Ultrafast mode, powered by Cerebras chips, promises 750 tokens per second. Nvidia's claimed 3,400 is a notable step beyond that, though real-world results rarely match benchmarks cleanly.

At the March unveiling Huang said he'd allocate roughly a quarter of data centre space intended for coding to Groq chips. The rest, he said, stays with Vera Rubin.

Should you worry about the price?

Not yet, but watch closely. None of this speed is free: cloud providers will charge more for low-latency tokens, and Nvidia has already telegraphed broader price rises. The benchmark gap between Groq and Cerebras looks enormous on paper, but the number that will actually shape your AI bill is whatever Nebius decides to charge per token when these racks go live. Nvidia reports earnings Wednesday, where investors will press for exactly that answer.

As for what to watch beyond the earnings call: Groq chips don't replace GPUs, they sit beside them. The real test is whether enterprises pay the premium or decide 750 tokens per second is fast enough.