AI Agents Are Quietly Draining a Resource Nobody Saw Coming: the Humble CPU
Amazon Web Services is rationing CPU capacity, Intel has sold out of server chips, and AMD has doubled its forecast. The AI boom just got a new bottleneck.
Key points
- Amazon Web Services told its engineers to conserve CPU capacity in 2025 as AI workloads caused a surge in wait times for CPU server resources.
- AMD doubled its server CPU forecast, and Intel has sold out of server CPUs through at least the end of 2025.
- Researchers from Intel and the Georgia Institute of Technology found that up to seven of eight stages in a typical AI agent pipeline run on the CPU, not the GPU.
- Increasing CPU core counts can cut the time a model takes to produce its first response by roughly 1.5 to 7 times in tests with longer inputs.
- Analyst Matt Kimball of Moor Insights & Strategy calls the CPU demand spike an "absolute tell" that the chip is now a key part of any serious AI system.
For the last few years, the AI hardware story was simple: GPUs, the specialised chips that do the heavy number-crunching AI needs, were scarce and expensive. CPUs, the general-purpose processors inside every laptop and server, barely got a mention. That is changing fast.
Amazon Web Services, the cloud-computing arm of Amazon that rents server capacity to businesses worldwide, has reportedly told its engineers to conserve CPU cycles, after a surge in demand stretched wait times for CPU-based servers. The story was first flagged by IEEE Spectrum AI.
Why do AI agents need so many CPUs?
AI agents are software programs that can carry out multi-step tasks on their own: searching the web, writing files, running code, and calling other agents for help. The GPU still does the heavy lifting of running the AI model itself, but almost everything else falls to the CPU.
Souvik Kundu, a senior research scientist at Intel, puts it plainly. "The CPU does the job of parsing output, figuring out which tool to invoke, making the API call or running the code, collecting the result, and feeding it back." AMD's vice president of compute and enterprise AI, Madhu Rangarajan, says AMD's own testing found seven of eight stages in a realistic agent pipeline run entirely on the CPU.
Think of it this way. An AI agent told to build a piece of software will ask the GPU to generate the code, then hand off to the CPU to write it to a file, download packages, run a compiler, and check the results. The CPU is busy the whole time the GPU is resting, and vice versa.
The maths gets uncomfortable at scale. Analyst Matt Kimball notes that a single enterprise deployment might start with 100 agents. Those agents spawn sub-agents. "One hundred become tens of thousands, hundreds of thousands, or millions," he says. Each one making tool calls. Each call landing on a CPU.
What is making the bottleneck worse?
Two research papers published recently, one co-authored by Kundu and one by PhD student Euijun Chung at Georgia Tech, point to a second problem: tokenisation.
Tokenisation is what happens before a model reads any text. It converts words and punctuation into numbered tokens, short chunks the model can process. For a single short question, this is trivial. For an AI agent handling a long, ongoing conversation plus the results of dozens of tool calls, it is anything but.
Chung explains that if a model is already working with 100,000 tokens and receives a tool result of 1,000 more, the tokeniser must process the whole sequence again from scratch. Every tool call repeats the process. His paper found that in tests using models including Meta's Llama 3.1-70B, a large open-source language model, adding more CPU cores cut the time to produce the first word of a response by 1.5 to 7 times, depending on sequence length.
Safety checks add further CPU load. Rules that inspect what an agent is about to do, and small AI models under a billion parameters that check an agent's intent, tend to run on the CPU because they are small and latency-sensitive.
What does this mean for prices and availability?
Intel has sold out of server CPUs through at least the end of 2025. AMD has doubled its server CPU forecast. Arm, Qualcomm, and Nvidia have all announced new CPU designs aimed at agentic AI workloads.
| Company | Move | When |
|---|---|---|
| Amazon Web Services | Rationed CPU capacity for engineers | Early 2025 |
| Intel | Server CPUs sold out | Through end of 2025 |
| AMD | Doubled server CPU forecast | 2025 |
| Arm / Qualcomm | New CPUs targeting AI agents | Announced 2025 |
| Nvidia | Prioritised Vera, its own Arm-based CPU | Part of Vera Rubin platform |
Kimball warns the crunch may trickle down to ordinary consumers. Intel has reportedly shifted production away from desktop and laptop chips in favour of server CPUs. If demand keeps climbing, expect the same story that played out with GPUs: tighter supply and higher prices.
Chung is blunt about the trajectory. "In the world of agentic AI, the average sequence length will grow and grow, so I'm expecting this problem to get worse in future workloads."
Common questions
Does this affect me if I just use ChatGPT or Claude?
Not directly today, but if CPU shortages push up cloud costs, AI services could get more expensive or slower over time. For now, nothing changes for everyday users.
Is a CPU the same as a GPU? Why do both matter?
No. A GPU handles thousands of simple maths operations at once, which is ideal for running AI models. A CPU handles fewer tasks but can juggle many different types of work, which is what AI agents need when they browse, write files, or run code.
Should businesses building on AI agents worry about this now?
It is worth factoring CPU capacity into any agentic AI project. If you rely on a cloud provider, check whether your plan includes guaranteed CPU resources, especially for workflows where agents call many tools in sequence.
