Your AI Data Center Has a Heat Problem. Here Is What the Industry Is Doing About It.
Modern AI chips now burn more than 1,000 watts each. Air conditioning can't keep up. A new white paper explains why liquid cooling is becoming the only practical answer.
Key points
- Individual AI processors now generate more than 1,000 watts of heat each, far beyond what air cooling can handle.
- A single server rack packed with AI hardware can release more than 100 kilowatts of heat.
- Single-phase direct liquid cooling pipes water or a water-glycol mixture directly over hot chips through metal plates called coldplates, then carries the heat away in a closed loop.
- The white paper is presented by IEEE Spectrum and Wiley, and sponsored by CoolIT Systems, a liquid cooling hardware maker.
- Chip power and rack density are expected to keep climbing, meaning the heat problem will get worse before it gets better.
AI chips run hot. That sentence used to be a footnote in a data centre engineer's report. Now it's the central problem for anyone building or running the computers that power AI tools.
Modern AI accelerators, the specialised processors that crunch the enormous calculations behind tools like ChatGPT, can each pour out more than 1,000 watts of heat. Slot dozens of them into a single server rack and that rack can shed more than 100 kilowatts. One rack, one data centre row, one serious headache.
Why air cooling is losing the battle
Air cooling simply cannot move enough heat fast enough at these densities. Fans push air across hot components, but air holds very little heat compared with liquid. Past a certain point, adding more fans just adds noise and electricity bills without solving the problem.
When chips overheat, they protect themselves through a process called thermal throttling: they automatically slow themselves down to reduce heat output. That means lower performance and wasted money on hardware that cannot run at full speed. It also shortens hardware life, a cost that rarely shows up in the vendor pitch.
What liquid cooling actually does
Single-phase direct liquid cooling works by running water or a water-glycol coolant mixture through metal coldplates mounted directly on top of the hottest chips. Heat from the chip surface transfers into the liquid, which carries it away through pipes to a coolant distribution unit, a cabinet-sized device that dumps the heat elsewhere and sends cool liquid back in a continuous closed loop.
Liquid absorbs and moves heat far more efficiently than air, letting data centres pack more powerful chips into a smaller physical space.
Two other approaches exist: two-phase cooling, where the liquid boils and turns to vapour to carry heat away, and immersion cooling, where entire server boards sit submerged in a special non-conductive liquid. Both work, but both carry higher complexity and cost. The IEEE Spectrum white paper compares all three directly.
AI2Day has been tracking the data centre infrastructure beat since July, and our 6 September story on Chinese-made components showed how deep the supply dependencies run before you even get to the cooling layer.
What this means for anyone whose job touches AI
If you run a business renting cloud computing time, or you're watching your company's AI infrastructure bill, the heat problem is yours too. Chips that throttle run AI models more slowly, and slower models cost more per task. Cooling costs will only grow as a share of total AI operating expenses.
You probably can't build your own liquid cooling system. But the next time a vendor pitches you on AI compute, ask how their data centres manage heat and whether thermal throttling is cutting into the performance you're paying for. That question alone will tell you a lot about how seriously they've thought this through.
