What is an AI data center?
An AI data center is a building packed with GPU or TPU accelerators wired together to train and run AI models. It differs from a normal data center in density: racks draw tens of times more power, need liquid cooling, and require ultra-fast interconnects so thousands of chips can act as one machine.
Why — the first-principles explanation
A conventional data center is a warehouse of servers doing loosely-related jobs — hosting websites, serving databases. If one rack fails, the others don't care. The design problem is reliability and floor space.
An AI data center inverts that. Training a large model is one calculation spread across thousands of chips that must stay in lockstep, exchanging results constantly. That single requirement rewrites the whole building.
Density. A traditional rack draws maybe 5-10 kW. A rack of AI accelerators can draw 40-130 kW. You're pushing an order of magnitude more electricity into the same floor tile, and essentially all of it becomes heat.
Cooling. Air physically cannot carry that much heat away — the fans would need to move impossible volumes of air. So AI facilities move to liquid cooling, running coolant directly to the chip. This is why AI and water get discussed together, though the relationship is widely misreported (more below).
Interconnect. If chip 900 waits on chip 12, everything stalls. So these buildings spend enormous money on networking — the wiring between chips is a first-class part of the machine, not an afterthought. This is the real reason you can't just rent scattered GPUs and call it a supercomputer.
Power siting. Because the constraint is megawatts, these get built where power is — next to nuclear plants, hydro, and large substations. The bottleneck for AI expansion in 2026 is grid interconnection queues, not chip supply.
Now the contested part, honestly. AI water and energy numbers vary wildly across sources, and the disagreement is almost entirely about boundaries, not arithmetic. Google's measurement of its own operations puts the median Gemini text prompt at 0.24 Wh, 0.03 gCO2e, and 0.26 mL of water — about five drops. That figure uses a broad boundary on energy (it includes idle machines, CPU, RAM, and cooling overhead) but the water number is on-site only. It excludes the water consumed generating the electricity somewhere else, which is often far larger. Studies that count off-site water report much bigger numbers, and they're not wrong — they're answering a different question. Anyone quoting you a single scary bottle-of-water figure without saying which boundary they used is not being straight with you.
The other honest caveat: per-prompt efficiency improved enormously (Google reports 33x less energy per prompt in 12 months) while total AI electricity demand rose. Both are true. Efficiency per unit and total footprint are different measurements, and conflating them is the most common error in this debate.
An example that makes it click
A normal data center is an office building — hundreds of people doing separate jobs, and if one person calls in sick, nothing else stops.
An AI data center is a rowing crew of eight thousand. Everyone pulls at once, on the same stroke, or the boat goes nowhere. That's why the wiring matters as much as the chips: the crew needs to hear the same call, instantly. And when eight thousand people row that hard in one room, the room gets hot enough that opening a window won't help — you need to run cold water through the building. That's liquid cooling, and it's a consequence of the rowing, not a design choice.
Key facts
- AI data centers are built around accelerators (GPUs/TPUs) rather than general-purpose CPUs, because the core operation is large-scale matrix multiplication.
- AI racks draw far more power than conventional racks — tens of kilowatts versus roughly 5-10 kW — which forces liquid cooling instead of air cooling.
- Google measured the median Gemini Apps text prompt at 0.24 watt-hours of electricity, 0.03 gCO2e, and 0.26 mL of water (about five drops) as of May 2025.
- Google's energy boundary includes idle machines, CPU and RAM, and data center cooling overhead — broader than estimates that count only active chip power.
- The 0.26 mL water figure is on-site data center consumption only and excludes off-site water used in electricity generation, which is the main reason published AI water estimates differ by orders of magnitude.
- Google reported per-prompt energy fell 33x and carbon footprint fell 44x between May 2024 and May 2025, driven mainly by model improvements (~23x) and machine utilization (~1.4x).
▶ The 60-second explainer (script)
An AI data center is a building full of specialized chips — GPUs or TPUs — wired together tightly enough that thousands of them act as a single machine. And that last part is what makes it different from a normal data center. A regular data center is an office building. Hundreds of people doing separate jobs. One calls in sick, nothing stops. An AI data center is a rowing crew of eight thousand. Everyone pulls on the same stroke or the boat goes nowhere. Training one model is one calculation, spread across thousands of chips that have to stay in lockstep. That single fact rewrites the building. Density: a normal rack draws five to ten kilowatts. An AI rack can draw forty to a hundred thirty. Almost all of it becomes heat. Cooling: air physically can't carry that away, so you run liquid coolant to the chip. Interconnect: if chip nine hundred waits on chip twelve, everything stalls — so the wiring costs as much as the chips. And siting: they get built where the megawatts are. Now, the water and energy numbers. Google measured its own median text prompt at zero point two four watt-hours and zero point two six milliliters of water — five drops. But that water figure is on-site only. It excludes the water used to generate the electricity elsewhere, which is much larger. Studies quoting bigger numbers aren't wrong; they drew a different boundary. Anyone giving you one scary bottle-of-water figure without telling you the boundary isn't being straight with you.
What authoritative sources say
People also ask
How is an AI data center different from a regular one?
Density and coupling. AI racks draw roughly ten times the power, require liquid cooling, and need ultra-fast interconnects because thousands of chips must work on one calculation together rather than separate jobs.
Do AI data centers really use a lot of water?
It depends entirely on the boundary. Google reports 0.26 mL per median text prompt on-site. Counting the water used to generate the electricity off-site gives a much larger number. Both are legitimate; they measure different things.
Why liquid cooling instead of fans?
Physics. At 40-130 kW per rack, air can't carry the heat away at any realistic fan speed. Liquid has vastly higher heat capacity, so coolant runs directly to the chip.
Is AI's total energy use going up or down?
Both, depending on what you measure. Google reports per-prompt energy fell 33x in a year, while total AI electricity demand has risen because usage grew far faster than efficiency improved.
What limits how fast AI data centers get built?
Electricity, primarily — grid interconnection and available megawatts, plus transformers and cooling infrastructure. That's why new sites cluster near nuclear, hydro, and major substations.