Why does AI use so much energy?
Because answering one prompt means multiplying billions of numbers, and every multiplication costs energy — mostly to move data, not to compute it. One prompt is still only 0.24 Wh. The totals get huge from volume: billions of prompts daily plus training runs, pushing U.S. data centers to 4.4% of the grid.
Why — the first-principles explanation
A language model is, mechanically, a giant pile of multiplication. The model's "knowledge" is stored as billions of numbers called weights. To produce even one word of a reply, the system multiplies your input against a large fraction of those weights and adds up the results — hundreds of billions of arithmetic operations for a single token. Then it does it again for the next word. A hundred-word answer means running that gauntlet a hundred times.
But here's the counterintuitive part that explains the hardware: the multiplication is not what costs the energy — moving the numbers is. Fetching a weight from memory can cost hundreds of times more energy than the arithmetic performed on it. This is why AI chips are strange-looking: they're mostly memory bandwidth and interconnect, wrapped around comparatively cheap math units, all trying to shove data around fast enough to keep the multipliers busy. And every joule spent moving those bits ends up as heat — a chip converts essentially 100% of its electricity into heat, since no energy is stored in the answer. That's why one accelerator draws 700–1,000 watts and a dense rack exceeds 100 kW.
Training versus inference are two different beasts. Training is doing all that arithmetic across a huge dataset, repeatedly, for weeks or months across thousands of chips — an enormous one-time spike per model. Inference is one pass, per query, tiny by comparison. But inference runs billions of times a day, so for a popular model, cumulative inference energy overtakes training within its service life.
Now the honest correction to the premise: per prompt, AI does not use much energy at all. Google measured a median Gemini text prompt at 0.24 Wh — less than nine seconds of television, and 0.10 Wh if you count only the AI chip. What drives the big totals is volume and duty cycle: millions of servers running every second, plus idle capacity held in reserve so responses feel instant. Idle reserve is a real and under-discussed cost — Google explicitly includes it, which is why their comprehensive number is 2.4x their chip-only number. And the trend runs the other way from the headlines: Google reports per-prompt energy dropped 33x in twelve months. AI's energy story is not inefficiency. It's scale.
An example that makes it click
Imagine looking up a word in a dictionary — except this dictionary has 200 billion entries, it's spread across a warehouse, and to answer one question you must physically walk to a few billion of those entries, read each one, and add the numbers up. Then, to say your second word, you do the whole walk again.
Here's the punchline. The reading is easy. The walking is what exhausts you. That's exactly how AI chips work: the arithmetic is cheap, and hauling numbers from memory to the math unit is what burns the power — sometimes hundreds of times more. So AI chips are built like freight yards, not calculators. And all that hauling comes out as heat, which is why a single one of these chips runs as hot as a small space heater, and why a room full of them needs a cooling tower.
Key facts
- A median Gemini Apps text prompt uses 0.24 Wh on a comprehensive boundary — or 0.10 Wh counting only the AI accelerator (Google, August 2025).
- Google reports per-prompt energy fell 33x and carbon 44x over a 12-month period.
- Modern AI accelerators draw 700–1,000 W each; dense racks exceed 100 kW, nearly all of it converted to heat.
- U.S. data centers used 176 TWh in 2023 — 4.4% of U.S. electricity — up from 76 TWh (1.9%) in 2018 (Berkeley Lab / DOE).
- IEA projects electricity in accelerated (AI) servers growing about 30% annually, driving nearly half the net increase in global data centre electricity to 2030.
- Google's comprehensive boundary includes idle machine capacity and data center overhead, which more than doubles the chip-only figure.
▶ The 60-second explainer (script)
Why does AI use so much energy? Start with what a model actually is: a giant pile of multiplication. Its knowledge is billions of numbers called weights. To produce one single word of a reply, it multiplies your input against a huge fraction of those weights — hundreds of billions of operations. Then it does the whole thing again for the next word. A hundred-word answer? A hundred rounds. But here's the part that surprises people. The multiplication isn't what costs the energy. Moving the numbers is. Fetching a weight from memory can burn hundreds of times more energy than the math you do with it. That's why AI chips look so strange — they're mostly memory bandwidth and interconnect wrapped around relatively cheap math units, all just trying to feed the beast. And every joule of that becomes heat, because no energy is stored in the answer. One chip: a thousand watts. A rack: over a hundred kilowatts. Now, the honest correction. Per prompt, AI doesn't use much energy at all — Google measured 0.24 watt-hours, less than nine seconds of TV. The totals get big from volume and duty cycle: millions of servers running every second, plus idle capacity held ready so it answers instantly. And the trend goes the opposite way from the headlines — Google reports per-prompt energy dropped 33-fold in a single year. AI's energy story isn't inefficiency. It's scale.
What authoritative sources say
People also ask
Why do AI chips use more power than regular processors?
They run far more arithmetic units in parallel and move vastly more data per second. Memory movement, not calculation, dominates the power budget — so bandwidth-heavy designs draw 700–1,000 W each.
Does a longer answer use more energy?
Yes, roughly proportionally — each output word requires another full pass through the model. Google's 0.24 Wh is a median text prompt; images, video, and reasoning-heavy requests cost considerably more.
Is AI energy use mostly training or inference?
Training is a huge one-time spike per model; inference is tiny per query but runs billions of times daily. For widely used models, cumulative inference typically exceeds training energy over the model's life.
Why does idle capacity count?
Because servers must sit powered and ready for instant responses. Google includes idle reserve in its comprehensive figure, which is a major reason it's 0.24 Wh rather than the 0.10 Wh chip-only number.