How was DeepSeek trained so cheaply?
DeepSeek trained cheaply through engineering efficiency, not luck. Key tricks: a Mixture-of-Experts design that activates only ~37B of 671B parameters per word, FP8 low-precision math, memory-saving Multi-head Latent Attention, and teaching its reasoning model with reinforcement learning instead of costly human-labeled data. The final V3 run reportedly used ~$5.6M of compute.
Why — the first-principles explanation
The cost of training an AI model is basically the number of math operations times the price of the hardware doing them. DeepSeek attacked both. The biggest lever is Mixture-of-Experts (MoE): instead of running all 671 billion parameters for every word, the model is split into many 'experts' and only routes each word to a small subset — about 37 billion parameters active at a time. That's like having a huge staff but only paying the few people actually working on each task, cutting compute roughly ninefold.
Second, DeepSeek trained in FP8, a low-precision number format. Using fewer digits per calculation halves memory and speeds up the math with little accuracy loss — like doing arithmetic to 3 decimal places instead of 8 when 3 is good enough. They paired this with Multi-head Latent Attention (MLA), which compresses the model's 'memory' of the conversation so it needs far less GPU RAM, and with custom low-level GPU code to squeeze more out of the export-limited H800 chips they were restricted to.
Third, for the R1 reasoning model, DeepSeek skipped the most expensive ingredient of all: human-labeled examples. Instead of paying people to write thousands of worked solutions, it used reinforcement learning, rewarding the model for correct answers so it taught itself to reason. Stacked together, these choices are why the final training run cost millions instead of the tens or hundreds of millions many assumed were required.
An example that makes it click
Imagine writing a huge encyclopedia. The expensive way is to make every one of your 671 writers work on every single page. DeepSeek's Mixture-of-Experts is like assigning each page only to the 2 or 3 writers who actually know that topic — the same giant team, but a fraction of the payroll per page.
Then, instead of printing every draft in full-color glossy (high precision), you print drafts in cheap grayscale that's still perfectly readable (FP8). And rather than hiring tutors to write model answers for the 'reasoning' volume, you just give the writers a gold star whenever they reach the right conclusion, and they practice until they're good (reinforcement learning). Same encyclopedia, a small fraction of the bill.
Key facts
- Mixture-of-Experts activates only ~37B of 671B parameters per token in DeepSeek-V3, cutting compute per word.
- FP8 mixed-precision training reduced memory use and sped up calculations with minimal accuracy loss.
- Multi-head Latent Attention (MLA) compressed the key-value cache, lowering GPU memory requirements.
- R1's reasoning was trained largely via reinforcement learning, avoiding expensive human-labeled datasets.
- The reported final V3 training run cost ~$5.6M in compute, excluding hardware, research, and salaries.
An open-weight Chinese model family that matched frontier quality at low cost.
Affiliate link — we may earn a commission at no cost to you.▶ The 60-second explainer (script)
How did DeepSeek train its AI so cheaply? Not by cutting corners — by clever engineering. The biggest trick is called Mixture-of-Experts. Instead of running all 671 billion parameters for every single word, the model only wakes up about 37 billion at a time — like having a huge staff but only paying the few people actually working on each task. That alone slashes compute. They also trained using a low-precision number format called FP8, which is faster and uses less memory with barely any accuracy loss, and a memory-saving attention method so the GPUs could do more with less. And for its reasoning model, instead of paying humans to write thousands of example answers, DeepSeek used reinforcement learning — rewarding the model for correct answers until it taught itself. Stack those together, and the final training run cost millions instead of hundreds of millions.
What authoritative sources say
People also ask
What was DeepSeek's single biggest cost-saver?
Its Mixture-of-Experts design, which activates only a small fraction of parameters per word, cutting compute dramatically.
Did DeepSeek use cheaper chips?
It used export-limited H800 GPUs and wrote custom low-level code to squeeze more performance out of them.
How did it save on reasoning training?
By using reinforcement learning to reward correct answers instead of paying humans to write labeled examples.
Does cheap training mean lower quality?
No — DeepSeek's models matched top competitors on many benchmarks; the savings came from efficiency, not weaker models.