How does chain-of-thought prompting work?
Chain-of-thought (CoT) prompting tells the model to write out its intermediate reasoning steps before the final answer, for example by adding 'Let's think step by step.' Showing the work raises accuracy on math, logic, and multi-step problems. It was introduced by Google researchers in 2022 and works by giving the model more computation per answer.
Why — the first-principles explanation
An LLM generates one token at a time, and each new token is conditioned on everything written so far, including its own previous words. So the text the model has already produced becomes part of its working space. If it jumps straight to an answer, it has almost no room to work. If it writes out steps first, each step becomes context that helps predict the next step, and finally the answer.
Chain-of-thought exploits this. By prompting 'show your reasoning' or 'let's think step by step,' you get the model to lay down a trail of intermediate results, like a student writing the middle lines of a math problem. Because more tokens are generated, more computation is spent on the problem, and one hard leap is broken into smaller, easier sub-steps.
There are two flavors. Few-shot CoT includes worked examples that demonstrate the reasoning style; zero-shot CoT just appends a trigger phrase like 'Let's think step by step.' CoT helps most on multi-step tasks (arithmetic, logic, planning) and less on simple lookups. Newer reasoning models do this internally by default, but the underlying mechanism, spend tokens reasoning before answering, is the same.
An example that makes it click
Ask a kid, 'What's 17 times 6?' If they blurt an answer, they might guess 92. Now say, 'Do it out loud, step by step.' They say: '17 times 6 is 10 times 6 plus 7 times 6, that's 60 plus 42, which is 102.' Same kid, same brain, but writing the middle steps caught the right answer.
Chain-of-thought prompting asks the model to show its work the same way. Just like the kid, breaking one hard leap into small steps makes it far more likely to land on the correct answer.
How to do it
- State the problem clearly, then add a trigger like 'Let's think step by step' or 'Show your reasoning before the final answer.'
- For harder tasks, include one or two worked examples that demonstrate the step-by-step style (few-shot CoT).
- Ask the model to put its final answer on a separate line (e.g., 'End with: Answer: ...') so you can extract it easily.
- Use it for multi-step problems, math, logic, and planning, and skip it for simple factual lookups where it adds little.
- If you don't want the reasoning shown to end users, ask the model to reason internally and output only the conclusion, or use a reasoning model.
Key facts
- Chain-of-thought prompting was introduced in a 2022 Google Research paper (Wei et al.) and shown to boost large-model performance on reasoning benchmarks.
- Zero-shot CoT uses the trigger phrase 'Let's think step by step' (Kojima et al., 2022).
- CoT works because LLMs generate token by token; written steps become context and add computation per answer.
- Gains are largest on multi-step tasks (arithmetic, logic, planning) and smallest on simple lookups.
- Modern reasoning models perform chain-of-thought internally by default, using the same reason-then-answer mechanism.
▶ The 60-second explainer (script)
How does chain-of-thought prompting work? It's simple: instead of letting the model jump straight to an answer, you ask it to write out its reasoning first, often just by adding the phrase 'Let's think step by step.' Here's why that helps. A language model writes one word at a time, and each new word is based on everything it's written so far. So if it answers in one leap, it has no room to work. But if it writes the middle steps, each step becomes context for the next, and it spends more computation on the problem. It's like a student doing 17 times 6 in their head versus writing 60 plus 42 equals 102. Same brain, better result, because the work is broken into small, easy pieces. Chain-of-thought was introduced by Google researchers in 2022, and it helps most on math, logic, and multi-step planning. Newer reasoning models now do this automatically, but the trick is the same: reason first, answer second.
What authoritative sources say
People also ask
What's the difference between zero-shot and few-shot chain-of-thought?
Zero-shot CoT just adds a trigger like 'Let's think step by step.' Few-shot CoT includes worked examples that show the reasoning style before your question.
Does chain-of-thought always improve answers?
No. It mainly helps multi-step reasoning like math and logic. For simple factual lookups it adds length without much benefit.
Do I need to show the reasoning to users?
No. You can ask the model to reason internally and output only the final answer, or use a reasoning model that hides its steps.
Why does writing steps make the model more accurate?
Each written step becomes context for the next and adds computation, turning one hard leap into several easy sub-steps.