Chain-of-Thought
A prompting technique that asks the model to produce intermediate reasoning steps before the final answer, improving accuracy on multi-step problems.
What Is Chain-of-Thought
Chain-of-thought prompting is the trick of asking an LLM to “think step by step” before answering. Instead of jumping to a conclusion, the model writes out its reasoning — sub-goals, calculations, considered options — and then states the final answer. The technique was introduced by Wei et al. in the 2022 Google paper “Chain-of-Thought Prompting Elicits Reasoning in Large Language Models.” Use it whenever a task involves arithmetic, logic, multi-hop questions, or anything where a plausible-looking wrong answer is worse than a slower right one.
How It Works
Two common variants:
- Zero-shot CoT — append “Let’s think step by step” to the prompt. Discovered by Kojima et al., 2022
- Few-shot CoT — show the model 2 to 8 worked examples with reasoning before the question
For models trained on chain-of-thought data, neither trigger is needed — they reason by default. Reasoning models like OpenAI o1, o3, and Claude’s extended-thinking mode bake CoT into the model itself and hide most of it from the user.
Why It Matters
- Accuracy gains — large boosts on math benchmarks like GSM8K and MATH, often 20+ points
- Auditability — you can read the reasoning trace and spot the exact step that went wrong
- Better tool selection — agents that reason before acting choose tools more carefully
Trade-offs
- Cost and latency — every reasoning token is billed and adds to wall-clock time
- Verbose output — for short factual queries, CoT is overkill
- Hidden hallucinations — fluent reasoning can be wrong; never trust the trace without checking
- Prompt injection surface — visible reasoning gives attackers more to grab onto
For simple lookups, skip CoT. For anything requiring multiple steps, turn it on.