Temperature
A sampling parameter that controls randomness in LLM output. Lower values produce focused, deterministic text; higher values produce diverse, creative text.
What Is Temperature
Temperature is a number you pass to an LLM API to control how random its output is. It rescales the probability distribution over the next token before sampling. At low temperature the model picks the most likely token almost every time. At high temperature it gives lower-probability tokens a real chance, which produces more variety — and more mistakes. Use it when you want to tune the trade-off between reliability and creativity.
How It Works
After the model assigns a probability to every possible next token, the logits are divided by the temperature value. Dividing by a number less than 1 sharpens the distribution toward the top candidates. Dividing by a number greater than 1 flattens it. The model then samples from the adjusted distribution.
Typical Ranges
- 0.0 — fully greedy, picks the highest-probability token every time. Best for classification, extraction, code, structured output
- 0.2–0.4 — focused but with light variation. Best for technical writing and Q&A
- 0.7–0.8 — default for general chat, balanced
- 1.0–1.2 — creative writing, brainstorming, marketing copy
- Above 1.5 — usually unstable, expect hallucinations and broken syntax
Temperature vs Top-p
Most APIs also expose top_p (nucleus sampling), which caps the cumulative probability mass instead of rescaling. Set one or the other — not both. For deterministic output, set temperature to 0 and ignore top_p.
Keep temperature at 0 for anything you’ll parse with code. Raise it only when human readers will judge the output.