Temperature (Sampling)

Temperature is a setting used when a language model generates text. The model assigns a probability to every possible next token; temperature rescales those probabilities before one is drawn. Low temperature sharpens the distribution toward the most likely tokens, while high temperature flattens it so less likely tokens are picked more often.

Why it matters

Temperature trades consistency for variety. Extraction, classification and code generation usually want low values so the same input gives nearly the same output. Brainstorming, naming and creative writing benefit from higher values. A temperature of 0 typically means always choosing the most likely token (greedy decoding), though some systems still show small run-to-run differences from hardware and batching.

Example

Prompt: "The capital of France is"
temperature 0.0 -> " Paris"  (almost every time)
temperature 1.5 -> " Paris", " the city of light", " a ..." (varies)

Most APIs accept a numeric value, commonly between 0 and 1 or 0 and 2, and the useful range depends on the provider.

Common mistakes

• Expecting temperature 0 to guarantee identical outputs or to stop hallucinations. It only removes sampling randomness.
• Changing temperature and top-p together without testing, since they interact.
• Using a high value for structured output such as JSON, which raises the risk of malformed results.

Related

• Top-p sampling
• LLM
• Hallucination
• Tokenization

This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.