Top-p, also called nucleus sampling, is a text generation setting that restricts the model to the smallest group of candidate tokens whose probabilities add up to at least p. The model then samples only from that group and ignores the long tail of unlikely tokens.
Why it matters
Top-p adapts to the situation. When the model is confident, a handful of tokens may already cover 90 percent of the probability, so the pool is tiny. When many continuations are plausible, the pool grows. This avoids the occasional bizarre token that pure random sampling can pick, while keeping more variety than always choosing the single best token. It is commonly used alongside, or instead of, temperature.
Example
Next-token probabilities: the 0.50, a 0.25, this 0.12, that 0.08, zebra 0.05
top_p = 0.90
Kept: the, a, this, that (0.50+0.25+0.12 = 0.87 < 0.90, so add "that")
Dropped: zebraThe model renormalises the remaining probabilities and samples from those four.
Common mistakes
- Setting top-p to 1.0 and expecting filtering. At 1.0 nothing is removed.
- Tuning temperature and top-p aggressively at the same time, which makes behaviour hard to reason about. Most providers suggest changing one.
- Confusing it with top-k, which keeps a fixed number of tokens rather than a probability mass.
Related
Get notified when this term updates
Plus occasional practical tips on uptime and monitoring, nothing else. Unsubscribe anytime.
This page is general information written by our editorial team and updated when documentation changes. Servers and providers differ, so test changes safely first. See our editorial policy and disclaimer.