Temperature
Scaling the logits to make sampling more or less random.
Overview
Temperature divides the logits before the softmax, so a low value sharpens the distribution toward the most likely token and a high value flattens it. It solves the problem that a model's raw distribution is often too peaked for open-ended text and too flat for factual answers. It exists as the simplest control over the trade between coherence and variety.
Technology Graph
How Temperature connects to the rest of the stack.