Constrained Decoding
Restricting which tokens a model may emit at each step.
Overview
Constrained decoding masks the logits so only tokens consistent with a grammar, schema or regular expression remain possible. It solves the problem that asking for valid output produces valid output most of the time, and a pipeline cannot be built on most of the time. It exists because enforcement during generation removes the need for retries and repair logic afterwards.
Technology Graph
How Constrained Decoding connects to the rest of the stack.