Constrained Decoding
Restricting which tokens a model may emit at each step.
Überblick
Constrained decoding masks the logits so only tokens consistent with a grammar, schema or regular expression remain possible. It solves the problem that asking for valid output produces valid output most of the time, and a pipeline cannot be built on most of the time. It exists because enforcement during generation removes the need for retries and repair logic afterwards.
Technologiegraph
Wie Constrained Decoding mit dem übrigen Stack zusammenhängt.