Constrained Decoding
Restricting which tokens a model may emit at each step.
Aperçu
Constrained decoding masks the logits so only tokens consistent with a grammar, schema or regular expression remain possible. It solves the problem that asking for valid output produces valid output most of the time, and a pipeline cannot be built on most of the time. It exists because enforcement during generation removes the need for retries and repair logic afterwards.
Graphe des technologies
Comment Constrained Decoding se rattache au reste de la pile.