Alignment
Making a model's behaviour match what people want, not just what is likely.
Overview
Alignment covers the methods that steer a trained model toward being helpful, honest and harmless rather than merely fluent. It solves the problem that next-token prediction on web text optimises for plausibility, which includes plausible-sounding harmful or false output. It exists because the objective a model is trained on and the behaviour people want from it are not the same thing, and the gap has to be closed deliberately.
Technology Graph
How Alignment connects to the rest of the stack.
Applied to
- Constitutional AI technique Safety & Alignment Constitutional AI used for Alignment High ✓ source
- Instruction Tuning technique Techniques Instruction Tuning used for Alignment High ✓ source
- LLM concept Models Alignment used for LLM High ✓ source
- RLAIF technique Safety & Alignment RLAIF used for Alignment High ✓ source
- Red Teaming technique Safety & Alignment Red Teaming used for Alignment High ✓ source
- DPO technique Techniques DPO enables Alignment High ✓ source
- RLHF technique Techniques RLHF enables Alignment High ✓ source