Безопасность и выравнивание
Alignment, red-teaming, moderation and the governance work around deployed models, including the failure modes that only appear at scale.
6 technologies
- Alignment technique Making a model's behaviour match what people want, not just what is likely.
- Guardrails technique Checks around a model's input and output that constrain what it can do.
- Constitutional AI technique Aligning a model using written principles rather than human labels alone.
- Red Teaming technique Adversarially probing a model for harmful or unsafe behaviour.
- RLAIF technique Reinforcement learning from AI feedback instead of human feedback.
- Jailbreak technique Prompt that circumvents a model's refusal behaviour.