Red Teaming
Adversarially probing a model for harmful or unsafe behaviour.
Aperçu
Red teaming is the deliberate attempt to make a model produce output it should not, using prompt injection, roleplay framing and other adversarial techniques. It solves the problem that a model which refuses obvious harmful requests may still comply when the request is disguised. It exists because the failure is found by looking for it: a model cannot be assessed for unsafe behaviour by asking it politely, and the findings are what safety training is then measured against.
Graphe des technologies
Comment Red Teaming se rattache au reste de la pile.
Utilise et dépend de
- Prompt Engineering technique Prompting Red Teaming uses Prompt Engineering High ✓ source
Appliqué à
- Alignment technique Safety & Alignment Red Teaming used for Alignment High ✓ source
- Guardrails technique Safety & Alignment Guardrails used for Red Teaming Medium ✓ source
- Jailbreak technique Safety & Alignment Jailbreak used for Red Teaming High ✓ source