Red Teaming
Adversarially probing a model for harmful or unsafe behaviour.
Resumen
Red teaming is the deliberate attempt to make a model produce output it should not, using prompt injection, roleplay framing and other adversarial techniques. It solves the problem that a model which refuses obvious harmful requests may still comply when the request is disguised. It exists because the failure is found by looking for it: a model cannot be assessed for unsafe behaviour by asking it politely, and the findings are what safety training is then measured against.
Grafo de tecnologías
Cómo se conecta Red Teaming con el resto del stack.
Usa y depende de
- Prompt Engineering technique Prompting Red Teaming uses Prompt Engineering High ✓ source
Aplicado a
- Alignment technique Safety & Alignment Red Teaming used for Alignment High ✓ source
- Guardrails technique Safety & Alignment Guardrails used for Red Teaming Medium ✓ source
- Jailbreak technique Safety & Alignment Jailbreak used for Red Teaming High ✓ source