Red Teaming
Adversarially probing a model for harmful or unsafe behaviour.
Überblick
Red teaming is the deliberate attempt to make a model produce output it should not, using prompt injection, roleplay framing and other adversarial techniques. It solves the problem that a model which refuses obvious harmful requests may still comply when the request is disguised. It exists because the failure is found by looking for it: a model cannot be assessed for unsafe behaviour by asking it politely, and the findings are what safety training is then measured against.
Technologiegraph
Wie Red Teaming mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
- Prompt Engineering technique Prompting Red Teaming uses Prompt Engineering High ✓ source
Angewendet auf
- Alignment technique Safety & Alignment Red Teaming used for Alignment High ✓ source
- Guardrails technique Safety & Alignment Guardrails used for Red Teaming Medium ✓ source
- Jailbreak technique Safety & Alignment Jailbreak used for Red Teaming High ✓ source