Red Teaming
Adversarially probing a model for harmful or unsafe behaviour.
Обзор
Red teaming is the deliberate attempt to make a model produce output it should not, using prompt injection, roleplay framing and other adversarial techniques. It solves the problem that a model which refuses obvious harmful requests may still comply when the request is disguised. It exists because the failure is found by looking for it: a model cannot be assessed for unsafe behaviour by asking it politely, and the findings are what safety training is then measured against.
Граф технологий
Как Red Teaming связана с остальным стеком.
Использует и зависит от
- Prompt Engineering technique Prompting Red Teaming uses Prompt Engineering High ✓ source
Применяется к
- Alignment technique Safety & Alignment Red Teaming used for Alignment High ✓ source
- Guardrails technique Safety & Alignment Guardrails used for Red Teaming Medium ✓ source
- Jailbreak technique Safety & Alignment Jailbreak used for Red Teaming High ✓ source