Jailbreak
Prompt that circumvents a model's refusal behaviour.
Überblick
A jailbreak is a prompt, or a sequence of them, that gets a model to produce output its safety training is meant to refuse. It solves nothing; it is the failure mode that red teaming exists to find. It exists as a named problem because refusals are learned behaviour layered over a model that can in principle produce the text, so the boundary is probabilistic rather than structural - which is why defences are applied around the model rather than only inside it.
Technologiegraph
Wie Jailbreak mit dem übrigen Stack zusammenhängt.
Nutzt und hängt ab von
- Prompt Engineering technique Prompting Jailbreak based on Prompt Engineering High ✓ source
Angewendet auf
- Red Teaming technique Safety & Alignment Jailbreak used for Red Teaming High ✓ source