MITRE ATLAS · Defense Evasion, Privilege Escalation
AML.T0054 · LLM Jailbreak
Adversaries may induce a large language model (LLM) to ignore, circumvent, or override its safety/alignment behaviors and/or guardrails to elicit outputs the model is intended to withhold. Once jailbroken, the LLM may be used in unintended ways by the adversary. Jailbreaks may be achieved via adversarial prompting, or by modifying model weights or safety mechanisms.
Mitigations
- Validate AI Model
- Generative AI Guardrails
- Generative AI Guidelines
- Generative AI Model Alignment
- AI Red Team
Detections 3
Related techniques
- LLM01 Prompt Injection OWASP LLM
In the news
-
Dressed in Authority: Practical Black-Box Jailbreaking of MLLMs via Visual Trust Bias
Researchers propose VTBA, a black-box jailbreak for multimodal LLMs that exploits their trust bias toward authoritative visual features.
-
Is 4-Bit Quantization More Than Noise to a Chat Model's Safety Behaviour? Attributing Behavioural Shifts with Matched Perturbations
Study finds 4-bit quantization shifts refusal behaviour in Qwen2.5-3B and Llama-3.2-3B chat models, with one model complying with jailbreaks more often and the other less.
Tools
Test or exercise it
- Augustus LLM red-teaming · Assess · Dual-use
- garak LLM red-teaming · Adopt · Dual-use
- llm-attacks (GCG) LLM red-teaming · Hold · Offense
- PyRIT LLM red-teaming · Adopt · Dual-use
Mappings are AI-assisted and checked against the framework definitions; corrections welcome.