MITRE ATLAS · Execution
AML.T0051 · LLM Prompt Injection
An adversary may craft malicious prompts as inputs to an LLM that cause the LLM to act in unintended ways. These "prompt injections" are often designed to cause the model to ignore aspects of its original instructions and follow the adversary's instructions instead.
Mitigations
- Validate AI Model
- Control Access to AI Models and Data in Production
- Generative AI Guardrails
- Generative AI Guidelines
- Generative AI Model Alignment
- AI Telemetry Logging and Monitoring
Detections 4
- YARA: invisible Unicode tag characters (U+E0000–U+E007F) in prompts, tool output or agent files
- AI coding agent spawns a shell that pipes a download into an interpreter
- Model output contains markdown images or reference links to external hosts
- Burst of Bedrock guardrail interventions from one principal and source
Related techniques
- LLM01 Prompt Injection OWASP LLM
- ASI01 Agent Goal Hijack OWASP Agentic
In the news
-
PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents
PIPETTE shows that high attack success rates in automated red-teaming do not guarantee robustness gains from adversarial training against indirect prompt injection in LLM agents.
ReviewEvaluate agent hardening against diverse injection techniques, not only high-ASR attacks
LLM01AML.T0051 arXiv:2611.24362 #prompt-injection#red-teaming#llm-agents 1 min read Brief In your stack -
AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents
AutoPI is an adaptive red-teaming tool that uses a prompt-injection taxonomy to find many distinct techniques that break LLM agents, not just frequent ones.
ReviewAssess agent prompt-injection test coverage across multiple technique classes
-
Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors
Paper introduces Evolution Under Fire, a framework that evolves safety harnesses and adaptive injection stressors to defend tool-using LLM agents against prompt injection.
Tools
Mappings are AI-assisted and checked against the framework definitions; corrections welcome.