threatdeck
Live 13:45 UTC ·

MITRE ATLAS · Execution

AML.T0051 · LLM Prompt Injection

An adversary may craft malicious prompts as inputs to an LLM that cause the LLM to act in unintended ways. These "prompt injections" are often designed to cause the model to ignore aspects of its original instructions and follow the adversary's instructions instead.

MITRE ATLAS

Mitigations

  • Validate AI Model
  • Control Access to AI Models and Data in Production
  • Generative AI Guardrails
  • Generative AI Guidelines
  • Generative AI Model Alignment
  • AI Telemetry Logging and Monitoring

Detections 4

Related techniques

In the news

  1. Attacks on AI Research arXiv (AI security papers) Medium

    PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

    PIPETTE shows that high attack success rates in automated red-teaming do not guarantee robustness gains from adversarial training against indirect prompt injection in LLM agents.

    ReviewEvaluate agent hardening against diverse injection techniques, not only high-ASR attacks

    LLM01AML.T0051 arXiv:2611.24362 #prompt-injection#red-teaming#llm-agents 1 min read Brief In your stack
  2. Offensive AI Research arXiv (AI security papers) Medium

    AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents

    AutoPI is an adaptive red-teaming tool that uses a prompt-injection taxonomy to find many distinct techniques that break LLM agents, not just frequent ones.

    ReviewAssess agent prompt-injection test coverage across multiple technique classes

    LLM01AML.T0051 arXiv:2611.24313 #prompt-injection#red-teaming#llm-agents 1 min read In your stack
  3. Defensive AI Research arXiv (AI security papers) Medium

    Evolution Under Fire: Evolving Agent Safety Harnesses with Adaptive Injection Stressors

    Paper introduces Evolution Under Fire, a framework that evolves safety harnesses and adaptive injection stressors to defend tool-using LLM agents against prompt injection.

    LLM01AML.T0051 arXiv:2611.22583 #prompt-injection#agent-safety#llm-agents 1 min read In your stack

Tools

Test or exercise it

Detect or mitigate it

Mappings are AI-assisted and checked against the framework definitions; corrections welcome.

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.