Learn / Securing LLM applications
Lesson 1 of 610 minReviewed
Prompt injection: direct, indirect, and the designs that contain it
Tell direct from indirect prompt injection, know which defences are probabilistic, and choose an architecture pattern that keeps injected text from triggering consequential actions.
The term dates from September 2022: Riley Goodside showed GPT-3 following instructions hidden in its input, and Simon Willison named the problem prompt injection by analogy with SQL injection, because both arise when untrusted input is concatenated into instructions [1]. OWASP lists it first in the 2025 Top 10 for LLM applications and covers two variants [3].
| Direct | Indirect | |
|---|---|---|
| Attacker | The person typing into the app | Whoever wrote content the app reads for someone else |
| Entry point | User message | Web pages, email, documents, tool output, RAG chunks |
| Typical goal | Bypass rules, extract the system prompt | Exfiltrate the victim's data, act on the victim's behalf |
| Who is harmed | Mostly the operator | The user, who never sees the attack text |
Greshake and colleagues showed the indirect form against real applications in 2023 [2]. It is the variant that matters most for agents, because the victim does nothing wrong: they ask for a summary and the document does the rest.
Layers that reduce the odds
Microsoft's published approach starts with probabilistic layers [4]. Hardened system prompts tell the model how to treat external text. Spotlighting marks untrusted content so the model can tell where it came from, using random delimiters, a data marker inserted throughout the text, or an encoding such as base64 [4]. In the original paper, spotlighting cut attack success on GPT-family models from above 50% to below 2% in the authors' tests [5]. Classifiers such as Prompt Shields then flag injection attempts in user input and documents [4].
Microsoft states that its design does not rely on blocking every injection; it aims to make a successful injection harmless through deterministic controls and user approval [4]. That is the right mental model: these layers lower the rate, architecture decides the impact.
Patterns that contain the impact
Beurer-Kellner and co-authors describe six patterns built on one principle: once an agent has read untrusted input, it must be constrained so that the input cannot trigger consequential actions [6].
| Pattern | How it constrains the model | Cost |
|---|---|---|
| Action-selector | Maps a request to one of a fixed set of actions and never sees their results | No open-ended tasks |
| Plan-then-execute | Commits to the sequence of tool calls before reading untrusted data | Data can still shape arguments |
| LLM map-reduce | Isolated instances each handle one untrusted item without dangerous tools | Extra calls, constrained outputs |
| Dual LLM | A privileged model with tools sees only references to text a quarantined model read | Complexity, weaker UX |
| Code-then-execute | The model writes a program; the program, not the model, touches untrusted data | Needs a safe interpreter |
| Context-minimisation | Removes earlier content from context once it has served its purpose | Less conversational memory |
Willison proposed the dual LLM pattern in 2023 and was candid that it adds complexity and is not complete [7]. CaMeL, from Debenedetti and colleagues, develops the idea: it derives control and data flow from the trusted user query, attaches capabilities to every value, and enforces policy when tools are called, so retrieved data cannot change what the program does. It solved 77% of AgentDojo tasks with provable security, against 84% for an undefended agent [8].
Try it on ThreatDeck
- Prompt Injection (LLM01) See recent stories and defences mapped to the OWASP entry.
- LLM Prompt Injection (ATLAS AML.T0051) Compare the ATLAS view: tactics, mitigations and linked case studies.
- Red-team an LLM application Turn the patterns into test cases for indirect injection.
- Labs & practice Practise indirect injection in a hosted lab before testing your own app.
Self-check
4 questionsPick an answer to see whether it is right and why. Only your score is saved, in this browser.
-
The instructions arrive inside content the app processes for someone else, written by a third party. That is indirect injection; the recruiter is the victim, not the attacker.
-
Spotlighting changes how untrusted content is presented (delimiters, data markers, encoding). It reduced attack success sharply in tests, but Microsoft itself groups it with probabilistic layers that sit in front of deterministic controls.
-
The sequence of calls is fixed before untrusted data is read, so injection cannot add a new action. Data can still flow into arguments, which is why outbound content needs its own checks.
-
The quarantined model reads untrusted text but has no tools. The privileged model plans with references such as $VAR1, and plain code performs the calls. Tainted text never reaches a model that can act.
References
- Prompt injection attacks against GPT-3. Simon Willison, 2022-09-12.
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. Greshake et al. (arXiv:2302.12173), 2023-02.
- LLM01:2025 Prompt Injection. OWASP GenAI Security Project, 2024-11.
- How Microsoft defends against indirect prompt injection attacks. Microsoft Security Response Center, 2025-07-29.
- Defending Against Indirect Prompt Injection Attacks With Spotlighting. Hines et al. (arXiv:2403.14720), 2024-03.
- Design Patterns for Securing LLM Agents against Prompt Injections. Beurer-Kellner et al. (arXiv:2506.08837), 2025-06.
- The Dual LLM pattern for building AI assistants that can resist prompt injection. Simon Willison, 2023-04-25.
- Defeating Prompt Injections by Design (CaMeL). Debenedetti et al. (arXiv:2503.18813), 2025-03.