New Technique Medium TD-2026-0084 For AI Eng, AppSec, Red Team, CISO AI-assisted
Agentic attacker reliably injects instructions into tool output across frontier models
The paper presents a synthetic prompt injection evaluation that uses random non-malicious actions as adversarial goals. It reports that an agentic attacker with increased capabilities can reliably get models to follow instructions found in tool output. The authors say this holds even for models previously considered robust on public benchmarks such as GraySwan Arena.
Why it matters
Organizations deploying tool-using AI agents may assume benchmark robustness implies protection against injected instructions. Attackers who automate injection attempts could make agents take unintended actions via untrusted tool output.
Do this
- Treat all tool and retrieval output as untrusted data and enforce instruction hierarchy in agent design.
- Restrict agent tool permissions so injected instructions cannot trigger high-impact actions without approval.
- Add human confirmation for sensitive tool calls initiated after reading external content.
- Run automated, recurring prompt injection red-teaming against agent deployments rather than relying on static benchmarks.
- Log and review agent tool calls that follow content retrieved from external sources.
Signals
- NoCISA KEV
- 23/100Priority
- 60%Confidence
Evidence
“We show that an agentic attacker of increased capabilities can reliably trick model to follow instructions in tool output in our evaluation”
Sources
Affected / related
Prompt injection via tool output, Agentic adaptive prompt injection attacks
Published 2026-11-09 04:34 UTC · updated 2026-11-12 18:05 UTC · AI-assisted summary, checked against quoted sources · How we score