New Technique Medium TD-2026-0043 For AI Eng, AppSec, Red Team, CISO AI-assisted
CREO: Self-evolving attacks on LLM search agents via injected web content
An arXiv preprint proposes an adversarial self-evolution framework that tests LLM search agents against attacks that adapt over time. The method, Contrastive Rollout Evolutionary Optimization (CREO), evolves an attack strategy population using contrastive signals, and the authors use a controlled sandbox to measure vulnerabilities. The authors report higher attack efficacy than static, human-crafted baselines across multi-hop QA benchmarks and frontier LLMs.
Why it matters
Search agents that act on retrieved web content can be steered by malicious information injection or context manipulation. Static defenses tuned to known templates may miss attacks that adapt to the defender's mitigations.
Do this
- Treat all web-retrieved content as untrusted input to the agent, and separate it from system instructions in agent prompts.
- Add source provenance and filtering checks for retrieved pages before they enter the agent context.
- Log search queries, retrieved sources, and agent tool calls so that anomalous answer shifts can be traced to specific sources.
- Include adaptive, multi-round adversarial testing in search agent red-team exercises, not only fixed attack templates.
- Track the paper's authors' released code or benchmarks, if published, before using them to test internal agents.
Signals
- NoCISA KEV
- 22/100Priority
- 55%Confidence
Evidence
“these agents are highly susceptible to various adversarial attacks (e.g., malicious information injection or stealthy context manipulation)”
“we propose an adversarial self-evolution framework that integrates bi-level optimization between the attacker's strategy evolution and the defender's adaptive mitigation”
“attack efficacy superior to static baselines, underscoring the urgent need to develop more robust defense frameworks”
Sources
Affected / related
Contrastive Rollout Evolutionary Optimization (CREO), Malicious information injection into search agent context
Published 2026-11-02 11:06 UTC · updated 2026-11-09 13:50 UTC · AI-assisted summary, checked against quoted sources · How we score