threatdeck
Live 14:45 UTC ·

New Technique Medium TD-2026-0043 For AI Eng, AppSec, Red Team, CISO AI-assisted

CREO: Self-evolving attacks on LLM search agents via injected web content

An arXiv preprint proposes an adversarial self-evolution framework that tests LLM search agents against attacks that adapt over time. The method, Contrastive Rollout Evolutionary Optimization (CREO), evolves an attack strategy population using contrastive signals, and the authors use a controlled sandbox to measure vulnerabilities. The authors report higher attack efficacy than static, human-crafted baselines across multi-hop QA benchmarks and frontier LLMs.

Why it matters

Search agents that act on retrieved web content can be steered by malicious information injection or context manipulation. Static defenses tuned to known templates may miss attacks that adapt to the defender's mitigations.

Do this

  1. Treat all web-retrieved content as untrusted input to the agent, and separate it from system instructions in agent prompts.
  2. Add source provenance and filtering checks for retrieved pages before they enter the agent context.
  3. Log search queries, retrieved sources, and agent tool calls so that anomalous answer shifts can be traced to specific sources.
  4. Include adaptive, multi-round adversarial testing in search agent red-team exercises, not only fixed attack templates.
  5. Track the paper's authors' released code or benchmarks, if published, before using them to test internal agents.

Signals

  • NoCISA KEV
  • 22/100Priority
  • 55%Confidence

Evidence

“these agents are highly susceptible to various adversarial attacks (e.g., malicious information injection or stealthy context manipulation)”

arXiv (AI security papers)

“we propose an adversarial self-evolution framework that integrates bi-level optimization between the attacker's strategy evolution and the defender's adaptive mitigation”

arXiv (AI security papers)

“attack efficacy superior to static baselines, underscoring the urgent need to develop more robust defense frameworks”

arXiv (AI security papers)

Sources

Affected / related

Contrastive Rollout Evolutionary Optimization (CREO), Malicious information injection into search agent context

Published 2026-11-02 11:06 UTC · updated 2026-11-09 13:50 UTC · AI-assisted summary, checked against quoted sources · How we score

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.