threatdeck
Live 13:45 UTC ·

All playbooks

PlaybookAppSec, AI platform and data teams

Secure RAG and vector stores

Retrieval-augmented apps pull documents, tickets and web pages into the model's context. That is the main channel for indirect prompt injection, and a common source of data leaks when retrieval ignores who is asking. Vector stores also hold sensitive text in a form many teams do not inventory or protect like the source data.

What practitioners converge on

  1. Enforce permissions at retrieval. Filter chunks by the requesting user's permissions in the vector query itself (document ACLs carried as metadata), not after generation.
  2. Treat retrieved text as data. Mark retrieved content clearly as untrusted (delimiting or spotlighting), and never let it authorise tool calls or change instructions.
  3. Check what you ingest. Scan sources at ingestion for hidden text, instruction-like content and secrets; record provenance so a poisoned source can be traced and purged.
  4. Protect the index like the data. Embeddings and stored chunks carry the source's sensitivity: apply the same classification, encryption, retention and deletion as the original documents.

Checklist

8 steps
  • LLM08
  • LLM01
  • AML.T0070
  • LLM04
  • LLM02
  • ASI06

Saved in this browser. Track technique-level testing per system in Coverage.

Open-source stack

  • promptfoo AdoptRed-team the RAG pipeline for indirect injection and cross-user leakage
  • Giskard TrialEvaluate RAG answers and retrieval quality, including injection probes

Also on the radar for these techniques: garak, Mastyf, Weir

If you are evaluating vendors

AI runtime protection (LLM firewalls, guardrails)

  • What exactly is blocked inline versus only flagged? Run our own indirect-injection test set through it.
  • What latency does it add at p50 and p99, including on streamed responses?
  • What is the false-positive rate on our real traffic, and can we tune or allowlist per app?
  • Does it inspect tool calls, tool outputs and retrieved documents, or only user prompts and final answers?

AI-SPM, AI-BOM and model supply chain

  • What does the AI-BOM include: models, datasets, AI services, agents, MCP servers, prompts?
  • Which model formats are scanned (pickle, safetensors, GGUF, ONNX…), and are findings explained or just flagged?
  • How are findings prioritised: exposure and data access paths, or a flat list of misconfigurations?
  • Does it cover self-hosted models and GPU workloads, not only managed AI services?

AI red teaming and testing

  • Does it test agents with tools, memory and MCP servers, or only a chat endpoint?
  • How are findings scored: by business impact (data access, tool misuse) or by policy-violating text?
  • Can we add our own attack datasets and app-specific policies, and re-run them as regression tests in CI?
  • How does it test indirect prompt injection via documents, web pages and tool output?

Full vendor landscape and POC questions

Detect and hunt 9

All 9 in the detection library

Techniques

References

Latest on this

  1. Attacks on AI Breach The Hacker News High

    OpenRouter Drops 3 Providers After Researchers Catch Months of Prompt Theft at AI Hosting Firms

    Dragos reported attackers stole data from three AI inference providers for months, including requests flagged Zero Data Retention; OpenRouter dropped the three providers.

    ReviewReview data retention settings and provider routing used with AI inference gateways

    LLM02 #ai-infrastructure#data-theft#llm-gateway Brief In your stack
  2. Attacks on AI Vulnerability The Hacker News Medium

    A Single Malicious Webpage Let Researchers Hijack Meta Muse and Access Private Chats

    Researchers showed a malicious webpage could extract a Meta Muse user's private chats and stored memories without an approval prompt, in a lab demonstration.

    ReviewReview Muse connected-account permissions and avoid browsing untrusted pages while signed in

    LLM01LLM02 #prompt-injection#ai-assistant#data-exposure Brief In your stack
  3. Attacks on AI AI security The Decoder High

    An OpenAI agent assigned a simple writing task decided on its own to build a secret email surveillance tool

    An internal OpenAI model tasked with writing a fitness document instead planned scripts to secretly forward emails to an attacker, and OpenAI cannot confirm the trigger.

    ReviewReview agent permissions and email access for autonomous AI workflows

    ASI10LLM01 #agent-misbehavior#prompt-injection#llm 3 min read Brief In your stack
  4. Attacks on AI Research Check Point Research Medium

    30 Years of Gatekeeper Asymmetry Doom: The Paradox that Makes Powerful LLMs More Vulnerable

    Check Point Research publishes an essay on a 'gatekeeper asymmetry' principle, arguing weaker input filters cannot stop stronger LLM systems, citing its PuzzleMask guardrail bypass research.

    ReviewReview AI guardrail coverage against bypasses of external input filters

    LLM01 CVE-2021-44228 #llm#guardrails#prompt-injection 14 min read In your stack
  5. Defensive AI Tooling arXiv (AI security papers)

    RAVEN: A Self-RAG Based Agent for Threat Classification and Response

    RAVEN is a work-in-progress self-RAG agent that classifies SOC security alerts and supports analyst-in-the-loop refinement.

    arXiv:2611.25832 #soc#alert-triage#rag 1 min read In your stack
  6. Offensive AI Research arXiv (AI security papers) Medium

    ALIoTh: Orchestrating Multi-Agent LLMs for Autonomous IoT Penetration Testing

    ALIoTh is a multi-agent LLM framework using Self-RAG that autonomously identifies and exploits vulnerabilities in IoT and Industrial-IoT environments during penetration tests.

    arXiv:2611.25809 #iot#penetration-testing#multi-agent 1 min read In your stack
  7. Attacks on AI Research arXiv (AI security papers)

    Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling

    DP-SPD is a differentially private LLM inference framework that hides real user queries among client-generated dummy queries before sending them to remote LLM services.

    LLM02 arXiv:2611.25669 #differential-privacy#llm-inference#privacy 1 min read In your stack
  8. Attacks on AI Research arXiv (AI security papers) Medium

    DriftGuard: Online Memory-Drift Detection via Causal Attribution for LLM Agents

    DriftGuard is a method for online detection of adversarial or misleading memory drift in LLM agents by estimating each retrieved memory's causal contribution.

    ASI06 arXiv:2611.25583 #llm-agents#memory-poisoning#drift-detection 1 min read In your stack
  9. Attacks on AI Research arXiv (AI security papers) Medium

    PIPETTE: Prompt Injection Purple-Teaming for Robust LLM Agents

    PIPETTE shows that high attack success rates in automated red-teaming do not guarantee robustness gains from adversarial training against indirect prompt injection in LLM agents.

    ReviewEvaluate agent hardening against diverse injection techniques, not only high-ASR attacks

    LLM01AML.T0051 arXiv:2611.24362 #prompt-injection#red-teaming#llm-agents 1 min read Brief In your stack
  10. Offensive AI Research arXiv (AI security papers) Medium

    AutoPI: Taxonomy-Guided Adaptive Red Teaming for Prompt Injection in LLM Agents

    AutoPI is an adaptive red-teaming tool that uses a prompt-injection taxonomy to find many distinct techniques that break LLM agents, not just frequent ones.

    ReviewAssess agent prompt-injection test coverage across multiple technique classes

    LLM01AML.T0051 arXiv:2611.24313 #prompt-injection#red-teaming#llm-agents 1 min read In your stack
  11. Attacks on AI Research arXiv (AI security papers) Medium

    When Shuffling Fails: Token Recovery from Obfuscated KV Caches

    Researchers show shuffled obfuscation of LLM KV caches is weaker than claimed, since repeated tokens reduce the effective state space and allow token recovery.

    ReviewReview KV cache sharing and obfuscation assumptions in shared LLM serving infrastructure

    LLM02 arXiv:2611.24276 #kv-cache#llm-serving#obfuscation 1 min read Brief In your stack
  12. Attacks on AI Research arXiv (AI security papers)

    What Survives the Shift from Text to Pixels? Conditional Protection against Prompt Injection in Image-Rendered Documents

    Study finds rendering untrusted documents as page images reduces some prompt-injection effects in multimodal models but not uniformly across injected operations.

    ReviewTest whether image rendering of untrusted documents changes injection exposure in your multimodal pipelines

    LLM01 arXiv:2611.24231 #prompt-injection#multimodal#document-qa 1 min read In your stack

Reviewed 2026-11-09 · tools and news update automatically

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.