threatdeck
Live 13:45 UTC ·

Learn / Securing LLM applications

Lesson 5 of 69 minReviewed

Guardrails and their limits

Know what input and output guardrails actually detect, why adaptive attackers get past them, and how to run them as a monitored layer rather than a security boundary.

A guardrail is a check that runs before or after a model call and blocks, rewrites or flags content. Most products combine several kinds. Amazon Bedrock Guardrails, for example, offers content filters (including a Prompt Attack category), denied topics, custom word filters, sensitive-information filters for PII and custom regular expressions, contextual grounding checks for RAG answers, and Automated Reasoning checks, applied to inputs and model responses [1].

Common guardrail layers
LayerExampleWhat it is good atWhere it falls short
Prompt-attack classifierPrompt Shields (user prompt and document attacks) [4]Known jailbreak and injection stylesParaphrase, new encodings, non-English text
PII and secret filtersBedrock sensitive-information filters [1]Masking structured identifiers in transitContext-dependent data, free-text secrets
Topic and word filtersDenied topics, word lists [1]Keeping an assistant on taskAnything phrased differently
Grounding checksBedrock contextual grounding [1]Answers that drift from retrieved sourcesPoisoned sources look grounded
Programmable railsNeMo Guardrails input, dialogue, retrieval, execution and output rails [2]Encoding app-specific flows and checksRails that call a model share its weaknesses

Open-source options exist on the same pattern. NeMo Guardrails (Apache 2.0) lets you define rails in Colang [2]. LLM Guard offers input and output scanners such as prompt-injection, secrets and malicious-URL detection, but its repository is now archived and no longer maintained, which matters if you depend on it [3]. Microsoft's Prompt Shields documentation is explicit about scope: the shields detect manipulation attempts, not whether content is harmful or grounded, document-attack detection is off by default in Foundry, and performance can vary outside English [4].

Why a high benchmark score is not a boundary

Guardrails are classifiers, and Microsoft groups them with the probabilistic layers of its own injection defence [5]. The more important evidence comes from testing defences against attackers who adapt to them. Nasr and colleagues tuned general optimisation methods (gradient descent, reinforcement learning, random search and human-guided exploration) against twelve recent jailbreak and injection defences and bypassed them, with attack success above 90% for most; the majority had originally reported near-zero attack success [6]. A defence evaluated only on a fixed set of known attack strings tells you little about a motivated attacker.

The UK NCSC adds two practical warnings: deny-list filtering of phrases such as "ignore previous instructions" fails because attackers can rephrase endlessly, and products that claim to stop prompt injection deserve scepticism; prefer those that explain how they reduce it [7].

Running guardrails well

  1. Put them in front of deterministic controls, never instead of them. A blocked attack is a bonus; an unblocked one must still hit authorisation checks and approvals.
  2. Measure false positives and false negatives on your own traffic and languages before choosing block over annotate [4].
  3. Send every intervention to your SIEM. A burst of blocks from one principal is a strong signal that someone is probing.
  4. Use versioned guardrail configurations [1] and alert on changes or deletion: anyone holding the right credentials can weaken a guardrail as easily as an administrator.
  5. Re-test after every model or configuration change, since classifier behaviour shifts with both.

Try it on ThreatDeck

Self-check

4 questions

Pick an answer to see whether it is right and why. Only your score is saved, in this browser.

  1. A vendor reports that its injection filter blocks 99.9% of attacks on a public benchmark. Which question matters most before relying on it?
  2. Why does a deny-list of phrases like "ignore previous instructions" fail?
  3. What is the most useful thing to do with guardrail intervention events?
  4. Where does a guardrail belong in a defensive design?

References

  1. Detect and filter harmful content by using Amazon Bedrock Guardrails. Amazon Web Services.
  2. NeMo Guardrails. NVIDIA (GitHub).
  3. LLM Guard (archived repository). Protect AI (GitHub).
  4. Prompt Shields in Azure AI Content Safety. Microsoft Learn.
  5. How Microsoft defends against indirect prompt injection attacks. Microsoft Security Response Center, 2025-07-29.
  6. The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses against LLM Jailbreaks and Prompt Injections. Nasr et al. (arXiv:2510.09023), 2025-10.
  7. Prompt injection is not SQL injection (it may be worse). UK National Cyber Security Centre, 2025-12-08.
Completing the self-check also marks the lesson complete. Saved in this browser only.

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.