Learn / Securing LLM applications
Lesson 5 of 69 minReviewed
Guardrails and their limits
Know what input and output guardrails actually detect, why adaptive attackers get past them, and how to run them as a monitored layer rather than a security boundary.
A guardrail is a check that runs before or after a model call and blocks, rewrites or flags content. Most products combine several kinds. Amazon Bedrock Guardrails, for example, offers content filters (including a Prompt Attack category), denied topics, custom word filters, sensitive-information filters for PII and custom regular expressions, contextual grounding checks for RAG answers, and Automated Reasoning checks, applied to inputs and model responses [1].
| Layer | Example | What it is good at | Where it falls short |
|---|---|---|---|
| Prompt-attack classifier | Prompt Shields (user prompt and document attacks) [4] | Known jailbreak and injection styles | Paraphrase, new encodings, non-English text |
| PII and secret filters | Bedrock sensitive-information filters [1] | Masking structured identifiers in transit | Context-dependent data, free-text secrets |
| Topic and word filters | Denied topics, word lists [1] | Keeping an assistant on task | Anything phrased differently |
| Grounding checks | Bedrock contextual grounding [1] | Answers that drift from retrieved sources | Poisoned sources look grounded |
| Programmable rails | NeMo Guardrails input, dialogue, retrieval, execution and output rails [2] | Encoding app-specific flows and checks | Rails that call a model share its weaknesses |
Open-source options exist on the same pattern. NeMo Guardrails (Apache 2.0) lets you define rails in Colang [2]. LLM Guard offers input and output scanners such as prompt-injection, secrets and malicious-URL detection, but its repository is now archived and no longer maintained, which matters if you depend on it [3]. Microsoft's Prompt Shields documentation is explicit about scope: the shields detect manipulation attempts, not whether content is harmful or grounded, document-attack detection is off by default in Foundry, and performance can vary outside English [4].
Why a high benchmark score is not a boundary
Guardrails are classifiers, and Microsoft groups them with the probabilistic layers of its own injection defence [5]. The more important evidence comes from testing defences against attackers who adapt to them. Nasr and colleagues tuned general optimisation methods (gradient descent, reinforcement learning, random search and human-guided exploration) against twelve recent jailbreak and injection defences and bypassed them, with attack success above 90% for most; the majority had originally reported near-zero attack success [6]. A defence evaluated only on a fixed set of known attack strings tells you little about a motivated attacker.
The UK NCSC adds two practical warnings: deny-list filtering of phrases such as "ignore previous instructions" fails because attackers can rephrase endlessly, and products that claim to stop prompt injection deserve scepticism; prefer those that explain how they reduce it [7].
Running guardrails well
- Put them in front of deterministic controls, never instead of them. A blocked attack is a bonus; an unblocked one must still hit authorisation checks and approvals.
- Measure false positives and false negatives on your own traffic and languages before choosing block over annotate [4].
- Send every intervention to your SIEM. A burst of blocks from one principal is a strong signal that someone is probing.
- Use versioned guardrail configurations [1] and alert on changes or deletion: anyone holding the right credentials can weaken a guardrail as easily as an administrator.
- Re-test after every model or configuration change, since classifier behaviour shifts with both.
Try it on ThreatDeck
- Detection: guardrail intervention burst Turn Bedrock guardrail blocks into a probing alert per principal and source.
- Detection: guardrail deleted or updated Alert when someone weakens or removes a guardrail.
- NeMo Guardrails on the radar Check project activity and how it fits with other guardrail tools.
Self-check
4 questionsPick an answer to see whether it is right and why. Only your score is saved, in this browser.
-
Adaptive attacks bypassed most of twelve defences that reported near-zero attack success on static tests. A score on fixed attack strings says little about an attacker who tunes against your filter.
-
Natural language offers unlimited rewordings. The NCSC calls out phrase deny-lists specifically as ineffective for this reason.
-
Blocks are evidence of attempts. Aggregated by principal and source they become a detection signal, which is how the related ThreatDeck detection works.
-
Guardrails lower the rate of successful attacks; deterministic controls bound the impact of the ones that get through. Treating a classifier as the boundary leaves nothing behind it.
References
- Detect and filter harmful content by using Amazon Bedrock Guardrails. Amazon Web Services.
- NeMo Guardrails. NVIDIA (GitHub).
- LLM Guard (archived repository). Protect AI (GitHub).
- Prompt Shields in Azure AI Content Safety. Microsoft Learn.
- How Microsoft defends against indirect prompt injection attacks. Microsoft Security Response Center, 2025-07-29.
- The Attacker Moves Second: Stronger Adaptive Attacks Bypass Defenses against LLM Jailbreaks and Prompt Injections. Nasr et al. (arXiv:2510.09023), 2025-10.
- Prompt injection is not SQL injection (it may be worse). UK National Cyber Security Centre, 2025-12-08.