threatdeck
Live 13:45 UTC ·

Learn AI security

Path 5 of 6Practitioner6 lessons · 50 minReviewed 2026-11-10

Red teaming AI systems

Plan and run security red team engagements against LLM applications and agents: scoping and rules of engagement, agent and indirect-injection testing, open-source tooling, impact-based findings and continuous regression testing.

For: Penetration testers, AppSec engineers and internal red teams adding AI systems to scope

6 lessons
  1. 01
    What AI red teaming is, and what it is not

    Define AI red teaming for a security team, separate security findings from safety benchmarking, and place the work in NIST AI 600-1 and the OWASP GenAI Red Teaming Guide.

    8 min · 4 questions

  2. 02
    Scoping an AI engagement and its rules of engagement

    Describe the system under test down to its tools, data and identities, write a threat model with impacts that count, and agree authorisation, test data and stop conditions before the first prompt.

    9 min · 4 questions

  3. 03
    Testing agents and indirect prompt injection

    Design agent tests where the attacker controls a tool output and the goal is an action, measure utility and attack success separately, and avoid the three ways agent results mislead: single attempts, aggregates and static attacks.

    9 min · 4 questions

  4. 04
    Choosing red team tooling: garak, PyRIT, promptfoo, Inspect

    Match open-source tools to the job (scanning an endpoint, orchestrating multi-turn campaigns, testing an application in CI, building custom evaluations) and know the limits of automated results.

    8 min · 4 questions

  5. 05
    Impact-based findings and reporting

    Rate AI findings by what the attacker achieved, make nondeterministic results reproducible, map them to OWASP and ATLAS ids, and hand over a fix plus a regression test.

    8 min · 4 questions

  6. 06
    Running it continuously: regression suites and retest triggers

    Turn findings into a regression suite, decide which changes trigger a retest, gate CI on results without flaky failures, and keep periodic human rounds for what automation misses.

    8 min · 4 questions

Start with lesson 1

Lessons cite primary sources and were last checked on the date shown in each lesson. Corrections are welcome by email (footer). Other paths: Fundamentals · LLM applications · Agents & MCP · AI coding assistants · Detect & respond

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.