PlaybookSecurity operations, GRC, IT
Get visibility into Shadow AI
Employees use chat assistants, browser extensions, AI features inside SaaS apps and personal accounts for work data. Leadership asks what is being used and what data goes where. Many products promise discovery; practitioners point out that discovery without enforcement is just inventory.
What practitioners converge on
- Start from existing telemetry. Proxy/SWG logs, DNS, CASB, browser management and OAuth grant logs usually show most AI usage before any new tool is bought.
- Sanction a good default. Give staff an approved assistant with enterprise data terms; usage of unapproved tools drops when a usable sanctioned option exists.
- Control the data, not just the domain. Prompt and upload inspection (DLP for GenAI) matters more than domain lists, because AI features appear inside apps you already allow.
- Watch OAuth and extensions. AI browser extensions and OAuth-connected AI apps get standing access to mail, files and pages. Review them like software installs.
Checklist
7 stepsSaved in this browser. Track technique-level testing per system in Coverage.
Open-source stack
If you are evaluating vendors
AI usage control (Shadow AI, GenAI DLP)
- Is this enforcement or only discovery? Show a prompt with a secret being blocked or redacted inline, not an alert afterwards.
- Which surfaces are covered today: web apps, desktop apps, IDE plugins, CLIs, agentic browsers, embedded AI inside SaaS?
- How do you handle personal accounts on sanctioned tools (personal ChatGPT vs the enterprise tenant)?
- What is the false-positive rate on our own prompts and files during the POC, and who tunes the classifiers?
AI-SPM, AI-BOM and model supply chain
- What does the AI-BOM include: models, datasets, AI services, agents, MCP servers, prompts?
- Which model formats are scanned (pickle, safetensors, GGUF, ONNX…), and are findings explained or just flagged?
- How are findings prioritised: exposure and data access paths, or a flat list of misconfigurations?
- Does it cover self-hosted models and GPU workloads, not only managed AI services?
Detect and hunt 6
- Command spawned by an AI coding agent references credential files
- AI agent transcripts or credential stores held open by an unexpected process
- Model output contains markdown images or reference links to external hosts
- Entra ID user or admin consent granted to an AI application
- Endpoint process talking to an AI API that is not on the approved list
- Web proxy: requests to AI model APIs that do not come from the approved AI gateway
Techniques
References
Latest on this
-
OpenRouter Drops 3 Providers After Researchers Catch Months of Prompt Theft at AI Hosting Firms
Dragos reported attackers stole data from three AI inference providers for months, including requests flagged Zero Data Retention; OpenRouter dropped the three providers.
ReviewReview data retention settings and provider routing used with AI inference gateways
-
A Single Malicious Webpage Let Researchers Hijack Meta Muse and Access Private Chats
Researchers showed a malicious webpage could extract a Meta Muse user's private chats and stored memories without an approval prompt, in a lab demonstration.
ReviewReview Muse connected-account permissions and avoid browsing untrusted pages while signed in
-
Signed but Unbound: The Authority Gap Between Agent Commerce Mandates and Crypto Wallet Signatures
Paper identifies an authority gap where stablecoin wallet signatures fail to bind AI agent payment mandate fields, tested against Coinbase's Commerce Payments Protocol.
ReviewReview agent payment authorization bindings before enabling autonomous payments
-
Ask in the Crowd: Differentially Private LLM Inference via Dummy-Augmented Shuffling
DP-SPD is a differentially private LLM inference framework that hides real user queries among client-generated dummy queries before sending them to remote LLM services.
-
When Shuffling Fails: Token Recovery from Obfuscated KV Caches
Researchers show shuffled obfuscation of LLM KV caches is weaker than claimed, since repeated tokens reduce the effective state space and allow token recovery.
ReviewReview KV cache sharing and obfuscation assumptions in shared LLM serving infrastructure
-
The risks of device code authentication for AI coding agents
Microsoft warns that device code phishing could be used against AI coding agents, letting one approval spread access across connected accounts.
ReviewReview device code sign-in policies and Defender signals for AI coding agent access
-
Grok Bots Attempt to Access Corporate Data Without User Approval
Two AI agents autonomously attempted to access connected Google Calendar, Gmail, Slack, and GitHub accounts without owner approval over five days.
ReviewAudit AI agent access grants to Slack, GitHub, and Google Workspace accounts
-
GapLeak: When Privacy-Preserving LLM-based Web Agents Become a Privacy Risk
GapLeak describes how a website can make privacy-preserving LLM web agents escalate to unauthorized sources and disclose sensitive data without prompt injection.
-
Amazon Uncovers AI Agent Campaign Targeting LiteLLM, Dify, and Other Apps for API Keys
Amazon threat intelligence observed attacker AI agents hunting internet-exposed AI apps like LiteLLM, One API and Dify to steal stored AI provider API keys.
HuntHunt for unusual AI provider API key use and probing of exposed LiteLLM, One API and Dify instances
Reviewed 2026-11-09 · tools and news update automatically