Learn / Detecting and responding to AI abuse
Lesson 2 of 69 minReviewed
The telemetry you actually have
Know which cloud, application and agent logs record AI use, which are off until you turn them on, and how to keep prompt logs from becoming a new data leak.
Before writing detections, find out what is actually recorded. For AI services the answer splits in two: audit logs that say who called which API, which are usually on, and content logs that say what was asked and answered, which are usually off.
| Source | What it records | Default |
|---|---|---|
| AWS CloudTrail, Bedrock | Caller, source IP, user agent, model id for InvokeModel, Converse and their streaming forms (management events); agent and knowledge-base calls are data events | Management events on; data events need event selectors [2] |
| Bedrock model invocation logging | Full request and response bodies, token counts, caller ARN, to CloudWatch Logs or S3 | Disabled [1] |
| Azure activity log | Control-plane operations on the resource, such as listing keys or changing deployments | Collected automatically [3] |
| Azure OpenAI resource logs | Request-level diagnostics routed by a diagnostic setting | Not stored until you create one [3] |
| Google Cloud Data Access audit logs, Vertex AI | GenerateContent, StreamGenerateContent, Predict and RawPredict calls (DATA_READ) | Disabled; must be enabled explicitly [4, 5] |
Two details catch teams out. CloudTrail never carries prompt or response text, so without invocation logging you can prove that a principal called a model but not what it generated; LLMjacking operators know this and read or delete the logging configuration first [6]. And the Converse API is logged under its own event name, not as InvokeModel, so a detection that only counts InvokeModel misses traffic attackers adopted within a month of the API's launch [6]. For streaming calls, Bedrock also writes a stream-completion event that carries token counts; count only AwsApiCall events when you count invocations, or you double them [2].
Application and agent telemetry
Cloud logs see the call, not the user behind it. Your own application should log, per request, the end user, the model, token counts, which tools were offered and called with which arguments, and the decision of any guardrail. The OpenTelemetry project maintains semantic conventions for generative AI clients, including MCP, so spans and metrics from different SDKs can share attribute names instead of each team inventing its own [7].
Coding agents run on endpoints, outside the cloud logs entirely. Claude Code, for example, exports OpenTelemetry events such as claude_code.user_prompt, claude_code.tool_decision and claude_code.tool_result once CLAUDE_CODE_ENABLE_TELEMETRY=1 is set; administrators can set the exporter for every user through managed settings, and a repository's own settings cannot enable telemetry or redirect it [8]. Content is opt-in: prompt text is redacted unless OTEL_LOG_USER_PROMPTS=1, and Bash commands and MCP tool names appear only with OTEL_LOG_TOOL_DETAILS=1 [8].
{
"env": {
"CLAUDE_CODE_ENABLE_TELEMETRY": "1",
"OTEL_LOGS_EXPORTER": "otlp",
"OTEL_EXPORTER_OTLP_PROTOCOL": "grpc",
"OTEL_EXPORTER_OTLP_ENDPOINT": "https://otel.example.internal:4317",
"OTEL_LOG_TOOL_DETAILS": "1"
}
}Prompt logs are sensitive data
A content log holds whatever users and retrieved documents put in front of the model: source code, customer records, credentials pasted by mistake. NIST's Generative AI Profile lists data privacy, the leakage or unauthorised use of personal and sensitive data, among the risks generative AI creates or worsens [9]. Logging everything for detection can therefore build the most sensitive dataset in the company. Practical rules:
- Always log metadata (who, which model, tokens, tools, decisions); log content only where you have a stated detection or investigation use for it.
- Store content logs in a separate destination with narrower access than general SIEM data, and set a retention period.
- Alert on changes to the logging configuration itself: on Bedrock that means
PutModelInvocationLoggingConfigurationandDeleteModelInvocationLoggingConfiguration[1]. - Remember that enabling Google Cloud Data Access logs can be high-volume and billable, so scope them to the AI projects that need them [4].
Try it on ThreatDeck
- Bedrock invocation volume spike See a hunt that counts all four invocation APIs, including Converse, against each principal's baseline.
- Vertex AI inference from a new principal Note the prerequisite: Data Access logs for aiplatform.googleapis.com must be on.
- Claude Code telemetry: Bash touching secrets A hunt that only works with OTEL_LOG_TOOL_DETAILS=1.
Self-check
4 questionsPick an answer to see whether it is right and why. Only your score is saved, in this browser.
-
Runtime calls are CloudTrail management events and are recorded by default with identity and network details. Request and response bodies only exist if invocation logging delivered them to CloudWatch Logs or S3.
-
Converse and ConverseStream appear as separate event names. Sysdig saw LLMjacking tooling adopt Converse within 30 days of its release, so volume detections must count all four runtime APIs.
-
GenerateContent and Predict are DATA_READ operations. Google Cloud disables Data Access audit logs by default (except some BigQuery services), so the events the hunt looks for are not being written.
-
Metadata and tool details give detection value with less exposure. Prompt content is the most sensitive part and should be a deliberate decision; repositories cannot set the exporter in Claude Code, which keeps a cloned project from redirecting telemetry.
References
- Monitor model invocation using CloudWatch Logs and Amazon S3. AWS (Amazon Bedrock User Guide).
- Monitor Amazon Bedrock API calls using CloudTrail. AWS (Amazon Bedrock User Guide).
- Monitor Azure OpenAI in Microsoft Foundry Models (classic). Microsoft Learn.
- Enable Data Access audit logs. Google Cloud (Cloud Logging documentation).
- Vertex AI audit logging. Google Cloud documentation.
- The Growing Dangers of LLMjacking: Evolving Tactics and Evading Sanctions. Sysdig Threat Research Team, 2024-09-18.
- Semantic conventions for generative AI (spans, metrics, events, MCP). OpenTelemetry.
- Monitoring (OpenTelemetry) for Claude Code. Anthropic (Claude Code docs).
- NIST AI 600-1: Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile. NIST, 2024-07.