Learn / Detecting and responding to AI abuse
Lesson 1 of 69 minReviewed
LLMjacking: stolen cloud credentials used against hosted models
Follow an LLMjacking operation from leaked key to resold access, and know which API calls give it away at each step.
LLMjacking is the name Sysdig's threat research team gave in May 2024 to the use of stolen cloud credentials to run hosted language models at the victim's expense [1]. In that first case the credentials came from a system running a Laravel version vulnerable to CVE-2021-3129; the attackers then tested the keys against ten AI services, including AWS Bedrock, Azure, Google Vertex AI, OpenAI and Anthropic, with the apparent aim of reselling access [1]. MITRE ATLAS tracks the behaviour as Cost Harvesting (AML.T0034) [6], and OWASP files the cost side under Unbounded Consumption (LLM10), which names Denial of Wallet as a risk of pay-per-use models [5].
How an operation runs
- Obtain a key. Exposed keys are enough. Permiso published an AWS key on GitHub as a honeypot on 25 June 2024 and saw attempts against Bedrock within minutes [3]. Microsoft says the Storm-2139 network used customer credentials scraped from public sources [4].
- Validate quietly. Sysdig saw
InvokeModelrequests withmax_tokens_to_sampleset to -1. AValidationExceptionproves the key reaches the model without paying for a completion;AccessDeniedshows it does not [1]. - Check, then remove, logging. Attackers called
GetModelInvocationLoggingConfiguration[1], and by September 2024 Sysdig also sawDeleteModelInvocationLoggingConfiguration[2]. Permiso notes that the oai-reverse-proxy tool is configured by default not to invoke models when invocation logging is on [3]. - Enable models.
ListFoundationModels,GetFoundationModelAvailability,PutUseCaseForModelAccessandPutFoundationModelEntitlementenumerate and switch on models the victim never used [2]. Permiso points out that several of these are normally driven by the console, so a long-term access key calling them stands out [3]. - Monetise. Keys sit behind reverse proxies and are sold or shared. Sysdig found that about 95% of the prompts it analysed were role-play, mostly adult-oriented, and described a Russian national reaching Claude on Bedrock with stolen credentials, a sanctions-evasion motive [2]. Permiso logged more than 75,000 successful invocations in two days, almost all of them sexual role-play [3].
Storm-2139 shows the same economy at larger scale: creators built the tools, providers modified and sold them, and users generated harmful imagery, including non-consensual intimate images of celebrities. Microsoft named four defendants in an amended complaint and obtained a court order to seize a website central to the operation [4].
Why it matters beyond the bill
Cost is the visible harm. Sysdig estimated more than $46,000 per day for a Claude 2.x quota across four regions in May 2024, and more than $100,000 per day for the most capable models by September [1, 2]. The less visible harm is content: your account becomes the origin of material you would not want attributed to it, and if invocation logging was never enabled, or was deleted, you cannot show what was generated [2, 3].
Questions to ask of your own accounts
- Is model invocation logging enabled in every region where models are reachable, and does a change to it raise an alert?
- Which long-term access keys can call
InvokeModelorConverse, and do any of them need to? - Are regions and model families you do not use blocked by policy?
- Would you notice a principal with no model history suddenly invoking one?
The next lesson covers which logs record these steps, which are off by default, and what each one can and cannot tell you.
Try it on ThreatDeck
- Bedrock invocation logging deleted Alert on the call attackers use to stop prompt and response logging.
- Bedrock reconnaissance burst Hunt for principals that enumerate models and read the logging configuration in one hour.
- Azure OpenAI keys listed by a new caller The Azure equivalent: who started reading model account keys this week.
- Cost Harvesting (AML.T0034) See the ATLAS entry, related detections and recent stories.
Self-check
4 questionsPick an answer to see whether it is right and why. Only your score is saved, in this browser.
-
An invalid parameter returns a ValidationException when the key is authorised and AccessDenied when it is not. The attacker learns whether the key is usable while generating no tokens and little noise.
-
Deleting the invocation logging configuration stops prompts and responses being written to CloudWatch Logs and S3. Enumeration and validation errors also occur in normal development; deleting the only content record rarely does outside planned changes.
-
Enabling model access is an administrative task usually done in the console. A long-term key doing it from a script or HTTP library is what LLMjacking tooling looks like, and it is rare in legitimate use.
-
The key is a compromised credential. Contain it first, then scope what else it touched; waiting to watch the abuse lets the attacker keep spending and may hide other actions taken with the same identity.
References
- LLMjacking: Stolen Cloud Credentials Used in New AI Attack. Sysdig Threat Research Team, 2024-05-06.
- The Growing Dangers of LLMjacking: Evolving Tactics and Evading Sanctions. Sysdig Threat Research Team, 2024-09-18.
- Hijacking AI infrastructure with non-human identities like access tokens. Permiso Security, 2024-10-03.
- Disrupting a global cybercrime network abusing generative AI. Microsoft Digital Crimes Unit, 2025-02-27.
- LLM10:2025 Unbounded Consumption. OWASP GenAI Security Project, 2024-11.
- MITRE ATLAS (Adversarial Threat Landscape for AI Systems). MITRE.