threatdeck
Live 13:45 UTC ·

Learn / Detecting and responding to AI abuse

Lesson 1 of 69 minReviewed

LLMjacking: stolen cloud credentials used against hosted models

Follow an LLMjacking operation from leaked key to resold access, and know which API calls give it away at each step.

LLMjacking is the name Sysdig's threat research team gave in May 2024 to the use of stolen cloud credentials to run hosted language models at the victim's expense [1]. In that first case the credentials came from a system running a Laravel version vulnerable to CVE-2021-3129; the attackers then tested the keys against ten AI services, including AWS Bedrock, Azure, Google Vertex AI, OpenAI and Anthropic, with the apparent aim of reselling access [1]. MITRE ATLAS tracks the behaviour as Cost Harvesting (AML.T0034) [6], and OWASP files the cost side under Unbounded Consumption (LLM10), which names Denial of Wallet as a risk of pay-per-use models [5].

LLMjacking: attacker steps and where each one shows up in logs Five attacker steps from left to right: a cloud credential is exposed, the key is validated with an invalid model request, the invocation logging configuration is read and then deleted, models are enumerated and enabled, and access is resold through a reverse proxy while the victim pays. Under each step is the detection opportunity: secret-scanning alerts and use from a new network, bursts of validation errors across models, calls to the logging configuration APIs, console-only model-access APIs called with a long-term key, and an invocation spike or one source address using many keys. Attacker step Detection opportunity Key exposed public repo, app flaw, scraped config Key validated invalid request; error type = access Logging checked read, then delete invocation logging Models enabled enumerate, submit use case, entitle Access resold reverse proxy, many keys; victim pays secret scanning; use from new network validation errors across models Get / Delete logging configuration calls console-only APIs from an AKIA key invocation spike; one IP, many keys
Every stage before the resale is a cloud control-plane call, so it is visible in audit logs even when prompt logging is off.

How an operation runs

  1. Obtain a key. Exposed keys are enough. Permiso published an AWS key on GitHub as a honeypot on 25 June 2024 and saw attempts against Bedrock within minutes [3]. Microsoft says the Storm-2139 network used customer credentials scraped from public sources [4].
  2. Validate quietly. Sysdig saw InvokeModel requests with max_tokens_to_sample set to -1. A ValidationException proves the key reaches the model without paying for a completion; AccessDenied shows it does not [1].
  3. Check, then remove, logging. Attackers called GetModelInvocationLoggingConfiguration [1], and by September 2024 Sysdig also saw DeleteModelInvocationLoggingConfiguration [2]. Permiso notes that the oai-reverse-proxy tool is configured by default not to invoke models when invocation logging is on [3].
  4. Enable models. ListFoundationModels, GetFoundationModelAvailability, PutUseCaseForModelAccess and PutFoundationModelEntitlement enumerate and switch on models the victim never used [2]. Permiso points out that several of these are normally driven by the console, so a long-term access key calling them stands out [3].
  5. Monetise. Keys sit behind reverse proxies and are sold or shared. Sysdig found that about 95% of the prompts it analysed were role-play, mostly adult-oriented, and described a Russian national reaching Claude on Bedrock with stolen credentials, a sanctions-evasion motive [2]. Permiso logged more than 75,000 successful invocations in two days, almost all of them sexual role-play [3].

Storm-2139 shows the same economy at larger scale: creators built the tools, providers modified and sold them, and users generated harmful imagery, including non-consensual intimate images of celebrities. Microsoft named four defendants in an amended complaint and obtained a court order to seize a website central to the operation [4].

Why it matters beyond the bill

Cost is the visible harm. Sysdig estimated more than $46,000 per day for a Claude 2.x quota across four regions in May 2024, and more than $100,000 per day for the most capable models by September [1, 2]. The less visible harm is content: your account becomes the origin of material you would not want attributed to it, and if invocation logging was never enabled, or was deleted, you cannot show what was generated [2, 3].

Questions to ask of your own accounts

  • Is model invocation logging enabled in every region where models are reachable, and does a change to it raise an alert?
  • Which long-term access keys can call InvokeModel or Converse, and do any of them need to?
  • Are regions and model families you do not use blocked by policy?
  • Would you notice a principal with no model history suddenly invoking one?

The next lesson covers which logs record these steps, which are off by default, and what each one can and cannot tell you.

Try it on ThreatDeck

Self-check

4 questions

Pick an answer to see whether it is right and why. Only your score is saved, in this browser.

  1. Why did LLMjacking operators send InvokeModel requests with max_tokens_to_sample set to -1?
  2. Which call is the strongest sign that someone is preparing to abuse a Bedrock account without leaving a record of prompts?
  3. Permiso recommends alerting when a long-term access key (AKIA…) calls PutUseCaseForModelAccess or PutFoundationModelEntitlement. Why is that combination unusual?
  4. You find a Bedrock usage spike from an IAM user key that no application owns. What should come first?

References

  1. LLMjacking: Stolen Cloud Credentials Used in New AI Attack. Sysdig Threat Research Team, 2024-05-06.
  2. The Growing Dangers of LLMjacking: Evolving Tactics and Evading Sanctions. Sysdig Threat Research Team, 2024-09-18.
  3. Hijacking AI infrastructure with non-human identities like access tokens. Permiso Security, 2024-10-03.
  4. Disrupting a global cybercrime network abusing generative AI. Microsoft Digital Crimes Unit, 2025-02-27.
  5. LLM10:2025 Unbounded Consumption. OWASP GenAI Security Project, 2024-11.
  6. MITRE ATLAS (Adversarial Threat Landscape for AI Systems). MITRE.
Completing the self-check also marks the lesson complete. Saved in this browser only.

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.