threatdeck
Live 14:45 UTC ·

New Technique Medium TD-2026-0031 For AppSec, AI Eng, CISO AI-assisted

CASE-Bench: repository context changes security of LLM-generated code (145 CVE tasks)

An arXiv preprint introduces CASE-Bench, 145 repository-level tasks built from real CVEs across 51 projects, each with functional tests and a validated proof-of-concept exploit. The authors tested 13 context construction methods and six controls with six LLMs. Functionality-oriented context raised secure pass rates by at most 3.7 pp, and withholding a safeguard lowered them by up to 8.1 pp.

Why it matters

Coding assistants and agents build repository context mostly for functional correctness, so they may miss existing authorization checks and copy insecure code. Organizations using these tools on security-sensitive code may ship vulnerable generated code that passes functional tests.

Do this

  1. Review how your coding assistants and agents select repository context, and confirm that files enforcing authorization and input validation are included for security-sensitive changes.
  2. Do not rely on functional test pass rates alone for AI-generated code; add exploit-based or security regression tests for changed endpoints.
  3. Audit existing repositories for insecure code patterns that could be imitated by AI tools, and fix or annotate them before they are used as context.
  4. Add repository-level secure-code evaluations to AI coding tool assessments, using tasks that include the project's own safeguards.

Signals

  • NoCISA KEV
  • 23/100Priority
  • 60%Confidence

Evidence

“For security-sensitive code, this context decides whether the model sees the project's safeguards (e.g., authorization checks) or insecure code to imitate.”

arXiv (AI security papers)

“functionality-oriented context raises the functional pass rate by up to 8.2 percentage points (pp) but the secure pass rate by at most 3.7 pp”

arXiv (AI security papers)

“withholding a safeguard lowers the secure pass rate by up to 8.1 pp”

arXiv (AI security papers)

“coding agents observe the relevant safeguard in only 27-36% of trajectories”

arXiv (AI security papers)

Sources

Affected / related

CASE-Bench, repository context construction for LLM coding assistants

Published 2026-11-04 10:29 UTC · updated 2026-11-08 17:52 UTC · AI-assisted summary, checked against quoted sources · How we score

Following

Follow the vendors, products, AI tools, techniques and actors you care about. Matching stories, CVEs and exploited bugs are marked, a Following tab appears in the feed, and returning visits show what changed. Saved only in this browser.

Presets

Get updates

A personal RSS feed of new stories on everything you follow. Paste it into any feed reader, Slack or Teams RSS app.