New Technique Medium TD-2026-0122 For AppSec, AI Eng, SOC / IR AI-assisted
Three frontier AI models flag mostly different code flaws in same open source audit
Echo tested Claude Mythos 5.1, GPT-5.6 Sol and GLM-5.3 on the same open source code, two Maven libraries and one npm package. Of 121 distinct potential issues, only one was flagged by all three models, and 100 came from a single model. Echo said the 42 findings that passed automated proof-of-concept testing were not necessarily distinct bugs.
Why it matters
Single-model vulnerability scanning may miss issues that other models find, so organizations relying on one AI tool may have blind spots. Echo warned that an attacker and a defender using different models could see different sets of vulnerabilities.
Do this
- Do not rely on a single AI model for code vulnerability review; cross-check findings with at least one other tool or method.
- Treat AI-reported severity ratings as inconsistent and triage findings with human review, since the models rated the same npm flaw Critical, High and as documented behavior.
- Confirm findings with project maintainers or established validation before acting, since automated validation is not maintainer confirmation.
- Track Echo's follow-up disclosures for the affected Maven and npm packages and review your dependency usage once details are published.
Signals
- NoCISA KEV
- 24/100Priority
- 70%Confidence
Evidence
“Of the 121 issues, 83% (100), came from a single model and 20 from the other two.”
“the defender might never even see the vulnerability the attacker is about to exploit.”
Sources
Affected / related
Claude Mythos 5.1, GPT-5.6 Sol, GLM-5.3, Echo, Anthropic, OpenAI, Z.ai
Published 2026-11-18 10:00 UTC · updated 2026-11-18 10:05 UTC · AI-assisted summary, checked against quoted sources · How we score