AI Model Risk Intelligence Know Which Models You Can Trust Before You Deploy

AI Model Risk Intelligence Know Which Models You Can Trust Before You Deploy August 4, 2026 0 mins readWhen we started thinking about how to surface AI model risk inside Evo, the obvious answer was to borrow from how we score everything else: find the issue, assign a severity, surface it. Done. The core of the new approach is a real risk score, built the way security teams already reason about risk: Likelihood × Impact. Likelihood comes from Attack Success Rate (ASR), the share of real adversarial attacks that succeed against a model. Impact is how much damage the attacker's goal does when it lands. We combine the two into a single score from 0 to 1000 (lower is better). Because the two are multiplied, neither side wins on its own: a rare-but-catastrophic attack gets discounted by its low likelihood, and a common-but-harmless one by its low impact. What rises to the top is what matters most: the attacks that both succeed often and cause real damage when they do. New Risk Score methodology What makes it different from a checklist or a vendor safety card is that ASR is derived from real adversarial testing, such as extraction prompts, multi-turn escalation, persona-based jailbreaks, and tree-of-attack strategies, with model-based judges confirming whether each attack actually succeeded. These aren't theoretical, and they aren't "naked" model numbers, either. Every attack in the benchmark runs against a baseline system-prompt hardening defense, so the risk score reflects what still gets through a standard guardrail already in place: the attacks that work in production, not just in a lab. What makes the score actionable is that it's impact-oriented: it's built around what actually goes wrong when an attack lands, not just whether the attack succeeded. And it doesn't stop at a number. Evo breaks every score down to the specific attacker's goal behind it. PII extraction, system-prompt extraction via injection, and insecure code generation. So you can see exactly where a given model is weak and build the right guardrail for it. That granularity is the signal you need before you make a deployment decision. This lands immediately with security teams. They see a model's risk broken down to the specific attacker goals it fails against, not a single abstract 'safe / unsafe' verdict. They don’t need to ask whether the model is safe in general and can start asking whether it's safe for what they're about to build. That first instinct, to find the issue, assign a severity, and move on, had a serious problem, and we heard it directly from customers. Security teams would look at a finding that said a model had an “information disclosure” issue, and the first question was always the same: What does that actually mean, and what do I do about it? The label told them something was wrong. It didn’t tell them how wrong, in what context, against what kind of attack, or what to build to fix it. So we rebuilt it. Here’s what changed and why. The gap showed up repeatedly on customer calls. When we started showing concrete numbers: GPT-3.5 at a 397/1000 Unsafe Content score, GPT-4 at 317/1000 for Bias & Fairness. Teams stopped asking whether a model was safe and started comparing. The problem isn’t the score, but the context. Most AI risk evaluations treat a model like a static artifact. Run some tests, assign a category label, and and publish a safety card. Teams look at that and try to make a deployment decision. But AI risk doesn’t work that way. The same model carries fundamentally different risks depending on how it’s deployed. A coding agent faces credential theft, backdoors, attacker-controlled commands, and dependency poisoning. A customer chatbot faces jailbreaks, harmful content generation, and prompt extraction. A personal assistant, with access to email, calendar, and documents, faces a different threat surface entirely. A risk score that doesn’t reflect deployment context isn’t a risk score. It’s a guess. Evo bakes that deployment context into how it weights impact, so the score reflects how a model will actually be used, not how it behaves in a vacuum. There’s a second problem: indirect attacks. Most model evaluations focus on direct prompts: a user types something malicious, and we see whether the model resists. That matters, but it overlooks the more dangerous class of attacks targeting agentic systems: indirect prompt injection embeds malicious instructions in data the agent reads, such as a retrieved document, a tool result, or a code comment. The user never sees it. The agent processes it as context. If the model follows those instructions, it can leak sensitive data, misuse tools, or take unauthorized actions. Model-only testing never catches this. This isn’t a theoretical concern. One enterprise came to us with an urgent incident driven specifically by LLM model risk, and at a global bank,, a similar issue escalated to the CEO. A national lender wanted to gain visibility into MCP usage and inject guardrails. These teams weren’t asking about model safety in the abstract; they were asking about behavior in a specific agentic context. We wanted Risk Intelligence to answer a different question: how does this model behave under attack in the way you’re actually planning to deploy it? What we built: Impact-based risk, down to the attacker's goal What models and AI skills are in use across the codebase? Security teams need visibility into where AI is used, which models are called, and which skills or tools are part of the system. How does each model behave under attack? Teams need risk profiles based on adversarial testing, not static claims or generic safety documentation. How can we act on the findings? A score is only useful if it helps teams compare models, prioritize guardrails, and enforce policy across applications and repositories. The headline output is a single risk score from 0 to 1000, weighted by the real-world impact of what an attacker can achieve. Attack Success Rate is the measured signal behind it. The score tells you how exposed a model is. The taxonomy tells you which attacks drove it, and why they matter for your deployment. Evo organizes findings in a three-level taxonomy built around impact, what actually goes wrong. A top-level impact category (say, information disclosure) rolls down to a sub-category, and then to the specific attacker goal at the most granular level (say, PII extraction). Direct and indirect attacks are tracked as separate surfaces across the whole taxonomy, because they call for different defenses. A coding agent with high ASR for indirect injection via code comments needs a different guardrail than a chatbot with high ASR for jailbreaks. The taxonomy makes that distinction explicit. And every attacker's goal maps to the frameworks your team already reports against: OWASP LLM Top 10, OWASP Agentic, MITRE ATLAS, and NIST. So a finding isn't a Snyk proprietary label you have to translate; it drops straight into the standards you're already governed by. You'll see that mapping in the risk-profile tab, alongside the score. That specificity is what moves teams from a score to a remediation plan. And it’s what lets Evo connect risk findings directly to policy enforcement, not as a separate workflow, but as the same motion. Skills coverage, not just models High-level risk labels are not enough. Telling a team that a model has an “information disclosure” or “unsafe content” issue may raise awareness, but it does not tell them what to fix. Useful risk intelligence breaks each finding down to the specific attacker goal. For example: system prompt extraction through injection, unauthorized tool execution, harmful content generation, insecure code generation, or sensitive data exfiltration. That level of detail helps teams decide what guardrails to build, what policies to enforce, and whether a model is appropriate for a given use case. It also helps leaders understand risk at different levels. A high-level category can show where the organization has exposure. A more detailed attack surface can show whether attacks are direct or indirect. A specific attacker goal can tell engineering teams exactly what failed. Risk doesn’t live in models alone. It lives in the combination of a model, the tools it can invoke, and the skills it uses. A model that scores well in isolation may behave very differently when paired with a file system tool, a code execution environment, or a sensitive data store. Evo maps skills coverage alongside model risk, giving teams a clear picture of which skills are in use, which models they’re attached to, and what the combined attack surface looks like. That’s the inventory view security teams need before they can make deployment decisions with confidence. This is especially relevant for coding agents, where the same model may handle pull request reviews one moment and execute deployment commands the next. The risk profile changes significantly based on what the model is doing and what tools it has access to when it does it. From evidence to enforcement Evo is designed to help teams understand how models behave under attack before they are trusted in production. When Evo scans a codebase, it identifies AI models and skills in use and enriches them with risk profiles built from adversarial testing. These profiles are built from Attack Success Rate, measured against real adversarial attacks and weighted by impact, so the score reflects consequences, not just attack counts. Risk Intelligence also organizes findings through a granular taxonomy, from broad categories down to specific attacker goals. Each one is mapped to OWASP LLM Top 10, OWASP Agentic, MITRE ATLAS, and NIST. That means teams can move from a headline score to the exact attack pattern that succeeded, and line it up with the framework they already report against. Because Risk Intelligence connects to Evo’s policy engine, teams can turn insight into enforcement. They can compare models before committing, see which models carry the most risk in specific deployment contexts, start with out-of-the-box policies, and customize thresholds based on their risk tolerance. The gap between a risk score and an enforcement policy is where most AI security programs stall. Teams get a finding. They don’t know what to do with it. Evo closes that gap by connecting Risk Intelligence directly to its policy engine. When a model shows a high risk score for a specific attacker goal or attack pattern, teams can set thresholds, apply policies, and block high-risk models from being used in sensitive contexts all within the same workflow. For Snyk customers, this extends a familiar pattern: discover risk where developers work, prioritize what matters, and apply policy consistently. The difference is that the policy now extends to AI model selection and agent configuration, not just code. Why this matters now AI security cannot rely on guesswork. As models and agents enter real-world applications, teams need to understand how those systems behave under attack. AI model risk intelligence enables organizations to measure model behavior, compare risk by use case, prioritize remediation, and govern AI adoption at scale. The question is no longer simply, “Is this model safe?” The better question is, “How does this model behave under attack in the way we plan to deploy it?” Most teams we talk to are operating with AI models they didn’t explicitly choose. They inherited them from a vendor, adopted them through a developer tool, or discovered them mid-audit. Snyk’s new State of Agentic Adoption report Volume II, found anonymized AI-BOM telemetry from 3,044 organizations. The pattern is consistent: for every model a team knows about, roughly 2.8 more AI components, libraries, MCP servers, and SDKs run unmanaged, and most organizations can't produce a full inventory of what they have. When one team scoped the job, a complete AI inventory would have taken 4 to 5 weeks and 10 to 12 stakeholders. The model inventory problem is real before the risk scoring conversation even begins. Demand for this type of capability is growing fast. Enhanced AI-SPM capabilities are available now to existing customers. Impact-based risk scores built on Attack Success Rate from real adversarial testing will be available at the end of August 2026. Book a demo today to learn more. You can’t govern AI you can’t see Start with Discovery. Start with Evo AI-SPM. Uncover every AI component hidden in your codebase and apply organization-wide governance.