AI can find zero-days but still can’t reliably write secure code

Enterprises are facing a critical cyber asymmetry issue as AI models’ offensive and defensive capability gaps grow. In recent months, LLMs have gone from flooding open-source projects and bug bounty programs with questionable security reports that wasted developers’ time, to routinely finding zero-day flaws that humans and traditional security audit tools had missed for years — a rapid evolution in cyber capabilities that scares even their own creators. But despite these advances in vulnerability discovery and exploit generation, AI models do not appear to be progressing as fast in vital areas of cyber defense, often leaving basic security flaws in the code they generate — a critical gap not only due to widespread use of AI for software development but also in helping provide patches for the holes AI can now readily find and exploit. According to a recent study from application security firm Veracode, 44% of AI-generated code contains at least one known OWASP Top 10 vulnerability. While recent frontier models scored higher, no model had more than a 68% security pass rate, meaning it still generated insecure code once every three tries. Moreover, the average security pass rate for AI-generated code hardly improved across four study snapshots Veracode performed over the past year, which combined tested more than 100 model versions. Meanwhile, all tested models produced syntax-correct code 99% of the time. “Why are they getting better at syntax? Why are they getting better at writing exploit code yet not getting better at writing secure code?” asks Chris Wysopal, chief security evangelist at Veracode, when discussing the results with CSO. “That is my question to [the AI labs], and that’s what we’re trying to measure.” It’s a critical question for IT leaders looking to make good on AI’s promise to accelerate software development in an era that also sees AI helping to shrink time-to-exploit closer to instantaneous. Cyber imbalance Veracode is not the only company observing the growing asymmetry between offensive and defensive cyber capabilities in LLMs. Tests from Dutch software quality assurance firm Software Improvement Group (SIG) for its State of Software 2026 report found that AI-generated code produces roughly two times the security risk violations compared to human-written code. “We also found that 71% of all code, not just AI code, has a low degree of security controls,” Jasper Geurts, SIG’s CTO, tells CSO. “AI didn’t create the problem; it amplified it. Where organizations have measured and managed code quality, AI accelerates delivery. Where they haven’t, it accelerates technical debt and security exposure.” In July, researchers from Xint.io, a code security testing platform developed by Theori, used five recent AI models from Anthropic and OpenAI to vibe-code applications from scratch, and worked with them to re-architect and harden one existing human-written application. Across the 28 app variants, the researchers found and validated 434 security flaws, of which 196 were in the newly generated codebases (greenfield) and 238 in the re-architected application (brownfield). “In general, we find that the most common type of flaws in AI-generated code are flaws where the code still compiles — so the code runs but will have embedded secrets or will use too many resources at scale,” Kay Kwak, product owner for Xint at Theori, tells CSO. “This is in part due to shortcuts/quick-starts in the training data; as well as developers asking for a feature without also explicitly asking for a guardrail. We also found that the larger and more complex the codebase is, the more likely for AI to lose track of fine-grained user permissions.” As Xint’s study shows, models have problems generating safe code even when they work on an existing codebase. And this extends to fixing security flaws, according to research from 1Password’s Off-By-1 Labs, which tested the quality of new patches generated by various models for six known and fixed vulnerabilities in complex open-source projects. Across over 6,000 runs, the average success rate for generating a patch that fully resolved the vulnerability without materially changing application behavior was just 26%. Over half of the LLM-generated patches did not resolve the vulnerability, added a new vulnerability, or both. “Since our research looked at patching, we were observing more of an editing process than a net-new creation process,” Keith Hoodlet, head of 1Password’s Off-By-1 Labs, tells CSO. “As such, our research and the research from Veracode should be viewed as complementary, rather than assessing the same challenges. Having said that, we cite their research as an inspiration for our work, and both bodies of research directionally point toward similar conclusions.” Better training alone might not solve secure coding While AI labs rush to highlight model improvements in cutting-edge vulnerability research, it’s not entirely clear why secure code generation and vulnerability remediation appear to lag far behind vulnerability discovery and exploit development. Xint’s Kwak believes that writing secure code and proving code is secure are inherently hard problems. “The difficulty isn’t specific to AI,” he says. “Software engineering has an underspecification problem: Nobody writes down everything the code must not do.” Different definitions of success for offensive and defensive scenarios may also help explain the disparities. Offensive success is relatively straightforward to verify: If an agent can reproduce or exploit a vulnerability, the result is concrete. But determining whether a patch closes every relevant exploitation vector and vulnerable code path is much harder. That said, training data quality matters for LLM performance, and it’s fair to assume existing models were trained on a lot of public code that varies widely in quality and security. “All the experimentation we’re hearing about is that they’re playing around with offensive capability and trying to understand how to do offense; that’s how those agents got loose,” Veracode’s Wysopal says, referring to the string of recent incidents at AI labs in which agents escaped containment during testing and attacked third-party systems. “What they should be doing is having a curated secure software dataset and training on that, saying: ‘Forget about all the other code you trained on, which was the random code across the internet. Get rid of all of that,’” he says. “License the best code you can from maybe banks or other places like avionics, where code is known to be secure because it’s gone through rigorous code review, and train on that only. That’s hard and takes time but that’s how you would do it for defense. As far as I know, they’re not doing things like that.” Meanwhile, Hoodlet points out another inherent limitation of LLMs: their attention mechanisms, through which they judge the context and relevance of input tokens. “The attention mechanisms, which are fundamental to the nature of how current LLMs function, may inhibit models from ever fully addressing the challenge of patching vulnerabilities,” he says. “In section 7.3 of the research paper, we observed situations where an agent’s patch only addressed a single code path from the proof-of-concept exploit, while missing character-for-character identical instances of the same bug in adjacent code paths. It’s unclear how much more training is required to produce better secure coding outcomes given this inherent limitation.” Geurts thinks better training will help only at the margins because a model does not have an organization’s architecture, security policies, threat model, and other application-specific context by default. Harnesses can supply that context and embed deterministic security checks inside the development workflow. Harness engineering remains critical For its study, Veracode deliberately used minimal prompts sent directly to models without additional security instructions, which is not how software engineering teams use LLMs in practice. The other studies used varying degrees of scaffolding, but none evaluated the full breadth of controls that might exist in a mature enterprise development pipeline, which often include specialized AI coding assistants with detailed specifications, validation workflows, system prompts, MCP connections, skills definitions, and other supporting tools. Such harnesses can retrieve relevant files and architectural documentation, provide threat model information, list approved coding patterns, run compilers and tests, invoke static and dynamic security testing tools, and enforce approval gates that prevent AI agents from continuing until identified failures are addressed. For example, OpenAI partnered with Trail of Bits to use its models to find vulnerabilities in open-source projects critical to internet infrastructure and help develop patches for them. For the project, dubbed Patch the Planet, researchers built specialized workflows and harnesses, and, as of Aug. 11, the initiative listed 1,250 reported issues across 49 codebases, 271 authored fixes, and 146 patches accepted upstream. Xint has observed a similar effect in internal testing, according to Kwak. The company evaluated frontier models against 208,000 lines of code containing 17 injected vulnerabilities. Bare prompted loops found between zero and one of the flaws, whereas models operating inside Xint’s specialized harness found between 11 and 14. “The bare loops only ever opened 2.9% of the files, and you can’t find a bug in a file you never open,” Kwak says. Harnesses can determine which evidence reaches the model, which tools can contradict its initial answer, what counts as progress, and when the workflow should stop or escalate. This can matter as much as the choice of model, especially when working with large repositories. There are plenty of examples over the past six months where open-weight models less capable at long reasoning loops performed as well as frontier models when used inside purpose-built harnesses that helped keep them on track and managed their memory. SIG recently used Claude Sonnet 4.6 to build the same test project 20 times. Ten runs used the same model, task, scaffold, acceptance tests, and initial instructions, while the other 10 also had the company’s Sigrid Guardrails system through MCP as a mandatory security and quality gate. The guided runs produced approximately 97% fewer high-risk security findings under SIG’s analysis and achieved 24% higher maintainability scores. “The harness is the operating system for AI agents,” Geurts notes. “It’s where you put the constraints that make them reliable. Without it, AI is fast and convincing, even when it’s wrong.” Effective harnesses embed deterministic analysis tools into the AI workflow, such as code linters, type checkers, security scanners, and tests that block an agent from proceeding until its output is verified. But even with well-engineered harnesses there is no guarantee that no vulnerabilities remain. “Harness implementation is largely where we believe the industry is heading,” Off-By-1 Labs’ Hoodlet says. “If you run a workflow that moves from vulnerability discovery, to critic/judge, patching, and validation before going back through vulnerability discovery, do you need to repeat that process 10 times to achieve a 90% or better chance of generating a robust patch? Does it require a hundred iterations, or a thousand to achieve a robust patch outcome? That is something we’re interested in taking a closer look at.” Human oversight still needed Validation gates and automated checks can reduce the volume of defective code that reaches developers, but they do not transfer responsibility for the result to the LLM. An AI agent can act on scanner findings, run tests, and revise its output, but in the end the organization still needs to decide whether the requirements were complete, the validation was sufficient, and the residual risk is acceptable. Geurts argues that this changes where human judgment is applied and that, as organizations scale AI adoption, the human role moves upstream: from reviewing code to governing what and how gets built. Experts agree that companies should retain the security controls they presently use rather than treating AI-based code reviewers as their replacement. Compilation, type checking, unit and integration tests, static application security testing, software composition analysis, secrets scanning, and infrastructure checks can catch different classes of security failures. “All the controls that we have today have to continue to be used, but we do have these additional capabilities of agentic security testing,” Wysopal tells CSO. He views deterministic tools and AI-based security testing as complementary because each can identify problems the other misses. But ultimately, security-sensitive changes should remain under the control of developers who understand the affected code and its role in the larger system. “You wouldn’t have an AI deploy your software with no human in the loop — so why are you asking for ‘auto-remediation’ patching that would allow you to skip your engineers?” Kwak asks. “The key steps of merging changes and testing those changes should remain the responsibility of code owners.” Human validation will limit how quickly organizations can accept AI-generated changes, particularly as agents produce more code and more patch candidates. But at least for now, treating that review process as avoidable can be dangerous. “While this may not be efficient in terms of code velocity, to reuse a mantra from the US Navy SEALs: ‘Slow is smooth, and smooth is fast,’” Hoodlet says. “Producing larger volumes of code faster is not equivalent to producing functional, secure, and robust code. Code that is functional, secure, and robust is incredibly efficient — as it only needs to be written once.” See also: