The future of AI security research isn’t autonomous, it’s human

HTTP Terminator generated, evaluated, and weaponized thousands of HTTP desync hypotheses, and surfaced a new technique that neither the AI or its human operator would have found alone. Meet HTTP Terminator, a new AI system that has identified hundreds of websites vulnerable to HTTP request smuggling, hacked them live at scale, and even identified a “genuinely new class” of vulnerability, dubbed “shared-parser confusion.” But it didn’t do it alone; it was guided by a human the entire time, which may be the most interesting finding of all. A researcher from security company PortSwigger used his own processes to design and build the AI, HTTP Terminator, posed narrow, high-value questions, ruled out weak answers, applied anomaly-detection logic, used deterministic code to restrict agent behavior, and applied findings to subsequent ‘cascade’ research. “This inverts the accepted narrative by showing an expert can be a massive amplifier for an AI research system,” James Kettle, PortSwigger’s director of research, explained in a white paper. “A human in the loop can still add significant value, as opposed to just building the loop, then stepping back.” Disrupting HTTP request flow HTTP desync attacks, also known as HTTP request smuggling, interfere with the way web sites process HTTP requests from various users. In this architecture, users send requests to a front-end server, which then forwards them to back-end servers. The issue is in the inherently “weak” isolation of requests, Kettle pointed out: The front-end server typically sends several requests at once over the same back-end network connection to improve performance and efficiency. “HTTP requests are sent one after another, and the receiving server has to determine where one request ends and the next one begins,” he explained. But front-end and back-end systems must “agree” on where each request ends, otherwise, attackers can send ambiguous messages that are interpreted as two requests by the back end. Through response query poisoning (RQP), those attackers can inject false information into subsequent requests or otherwise manipulate data flows so that systems lose track of which responses came from where. They can then intercept responses intended for other users, which can sometimes include sensitive information, credentials, or API keys. While request smuggling is typically associated with HTTP/1 requests, websites that support HTTP/2 might also be vulnerable, depending on their back-end architecture, Kettle noted. “Request smuggling vulnerabilities are often critical in nature, allowing an attacker to bypass security controls, gain unauthorized access to sensitive data, and directly compromise other application users,” he said. HTTP Terminator’s methodology Kettle built HTTP Terminator around his own research methodologies. The initial phase is ideation: It autonomously generates hypothetical testable RQP attacks, such as, for instance, desync triggers, patterns, or weaponization techniques. The next step is evaluation: hypotheses are tested at large scale to identify which actually worked. HTTP Terminator used live websites that allow security testing via bug-bounty and Vulnerability Disclosure Programs (VDPs). A built-in anomaly detection layer flagged unusual responses. Following that is a weaponization phase, which determines whether findings actually have real-life impact. In some cases, they did: HTTP Terminator initially generated 30,000 unique attack vectors that it tested continuously, soon identifying 700 vulnerable targets, including financial institutions and government infrastructure. The system even stole a live API key from one of these banks, and other breaches were traced back to flaws in popular enterprise products like Apache Traffic Server, Citrix NetScaler, and BeyondTrust. Finally, there’s a step Kettle calls ‘cascade.’ Each finding may be a clue to an overlooked target or detection path. “When you discover something, if you explore back up the tree you may find other undiscovered branches,” he explained. For instance: How could you detect similar behavior elsewhere? And could the behavior enable other types of attacks? Essentially, each finding becomes “the seed for the next.” “That might not look like much, but it creates a positive feedback loop which can spiral into a cascade of discoveries taking you beyond predictable findings, into the unknown,” Kettle said. Ultimately, “I realized that autonomous vs human is the wrong framing,” he explained. “It’s better to frame system design as AI vs code vs human.” A new tool for security teams HTTP Terminator was able to invent and prove several novel desync triggers, one novel desync pattern, and a desync weaponization technique. Notably, though, it identified a new “shared-parser confusion” technique that allows attackers to parse requests and responses and thus increase their attack surface. “That, by itself, is absolutely huge,” Kettle said. But, he emphasized, “this discovery was not fully autonomous — the HTTP Terminator proposed it, and I validated it. Neither of us would have discovered it alone.” Kettle has open-sourced HTTP Terminator along with his research blueprint so other security teams can “turn their own methodology and instincts into an autonomous research weapon.” The researcher advised tackling tasks in four steps: Set an objective, create an evaluation strategy, establish inspiration sources, and explore cascade routes for new discoveries. “Evaluation is the first concrete step for both design and implementation,” he said, “because any issues there will derail the entire project.” It’s also important to “aggressively” identify and resolve data quality issues, as those will be difficult to correct later on. Further, ask high-value questions without being too broad; review outputs in initial test runs and use them to rule out low-value hypotheses; and remember that “every extra sentence of prompt risks context-contamination,” Kettle said. When a system is fully AI-driven and also reliant on AI-built code, it’s near-impossible for it to improve over time, he said. It’s better to start quickly with an AI-heavy approach, then move responsibility to deterministic code for speed and accuracy. “Fully autonomous research is real,” Kettle acknowledged. However, “humans are a massive power amplifier for AI research systems.”