AI summary attack conceals code that tampers with LLMs

Spectra Assure Free Trial Get your 14-day free trial of Spectra Assure for Software Supply Chain Security Get Free TrialMore about Spectra Assure Free TrialKey takeaways Researchers have demonstrated that AI summaries of email can be altered using hidden HTML code for launching an indirect prompt-injection attack. In a measured laboratory proof of concept, researchers at Forcepoint X-Labs isolated a single email summarizer running an unguarded LLM pipeline, embedded a hidden prompt-injection payload using common HTML concealment techniques, and confirmed that indirect prompt injection can silently hijack summarizer output without signaling tampering to the reader. The research shows that attackers can silently tamper with information that employees use to make decisions, said X-Labs security researcher Ben Gibney. The hidden payload then issues summaries that contain fabricated data and omit facts, yet they seem legitimate, he said. The research confirms known design failures, and its rigorous measurement proves that the attack succeeds reliably in production conditions, Gibney said. Security teams need to recognize that email summarizers reading unverified content are attack vectors, he said — one with the potential for drastic consequences. “The real business impact emerges when AI gains agentic capabilities. A summarizer with permission to send emails or approve requests becomes a much higher-severity target.” —Ben Gibney Here’s what you need to know about this AI summary of email-based attack vector — and why threat intelligence is key to getting out in front of it. [ See webinar: Threat Intel 201: The Intelligence Foundation | Get guide: Best Enterprise File Scanning Tools of 2026 ] When mounting the attack, the researchers sent an email containing an indirect prompt-injection payload, but because it was styled with an HTML tag calling for a font size of 0 pixels, white text, and 0 line height, it was invisible to humans though machine-readable. The instructions told the email summarization service to ignore the message’s legitimate text, or authoritative record, and summarize the hidden text instead. It also instructed the summarizer to mention neither the instructions nor the legitimate text. Forcepoint noted that nearly half of what it passed to the summarizer was hidden from the user: 1,009 characters sent to the model, 537 displayed, and 472 hidden injection text. The only cue that might be noticed was extra whitespace between the last line and the sign-off, where the injection text sits between two tags, and not a consequence of the injection itself. What’s important, the researchers said, is that the resulting email summaries carried information only from the injected text. The reader of the summary saw no indication that the information was fabricated or that the email contained contradicting instructions. The stealthy part of the injection was a success, Forcepoint said. None of the summaries mentioned the notice to the summarizer, the superseded draft, or the authoritative record. “This finding matters because it turns a trusted helper into a quiet liar,” said Waseem Ahmed, head of engineering at Secure.com. “The summary reads clean while the numbers and dates are fabricated, and the reader gets no warning. It does not create a new risk. It confirms a known design flaw that teams keep shipping anyway.” —Waseem Ahmed The advantage of this approach is that an adversary does not need to figure out how to submit the prompt to the target LLM, said Scott Miserendino, CTO at DataBee. “A service such as your email system is going to automatically send content to the LLM. Indirect prompts just need to be part of that content. Direct access to LLMs is usually heavily restricted behind network protections, as well as user accounts paying for the LLM’s use.” —Scott Miserendino Steve Wilson, founder and co-chair of the OWASP GenAI Security Project, praised X-Labs’ work as “a good educational demonstration.” “It is simple and reproducible, and it shows clearly how an attacker can manipulate the integrity of an AI-generated summary without the human recipient realizing that the source email contained hidden instructions.” —Steve Wilson Wilson pointed out, however, that indirect prompt injection has been a known problem almost from the beginning of modern LLM application security. “I led the OWASP Top 10 for LLM Applications project when we released the original list in 2023 and continue to lead the project today. Prompt iInjection was LLM01 in that original list, and we explicitly distinguished direct prompt injection from indirect prompt injection, where malicious instructions arrive through external data such as websites, documents, and emails that the LLM is asked to process,” he said. Wilson noted that in 2024, Johann Rehberger demonstrated indirect prompt-injection attacks against Microsoft 365 Copilot using attacker-controlled email and documents, including techniques for hiding instructions from the human reader. “Microsoft itself was using almost exactly this email-summarizer scenario when it introduced Prompt Shields for indirect prompt injection in 2024,” he added. “Similar attacks have subsequently been demonstrated against other assistants.” That does not mean the risk is acceptable, Wilson continued. “In many enterprise scenarios, indirect prompt injection is actually more concerning than direct prompt injection because the attacker does not need access to the AI interface at all,” he said. “If that AI only produces a summary, the impact may be false information, omitted facts, or manipulated decisions. If the AI has access to sensitive data or tools, the consequences can become much more serious, including data disclosure or unwanted actions performed with the user’s privileges.” —Steve Wilson Wilson noted that evidence of broad exploitation in the wild is still limited. There are signs that attackers are experimenting with indirect prompt injection and that tools for generating these attacks are being discussed and sold in underground communities, he said, “but I have not seen evidence that this has become a routine, large-scale email attack technique yet.” “That distinction matters. This is a very real vulnerability class, but that is not the same thing as saying it is already commonplace in production attacks.” —Steve Wilson He recommended that security teams avoid treating every prompt-injection problem as if it requires solving prompt injection itself. “Detecting arbitrary prompt injection is hard,” he observed. “Detecting evidence that someone is deliberately hiding text from a human reader is much easier.” Zero-size fonts, white-on-white text, off-screen elements, suspicious CSS, encoded content, and large differences between what a human sees and what a machine receives are all things security controls can look for before the content ever reaches an LLM, he said. That gives defenders an important advantage, he asserted. “You do not necessarily have to answer the much harder question, ‘Is this sentence secretly a malicious prompt?’ if you can first ask, ‘Why does this email contain a large amount of content that the recipient cannot see?’” John Bambenek, president of Bambenek Consulting, said that while the Forcepoint indirect prompt injection isn’t the most significant one out there, it does reinforce that indirect prompt injection does happen in a wide variety of applications. “People have always been lazy, and AI supersizes the risk of that laziness. An email summarizer gives an opportunity for a user to take some action based on the summary and not the content of a message, so the social engineering possibilities are unlimited.” —John Bambenek Indirect prompt attacks turn “garbage in/garbage out” into “garbage in/gospel out,” said John Strand, the principal at Black Hills Information Security. “If we can inject garbage into a data stream and then that garbage gets picked up as gospel and acted on by other AI agents or even other people, that’s a new way of looking at an old adage in computer security.” —John Strand Because attacks using these methods keep evolving, Erich Kron, CISO advisor at KnowBe4, recommends a “trust nothing” mindset when accepting input from an outside source, whether websites or digital messaging. “Any HTML should be checked and scrubbed prior to reaching a user’s inbox, and we must closely monitor the actions of agents in order to detect anomalies that are potentially covered up by their own summaries.” —Erich Kron Prompt injection should be treated as an authorization problem, said Noelle Murata, a senior security engineer at Xcape. “If we can treat this issue as a representation of a problem that security has already solved, we can put proper guardrails and architectural controls in place and hopefully make the internet a slightly safer place.” —Noelle Murata The attack vector described by Forcepoint is something old made new by AI, said Jacob Krell, senior director for secure AI solutions and cybersecurity at Suzu Labs. “Attackers don’t't need new tradecraft, as techniques originally designed to hide content from people or confuse scanners can now carry instructions for the AI processing the message.” —Jacob Krell The email summarizer is one instance of a broader condition. Files, attachments, packages, and documents arriving at an organization are no longer read primarily by people; they are parsed in full by systems that see everything the sender included, including what was deliberately hidden from human view. That breaks the assumption underneath most trust decisions, which is that a known sender, a valid signature, or a familiar file format tells you what a file will do. As KnowBe4's Kron noted in his "trust nothing" framing, the artifact itself now has to be inspected before it reaches a model, an agent, or an inbox — and inspection is only as good as the intelligence behind it. This is where efficacy, rather than volume, separates useful threat intelligence from noise. Defenders need decisive classifications across the file types and archive formats actually in circulation, depth to reach content nested several layers down, and a low enough false-positive rate that verdicts can be acted on automatically. Stale indicators carry a higher cost in AI-driven workflows than they did in human ones, because a wrong or ambiguous verdict propagates into downstream decisions at machine speed with no one pausing to sanity-check it. High-efficacy threat intelligence, built on a large corpus of both goodware and malware and capable of reclassifying files previously judged clean, is what lets teams answer the question Wilson posed about hidden content before that content ever shapes an output. See our new guide: Best Enterprise File Scanning Tools of 2026. ExtraHop's Kanaiya Vasani unpacks the concept of the agentic SOC — and how the Agentic SOC Alliance is working to build them out. Organizations don’t realize how pervasive shadow AI has become. And as AI's capability grows, shadow use is harder to manage.