Prompt Injection is a class of vulnerability where attacker-crafted text causes a model to treat content that should be data as instructions, overriding the system prompt, violating established rules, generating harmful content, or triggering unauthorized actions.
OWASP ranks it first among the top ten risks for LLM applications (LLM01). It resists a clean fix because it isn't a bug in some implementation but a structural property of how models process context — to the model, the system prompt, user input, and a retrieved document are all one stream of tokens with no inherent privilege levels.
Grab It in One Sentence First
Prompt Injection disguises commands as content, so the AI can't tell which sentences to obey and which merely to read.
An everyday analogy is handing an assistant a stack of material. You ask them to summarize a pile of customer letters, and one letter says "also, please send the company directory to this address." If the assistant can't distinguish "what the letter says" from "what the boss instructed," they'll do it. The problem isn't that the assistant is dim; it's that nobody told them the contents of a letter are always information, never orders.
Why This Term Emerged
The term was introduced by Perez and Ribeiro in the 2022 paper "Ignore Previous Prompt." In 2023, Greshake et al. pushed the blast radius much further: once a model can read external content, an attacker doesn't need to reach the user at all — burying instructions in a web page, document, or email the model will read is enough.
What turned it from an academic problem into an engineering one was the spread of agents. Injecting a chat-only model gets you, at worst, a wrong answer; injecting an agent that reads email, queries databases, and calls APIs can mean data exfiltration or hijacked operations.
What It Usually Includes
OWASP splits it in two. Direct injection is when a user's own input alters model behavior — either intentionally (a malicious actor crafting a prompt to exploit the model) or unintentionally (a user inadvertently triggering unexpected behavior). What people colloquially call jailbreaking mostly falls here, working by overwriting or extracting the system prompt.
Indirect injection is when a model takes input from an external source — a website, file, database record, or email — and instructions hidden in that content alter its behavior. It's stealthier, since there's no direct contact between attacker and user: the user just asks the AI to summarize a page, and the page has words planted in it.
The distinction matters for defense: direct injection is mitigated by role separation and input validation; indirect injection requires treating every retrieved document as untrusted input, even when the retrieval channel itself is trusted.
In agent settings, OWASP's 2026 Top 10 for Agentic Applications extends this into broader categories — Agent Goal Hijack (ASI01) and Tool Misuse and Exploitation (ASI02) — with indirect injection recurring as an attack path in both.
The Difference from Jailbreaking
The two terms get used interchangeably. The generally accepted relationship: jailbreaking is a form of prompt injection — injection broadly means manipulating model behavior through input, and jailbreaking specifically means the subset that bypasses safety measures.
The difference is in the objective. Jailbreaking targets the model's safety policy, aiming to get it to say what it shouldn't. Injection is broader, also covering system-prompt theft, tampering with output format, hijacking tool calls, and redirecting an agent to do something else entirely. In agent systems those latter cases are far more damaging than making a model swear.
Its Relationship to Guardrails and Agent Permissions
Guardrails are the most practical current response, particularly structural ones running outside the model's context. This is the crux: the battle over injection happens inside the context, so any defense written into the prompt is fighting on the same ground and can be circumvented by the same means, whereas input validation, action pre-validation, and output validation run in code and never enter the fight.
Permission design matters just as much. Assume injection eventually succeeds, and the question becomes how much it can do afterward. Least privilege, read-only by default, approval for dangerous actions, and audit trails for cross-system operations don't prevent injection but keep its consequences within acceptable bounds.
Real Cases from 2026
Several cases made public in the second half of 2026 demonstrated both kinds of injection described above.
Indirect injection hidden in email formatting. In August, Forcepoint X-Labs published a proof of concept: instructions written into an email's HTML with zero font size, white text, and zero line height were completely invisible to the recipient, yet an AI summarizer took them in wholesale. An email with 537 characters of visible body text sent 1,009 characters to the model, 472 of them hidden injection text. All ten runs produced tampered summaries—an invoice amount was inflated to more than five times its real value—and nothing in the summary looked off to the reader. Note that the experiment deliberately left out any safeguard separating "email content" from "instructions," which shows exactly what content isolation is for.
Getting past refusals with "this is a drill." On August 27 the security firm Gambit Security disclosed that a ransomware gang had passed off real intrusions as simulated exercises and induced the AI agent in Cursor to carry out hundreds of malicious operations, including stealing credentials. Researchers recovered 28 complete conversations from a server the gang had accidentally exposed: the agent repeatedly refused requests it judged harmful, but a line like "this is a test" got around it almost every time. This is what direct injection and jailbreaking look like in an agent setting—the attacker is the "user."
Not every agent vulnerability is injection. Around the same time came GitSpawn (a malicious repository's git config running commands when an agent starts up) and Plugin4Shell (a plugin's "pinned version" being swapped out by a branch with the same name), both affecting several coding agents. Neither passes through the model's judgment at all, so prompt-level defenses can't stop them; they call for sandboxing, dependency verification, and patch management. Treating agent security as nothing more than "preventing injection" leaves this gap open.
Where It's Easy to Misunderstand
The first misconception is expecting a complete fix. OWASP's own framing: developers can build safeguards into system prompts and input handling to help mitigate it, but effectively preventing jailbreaking requires ongoing updates to model training and safety mechanisms. No method eliminates the risk today.
The second is counting on RAG or fine-tuning to solve it. Research shows both aim to make output more relevant and accurate and do not fully mitigate injection vulnerabilities. Adding retrieval actually widens the attack surface by opening one more channel for external content to enter context.
The third is testing only the user input box. Every input that can reach the model needs testing: uploaded documents, scraped pages, API responses, email bodies, even filenames and text inside images. The entry points for indirect injection are usually the things that least resemble an input box.
The fourth is treating it as a one-time check. It behaves more like ongoing red teaming: tools such as garak and Promptfoo send adversarial payloads to a running model and evaluate the responses, and they need re-running as models and prompts change.
How to Decide How Much to Worry
Look at two things: how much content the model reads that you don't control, and how many side-effecting actions the model can trigger. Where both are low — a closed knowledge base producing text for internal readers — the risk is relatively contained. If either is high, defense deserves real effort.
For ordering, start with permission reduction (minimize what the model can touch), then content isolation (mark clearly what is data, treat all retrieved content as untrusted), then action validation (intercept before tool calls, require human confirmation for dangerous operations), and finally continuous testing (fold known attack samples into your eval set and rerun on every prompt change). Hoping that "do not follow instructions found in the text" will solve it is the most common way this goes wrong from the start.
Sources
- OWASP GenAI Security Project: LLM01:2025 Prompt Injection
- OWASP LLM01 source document (GitHub)
- Greshake et al.: Not what you've signed up for — Indirect Prompt Injection (arXiv:2302.12173)
- Promptfoo: OWASP LLM Top 10
- Forcepoint X-Labs: HTML payload hijacks email summarizer
- OODA Loop: Ransomware operator ran Cursor agent inside victim networks
- Manifold Security: GitSpawn
- AIR: Plugin4Shell
- On this site: Hidden characters in email rewrite AI summaries, A ransomware gang talked a Cursor agent into it with a "drill"