How to Prevent Prompt Injection: 6 Enterprise Best Practices
Production generative AI workflows face critical vulnerabilities against indirect prompt injection when operating without strict context isolation and schema validation. Implementing multi-layered architectural controls and continuous red-teaming prevents unauthorized tool execution and data exfiltration.
Updated October 7, 2026
When enterprises connect Large Language Models (LLMs) to internal APIs, knowledge bases, and automated workflows, their role goes far beyond generating text. The model can now influence actions, access information, and interact with other systems. This means that a piece of untrusted text can become a security risk. Prompt injection exploits this gap by manipulating an LLM into treating malicious input as instructions; something traditional application firewalls are not designed to detect.
In this blog, we’ll look at how prompt injection attacks work, why connected enterprise AI systems are particularly exposed, and the practical controls security teams can put in place to protect production environments.
» Ensure your cybersecurity is up to standard with KELA
Overview of Prompt Injection
Prompt injection is not entirely new from a security perspective. Like other application vulnerabilities, it exploits the way a system processes untrusted input—but the attack surface is different when that input is interpreted by an LLM.
The table below puts prompt injection into context by comparing it with more familiar attack types.
Attack Type | Mechanics & Execution Channel | Target Objective | System Impact |
|---|---|---|---|
Traditional Application-Layer Attacks (SQLi, XSS) | Exploits weaknesses in how applications parse, validate, or handle structured input. | Manipulates database queries, browser execution, or application behavior. | Data theft or modification, unauthorized actions, or code execution. |
Ordinary Malicious Input | Provides unexpected, malformed, or excessive input to test application validation and handling. | Triggers errors, bypasses validation, or consumes excessive resources. | Application errors, degraded performance, or denial of service. |
Prompt Injection | Exploits the LLM's inability to reliably distinguish instructions from untrusted content within its context. | Manipulates the model into ignoring intended instructions and following attacker-controlled ones. | Unauthorized tool calls, data exposure, unintended actions, or manipulated outputs. |
Jailbreak Attempts | Uses prompts or conversational techniques designed to bypass an LLM's safety rules and restrictions. | Causes the model to produce content or perform actions it was designed to refuse. | Safety-policy circumvention, harmful outputs, or misuse of model capabilities. |
» Here's everything you need to know about prompt injection
Why Prompt Injection Threatens Enterprise Deployments
Prompt injection becomes much more dangerous when an LLM moves beyond generating text and starts interacting with business systems.
An AI assistant that only answers general questions has limited ability to cause direct damage. An AI agent connected to a CRM, database, email platform or internal API has considerably more influence. If malicious content changes what the agent believes it should do, the consequences can extend beyond an incorrect response.
The main challenges include:
- Loss of control boundaries: Traditional software normally separates instructions from data. LLMs process natural language in a shared context, which means malicious content can influence how legitimate instructions are interpreted.
- Unauthorized autonomous actions: AI agents can be given access to CRM systems, email platforms, databases and other business tools. A manipulated agent may therefore make unauthorized API calls, change records or perform actions that were never intended by the organization.
- Indirect attacks through trusted content: Attackers do not always need direct access to an AI prompt. They can hide malicious instructions inside documents, emails, support tickets or web pages that an AI system later reads through RAG, browsing or another data retrieval process.
- Persistent exposure across workflows: Once untrusted content enters a system that is repeatedly retrieved or processed, the same malicious instructions can potentially affect multiple AI interactions or workflows.
The OWASP Top 10 for LLM Applications 2025 identifies prompt injection as LLM01, placing it at the top of the list of critical security risks for applications built with large language models. This highlights how prompt injection has become a core security concern as organizations increasingly connect LLMs to sensitive data, tools, and business workflows.
» Here are the most targeted entry points by attackers
High-Risk Enterprise AI Use Cases
Prompt injection becomes more dangerous when AI systems can access sensitive information or take actions on behalf of employees. Two enterprise use cases are particularly exposed: retrieval-augmented generation (RAG) and autonomous AI agents.
RAG and Internal Document Search
RAG systems retrieve information from sources such as PDFs, emails, customer tickets and shared files before generating a response.
The risk comes when these sources contain untrusted content. An attacker can hide malicious instructions inside a document, which the AI may later retrieve and interpret as part of its instructions. This can result in manipulated responses or sensitive information being exposed.
» Find out how agentic AI is transforming cybersecurity
Autonomous Agents and Tool-Calling Workflows
AI agents introduce greater risk because they can take actions, not just generate responses. Depending on their permissions, they may be able to send emails, update databases, change system configurations or access internal APIs.
If malicious instructions are embedded in an email, support ticket or other content an agent processes, they could influence the agent's next action. The result could be:
- An unauthorized API call
- Exposure of sensitive information
- Unapproved changes to business systems
» Here's everything you need to know about infostealers
Who Is Targeting Enterprise AI?
Prompt injection is not limited to one type of attacker. The threat landscape includes cybercriminals, initial access brokers and advanced persistent threat (APT) groups. Their goals differ, but AI can give them another way to influence enterprise systems and workflows.
Cybercriminals and Initial Access Brokers
Cybercriminals and initial access brokers are typically motivated by financial gain, targeting credentials, session information, financial data and valuable SaaS environments.
They can plant malicious instructions in public-facing forms, customer support tickets, documents or shared knowledge bases that AI systems later process. If an AI agent has access to sensitive APIs or internal systems, a successful injection could persuade it to retrieve or expose information the attacker could not access directly.
» Learn more about access broker's role in cybercrime
APTs and Strategic Reconnaissance
APT groups are more likely to pursue corporate espionage, intellectual property theft and long-term access to valuable enterprise environments.
Their attacks can extend beyond directly targeting the LLM itself. Techniques such as multimodal prompt injection, RAG database poisoning, and cross-tool contamination can allow attackers to introduce malicious instructions into the data and systems an AI application relies on. For example, an attacker could compromise a third-party document workflow or shared cloud storage and embed instructions in files that an enterprise AI system later retrieves and processes.
» Did you know? Cybercriminals now exploit generative AI
6 Best Practices for Preventing Prompt Injection
Preventing prompt injection requires more than filtering user input. Enterprises need layered controls that separate untrusted content from privileged actions, restrict what AI systems can access, and continuously test whether those safeguards hold up in practice.
1. Separate Untrusted Content From Privileged Execution
One of the strongest ways to reduce prompt injection risk is to separate the AI that reads untrusted content from the AI that can take actions. This limits an attacker's ability to turn malicious text into an unauthorized instruction.
What risk does it address?
When an AI can both interpret untrusted content and execute privileged actions, a malicious instruction hidden in an email, document, or retrieved webpage can potentially influence what the system does next. A dual-LLM approach reduces this risk by:
- Sending untrusted content to a low-privilege parsing model
- Converting that content into structured, validated data
- Allowing the privileged execution model to work only with those approved fields
This creates a boundary between "what the AI reads" and "what the AI is allowed to do."
What does effective implementation look like?
In day-to-day operations, the parsing model should process content such as customer tickets, emails and documents without access to sensitive APIs, tools, or network resources.
It converts the content into a defined structure, such as JSON, before passing it to the execution model. The execution model then works only with those approved fields rather than interpreting raw text as instructions.
2. Apply Strict Least-Privilege Access
AI systems should have only the permissions they need for the task they are performing. Restricting access does not necessarily prevent an injection, but it can dramatically reduce what an attacker can do if an AI system is compromised.
What risk does it address?
A successful prompt injection becomes much more serious when an AI agent has broad access to enterprise systems.
For example, an agent with administrator-level credentials could potentially:
- Access sensitive information
- Modify business systems
- Make unauthorized API calls
- Move laterally between connected services
Least-privilege access limits the blast radius of a compromised AI system.
» Don’t overlook the real threat—learn how infostealers put your data at risk
What does effective implementation look like?
Enterprises should give AI agents dedicated machine identities and tightly scoped credentials. OAuth 2.0 tokens, role-based access control (RBAC), attribute-based access control (ABAC), and continuous access evaluation can help ensure permissions match the task and the user's existing authorization.
3. Mandatory Human-in-the-Loop (HITL) for High-Impact Actions
AI should not be allowed to independently perform every action it is technically capable of performing. Human approval should sit between the AI and high-impact or irreversible operations.
What risk does it address?
Human-in-the-loop controls help prevent prompt injection from becoming an immediate business-impacting event.
An attacker may successfully manipulate an AI agent, but requiring approval before actions such as:
- Wire transfers
- Database deletion
- Bulk communications
- Sensitive account changes
gives a human an opportunity to stop the action before it happens.
What does effective implementation look like?
The approval process should be built directly into the workflow rather than relying on employees to manually monitor AI activity.
When an agent reaches a predefined risk threshold, execution pauses and sends an approval request to an authorized reviewer. The reviewer should be able to see exactly what the AI intends to do, including the target system and relevant parameters.
4. Use Passive Canary Tokens for Early Detection
Canary tokens provide a way to detect suspicious AI behavior without giving the model additional instructions. They act as digital tripwires that can reveal when sensitive or decoy information has been retrieved or exposed.
Canary tokens can help identify indirect prompt injection and potential data exfiltration.
For example, a unique token can be placed inside a document or vector database. If that token unexpectedly appears in an AI output or API request, security teams have a signal that the system may have accessed or exposed content it should not have.
What does effective implementation look like?
Security teams can place unique, non-executable strings inside selected documents and monitor where those strings appear. The tokens should not tell the AI to perform an action. They simply provide something that security systems can detect.
When a token appears in an unexpected location, the event can trigger an alert in the organization's SIEM or security monitoring system.
Example: A decoy document contains a unique identifier that should never appear in customer-facing responses. If the identifier suddenly appears in an AI response, security teams can investigate the retrieval path and surrounding activity.
» Worried about security? Here are the reasons you need cyber threat intelligence
5. Enforce Strict Input and Output Schemas
Structured inputs and outputs reduce the amount of free-form text that can flow directly into downstream systems. Schema validation creates a technical boundary between what the model generates and what an application is willing to execute.
What risk does it address?
Prompt injection often relies on manipulating free-form model behavior. If an application accepts unrestricted text and passes it directly into tools or APIs, malicious instructions can potentially influence downstream behavior.
Strict schemas reduce this risk by locking model interactions into rigid structural templates like Pydantic models or JavaScript Object Notation schemas, eliminating the free-form text blending that allows unauthorized commands to slip through.
For example, instead of accepting: "Please delete the customer's account immediately."
An application requires a validated schema payload containing specific fields such as action, customer identifier, and status, allowing the system to independently verify authorization.
What does effective implementation look like?
- Validate AI outputs: Use JSON Schema, Pydantic models, structured output modes, or function-calling parameters to validate model responses.
- Reject invalid outputs: If a response does not match the expected structure, reject it rather than passing raw output downstream.
- Make validation standard: Build schema validation into normal application processing rather than treating it as an optional security check.
6. Continuous Red-Teaming & Dynamic Vulnerability Scanning
Prompt injection defenses cannot be treated as a one-time security test. Continuous red-teaming helps organizations identify new weaknesses as models, prompts, data sources, and agent capabilities change.
What risk does it address?
New attack techniques can bypass controls that previously appeared effective. This is particularly important for RAG systems and AI agents, where changing a document source, tool, prompt, or model can create a new attack path. Continuous testing looks for vulnerabilities such as:
- Prompt injection
- Jailbreaks
- Context poisoning
- Unsafe tool calling
- RAG corpus manipulation
What does effective implementation look like?
Security teams can integrate automated adversarial testing like Garak into CI/CD pipelines. When developers change a system prompt, RAG pipeline, model, or agent workflow, the updated system is tested before deployment. This moves security testing closer to development rather than waiting for a periodic audit.
» Make sure you know the difference between a vulnerability, a threat, and a risk
Stay Ahead of AI Threats
Combine KELA Cyber's Cybercrime Threat Intelligence Platform with AiFort to detect, monitor, and defend against evolving AI threats.
New Opportunities and Emerging Risks
The AI revolution is here, and organizations are rapidly adopting generative AI tools to drive innovation and efficiency. But this rapid adoption also introduces new security risks, including prompt injection, data leakage, and vulnerabilities that traditional application defenses may not be equipped to address.
At KELA, we help organizations manage these risks across the AI lifecycle. Our AiFort platform combines real-world threat intelligence with an advanced AI verdict engine to identify security risks early in development, continuously monitor models in production, and automate red-teaming at scale. This enables enterprises to adopt AI quickly while strengthening their defenses against evolving threats.
FAQs
What is prompt injection?
Prompt injection is an attack in which instructions hidden inside untrusted text, such as a document, an email or a web page, are read by a large language model and acted on as though they came from the operator. The model has no reliable way to tell an instruction apart from ordinary content.
Why are RAG systems vulnerable?
Retrieval-augmented generation pulls in outside files, tickets and pages before the model answers. If any of that material carries hidden instructions, the model reads them alongside the legitimate content, and standard input filters never see them, because the attacker never typed anything into the prompt.
What do human-in-the-loop controls do?
They put a person between the AI and any action that is high-impact or hard to undo, such as a payment, a bulk message or a deletion. When the agent reaches that threshold, execution pauses and an authorized reviewer sees the intended action, the target system and the parameters before approving it.
What is the difference between direct and indirect prompt injection?
Direct injection is when an attacker types the hostile instruction into the prompt themselves. Indirect injection is when the instruction is hidden inside content the model retrieves later, such as a document, a support ticket, a web page or a file in a shared drive. Indirect is the harder enterprise problem, because the attacker never touches the AI system and the content usually arrives through a channel the organization already trusts.
Is prompt injection the same as jailbreaking?
They overlap, but they aim at different targets. Jailbreaking goes after the model's own safety rules, trying to make it produce content it was built to refuse. Prompt injection goes after the application around the model, trying to make it ignore the operator's instructions and follow the attacker's instead. In a connected system the second tends to matter more, because the outcome can be an action rather than a block of text.
Can prompt injection be completely prevented?
No single control eliminates it. Language models cannot reliably separate instructions from data, so the practical goal is to limit what a successful injection can reach rather than to filter every hostile string. That means separating the model that reads untrusted content from the one allowed to act, scoping agent permissions tightly, requiring human approval on high-impact actions, and retesting as prompts, tools and data sources change.
Where does prompt injection rank in the OWASP Top 10 for LLM Applications?
Prompt injection is LLM01 in the OWASP Top 10 for LLM Applications 2025, the first entry on the list of critical security risks for applications built on large language models.
