Large language models opened up a new attack surface — prompts, embeddings, tools, and agents. This guide walks through the threats security teams see in production and the controls that actually work.
Traditional appsec assumes deterministic code paths and structured input. LLMs invert both. Every prompt is untrusted input, every response is generative code or content, and every tool call the model can invoke becomes an attack primitive. Input validation, output encoding, and least-privilege still apply — but you also have to defend the model itself, the context you assemble, and any agent loops you build on top.
OWASP maintains a Top 10 specifically for LLM apps. These are the categories worth mapping every LLM feature against.
Direct or indirect instructions that override the system prompt or exfiltrate context.
Rendering model output as HTML, SQL, or shell without sanitization — classic XSS/RCE via generation.
Malicious data in fine-tune sets, RAG corpora, or feedback loops that shifts model behavior.
Adversarial prompts that force expensive generations, long contexts, or infinite tool loops.
Compromised base models, adapters, plugins, or embedding providers.
The model leaks secrets, PII, or system prompts baked into its context.
Tools with over-broad scopes, unvalidated arguments, or missing auth on downstream APIs.
Agents given write access to systems where a hallucinated action becomes a real incident.
Users or downstream code trusting LLM output without human or programmatic verification.
Extraction of proprietary weights, prompts, or fine-tunes via API abuse.
Direct injection lives in the user turn. Indirect injection hides in tool output, web pages the model reads, PDF metadata, or email bodies in an inbox assistant. Assume any text the model ingests can carry instructions.
System prompts, retrieved documents, and cached embeddings can all leak. Adversaries ask the model to repeat its instructions, quote its sources, or emit tokens that reconstruct the corpus.
An agent with shell, browser, or database tools is a confused deputy waiting to happen. A single crafted document can turn a helpful assistant into an exfiltration pipeline.
Model output rendered as HTML enables XSS. Passed to eval, exec, or a SQL driver, it enables RCE and injection. Treat generations as untrusted the same way you treat user input.
Repeated long-context requests, recursive tool calls, and image inputs can burn credits and starve legitimate traffic. Budgets and rate limits are security controls, not billing controls.
Lookalike apps, jailbroken clones, and phishing pages wrap your model or brand. Monitor the surface, not just the model.
Separate system, developer, and user roles. Mark retrieved content as data, not instructions. Strip or escape control tokens before they hit the model.
Every tool the model can call gets its own scope, its own auth, and its own audit trail. No tool should be able to act on data outside the current user's tenant.
Input filters for injection patterns, output filters for PII, secrets, and policy violations. Fail closed when a filter is unavailable.
Automated jailbreak suites, adversarial prompts, and regression evals gate deploys the same way unit tests do. Track pass rates over time.
Sign your corpus. Validate sources. Chunk with provenance metadata so the model — and your logs — always know where a claim came from.
Log every prompt, tool call, and response with tenant, user, and cost. Alert on injection signatures, cost anomalies, and tool-call spikes. Rehearse the LLM-incident runbook.
Not fully — it is the SQL injection of a probabilistic system. What you can do is reduce blast radius: separate roles, scope tools, sanitize output, and monitor for signatures. Defense in depth wins.
For anything user-facing or agentic, yes. A cheaper classifier or rules layer in front of and behind the main model catches most policy and injection failures without adding user-visible latency.
Treat the corpus as code. Sign it, review changes, and store provenance. At query time, mark retrieved chunks as data, filter secrets, and never let retrieved content silently become instructions.
OWASP Top 10 for LLM Applications, the NIST AI Risk Management Framework (AI RMF), and — in the EU — the AI Act obligations for your risk tier.
TrendGuru AI ships the scanner, brand monitor, and training modules teams use to close these gaps in production.
Explore the platform