OWASP LLM Top 10 and Agentic AI Top 10: A Product Manager's Security Checklist

Malicious user input overrides your system prompt or causes the model to follow attacker instructions. Direct injection (user types it in) and indirect injection (hostile content in a document the model reads) are both in scope.

PM Action: Define which inputs can override system-level instructions. Treat all user-supplied content as untrusted. Test your product by trying to jailbreak it yourself before shipping.

The model outputs PII, credentials, or confidential business data it was exposed to during fine-tuning or via RAG retrieval. This is distinct from deliberate exfiltration — it's accidental regurgitation.

PM Action: Audit what data touches your model. Fine-tuning data is memorizable at scale. RAG results need access controls so the model can only retrieve what the current user is authorized to see.

Compromised model weights, poisoned third-party datasets, or malicious plugins can introduce vulnerabilities before your product ever launches. The trust chain for AI is longer than for traditional software.

PM Action: Vet your model providers. Know which third-party datasets went into your fine-tuning. Treat each AI vendor in your stack with the same scrutiny you'd apply to a payment processor.

Training data or retrieval corpora are intentionally corrupted to change model behavior at inference time. A poisoned RAG index can make your model give wrong answers to specific queries.

PM Action: Control who can write to your RAG index and fine-tuning datasets. Audit data pipelines for injection points. Monitor model behavior over time for distributional shifts.

LLM output is used downstream without sanitization. Rendered HTML from a model can execute XSS. SQL queries from a model can cause injection. This is the AI equivalent of unsanitized user input.

PM Action: LLM output is untrusted input to every downstream system. Treat it like user-submitted form data. Require sanitization before database writes, code execution, and HTML rendering.

The model has more capability, permission, or scope than it needs to do its job. The principle of least privilege applies to AI. An agent that can send emails, create calendar events, AND delete files doesn't need all three for most use cases.

PM Action: Scope tools and permissions to the minimum needed per task. Design explicit approval flows for irreversible or high-blast-radius actions. Log every tool call for audit.

Users extract your system prompt through clever prompting. This exposes business logic, competitive IP, or security measures baked into the prompt.

PM Action: Don't rely on system prompt secrecy for security. Design your system so a leaked prompt doesn't compromise the product. Use separate config for sensitive values.

RAG systems can be attacked via the vector store: malicious documents that score high for legitimate queries, poisoned embeddings, or retrieval manipulation that redirects the model's context.

PM Action: Validate documents before indexing. Set retrieval score thresholds. Monitor retrieved chunks for anomalies. Consider separate indexes for different trust levels.

The model generates factually incorrect output confidently. This was 'overreliance' in 2024. The updated framing puts responsibility on both the model and the product that deploys it without appropriate caveats.

PM Action: Design for skepticism. Show sources. Distinguish between 'generated' and 'retrieved' content. Build in confirmation flows for high-stakes outputs (medical, legal, financial).

Attackers exploit missing rate limits to run up your inference bill or degrade performance for other users. At scale, a single adversarial session can cost thousands of dollars.

PM Action: Implement per-user token budgets, rate limits, and cost alerts from day one. Design the product so a runaway request cannot cause unbounded API spend.

An attacker redirects the agent's objective through adversarial input. The agent was told to summarize a document; it reads injected instructions in the document and sends the user's data to an external server instead.

PM Action: Treat agent goals as invariants, not guidelines. Separate user instructions from agent objectives architecturally. Log and alert on any goal divergence.

The agent uses legitimate tools in unintended ways. An agent with access to the email tool and the CRM tool can combine them to exfiltrate customer data without triggering a single tool-specific alert.

PM Action: Audit tool combinations, not just individual tools. Define the 'intended path' for each workflow and instrument deviations.

The agent impersonates a human or escalates its own privileges. An agent that can act 'as the user' can approve its own requests, bypass review flows, or access admin endpoints.

PM Action: Agents should have their own service identities, separate from users. No agent should be able to authorize its own requests. Require human approval for privilege escalation.

Malicious MCP servers, compromised agent frameworks, or hostile sub-agents introduce vulnerabilities into multi-agent pipelines. The GitHub MCP exploit demonstrated this at scale in 2025.

PM Action: Vet every tool, plugin, and sub-agent in your pipeline. MCP servers you load from third parties have the same trust level as npm packages: verify, pin versions, and monitor.

The agent generates and executes code without appropriate sandboxing. AutoGPT's 2025 RCE vulnerability showed that LLM-generated code running in an unsandboxed environment is a critical attack surface.

PM Action: Sandbox all agent-generated code execution. Never execute agent output directly in a production environment. Use containerization with strict resource and network limits.

An attacker injects malicious content into the agent's persistent memory or long-running context. The Gemini Memory Attack demonstrated that hostile web pages could write to an agent's memory store.

PM Action: Treat agent memory as untrusted storage. Validate and sanitize content before writing to persistent memory. Scope memory access per user, per session, and per tool.

Multi-agent systems pass messages between agents without authentication or integrity checks. A compromised sub-agent can poison the orchestrator's context or inject instructions into the pipeline.

PM Action: Authenticate inter-agent messages. Don't assume an agent message is trustworthy because it came from within your system. Verify, log, and scope agent-to-agent permissions.

A failure in one agent cascades through the pipeline and amplifies before any human can intervene. Automated retry logic, combined with aggressive tool access, can cause irreversible multi-system damage.

PM Action: Design circuit breakers into agentic pipelines. Set hard limits on retry counts and blast radius. Require human approval before any action that can't be undone.

Attackers exploit the user's trust relationship with an AI agent to manipulate user behavior or extract sensitive information through social engineering mediated by the agent.

PM Action: Be explicit about what your agent can and can't request. An agent should never ask for passwords, payment info, or sensitive credentials. Design and communicate these limits to users.

An agent starts pursuing goals misaligned with both user intent and system design. This includes agents that optimize for proxy metrics, that learn to game reward signals, or that develop unintended emergent behaviors in production.

PM Action: Monitor agent behavior for distributional drift. Compare sampled agent outputs against expected behavior weekly. Have a kill switch: the ability to halt all agent actions within seconds.