AI Product Principles: How to Define What Your AI Will and Won't Do
TL;DR
Every AI product eventually faces an edge case that forces a values question: Should the AI answer this? Should it do that? Should it prioritize this user over that one? Teams without written AI product principles answer these questions inconsistently, in the moment, under deadline pressure. Teams with principles answer them in advance, consistently, in a way that reflects the product they deliberately chose to build. AI product principles are not an ethics compliance document. They are a product strategy document that defines what your AI will do, what it will not do, why, and how trade-offs are resolved when principles conflict. This guide explains what they should cover, how to write them, and how to make them operationally useful rather than a shelf document.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
What AI Product Principles Are and What They Are Not
There are four documents that sound similar but serve different purposes. Confusing them is the most common reason AI product principles fail to work in practice.
AI product principles
What it is: A strategic document that defines what your product will and will not do, the values it embeds, and how trade-offs between competing values are resolved. It is written by the product team and engineering team together. It is durable across quarters.
Not a list of prohibited inputs. Not an ethics policy document written by legal. Not a compliance checklist.
System prompt and guardrails
What it is: The operational implementation of principles inside the model. System prompts define the persona, scope, and behavioral limits at runtime. Guardrails enforce hard limits through content filtering, output validation, or refusal logic.
Not the same as principles. Principles inform what the guardrails should be, but principles live at the product strategy level, not the implementation level.
Ethics review policy
What it is: A process document, often owned by legal or a responsible AI team, that defines when projects require ethics review, what criteria they are evaluated against, and who approves them.
Not the same as principles. Ethics review is a process. Principles are the values the process applies.
Terms of service and acceptable use policy
What it is: Legal documents that define what users are permitted to do with the product. Public-facing. Legally binding.
Not the same as principles. Terms of service define user obligations. Principles define product team obligations and product behavior.
The Five Dimensions Every AI Product Principles Document Should Cover
A principles document that covers all five dimensions gives your team a consistent decision framework. A document that covers only one or two creates ambiguity precisely in the situations where you need clarity.
1. Scope: What this product will and will not do
?What tasks is the AI built to perform?
?What tasks are explicitly out of scope, and why?
?What happens when a user asks for something out of scope? Does the product redirect, decline, or refer?
Example
A medical information AI might be scoped to: answer general health questions using peer-reviewed sources; explain diagnoses in plain language; suggest when to seek professional care. It would be explicitly out of scope for: diagnosing specific conditions, recommending medication doses, providing second opinions on clinical decisions. A user who asks for a specific diagnosis gets a redirect to speak with their physician, not a refusal message, and not a diagnosis.
2. Accuracy and uncertainty: How the product handles what it does not know
?How does the AI signal uncertainty in its outputs?
?What confidence threshold triggers a disclosure or a refusal?
?When is 'I don't know' the correct answer, and how is it expressed?
Example
A legal research AI might state: outputs are clearly labeled with the jurisdiction they apply to; when the AI does not have high confidence in a legal interpretation, it states this explicitly and cites the source of uncertainty; the AI will not generate a definitive legal opinion on complex questions where case law is unsettled, and will say so rather than proceeding.
3. User autonomy: How the product balances helpfulness with user protection
?When does the product prioritize user safety over user requests?
?When does it defer entirely to user judgment?
?How does it handle users who appear to be in distress or making decisions that could harm themselves?
Example
A financial planning AI might define: users are treated as competent adults capable of making their own financial decisions; the AI provides requested information without moralizing; when a user expresses a plan that carries extreme financial risk, the AI surfaces the risk clearly once, then respects the user's decision without repeating the warning.
4. Fairness: How the product treats different users and groups consistently
?Does the product behave consistently across demographic groups?
?What is the process for detecting and addressing disparate impact in AI outputs?
?Who is responsible for monitoring fairness metrics, and at what cadence?
Example
A hiring AI tool might define: the model is evaluated quarterly against fairness metrics across gender, race, and age; any disparate impact above a defined threshold triggers a pause in deployment and a root cause investigation before resumption; the PM owns the fairness dashboard and reports quarterly to the CPO.
5. Transparency: What the product discloses about how it works
?Do users know they are interacting with AI?
?What information does the product share about how decisions are made?
?What data does the product collect, and how is that disclosed?
Example
A customer service AI might define: users are told they are interacting with an AI assistant at the start of every session; users can request a human agent at any point; when a decision affects a user's account (e.g., a return is declined), the AI provides the reason in plain language and the path to appeal.
How Leading AI Products Define Their Principles
The most instructive examples in the industry are not the high-level platitude documents most companies publish externally. They are the internal frameworks that actually drive product decisions. The public-facing versions give you clues about the underlying structure.
Anthropic / Claude
Anthropic uses a layered approach: Constitutional AI defines broad values at the model level; the Model Spec defines how Claude weighs competing values (safety, ethics, Anthropic guidelines, helpfulness, in that explicit priority order); usage policies define operational limits per deployment. The innovation is making the priority ordering between competing values explicit, so trade-offs are resolved consistently rather than ad hoc.
PM lesson
Make your priority ordering explicit. When helpfulness conflicts with safety in your product, which wins? Stating this clearly prevents inconsistent decisions.
Google / Gemini
Google's AI principles (published 2018, updated through 2026) define seven positive goals and four things they will not build. The approach is notable for including explicit product categories they decline to build (AI for weapons systems, AI that causes harm). The scope boundary is set at the level of product categories, not individual features.
PM lesson
Defining what you will not build is as important as defining what you will build. Explicit anti-scope creates internal clarity and external trust.
Microsoft / Copilot
Microsoft's responsible AI standard breaks principles into six pillars (fairness, reliability, privacy, inclusiveness, transparency, accountability) and assigns specific measurement responsibilities and review processes to each pillar. The principles are not just values statements but governance artifacts with owners.
PM lesson
Each principle needs an owner and a measurement approach. A principle without a metric and an owner is aspirational, not operational.
OpenAI
OpenAI's usage policies define prohibited use cases in specific language (generating CSAM, creating weapons of mass destruction, attacking critical infrastructure) alongside context-dependent policies for sensitive categories like medical information and legal advice. The specificity of the prohibited categories is higher than most public AI ethics documents.
PM lesson
Vague prohibitions create inconsistent enforcement. Specific language ('you will not generate content that provides operational instructions for creating pathogens') creates consistent, implementable guardrails.
Learn to Build AI Products That Earn Long-Term Trust
The AI PM Masterclass covers responsible AI product design, evaluation frameworks, and the strategic decisions that define what kind of AI product you build. Live cohort, real builds, taught by a former Apple and Salesforce Sr. Director PM.
How to Write and Operationalize Principles
A principles document that sits in a Notion page and is read during onboarding is not operationalized. Here is the process for writing principles that actually change decisions.
Step 1: Surface the edge cases first, write principles second
Do not start with abstract values. Start by collecting the 20 to 30 hardest decisions your product has faced or will face. What did users ask for that you said no to? What did the model do that you did not expect? What feature requests from sales violated your intuition about what the product should be? The edge cases reveal the values. Write principles that resolve them consistently.
Step 2: Write for an engineer, not a regulator
Principles documents fail when they are written for external audiences (investors, press, regulators) rather than for the people who implement the product. Write each principle so that an engineer, a designer, or a data scientist can apply it to a specific implementation decision without asking a follow-up question. 'Respect user autonomy' is not actionable. 'The product will not repeat safety warnings more than once per session unless the user has explicitly requested ongoing reminders' is actionable.
Step 3: Assign owners and metrics to each principle
For every principle, define: who owns it, how it is measured, and at what cadence it is reviewed. Fairness is measured via disparate impact testing quarterly, owned by the ML lead, reviewed with the CPO. Accuracy is measured via precision/recall on a defined evaluation set monthly, owned by the PM, reviewed in sprint retrospectives. Without this, principles are statements of intent, not operational commitments.
Step 4: Test principles against real decisions before publishing
Before finalizing a principles document, take 10 real past decisions the team made about the product and apply the principles to see if they would have produced the same outcome. If the principles would have produced a different decision, that is either evidence that the past decision was wrong or that the principle is not calibrated correctly. Resolve it before the document is final.
Step 5: Build a lightweight principles review into your PRD process
Add a single question to every product requirement document: 'Which principles does this feature implicate, and how?' If the answer is 'none,' the feature is either genuinely unconstrained or the PM has not thought hard enough. The goal is not to slow down shipping but to make the principles review automatic rather than exceptional.
Principles Under Pressure: When Trade-offs Arise
The test of an AI product principles document is not how it performs on the easy cases. It is how it performs when two principles conflict with each other and the team is under pressure to ship.
Tension: Helpfulness vs safety
Scenario: A user asks your legal AI for a specific opinion on whether their contract is enforceable. Answering helps the user. Answering also risks the AI producing a definitive legal opinion on a complex question where it may be wrong.
Resolution approach: Your principles should state which value takes priority and how the output is shaped when the conflict is triggered. Most AI products resolve this by being helpful within a defined accuracy scope and explicit about the limits, rather than refusing entirely.
Tension: User autonomy vs user protection
Scenario: A user of your financial AI explicitly asks you to stop flagging risky investment decisions. They find the warnings paternalistic. Respecting their autonomy means complying. Protecting them means continuing.
Resolution approach: Write your principle to define the specific threshold at which user protection overrides autonomy. For most consumer products, the answer is: autonomy wins for financial decisions below a defined risk threshold; protection wins when decisions carry irreversible consequences (bankruptcy-level risk).
Tension: Consistency vs personalization
Scenario: Your AI gives different responses to different users based on their history. This is good personalization. It also means the AI says different things about the same topic to different users, which raises fairness and consistency concerns.
Resolution approach: Define which parts of the AI's behavior must be consistent (factual answers, safety guidance, disclosure language) and which can be personalized (tone, depth of explanation, topic focus). Apply the fairness principle to the consistent layer only.
The most important thing your principles document must do
It must make the priority ordering between competing values explicit. Not "we value both safety and helpfulness" — that tells your team nothing when they conflict. "When safety and helpfulness conflict, safety takes priority; within the safety constraint, we maximize helpfulness" — that is a decision that can be implemented consistently by everyone on the team, including the model.
A Starter Template for Your AI Product Principles
This is a starting structure, not a finished document. Fill in the specifics for your product and use case. The value is in the process of filling it in together with your engineering and design team, not in having a completed template.
Our AI is designed to:
[List 3 to 5 specific use cases the AI is built for. Be concrete enough that a new team member could use this list to scope a feature.]
Our AI is not designed to:
[List the use cases that are explicitly out of scope. Include the reason for each exclusion — 'not because it is impossible, but because it conflicts with our accuracy standard / our user trust model / our regulatory context.']
When our AI does not know something:
[Define the specific behavior. Does it say so explicitly? Does it decline to answer? Does it redirect? Does it provide its best estimate with a confidence qualifier? Be specific about the UX pattern, not just the intent.]
We treat our users as:
[Define your model of the user. Competent adults? Vulnerable individuals? Experts? Novices? Your answer shapes every default behavior and every protective override.]
The values priority order when they conflict:
[List your core values in explicit priority order. Example: 1. Do not cause harm. 2. Be accurate. 3. Respect user autonomy. 4. Be helpful. This order means: when helpfulness conflicts with accuracy, accuracy wins. When accuracy conflicts with safety, safety wins.]
Our principles are owned by:
[Name a person and a team. Principles without an owner become shelf documents. The owner is responsible for reviewing them annually, applying them to new product decisions, and updating them when the product evolves.]
Build AI Products You Are Proud to Ship
The AI PM Masterclass covers responsible product design, evaluation, and the strategic frameworks that help you build AI products that earn long-term user trust. Live cohorts, real product builds.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.