How to Write an AI Model Brief: Communicating What You Want From Your ML Team
TL;DR
- A model brief is NOT a PRD. It is a one to two page spec that tells your ML team what a model needs to do, not how to build it.
- A good model brief defines the task framing, ground truth labels, success metrics, constraints, and failure modes before any training begins.
- Most model-product misalignments trace back to skipped or vague briefs, not bad engineering.
- This guide provides a template and common mistakes drawn from production ML teams at Salesforce, Meta, and Google.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
Why a Model Brief Is Not a PRD
Product requirements documents define what a feature does for users. Model briefs define what a machine learning model needs to output given a set of inputs. The difference sounds subtle. In practice it determines whether your ML team spends six weeks building the right thing or six weeks building something that works technically but ships nothing.
A PRD written for a recommendation feature might say: "Users should see content relevant to their interests, leading to a 15% increase in session depth." That is a perfectly fine product spec. It tells engineering what outcome to target. But it leaves your ML team answering questions that should have been answered before they opened a Jupyter notebook:
- What is the label? What makes one piece of content better than another?
- What signals are available at inference time?
- Is false positive or false negative more costly?
- What latency budget does the model have?
- What are the constraints on explainability or fairness?
When PMs leave these questions unanswered, ML leads fill in the blanks. Sometimes they guess correctly. More often, you get a model that maximizes the wrong proxy metric, ignores critical edge cases, or requires a retraining cycle before it can ship.
The model brief is the document that closes this gap. It lives between the PRD and the ML implementation plan. The PM owns it.
The Five Components of a Model Brief
A complete model brief contains five sections. None of them require ML expertise to write. They require you to think carefully about the problem before asking someone else to solve it.
Task framing
State the ML problem in one sentence. Not the product goal: the model task. Examples: 'Given a user query and candidate product, predict the probability the user clicks within 24 hours.' 'Given a 30-second audio clip, classify the dominant emotion from sad / neutral / happy / angry.' 'Given the last 10 user actions, rank the next 50 candidate items by predicted long-session engagement.' Task framing forces precision. If you cannot write this sentence, the model cannot be built.
Labels and ground truth
What is the definition of a positive example? A negative? How will labels be generated: explicit signals (thumbs up/down), implicit signals (30s watch time, share, save), human raters, or programmatic heuristics? What is the label quality budget? How many labels can be acquired per week? Ground truth is often the hardest part of any ML project. If you leave it ambiguous, the team will pick a proxy that is easy to generate, not one that reflects your actual goal.
Success and guardrail metrics
Separate success metrics from guardrail metrics. Success metrics are what the model is optimizing: precision at k, AUC, NDCG, RMSE. Guardrail metrics are hard floors you will not cross: p95 inference latency under 50ms, false positive rate below 2% on protected attribute slices, no more than 10% of predictions pointing to a single creator. Write specific numbers. 'Good performance' is not a metric.
Input features and constraints
List the signals available at inference time. Flag any that are legally restricted (age, location), operationally unavailable (cold start users), or subject to lag (engagement aggregates that are 24 hours stale). Also state hardware and latency constraints: edge vs. cloud inference, model size budget, batch vs. real-time serving. ML engineers can work around constraints you name. They cannot work around constraints you forgot to name.
Failure modes and out-of-distribution behavior
What happens when the model is wrong? High-severity failures need explicit callouts: incorrectly flagging a post as hate speech, recommending a creator who has since been banned, pricing a product 10x above market. Also describe the out-of-distribution cases the model will encounter that were not in training: new users, new item categories, traffic spikes during events. The ML team will handle these better if you have named them.
Writing Success Criteria Your ML Team Can Use
The most common model brief failure: the PM writes business outcomes and calls them ML metrics. "We want a 15% lift in engagement" is a business goal. It is not a training objective or an evaluation benchmark. Your ML team needs to know what to measure on a held-out test set before you run a single A/B test.
The translation process works like this:
| Business goal | Offline ML metric | Why this translation |
|---|---|---|
| Increase click-through on recommendations | AUC-ROC on held-out click labels, precision@10 | Ranking by click probability is the direct proxy for CTR lift |
| Reduce spam in comments | F1 on spam / not-spam; false positive rate on non-spam | F1 balances precision and recall; FPR guardrails protect legitimate users |
| Improve search relevance for long queries | NDCG@5 segmented by query length bucket | NDCG rewards rank-ordering; segmentation surfaces long-query degradation |
| Predict 30-day churn before it happens | AUC-PR (precision-recall) at decision threshold; calibration error | Class imbalance makes AUC-ROC misleading; calibration matters for business decisions on outputs |
| Personalize pricing per user | RMSE on willingness-to-pay, max price error as guardrail | Regression task; max error guardrail prevents catastrophic mispricings |
Once you have offline metrics agreed, define what "good enough to ship" looks like as a number. This is the minimum bar for the model to proceed to A/B test. Not every model will beat its baseline on day one. Without a go/no-go threshold, the team will keep iterating indefinitely or ship something that is not ready.
AI Product Management Masterclass
Learn to lead ML teams as a product manager
The Sept 15 cohort covers model briefs, PRD-to-spec translation, and the full PM-to-ML collaboration workflow. Live, instructor-led, with real projects.
Five Mistakes That Waste ML Cycles
These mistakes appear in most model briefs written by PMs who have not shipped ML features before. Recognize them in your drafts before your ML lead points them out in review.
Specifying the algorithm
Writing 'use a transformer' or 'build a collaborative filter' is premature. The brief defines what the model does, not how it does it. Algorithm choice belongs to ML engineering. If you constrain it without a reason (latency, interpretability), you reduce your team's design space.
Conflating product metrics with model metrics
Engagement rate is a product metric. It is what you measure after you ship. It is not what the model optimizes. The brief needs both, and it needs to make the connection between them explicit: 'We believe optimizing for [model metric] will produce [product metric] lift based on [evidence or assumption].'
Leaving label definition to the ML team
Label definitions are product decisions. What counts as a relevant result, a harmful post, or a high-intent user is a judgment call with business consequences. If your ML team defines labels without you, they will optimize for what is easy to measure, not what you actually care about.
No discussion of edge cases
New users, rare categories, adversarial inputs, and traffic spikes during live events are predictable. If they are not in the brief, they will not be in the training data or evaluation suite. You will discover the gaps in production.
Treating the brief as a one-way document
The model brief should trigger a conversation, not end one. Your first draft will have incorrect assumptions about what signals are available, what latency is achievable, and what label quality is realistic. Schedule a 30-minute brief review with the ML lead before any work begins.
A One-Page Model Brief Template
This template is designed to fit one to two pages in a shared doc. Fill it out before your kickoff with the ML team. Treat blank fields as open questions, not skippable sections.
Copy this into a Google Doc or Notion page. Share it with the ML lead at least 48 hours before the kickoff meeting. Ask them to annotate the fields where your assumptions look wrong or where constraints you listed are not realistic.
Getting ML Team Buy-In on Your Brief
ML engineers and researchers often distrust product-authored specs. The concern is reasonable: most product briefs they have seen either micromanaged the algorithm or left the hardest decisions undefined. Your brief will earn trust faster if you do three things.
Separate what from how explicitly. Open your brief with: "This document defines the task, metrics, and constraints. It does not specify modeling architecture or training approach. Those are your calls." That sentence resets the relationship from the first line.
Show your work on label definition. Do not just assert that "a click within 30 seconds is a positive signal." Walk through why: your prior A/B tests, user research, benchmark comparisons. When ML leads understand the reasoning behind a label definition, they can help you refine it rather than working around it.
Own the tradeoffs explicitly. If your guardrail metric is p95 latency under 100ms, write out why: "The feature surfaces inline in search results, and above 100ms we see user-perceived lag in user testing." This gives the ML team context to push back if the constraint is unachievable with the required model quality, rather than silently accepting a tradeoff that ships a slower or weaker model.
The brief review meeting
Schedule 30 minutes with the ML lead after sharing the draft. Walk through each section. Ask specifically: "Which labels are going to be hard to get?" "Which constraints might force a quality tradeoff I should know about?" "What did I leave out?" Take notes. Update the brief the same day. The meeting is not a sign-off ceremony. It is a shared debugging session.
Model briefs are most valuable on your first two or three ML features with a given team. Once the team trusts that you know what you want and how to ask for it, the brief becomes lighter: a half-page that catches the non-obvious constraints and leaves the rest to the ML lead's judgment. The full template is the starting point, not the permanent format.
Lead ML teams with the confidence of a technical PM
The AI Product Management Masterclass teaches model briefs, PRD-to-spec translation, ML-team collaboration, and the full lifecycle of shipping AI features. Next cohort starts Sept 15.
Related Articles
AI Feature PRD Template
A structured PRD format for AI features, covering data requirements, model specs, and evaluation criteria.
ML System Design for Product Managers
How to reason about model serving, feature pipelines, and retraining cadences without building them yourself.
Working With Data Scientists as a PM
Communication patterns, meeting structures, and decision-making frameworks for cross-functional AI teams.
AI Product Metrics Framework
How to connect model-level metrics to product-level outcomes without misrepresenting what the model can control.
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.