Working With Data Engineers and ML Teams as an AI PM: The Collaboration Playbook
TL;DR
The most common source of friction between AI PMs and data or ML teams is not technical disagreement. It is the PM asking for things before they are defined, asking for status when the team is mid-experiment, or treating an ML milestone like a feature ticket. Effective AI PM collaboration with data engineers and ML engineers requires a different operating model than traditional software PM. This article covers: how to unblock ML teams by writing crisp problem definitions, the five artifacts every AI PM should be able to read, how to run an ML planning conversation that produces useful estimates, and the specific mistakes that slow every AI sprint.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
What Data Engineers and ML Engineers Actually Care About
These are two distinct roles with different concerns. PMs who lump them together miss the collaboration opportunity. Understanding each team's primary concerns lets you bring the right questions to the right conversation.
Data Engineers
- Pipeline reliability: does the data flow work end to end, on schedule, without silent failures?
- Schema stability: does upstream data change shape without warning, breaking downstream transforms?
- Data quality: are there nulls, duplicates, or incorrect values that invalidate model inputs?
- Computational cost: is the pipeline running efficiently, or are we querying and transforming more data than needed?
When you need data from a data engineer, define what you need in terms of schema (which columns), freshness (how recent the data must be), volume (how many rows), and quality (what makes a row invalid). Vague requests like 'can we get the user behavior data' are unactionable and will stall for weeks.
Applied Scientists and ML Engineers
- Problem definition clarity: what is the input, what is the output, and what makes an output correct vs. incorrect?
- Label quality: is the ground truth data clean, consistent, and representative of real production inputs?
- Evaluation metrics: how will we measure whether the model works, and is that metric actually aligned with the business goal?
- Experimentation time: can they run multiple model iterations and compare results systematically?
ML work is fundamentally experimental. A sprint goal for an ML team is rarely 'ship the feature' — it is 'test whether approach A beats approach B on metric X.' The PM's job is to define what metric X is and what business outcome it connects to, not to predict whether the model will work.
The Five Artifacts Every AI PM Must Learn to Read
You do not need to be able to write these documents from scratch. But you need to be able to read them, understand what they are telling you, and ask good questions about them. An AI PM who cannot read an evaluation report is flying blind on the most important product decisions.
Evaluation report (eval report)
What it tells you: How the model performed on a test set, broken down by metric, slice, and often by failure category. The most important doc in AI product development.
Questions to ask: What does this metric actually measure, and is it correlated with what users care about? What are the hardest slices where performance is worst? What did we not test?
Model card
What it tells you: Intended use, training data, known limitations, performance across demographic groups, evaluation methodology. The model's official documentation.
Questions to ask: Which of these limitations will we encounter in our specific use case? What are the gaps between their evaluation conditions and our production conditions?
Data quality report or data card
What it tells you: Distribution of training or fine-tuning data: volume, label distribution, provenance, known quality issues.
Questions to ask: Does the training distribution match our production input distribution? What percentage of examples were filtered or rejected, and why?
Drift alert or monitoring dashboard
What it tells you: Whether the distribution of real production inputs has shifted relative to what the model was trained on, which predicts impending performance degradation.
Questions to ask: What inputs are shifting? Is this a leading indicator of user behavior change, or a technical upstream data issue? How long before we expect to see performance impact?
Inference cost dashboard
What it tells you: Actual cost per model call, broken down by model, endpoint, and often by token count or feature. The financial health of your AI feature.
Questions to ask: What is the cost trend over the last 30 days? What is our cost at the 10x traffic scenario? Is there a long tail of expensive calls we could cap or route to a cheaper model?
Running ML Planning Conversations That Get Useful Estimates
Asking an ML engineer "how long will this take?" is almost never useful. It is not evasiveness — it is that the answer genuinely depends on how hard the problem turns out to be, which they do not know until they try. Here is a planning format that gets more actionable output.
Start with the problem definition, not the solution
Open with: here is what the user needs to accomplish, here is what a correct output looks like, here is what an incorrect output looks like. Ask the ML team to identify what kind of problem this is technically. Their framing of the problem reveals feasibility assumptions you cannot see.
Ask for the simplest baseline first
Before discussing the best possible model, ask: what is the simplest thing we could try that has a 50% chance of being good enough? This shifts the conversation from aspirational to concrete. A decision-tree or keyword rule might solve 60% of the problem in one week versus 80% with a fine-tuned LLM in six weeks.
Separate exploration time from delivery time
ML work has two modes: exploration (we do not know if this will work) and delivery (we know it works, we are productizing it). Get estimates for each separately. Exploration should be timeboxed (two-week spike) with a defined signal for whether to continue. Delivery estimates are more reliable.
Identify the data blocker first
Ask: before we can start training or eval, what data do we need and what state does it need to be in? Data collection and labeling are the most underestimated time sinks in AI projects. Surface this dependency before the sprint starts, not during it.
Define the exit criteria before you start
What metric value on what evaluation set constitutes success for this sprint? If the team cannot answer this, the sprint has no definition of done. Without clear exit criteria, ML work expands indefinitely toward the theoretical optimum.
Learn to Lead AI Teams With Confidence
The AI PM Masterclass covers exactly how to work with data science and ML engineering teams effectively, taught live by a Salesforce Sr. Director PM.
The Five PM Mistakes That Slow Every AI Sprint
Mistake 1: Specifying the model architecture instead of the output behavior
Fix: Do not write 'use GPT-4 for this.' Write 'given a customer support ticket, produce a draft response that is factually accurate, matches our brand tone, and is under 150 words.' How to achieve it is the ML team's call. What the output should look like is yours.
Mistake 2: Asking for status when the team is mid-experiment
Fix: ML experiments have natural sync points: after baseline evaluation, after first model iteration, after A/B test results. Schedule your check-ins around those points, not on a fixed calendar cadence. Interrupting an experiment in progress costs more time than the check-in saves.
Mistake 3: Treating model improvements as on-off features
Fix: A model improvement from 71% to 76% accuracy is not a binary ship or no-ship. It is a question of whether the delta is large enough to make a product difference. PMs who think in feature-complete terms create artificial pressure that leads to shipping premature models or indefinitely blocking on marginal gains.
Mistake 4: Defining success only in user metrics, never in model metrics
Fix: User retention and model accuracy are both necessary. If you only define success in user metrics, the ML team cannot evaluate whether a model experiment is working before it ships to users. Define a model metric (precision, recall, BLEU, win rate in human eval) that you believe predicts the user outcome. That metric becomes the sprint target.
Mistake 5: Not involving data engineers in scoping
Fix: PMs often loop in ML engineers during scoping but not data engineers. Then a week into the sprint, the ML team discovers the required data does not exist in a usable form. Always ask: what data does this model need, and can data engineering confirm that data is available, clean, and freshly pipelined? Involve data engineers in the initial scoping conversation, not the follow-up.
Building Long-Term Credibility With Data and ML Teams
Credibility with ML teams is earned differently than with software engineering teams. Software engineers trust PMs who write clear requirements and respect technical decisions. ML engineers trust PMs who understand uncertainty, write clear evaluation criteria, and bring business context the team cannot see themselves.
Learn to read their eval report in public
In your next sprint review, ask the ML engineer to walk you through the evaluation results. Ask clarifying questions. Do not pretend to understand things you don't — but do not disengage either. Genuine curiosity is the fastest path to being trusted as a collaborative partner.
Bring real user examples, not hypothetical edge cases
The most useful thing a PM can give an ML team is a set of real, labeled examples from production that the model got wrong. Collect these systematically from customer support tickets, session replays, or user research. Real failure examples are more actionable than hypothetical ones.
Advocate for evaluation investment, not just model investment
ML teams spend most of their time training and improving models. PMs should actively advocate for dedicating engineering time to building better evaluation pipelines. A better eval suite pays dividends on every future model improvement. It is underinvested at almost every company.
Protect exploration time in sprint planning
When you own the sprint priorities, preserve dedicated time for ML exploration that has no guaranteed output. If every sprint is committed to deliverables, there is no time to find the 2x improvement that requires trying something that might not work. Protect one research-mode sprint per quarter at minimum.
Eval coverage
Percentage of real production failure types that appear in your evaluation set. A target of 80%+ means your evals are representative.
Labeling agreement rate
If two people label the same example, how often do they agree? Below 80% means the task definition is ambiguous and the PM needs to clarify criteria.
Experiment velocity
How many model iterations can the team run per sprint? This is a proxy for how well the data pipeline, eval infrastructure, and training infra support rapid iteration.
What the Best AI PMs Contribute That ML Teams Cannot Get Elsewhere
ML engineers build models. Data engineers build pipelines. The AI PM's unique contribution is not technical — it is contextual. These are the inputs that only a PM can provide and that determine whether the AI investment is pointed in the right direction:
The cost of a false positive versus a false negative
In your product, is it worse for the model to miss a real case or to flag a false one? This determines the precision-recall tradeoff the ML team should optimize for. Without this input from the PM, the team optimizes for aggregate accuracy, which can be the wrong objective entirely.
Which failure modes users will actually notice
Model benchmarks are averages. Users encounter specific failures. The PM knows from user research which failure categories cause users to lose trust, churn, or complain publicly. This knowledge should directly shape which failure slices the eval set prioritizes.
The business context behind a metric target
When you tell an ML team you need 90% accuracy, they ask why. The PM knows: because below 90%, support ticket volume goes up by 30%, which costs more than the feature saves. That context helps the team understand what accuracy improvement is worth pursuing and what shortcuts are unacceptable.
Prioritization across competing use cases
An ML team serving multiple product features will be pulled in competing directions. The PM knows which features have the highest business value and which user segment matters most at this stage. That prioritization is invisible to the ML team without explicit communication from the PM.
Build the Skills to Lead AI Product Teams
The AI PM Masterclass covers how to work with ML and data teams, write AI product specs, and ship AI features that hold up in production.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.