How to Work With ML Engineers as an AI Product Manager

The ML Engineer's Actual Job (Not What PMs Think It Is)

Most PMs come in with a mental model of ML engineers as software engineers who build machine learning features. That framing causes almost all the friction. ML engineering is fundamentally an experimental discipline. The job is not to build a known thing. It is to discover whether a thing is buildable and then build it.

Research phase

Reviewing literature, reproducing results, identifying the right modeling approach. This can take days or weeks before a line of product code is written. PMs who skip this in their planning create timeline surprises downstream.

Data work

Sourcing, cleaning, labeling, and validating training data. Often 40 to 60% of total project time. Not glamorous, deeply important. When data is wrong, the model is wrong. PMs who treat data work as a minor prereq consistently underestimate timelines.

Experiment tracking

Running controlled experiments to test hypotheses about model architecture, training strategy, and hyperparameters. Each experiment has a question, a setup, and a result. ML engineers who skip this discipline produce unreproducible models.

Evaluation design

Building the infrastructure to measure whether a model is good. Automated metrics plus human evaluation plus slice analysis. This is ongoing, not a one-time gate at the end of the project.

Model deployment and monitoring

Exporting the model, optimizing inference, integrating with the product backend, setting up monitoring for quality drift. This is where ML and software engineering overlap most heavily.

Each phase has different PM involvement requirements. Research and experiment tracking benefit from your product direction and user signal. Data work benefits from your prioritization of which slices matter most. Evaluation benefits from your definition of what "good enough" actually means for users.

Common Friction Points That Kill PM and ML Collaboration

These are not personality clashes. They are structural mismatches between how product and ML engineering work. Knowing them makes them fixable.

Deadline without uncertainty acknowledgment

PMs give ML engineers a ship date without a plan for what happens if the experiment fails. ML engineers know experiments fail. They absorb the deadline pressure silently, work around results that don't support the product direction, and ship something that looks good but degrades in production.

Success criteria stated as features, not outcomes

"Add a confidence score to every response" is a feature. "Users should not feel misled by outputs more than 5% of the time" is a success criterion. ML engineers need the outcome to decide how to achieve it. The feature spec often constrains the solution space without adding value.

Treating model quality as binary

"Is the model good enough?" is not a useful question. "Which user segments does it perform well for, at what quality level, and how does that compare to our target?" is the question that drives decisions. PMs who can't read an eval breakdown create pressure that pushes ML engineers toward overall average optimization.

Scope changes mid-experiment

Changing the target domain, evaluation criteria, or output format while an experiment is running can invalidate all prior results. ML experiments are not sprints. Changing the acceptance criteria midway is not a minor adjustment. It restarts the work.

Not differentiating model work from feature work

Some ML work is building a new model capability. Some is adding model-based features to an existing product. These have very different timelines and uncertainty profiles. Treating them the same in sprint planning creates misaligned expectations across the whole team.

Skipping the failure analysis conversation

When a model underperforms, PMs often move straight to "what do we do next" without understanding why it failed. ML engineers who aren't given space to analyze failures are likely to repeat them.

Writing ML Requirements That Engineers Can Actually Use

The most valuable thing a PM can give an ML engineer is a clear, actionable problem statement. Here is the structure that works:

User context and use case

Who is using this, in what workflow, and what are they trying to accomplish? Not the feature you want to build. The task the user is trying to complete.

Example: Enterprise sales reps use our product to draft follow-up emails after customer calls. They have 10 to 15 minutes between calls and need a draft that captures the key commitments made in the meeting.

Input and output description

What does the model receive? What does it produce? Include edge cases that matter to your users.

Example: Input: call transcript (avg 3,000 words, sometimes 8,000+), CRM context for the account. Output: email draft under 200 words, first-person from the rep's voice, covering only the commitments explicitly made.

Success criteria with thresholds

What does 'good enough' look like? State it in measurable terms. Include floor quality (minimum acceptable) and target quality (what you want at launch).

Example: Floor: 70% of drafts need zero or one user edits before sending. Target: 85%. Non-negotiable: zero hallucinated commitments that weren't in the transcript.

Failure modes that matter most

Which errors are unacceptable versus annoying? This drives evaluation design and helps ML engineers prioritize what to optimize against.

Example: Worst failure: email that fabricates a discount or delivery commitment not made in the call. Moderate failure: email that misses a key commitment. Minor failure: email that needs rewording but facts are correct.

Latency and cost constraints

At what latency does the feature become unusable? What is the cost ceiling per call processed? These are real engineering constraints, not afterthoughts.

Example: Draft must appear within 8 seconds of call end. Cost must stay under $0.03 per draft at full volume (50,000 calls/month).

Build Real ML Collaboration Skills in the Masterclass

The AI PM Masterclass includes live sessions on working with ML teams, writing eval-driven requirements, and navigating the experiment-to-production lifecycle. Taught by a Salesforce Sr. Director PM.

Navigating ML Timelines and Experiment Uncertainty

Timeline conversations with ML engineers are hard because the honest answer to "when will this be done?" is often "we don't know yet." That's not evasion. It is an accurate description of experimental work. Here's how to plan around it:

Plan for a decision gate, not a ship date

Structure the first phase as "by [date] we will know whether this approach is viable." That is a predictable milestone. The ship date follows from the viability decision, not the other way around.

Establish a stage-gate structure

Break the work into feasibility (can we beat a baseline?), quality (can we hit the floor threshold?), and production (can we deploy at cost and latency targets?). Each gate is a real decision point, not a bureaucratic checkpoint.

Ask for a range, not a point estimate

"When will this be done?" → "What's the best-case timeline if the first experiment works? What's the realistic timeline if we need two more experiment cycles?" Ranges build in the uncertainty honestly rather than hiding it.

Name the assumptions explicitly

"This timeline assumes labeled data is available by [date], the third-party embedding API stays under [latency], and we don't change the target output format mid-project." Naming assumptions surfaces risks before they become emergencies.

Create a kill criteria before you start

Define the threshold at which you stop investing and change approach. If by experiment 3 we haven't beaten the baseline by X%, we revisit the approach. Having this agreed upfront removes the pressure to chase sunk costs.

When to Push Back and When to Defer

One of the hardest parts of the PM and ML engineer relationship is knowing when to advocate hard for the product position and when to defer to the technical judgment. Here is the distinction that matters:

Push back: when the success criteria are being redefined

If an ML engineer proposes changing what 'good enough' means because it's hard to achieve, that is a product conversation, not a technical one. You own the quality bar. You can revise it, but do it consciously after understanding the user implication.

Defer: which model architecture to use

If the ML engineer says "we should use a seq2seq approach rather than decoder-only for this task," that is their call. Understanding the tradeoff is valuable. Overriding it is not.

Push back: when timeline pressure is creating silent quality shortcuts

If you sense the team is shipping something that doesn't meet the floor criteria because of deadline pressure, surface it explicitly. "Are we confident this meets the floor we agreed on, or are we accepting risk to hit the date?" is a fair question.

Defer: evaluation methodology

The ML engineer will have strong opinions on how to measure model quality. You should understand and agree on the metric, but not redesign the evaluation protocol without their buy-in.

Push back: when the product direction is being assumed instead of asked

ML engineers sometimes make implicit product decisions during experiments (optimizing for one metric when another matters more, training on one user segment when another is higher priority). These are your decisions. Create space for them to surface these choices rather than decide alone.

Defer: data augmentation and training decisions

How to handle class imbalance, what data augmentation to apply, when to use synthetic data vs real data. These are deeply technical calls with real quality implications. You can ask why. You should not direct.

Building a Long-Term Partnership With Your ML Team

The PM and ML relationship compounds over time. An ML engineer who trusts you will bring you into experiments early, flag risks before they become crises, and advocate for your product priorities with the rest of the engineering team. Building that trust takes consistent behavior over months, not a single good kickoff.

Share user signal directly and often

Don't filter and summarize. Give ML engineers raw user feedback, support tickets, and session recordings. They will often notice patterns you didn't and connect them to model behaviors you weren't aware of.

Show up for experiment reviews, not just results readouts

Attend the experiment design meeting, not just the results presentation. This signals that you care about the method, not just the outcome, and gives you earlier visibility into risks.

Publicly credit technical decisions when they work

When a model performs well, attribute it specifically to the team's choices. Not just "the team did great work" but "the decision to use contrastive fine-tuning on hard negatives is what closed the quality gap." Specificity shows you were paying attention.

Document and honor past commitments

Write down what you agreed to (success criteria, timelines, scope limits) and don't revise them without explicit discussion. ML engineers who have been burned by shifting requirements treat every new commitment as provisional unless you demonstrate otherwise.

Make the business context visible

ML engineers are often working with one layer of abstraction between their work and the user. Connect their experiments to the business outcome explicitly. "This feature will reduce the average call handle time by X minutes, which is the top support team request." That context changes how they prioritize tradeoffs.

The real test

You know the relationship is working when your ML lead brings you a problem before it becomes a crisis, or tells you "we're not going to hit the quality bar and here's why" with two weeks to spare rather than two days before launch. That candor only happens when they trust you won't shoot the messenger.