AI Quality Engineering: The PM's Strategic Guide to Shipping with Confidence

Why AI Products Break Traditional QA

Classical software testing assumes determinism: given the same input, the system produces the same output. You can write a test, run it, and get a binary pass or fail. AI products violate this assumption at every layer. The same prompt to an LLM produces different outputs on successive calls. A model update from your API provider changes behavior across your entire product without you deploying a single line of code. The edge case that fails today might work tomorrow after a prompt tweak, and vice versa.

The consequence: teams that import their existing QA process into AI product development find that tests pass green while user-facing quality silently degrades. Forty percent of AI product teams discover quality regressions from user complaints rather than internal testing, according to a 2026 survey by Tricentis. By the time the complaint reaches the PM, the damage is done.

Non-determinism

The same input produces different outputs. Unit tests for deterministic outputs don't apply. You need distribution-level testing: does the distribution of outputs stay within acceptable bounds?

Silent model updates

API providers update base models without versioning guarantees. A behavior you relied on may change without warning. You won't catch it unless you have continuous eval running against production.

Long-tail failure modes

LLMs fail on edge cases that don't appear in standard test suites. A user input you never anticipated can produce a harmful or absurd output. Coverage must be designed, not assumed.

Evaluation is hard

You can't always write an assertion. 'Is this summary good?' requires a judge, not a string comparison. AI-based evaluation introduces its own accuracy and cost considerations.

Quality vs. cost tradeoffs

More quality often means more expensive models or more inference calls. A quality strategy that ignores cost ships products with unsustainable unit economics.

The Three Quality Layers Every AI Product Needs

A complete quality strategy for an AI product covers three distinct layers. Missing any one of them creates blind spots that reach users.

Layer 1: Infrastructure and Reliability

What it covers: Latency, uptime, error rates, and cost per call. These are standard APM metrics but they matter more for AI products because model calls are slow and expensive. A 5% error rate on a standard API is a crisis; on an LLM call chain, it compounds across steps.

PM action: Set SLOs for P95 and P99 latency by feature. Define cost-per-successful-task targets. Set up alerts for model provider error rates, not just your own infrastructure.

Layer 2: Model Output Quality

What it covers: Does the model produce outputs that meet your quality bar? This includes factual accuracy, format adherence, tone, safety, and task completion. This layer is the hardest to measure because it often requires human or AI-based judgment.

PM action: Define a golden set of 50 to 200 test inputs with expected output characteristics (not exact strings). Run this eval on every prompt change and on a schedule to catch silent model drift. Use an LLM judge for dimensions like tone and helpfulness.

Layer 3: Product Behavior and User Outcomes

What it covers: Does the product do what users need it to do, and does that outcome match what the business needs? A model that produces technically correct outputs can still fail to move user behavior in the right direction.

PM action: Instrument user actions after every AI feature interaction: did they accept the output, edit it, discard it? Track edit distance between AI output and user's final version. High edit rates signal low utility even when model quality metrics look fine.

Shift-Left Quality for AI: Catching Problems Before They Ship

"Shift left" means moving quality work earlier in the development cycle. For traditional software, that means writing tests before code (TDD) and running them in CI. For AI products, it means building quality gates into the prompt engineering, model selection, and eval design stages, not just the post-launch monitoring phase.

Define failure modes before writing prompts

Before writing a single prompt, enumerate the five most dangerous failure modes for this feature. Hallucination on medical claims? Tone that sounds dismissive? Leaking system prompt content? Each failure mode should have a corresponding eval case before the first prompt draft.

Red-team during prompt review, not after launch

Add an adversarial review step to your prompt review process. Have one engineer or PM spend 30 minutes trying to break the feature with edge inputs. Inputs that produce bad outputs at review are cheap to fix; the same bugs in production are reputational damage.

Gate on eval scores before merge

Treat prompt changes like code changes: they go through review, they run evals in CI, and they don't merge if they regress the golden set score below your threshold. Set the threshold conservatively at first (e.g., no more than 5% regression on any dimension) and tighten over time.

Model selection is a quality decision

The decision to use GPT-4.1 vs. Claude Sonnet 5 vs. a fine-tuned open-weight model is a quality decision as much as a cost decision. Run your eval set on candidate models before committing to one. Document the quality-cost curve so future model selection decisions have a baseline.

Build Your AI Quality Playbook in the Masterclass

The AI PM Masterclass covers eval design, quality strategy, and production monitoring for AI products, taught live by a Salesforce Sr. Director PM.

Coverage Planning for Non-Deterministic Systems

In classical software, you can achieve 100% branch coverage and know you tested every path. AI products don't have enumerable paths. A language model can be prompted in an infinite number of ways. So coverage planning for AI is about taxonomies, not exhaustive enumeration.

1. Build an input taxonomy

Categorize the inputs your feature will receive into 8 to 12 buckets. For a customer support AI: simple questions, complex multi-part questions, questions with missing context, abusive inputs, questions in other languages, requests that are out of scope. Ensure you have test cases in every bucket.

2. Identify severity tiers

Not all quality failures are equal. Classify failures as Critical (user harm, regulatory risk, severe reputation damage), Major (task not completed, user confused), and Minor (suboptimal phrasing, extra verbosity). Allocate testing depth by severity: exhaustive for Critical, thorough for Major, sampled for Minor.

3. Build and maintain a golden set

A golden set is a curated collection of 50 to 300 inputs where you know what 'good' looks like. It doesn't have to be exact expected outputs. It can be rubric-based: does the output answer the question, stay on topic, avoid harmful content? Maintain this set as the product evolves.

4. Add real failure cases

Every production failure you catch should be added to your golden set as a regression test. This is the highest-leverage coverage activity over time: the golden set grows by learning from real problems, not just hypothetical ones.

5. Schedule blind spot audits

Quarterly, review your input taxonomy and ask: what inputs are we probably not testing for? Common blind spots include very long inputs, inputs mixing languages, inputs that start coherently and become incoherent mid-message, and inputs that probe your system prompt.

The Five Metrics That Tell You Quality Is Actually Improving

Most AI teams track model-level metrics that don't connect to business outcomes. These five metrics span all three quality layers and give PMs a meaningful signal on whether the product is getting better or worse.

Eval pass rate (trended)

The percentage of your golden set inputs that pass your quality rubric, tracked week over week. The trend matters more than the absolute number. A score that stays flat while you're shipping new features is a warning sign: you're adding complexity without adding quality.

User acceptance rate

What percentage of AI outputs do users accept, edit, or discard? For a writing assistant: did they use the suggested paragraph, modify it, or delete it? High discard rates signal the model is generating technically correct but practically useless output.

Edit distance on accepted outputs

For outputs that users do use, how much do they change? Low edit distance means the output is landing close to what users want. Rising edit distance signals quality drift: the model is producing outputs that technically address the prompt but miss user intent.

Escalation and retry rate

What fraction of AI tasks result in users retrying the action or escalating to a human? In a support bot, how often do users say 'talk to a human'? Tracking this against the baseline tells you whether new deployments are making the product more or less self-sufficient.

Cost per successful task

What does it cost in model inference to complete a task that the user actually accepted? This ties quality to economics: a team that improves eval pass rate but increases per-task inference cost by 50% has traded one problem for another.

The PM's Role in Quality Engineering Sprints

Quality engineering for AI is not purely an engineering function. PMs who hand quality ownership entirely to engineering produce AI products that optimize for technical metrics at the expense of user outcomes. The PM's job is to make quality a shared discipline with clear ownership across the team.

What PMs should own specifically

Quality definitions: You define what "good" means for each AI feature. Engineers can implement the eval, but they can't decide what quality looks like for a user-facing product. Write the rubric.
Golden set curation: PMs are best positioned to identify the inputs that matter most to users. Maintain a living document of representative inputs, prioritized by frequency and business criticality.
Failure triage priority: When the eval catches failures, the PM decides which ones to fix in the current sprint. Not every failure is equal. Apply the severity taxonomy you defined.
User feedback loop design: Every AI feature ships with a mechanism to capture user feedback on quality (thumbs, stars, edit detection). PMs spec this as a first-class requirement, not an afterthought.
Quality OKRs: Set quality targets at the OKR level so they have weight against feature velocity. "Increase eval pass rate from 82% to 90% in Q3" is a real objective that competes for sprint capacity the same way a feature does.

The team that gets quality right in AI products is the team where the PM and engineering lead do a weekly 30-minute quality review together. Not to assign blame but to read the signals: what did the eval catch this week, what did users flag, and what do those signals tell us about what to prioritize? That cadence, more than any specific tool or framework, is what separates AI products that users trust from those that feel inconsistent.