Agentic Workflow Design Patterns: A PM's Guide to Multi-Step AI Automation
TL;DR
An agentic workflow is a sequence of AI-driven steps where each step can call tools, read and write state, and make decisions that affect subsequent steps. With OpenAI Agents API now generally available and Anthropic, Google, and AWS all shipping managed agent infrastructure, the pattern has crossed from research to production. But most teams shipping agentic features today are improvising the design. This guide covers the six foundational design patterns, where to place human-in-the-loop checkpoints, how to design for error recovery, and what state management looks like at each complexity level. Understanding these patterns is what separates an agentic feature that runs reliably from one that compounds errors until a user calls support.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
Single-Turn vs. Multi-Turn vs. Agentic: What Actually Changes
Before choosing a design pattern, make sure you actually need one. Agentic workflows carry real complexity costs. Here is how to think about the three tiers.
Single-turn
Definition: One prompt, one response. No tool calls, no state, no iteration. Works for: summarization, classification, extraction, generation with a fixed input.
PM note: If your feature can be solved with a single well-designed prompt, it should be. Agentic complexity is a cost, not a feature.
Multi-turn with memory
Definition: A conversation that builds on prior turns. State lives in the message history. Tool calls may be present but are triggered by explicit user requests, not by the model deciding independently.
PM note: Most chat assistants, copilots, and Q&A features fall here. The model is reactive, not proactive.
Agentic
Definition: The model decides what steps to take, calls tools without user prompting, and runs a loop that continues until a goal is met or a limit is hit. The model has agency over the sequence of actions.
PM note: Reserve for tasks where the right sequence of steps is not known in advance, where multiple tools need to be coordinated, or where the task duration exceeds a single inference call.
The Six Core Agentic Workflow Design Patterns
Every production agentic system uses one or more of these six patterns. They are not mutually exclusive. Complex workflows chain them together.
Pattern 1: Sequential chain
Steps execute in a fixed order. Output of step N becomes input to step N+1. No branching, no loops.
Example: Research query -> summarize sources -> draft a report section -> format for Slack
Use when: The task decomposition is known in advance and stable. Each step's output fully determines the next step.
Failure mode: A bad output in step 2 silently propagates through steps 3 and 4. Add validation checkpoints between steps for any chain longer than 3.
Pattern 2: Parallel fan-out
A planner step spawns multiple independent sub-agents that run concurrently. Results are aggregated.
Example: Competitor analysis: spawn one agent per competitor, each researches independently, results are merged into a comparison table
Use when: Tasks can be parallelized for speed. Sub-tasks are independent and their outputs are combinable.
Failure mode: One failed sub-agent can silently degrade the aggregation. Design the merge step to be explicit about missing inputs.
Pattern 3: Conditional branching
The model evaluates a condition and takes different next steps based on the result.
Example: Document router: classify incoming document -> if contract, route to contract review workflow; if invoice, route to AP automation
Use when: Different task types require different tool sets or different handling paths.
Failure mode: The model mis-classifies the condition. Always provide a fallback branch for the 'none of the above' case.
Pattern 4: Iterative refinement
The agent generates output, evaluates it against a criterion, and loops until the criterion is met or an iteration cap is hit.
Example: Code generation loop: write code -> run tests -> if tests fail, read error and revise code -> repeat
Use when: Output quality can be measured programmatically. The task has a definable 'done' state.
Failure mode: Infinite loops. Always set a hard iteration cap (5 to 10 for most use cases). Log every iteration so users can see the reasoning.
Pattern 5: Orchestrator with sub-agents
A planner model coordinates specialized sub-agents. Each sub-agent has a narrow tool set and a focused role.
Example: Sales pipeline agent: orchestrator delegates to 'research agent' (web search), 'CRM agent' (Salesforce read/write), 'email agent' (drafts outreach)
Use when: The task requires tool sets that would overload a single model context. Specialization improves quality and reduces context cost.
Failure mode: The orchestrator's planning quality determines overall quality. Test the planner with hard edge cases before optimizing sub-agents.
Pattern 6: Reactive event loop
The agent waits for an external event (webhook, schedule, user action), acts on it, and returns to waiting.
Example: Support ticket triage: new ticket arrives -> classify priority -> route to queue -> notify on-call if severity is critical
Use when: The agent's job is to respond to a stream of inputs rather than complete a single defined task.
Failure mode: Event ordering problems. If two events arrive simultaneously and both require writing to shared state, you need idempotency and locking.
Human-in-the-Loop Placement: Where to Put the Checkpoint
Human-in-the-loop (HITL) is not a binary choice between fully autonomous and always-ask-first. The decision is about which specific actions require human confirmation, not whether to include humans at all. Use this framework to place checkpoints correctly.
Irreversibility
Any action that cannot be undone in under 30 seconds requires a confirmation step. Sending an email, making a payment, deleting a record, or publishing to a public channel are all irreversible. Reading, drafting, and summarizing are not.
Dollar value
Set a threshold above which the agent pauses for human approval before spending. Most production deployments use $10 to $100 per action as the threshold. Wire this as a rule, not a judgment call for the model.
External communication
Any communication that goes outside your system to a third party (customer email, vendor request, regulatory filing) should have a review gate. Internal drafts do not need one.
Novel situation
Design the agent to recognize when it encounters a situation outside its training distribution and escalate rather than proceed. This is one of the hardest engineering problems in agentic systems, but a simple version is: if confidence is below threshold, pause.
First run of a new capability
For the first 50 to 100 runs of any new agentic feature, require human review of outputs even if the action is reversible. Use those runs to validate the model's decision quality before loosening the gate.
Audit requirements
Any domain with compliance obligations (financial services, healthcare, legal) should have human sign-off as a documented step, even if the underlying action is low-stakes. The audit trail, not the check itself, is the requirement.
Ship Agentic Features That Actually Work
The AI PM Masterclass includes hands-on modules on agentic system design, HITL architecture, and how to spec agentic features without surprising your engineering team. Taught live by a Salesforce Sr. Director PM.
Error Recovery Design: What to Do When a Step Fails
Agentic workflows fail in ways that single-turn AI does not. A single model error is a bad response. An agentic error can be a corrupted state, a partially executed action, or a compounding failure across six steps. Design error recovery explicitly, not as an afterthought.
Tool call failure
Retry with exponential backoff for transient failures (rate limits, timeouts). On hard failure (resource not found, permission denied), surface a structured error to the orchestrator with enough context to decide whether to abort, reroute, or escalate to a human.
Model reasoning error
Do not assume the model will self-correct. Build an external validator that checks key invariants after each model decision: did it choose a valid tool? Is the parameter format correct? Is the action within scope? Reject and retry on validation failure.
Partial execution
Design every multi-step action as a transaction: either all steps complete or none of them take effect. If a rollback is not possible (external system has already been written), record the partial state and route to a human with a clear description of what completed and what did not.
Context window overflow
Long-running agents can exhaust context. Summarize completed steps periodically and store them externally. Pass a summary of prior steps plus the current step into each model call rather than the full history.
Loop detection
Track the action history and detect when the agent is repeating the same action sequence (same tool, same parameters, same outcome). This is a stuck loop. Abort and surface to the user with the loop log.
State Management: Where Does the Agent Remember Things?
Single-turn AI is stateless. Agentic systems are not. Every agentic workflow requires decisions about what state to persist, where to store it, and how long to keep it. Get this wrong and your agent forgets what it did, re-does completed work, or makes contradictory decisions across runs.
In-context memory
Best for: Current run only, short-term reasoning
Bounded by context window. Suitable for 1 to 5 minute task runs. Disappears when the run ends.
Tradeoff: Zero engineering overhead. Complete loss of state on run end or failure.
External key-value store
Best for: User preferences, session state, intermediate results
No practical size limit. Survives run failures and restarts.
Tradeoff: Read/write latency. Requires cache invalidation strategy for frequently changing values.
Vector database
Best for: Semantic retrieval of prior runs, episodic memory, knowledge accumulation
Designed for retrieval by similarity, not exact key lookup.
Tradeoff: Retrieval is probabilistic, not deterministic. Great for 'find similar past experience'; bad for 'look up the status of task ID 12345'.
Relational database
Best for: Structured records, audit logs, multi-agent coordination
Requires schema design upfront.
Tradeoff: Strong consistency and queryability. Right choice for any state that needs to be audited or shared across multiple agent instances.
Testing and Observability for Agentic Workflows
Testing agentic workflows is harder than testing single-turn AI. The evaluation space is larger (more steps, more branching), the execution is non-deterministic, and failures can be subtle (wrong path taken, not just wrong output). Here is the minimum observability stack every agentic feature needs before launch.
Step-level trace logging
Log every model call: the input, the model used, the tool chosen, the parameters, the output, the latency, and the token cost. Without this, debugging a failed 8-step workflow is guesswork.
Goal completion rate
Did the agent reach the defined end state? Track this as the primary metric, segmented by task type. Separate it from user satisfaction, because an agent can technically complete a goal and still produce a useless output.
Step success rate by step
Break goal completion down by step to identify where the workflow is leaking. Step 3 might be failing 40 percent of the time even though overall completion looks fine because users are retrying.
Human escalation rate
Track how often the agent hits a human-in-the-loop checkpoint. A sudden spike means the model is encountering a new class of inputs it is not handling. A drop to zero means checkpoints may be placed wrong.
End-to-end latency distribution
P50, P90, and P99 for the full workflow. Agentic workflows can have long-tail latency from iterative patterns or slow external tools. Know your latency distribution before users surface it in support tickets.
Adversarial test cases
Build a set of test inputs specifically designed to trigger the known failure modes: ambiguous inputs, missing required context, external tool errors, contradictory instructions. Run these on every deploy.
Go Deeper on Agentic Product Engineering
The AI PM Masterclass covers agentic system design, spec writing for multi-step AI features, and how to work with engineering teams building production agent infrastructure. Instructor led, live sessions, with a hands-on build.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.