Designing for the Wait: UX Patterns for Reasoning Model Latency
TL;DR
Reasoning models like o3, Claude 3.7 Sonnet, and Gemini 2.5 Pro routinely take 30 to 90 seconds to respond on complex tasks. That is not a bug to fix with faster inference. It is the cost of the accuracy that makes these models worth using. The product design problem is different from general latency optimization: you are not trying to make the wait shorter, you are trying to make the wait acceptable, informative, and trust-building. This guide covers the UX patterns, streaming strategies, and architecture decisions that determine whether your users stay or abandon during those 60 seconds.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
Why Reasoning Latency Is a New Design Problem
Standard LLM latency and reasoning model latency require different product responses. For a standard model returning a first token in under a second and a full response in 3 to 5 seconds, the solution is streaming: start outputting tokens immediately and the experience feels fast. For a reasoning model that spends 45 seconds processing before producing any output, streaming the final answer does not solve the user's experience during those 45 seconds of silence.
This distinction matters for product decisions. The conventional wisdom in AI UX is "stream everything as fast as possible." For reasoning models, that advice addresses the wrong half of the experience. The harder half is the extended thinking period, and it requires a different set of design primitives.
Standard LLM latency (1 to 5 seconds)
Challenge: Time to first token. User sees a blank screen briefly.
Design response: Streaming. Token-by-token output makes the experience feel fast and continuous. First token in under 500ms is the target.
Reasoning model latency (15 to 90 seconds)
Challenge: Extended thinking period before any output. User cannot tell if the model is working, hung, or slow.
Design response: Not just streaming. You need active signals during the thinking period: progress indicators, partial reasoning summaries, or intermediate state updates that prove the system is alive and working.
The practical threshold: once your median response time exceeds 8 seconds, users start to abandon at rates that materially affect engagement metrics. At 30 seconds, you need an active strategy for the wait, not just faster infrastructure.
The Abandonment Problem: What Users Do During Long AI Waits
User behavior during extended AI waits follows a predictable pattern. Understanding it is the foundation for designing against it.
0 to 3 seconds
Users wait with no significant anxiety. The UX equivalent of a normal page load. No special design required.
3 to 8 seconds
Users begin scanning for visual feedback. A simple spinner is enough. Users check their input to see if they sent the right thing.
8 to 20 seconds
The critical zone. Users begin to doubt: is the system working? Did my request go through? Without active feedback, 20 to 35% of users abandon in this window depending on the task type.
20 to 60 seconds
Users who have not abandoned become invested. They switch to another tab or continue other work, expecting the task to be done when they return. Progress signals matter here: they need to know when to come back.
60+ seconds
Users who stay are highly task-committed. The experience now needs to feel like a background job with a clear completion event, not like a hanging request.
Design principle
The 8 to 20 second window is where most abandonment happens and where design has the most leverage. A spinner is not enough. You need a signal that communicates specifically that the model is still working and why it is taking this long.
The UX Pattern Library for Reasoning Model Latency
Five distinct UX patterns address reasoning latency. They are not mutually exclusive. The right combination depends on your task type, user context, and how much of the reasoning chain you want to expose.
Thinking state indicator
Best for: All tasks over 8 seconds. The baseline pattern.
How it works: A visible state change that says the model is working: animated dots, a pulsing ring, a filling progress bar, or an explicit text label such as 'Thinking...' or 'Analyzing your request'. The indicator must be clearly different from a page loading state so users understand the system is active, not stuck.
Watch out for: A spinner alone is the minimum viable response, not the goal. Without additional signals, it does not convey progress or completion timing.
Reasoning trace summary
Best for: Tasks where showing intermediate reasoning increases user trust and output quality judgment.
How it works: Surface a condensed, readable version of the model's reasoning chain in real time as it processes. Not the raw chain of thought (too noisy) but a curated summary: 'Reviewing the three contracts you uploaded... Comparing clause 7.2 across documents... Identifying conflicts in termination language...' This pattern is used effectively by Perplexity, Claude.ai in extended thinking mode, and legal AI tools.
Watch out for: Users who see reasoning traces develop more calibrated trust. They also become better at catching errors. This is a feature, not a risk, but it requires that your reasoning trace actually reflects the model's work, not a scripted animation.
Chunked delivery
Best for: Long responses that can be structured into discrete sections.
How it works: Structure the output so the model streams completed sections rather than token by token. The user sees Section 1 appear complete, then a brief thinking state, then Section 2 appears complete. This creates a sense of progress and lets users read completed content while the next section processes.
Watch out for: Requires prompt engineering and output structure discipline. Works better for document-like outputs than conversational responses.
Async background job
Best for: Tasks over 60 seconds, or tasks where the user does not need to watch the model work.
How it works: Treat the request as a background job with a completion notification. The user submits, receives a job ID, gets a notification or badge count when done, and returns to results. Email generation, long document analysis, and batch operations work well in this pattern.
Watch out for: Users need a clear contract: 'This will take 2 to 3 minutes. We will notify you when it is done.' Vague time estimates produce anxiety. No estimate at all produces abandonment.
Confidence-gated streaming
Best for: Tasks where partial output could mislead if the model changes direction mid-reasoning.
How it works: Buffer output until the model reaches a confidence threshold in its reasoning direction, then stream the rest. The user sees a longer thinking state followed by fast, confident output rather than watching the model backtrack in real time.
Watch out for: Requires inference infrastructure that supports buffered streaming with confidence signals. Available in some model APIs but not all.
Design AI Products That Users Actually Trust
The AI PM Masterclass covers AI UX design, reasoning model tradeoffs, and how to make architecture decisions that translate into better products, taught live by a Salesforce Sr. Director PM.
Architecture Decisions That Affect the Wait Experience
Some latency UX problems are design problems. Others are architecture problems. The PM needs to know the difference before going to engineering with requests.
Streaming vs. buffered response delivery
PM implication: Streaming (tokens delivered as generated) is the default for most model APIs and is almost always correct for standard models. For reasoning models, check whether the API supports streaming the thinking trace separately from the final answer. Anthropic's extended thinking API does. OpenAI's o-series returns the reasoning summary after processing. The choice affects your thinking-state UI options significantly.
Client-side vs. server-side timeout handling
PM implication: Most clients have a default timeout of 30 to 60 seconds. Reasoning model responses routinely exceed this. Your engineering team needs to configure appropriate timeouts for AI reasoning endpoints, often 120 to 300 seconds, and handle the timeout gracefully if it is exceeded. Users should never see a generic error when a reasoning model just took longer than expected.
Synchronous vs. asynchronous architecture
PM implication: Synchronous architectures block the user's browser or app thread while waiting for a response. For reasoning model tasks over 30 seconds, this often causes connection issues, browser tab freezes, and mobile app backgrounding. Async architectures with a job queue and push notification on completion handle long reasoning tasks more reliably. The tradeoff is implementation complexity and a less immediate feel.
Intermediate state persistence
PM implication: For tasks over 60 seconds, if the user's connection drops or they accidentally close the tab, the reasoning work in progress is typically lost. Building a persistence layer that checkpoints reasoning state and allows resumption is expensive but dramatically improves user experience for high-value long-running tasks.
Model routing for latency tiers
PM implication: Not every task needs a reasoning model. Implement a routing layer that sends tasks requiring deep analysis to reasoning models and routes simpler tasks to faster standard models. The user experience can dynamically calibrate: routine tasks respond in seconds, complex tasks are correctly framed as a longer background operation. This requires task complexity classification, which is itself a product and engineering challenge.
When to Show the Reasoning Chain and When Not To
One of the most consequential product decisions for reasoning model features is how much of the model's thinking process to expose to the user. There is no universal right answer. The decision depends on your users, your task type, and what kind of trust relationship you are building.
Show reasoning when the task is high-stakes
Legal, medical, financial, and compliance tasks benefit from visible reasoning. Users who can see how a conclusion was reached are better able to catch errors before acting on the output. The accuracy improvement from user review outweighs the added complexity.
Hide reasoning when speed perception matters
For conversational tasks and quick questions, showing a 45-second reasoning trace before a 3-sentence answer trains users to expect slowness for simple tasks. Route these to faster models or buffer the reasoning and show only the final answer.
Show reasoning when you want power user adoption
Users who can see the model's reasoning chain develop more sophisticated understanding of what it can do. They write better prompts, judge outputs more accurately, and become product advocates. Consider reasoning trace access as a feature tier rather than a default.
Hide reasoning when the audience is non-technical
Raw reasoning traces (especially from o-series models) are verbose, technical, and sometimes alarming in how they explore wrong answers before correcting. Non-technical users who see this process often lose confidence in the model even when the final answer is correct. Use summarized thinking states instead.
Show partial reasoning as a trust signal during the wait
Even if you hide the full chain of thought, surfacing a high-level summary of what the model is doing during the wait reduces abandonment significantly. 'Reviewing your contract... Checking clause precedents... Drafting your analysis...' communicates competence without exposing the full reasoning overhead.
Test both before committing
Run an A/B test with and without reasoning trace visibility on your specific user population and task type. The research on this is not uniform. Consumer users generally prefer faster-appearing results. Enterprise users doing analytical work generally prefer visible reasoning. Your users may not match the average.
How to Test Your Wait-State Design
Most product teams test AI output quality but not the experience of waiting for that output. These are different things. A user can receive a perfect answer and still have a poor experience if the wait was badly designed. The following tests belong in your QA and user research process for any reasoning model feature:
Artificial latency user testing
Add artificial delays to your staging environment to simulate 30, 60, and 90 second response times. Run user sessions with think-aloud protocol. Record when users verbalize doubt, move away from the task, or attempt to retry. These are your design failure points.
Abandonment rate measurement by latency bucket
Instrument your production analytics to measure task abandonment rate segmented by actual response time. Track P50, P90, and P99 response times for reasoning model endpoints. If abandonment spikes after a certain latency threshold, that is your design intervention priority.
A/B test thinking state variants
Test your spinner vs. a reasoning trace summary vs. a progress bar with estimated time remaining. Measure task completion rate, not just abandonment. Users who see verbose reasoning may complete the task less often if the reasoning content itself increases their uncertainty.
Return-to-tab testing for async patterns
For tasks routed to async background processing, measure how often users return to the completed result vs. abandon the session entirely after submission. A low return rate indicates your completion notification is not reaching users effectively.
Trust calibration survey after wait states
After users complete a reasoning model task, ask: 'How confident are you that this output is accurate?' Compare confidence between users who saw a reasoning trace and users who saw only a thinking indicator. Calibrated confidence (neither over-confident nor under-confident) is the goal.
Turn Technical Depth Into Better Product Decisions
The AI PM Masterclass bridges model architecture, UX design, and product strategy. Learn how the technical choices made at inference time become the user experience your customers live with.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.