AI Strategy for Spatial Computing: Building for Apple Vision Pro and the XR Stack
TL;DR
Spatial computing (XR: AR, VR, and mixed reality devices) is not the same as the VR hype cycle of 2016. Apple Vision Pro, Meta Quest 3, and the emerging XR developer ecosystem have put AI at the center of the platform: computer vision for scene understanding, on-device inference for low-latency interaction, and multimodal input as the default interface. AI PMs who understand this platform shift now are positioning themselves for the next major product surface.
The AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.
What Spatial Computing AI Actually Is
Spatial computing refers to systems that perceive and interact with the three-dimensional physical world rather than a flat 2D screen. In a spatial computing device, your environment becomes the canvas. Digital objects can be placed on physical surfaces, instructions can appear in your field of view while your hands are occupied, and AI can understand not just what you say but what you are looking at, pointing to, and physically doing.
This shifts AI from a conversational interface to a contextual, ambient one. The primary input is not typing; it is gaze, gesture, voice, and spatial context. The primary output is not text on a screen; it is overlaid information in the environment, spatial audio, and haptic feedback.
Computer Vision
The AI capability that makes spatial computing useful. Real-time scene understanding: what objects are in the environment, what surfaces are available, what the user is looking at. On Apple Vision Pro, Persona (real-time facial expression capture) and environment understanding run continuously via dedicated neural engines. This is AI at the OS layer, not the app layer.
On-Device Inference
Network latency is incompatible with real-time spatial interaction. A 200ms delay between looking at an object and getting information about it breaks the immersive experience. The spatial computing platforms that win are the ones with the most capable on-device AI silicon: Apple's M-series chips, Qualcomm's XR platforms, and Meta's custom silicon in Quest Pro.
Multimodal Input Understanding
Spatial AI must fuse gaze (eye tracking, which indicates intent and attention), voice (primary text input in a glasses-form-factor world), gesture (precision object manipulation, navigation), and spatial context (where the user is, what is around them). No single-modality model is sufficient; the platform must route correctly across them.
Semantic Scene Understanding
Beyond 'there is a table here,' advanced spatial AI understands what is happening in a scene: 'this person is working on a laptop and is looking at the document in the upper left of their screen.' This enables AI assistance that is contextually appropriate rather than requiring the user to describe their context from scratch.
The AI Capabilities That Power Spatial Products
Building a spatial AI product is not just about porting a mobile app to a headset. The AI requirements are fundamentally different. Here is the capability stack that determines what you can and cannot build on current XR platforms.
Real-time object recognition
The ability to identify objects in the user's environment as they move through space. This enables: 'show me the manual for this machine,' product lookup by pointing, and AI assistance that understands physical context. Current state in 2026: reliable for known object categories, still inconsistent for novel or occluded objects.
Apple Vision Pro (visionOS ARKit), Meta Quest (Presence Platform), Google Glass Enterprise Edition.
Gaze-driven intent inference
Eye tracking in XR devices creates a rich intent signal. What you look at for more than 200ms is usually what you care about. AI can use this to preload relevant information before you ask for it, rank menu options by visual attention, and eliminate the 'selection problem' in hands-free contexts. Privacy consideration: gaze data is deeply personal.
Apple Vision Pro (eye tracking used for select), Meta Quest Pro (eye tracking), Pico 4 Enterprise.
Voice as primary input
In form factors where typing is impossible (AR glasses, VR headsets), voice becomes the primary text and command input. The quality of on-device speech recognition and LLM integration determines the UX ceiling. Low-latency, privacy-preserving voice AI is a key differentiator for spatial products.
All major XR platforms integrate voice through OS-level APIs. App developers access this via ARKit, OpenXR, or platform SDKs.
Spatial audio and voice output
Output in spatial computing is not a screen; it is positional audio, visual overlays, and haptic feedback. AI-generated content must be adapted for these modalities: responses should be shorter, more conversational, and anchored to objects in the environment rather than presented as floating text blocks.
Apple visionOS Spatial Audio APIs, Meta Presence Platform audio APIs.
Product Opportunities in XR AI
The spatial computing product space is early enough that clear patterns of what works are only now emerging. The highest-traction categories in 2026 share a common characteristic: they replace or enhance something that is genuinely difficult on a phone or laptop because the relevant context is in the physical world.
Industrial and field service AI
Most mature. Apple Vision Pro and Meta Quest Enterprise are already deployed in manufacturing, healthcare, and field service. This is the most immediate enterprise AI PM opportunity in spatial.Technicians repairing complex equipment, surgeons reviewing imaging during procedures, field engineers inspecting infrastructure. AI overlays on the physical object they are working on, hands-free voice interaction with technical documentation, and real-time AI guidance reduce errors and speed workflows.
Spatial meeting and collaboration AI
Early to mid. Apple Persona and spatial FaceTime show what is possible. Enterprise productivity suites are beginning to ship spatial versions. The AI layer (contextual meeting intelligence) is largely unbuilt.Real-time language translation overlaid on speakers' faces, AI meeting notes and action items in your peripheral vision, spatial whiteboards with AI-generated content. Remote teams collaborating in shared virtual spaces with AI that understands both the conversation and the shared artifacts.
AI-powered training and simulation
Early. Military and aviation already use VR training at scale. AI observation and adaptive feedback within simulations is the next layer being built.Medical training, safety training, complex skills training that requires hands-on practice. AI that observes the trainee's physical actions, provides real-time feedback on technique, and adapts the simulation difficulty. Safer than real environments, cheaper than physical simulators.
Spatial retail and commerce
Mid. Shopify, IKEA, and major apparel brands have AR try-on. AI personal shopping within the spatial experience is largely unrealized.Try-before-you-buy for furniture, clothing, and products in your actual space. AI personal shopping assistant that understands your style preferences, room dimensions, and budget simultaneously. Reduced returns are the business model: a 2024 study showed 40 percent lower return rates for furniture bought through AR try-on.
Build Strategy for the Next Platform Shift
The AI PM Masterclass covers emerging AI platforms, on-device AI, and how to build strategy for products that are not yet mainstream. Taught live by a Salesforce Sr. Director PM.
The Unique Challenges of Spatial AI Products
Spatial computing AI products fail in ways that are different from standard AI products, and most of those failure modes are not obvious until you have shipped something. Here are the constraints that experienced spatial AI PMs treat as first principles.
Latency tolerance is near zero
A conversational AI that takes two seconds to respond is mildly annoying. A spatial AI that takes two seconds to overlay information when you look at something is unusable. The tolerance for perceptible AI latency in spatial experiences is under 100ms for visual overlays and under 300ms for voice responses. This means on-device inference is not optional for most interactions.
Physical comfort and safety are PM concerns
AI that causes excessive eye movement, places overlays in dangerous visual fields, or generates content that obscures physical hazards has real-world consequences. Safety testing for spatial AI must include physical context (are users looking away from traffic? Are they distracted during medical procedures?), not just output accuracy.
Battery and thermal constraints limit AI compute
Current XR headsets have 2 to 3 hour battery life under normal use. Running continuous AI inference at full capacity can cut that in half. PMs must be involved in the AI inference budget: how much compute runs continuously vs. on-demand, which models run on-device vs. in the cloud, and what the thermal ceiling is before the device becomes uncomfortable to wear.
Privacy is structurally more sensitive
A spatial computing device sees your home, your physical environment, and your face. The AI capabilities (gaze tracking, scene understanding, person recognition) are surveillance-grade by default. The trust bar for users is higher than for any prior device category. Privacy-by-design is not a feature; it is a prerequisite for adoption in consumer and enterprise markets.
User research methodology needs to change
Standard user research (sit down, look at a screen, tell me what you think) does not transfer. Spatial computing requires in-context research: users wearing the device in their actual work environment, performing real tasks. Recruiting is harder, sessions are shorter (physical fatigue), and your standard analytics dashboard does not capture the spatial interactions that matter.
Platform Strategy: Apple vs. Meta vs. Open Standards
Spatial computing in 2026 has two dominant platforms, one open standard, and a group of enterprise-specific hardware players. Your platform choice determines your AI capability constraints, your distribution options, and your competitive exposure.
Apple visionOS (Vision Pro)
AI Strengths
Best-in-class on-device AI silicon (M4 + R1 chip), deeply integrated eye and hand tracking, Apple Intelligence (on-device LLM), and the Persona framework for avatar-based collaboration.
Distribution
App Store distribution, $3,499 device price. Premium consumer and enterprise. Current install base small but growing. Developers earn outsized revenue per user.
Strategy: Build for the premium enterprise segment. The Vision Pro user is a professional who paid $3,499; they expect and will pay for productivity-grade AI tools. Avoid consumer entertainment plays until the price point drops significantly.
Meta Quest (Horizon OS)
AI Strengths
More mature developer ecosystem, broader install base, Meta AI integration (Llama models on-device and cloud), and Presence Platform for mixed reality scene understanding.
Distribution
Meta Quest Store, lower device prices ($299-$999). Larger consumer install base. More diverse content types including games, fitness, and social.
Strategy: Consumer and prosumer plays work here. Training, fitness, social experiences, and creative tools. Enterprise deployments possible but the ecosystem is less polished than Apple.
OpenXR (open standard)
AI Strengths
Khronos Group OpenXR standard enables apps to run across platforms without rewriting. Reduces lock-in but also means you cannot use platform-specific AI APIs that are often the most capable.
Distribution
Works across Quest, Pico, Valve Index, and enterprise headsets. Essential for enterprise B2B where customers have heterogeneous hardware.
Strategy: If your customer base spans multiple hardware types (typical in enterprise), build on OpenXR from day one. Accept that you will sacrifice some platform-specific AI capabilities for portability.
Enterprise hardware (HoloLens, RealWear, Vuzix)
AI Strengths
Purpose-built for specific industrial environments: dust-resistant, certified for hazardous locations, integrated with enterprise systems (SAP, Salesforce). AI capabilities are typically more limited but deployment contexts are well-defined.
Distribution
Direct enterprise sales. Long sales cycles, high ACV. Integration with existing enterprise software stacks is the key PM challenge.
Strategy: Deep vertical integration. The product wins not because the AI is the best but because it integrates with the specific workflows, safety systems, and enterprise software that the customer already runs.
PM Playbook: How to Get Started
Most AI PMs approach spatial computing the same way they approached mobile in 2010: by waiting until it becomes obvious. That is a valid strategy for companies that cannot afford early bets. But for individual PMs and teams building for the long term, the investment to develop spatial computing fluency now is low relative to the potential value.
Get hands-on with a device
If you have not used Apple Vision Pro or Meta Quest 3 for more than a few hours, you do not have a mental model of what spatial computing products actually feel like to use. Rent one for a weekend through Humbird or Meta's trial program. Build one simple app, even a hello-world spatial experience, before forming opinions about the platform.
Identify the spatial problem in your domain
Ask: what workflow in my target market requires the user to look at a physical object, move through a physical space, or use both hands simultaneously? Those workflows are where spatial computing has an unfair advantage over a phone or laptop. Start your product thesis there, not from 'what can I build in XR.'
Evaluate on-device AI constraints early
Whatever AI capability you are planning to use, validate on-device performance before committing to your architecture. A feature that requires 300ms of cloud inference may need to be redesigned as a progressive disclosure pattern, or replaced with a simpler on-device model that runs in 50ms.
Build a spatial-specific eval suite
Standard AI product evals (accuracy, latency, cost) miss the spatial-specific failure modes: overlay placement errors, gaze inference misfires, voice recognition failures in noisy environments, and performance degradation during sustained wear. Build an eval framework that captures the physical context of your product.
Follow the developer ecosystem, not the consumer press
Consumer press coverage of XR is still dominated by hype and skepticism cycles. The developer ecosystem (visionOS developer forums, Meta Horizon OS developer blog, XR Association research) shows what is actually being built and where the real constraints are. That is where to calibrate your product bets.
Build AI Products for the Next Platform Wave
The AI PM Masterclass gives you the technical depth and strategic frameworks to build AI products on emerging platforms, not just today's mainstream stack.
Related Articles
Before you go: get the AI PM Minute
One tactic to make you a sharper AI PM, twice a week. 60 seconds to read. Free.
No fluff. Unsubscribe anytime.