Post-Transformer Architectures Explained for Product Managers

The transformer has dominated AI since 2017 because it works — astonishingly well at scale. But it carries a structural inefficiency that becomes a product constraint the moment you need long context windows, low-latency inference, or affordable per-token pricing at volume.

That inefficiency is quadratic attention complexity. Every attention operation computes relationships between every pair of tokens. Double the context, quadruple the compute. At 128K tokens, the attention computation alone requires roughly 16 times more work than at 32K tokens. This is why frontier-model providers charge more for long contexts and why latency spikes when users paste in long documents.

The product consequence: building features that require very long context windows is expensive today. The architectures below aim to collapse that cost curve. How fast they do so — and how much quality they sacrifice along the way — is what every AI PM needs to understand.

State Space Models (SSMs) are a class of sequence architectures rooted in control theory rather than neural network attention. Instead of computing explicit pairwise token relationships, an SSM maintains a compact hidden state that is updated as each token arrives. Where a transformer asks "how does token 1 relate to token 10,000?" and computes every pairwise score, an SSM asks "how do I update my running summary of what I've seen given this new token?" The information is carried forward in the hidden state rather than recomputed from scratch.

Mistral's Codestral Mamba (2024) was the first production-scale Mamba deployment, validating that SSM backbones can handle code generation at competitive quality with linear inference time. That release marked the moment SSMs moved from research curiosity to production-viable architecture.

The original Mamba paper (December 2023) introduced selective state spaces — the key innovation that addresses the core weakness of prior SSMs. Classical SSMs used fixed state-transition matrices, meaning the model updated its hidden state the same way regardless of the input. Mamba made the state transitions input-dependent: the model can learn to "ignore" irrelevant tokens and retain salient ones.

This selectivity is what closes the quality gap. A classic SSM processing a long document blends all information together equally. Mamba's selective scan can learn to retain "the defendant's name mentioned on page 1" while processing page 50 — approximating the behavior of attention without the quadratic cost.

On May 5, 2026, a Miami-based startup called Subquadratic launched SubQ with $29M in seed funding and a striking claim: the first fully subquadratic LLM — not a transformer with attention replaced, but a ground-up redesign of how language modeling works at the architecture level. The headline number: a native 12 million token context window with linear scaling.

For context, the frontier transformer models (Claude 3.5 at 200K, Gemini at 2M, Llama 4 Scout at 10M) represent the high end of what's viable with attention-based architectures. SubQ's 12M context is achieved with linear scaling — adding more tokens adds proportional, not exponential, compute.

SubQ's architecture is described as "sparse, subquadratic attention end to end" — even the attention-like operations are subquadratic by design, not just approximated. This is architecturally distinct from FlashAttention (which reduces memory footprint of standard attention) or linear attention approximations (which trade off quality for efficiency). Whether real-world quality holds across diverse workloads remains to be validated at scale, but the launch benchmarks are the strongest posted by any non-transformer architecture to date.

Most AI PMs won't choose architectures directly — you'll choose providers and models. But understanding the architectural trade-offs helps you ask the right questions when evaluating models, predict where limitations will appear, and anticipate the cost curves your product will face at scale.