State Space Models (SSMs), popularized by the Mamba architecture, offer a compelling alternative to Transformers for sequence modeling. SSMs replace the attention mechanism with a learned state-space dynamics model that compresses the entire sequence into a hidden state, achieving linear-time inference and theoretically unlimited context length.
Key Characteristics
Linear-Time Inference: Processes sequences in O(n) time rather than O(n^2), enabling very long context windows
Recurrent Formulation: Operates as a recurrent neural network at inference time for constant-memory generation
Selection Mechanism: Modern SSMs (Mamba) learn to selectively propagate or ignore information based on input content
Hardware-Aware Design: Optimized for GPU memory hierarchy with parallel scan algorithms
Popular Models
Mamba: The pioneering selective SSM that matches Transformer quality with linear-time inference
Mamba-2: Simplified architecture using state-space duality theory for improved throughput
Jamba: Hybrid architecture combining Mamba layers with Transformer attention layers
S4 (Structured State Space Sequence Model): Foundational SSM introducing structured state-space parameterization
Build This Pattern
Copy this prompt and paste it into Claude Code, OpenCode, Codex, or Cursor to implement this pattern.
Explain State Space Models (SSMs) as an alternative to transformer attention for sequence modeling. Architecture: describe the structured state-space formulation originating from control theory, where a continuous-time system is discretized into recurrence: h_t = Abar * h_{t-1} + Bbar * x_t, y_t = C * h_t. Explain how S4 (Structured State Space Sequences) initialized A using HiPPO matrices to enable long-range memory, and how this yielded the dual representation — SSMs can be computed either as a linear recurrence (O(n) per step, ideal for autoregressive inference) or as a global convolution (O(n log n) training via FFT). Then describe Mamba's key innovation: selective state spaces where parameters (A, B, C) become input-dependent rather than fixed, breaking the convolution-convolution duality. Explain the hardware-aware parallel scan algorithm that enables efficient training despite this selectivity, and how Mamba simplifies the architecture by integrating SSM recurrence with MLP blocks in a homogeneous stack. Compare to transformers: SSMs achieve O(n) time and O(1) memory per step versus O(n^2) for attention, with Mamba achieving 5x inference throughput over transformers at 16K sequence length. Reference hybrid models (Jamba, Nemotron-H, Falcon-H1R) that combine 90%+ SSM layers with sparse attention for best-of-both-worlds performance. Error handling: discuss the retrieval gap where fixed-size state compresses too aggressively for precise needle-in-a-haystack recall, mitigation through Mamba-2's structured state space duality connecting SSMs to linear attention, and Mamba-3's exponential-trapezoidal discretization improving precision at smaller state dimensions. Cover training instabilities in deep SSMs including recency bias versus over-smoothing at different layer depths, and the ongoing challenge of scaling recipes for very large SSMs. Edge cases: behavior on short sequences where transformer attention overhead is negligible and custom SSM kernels may be slower, handling sequences requiring perfect recall of distant tokens (SSMs trade recall for efficiency), interaction with positional encoding (Mamba eliminates explicit positional encodings unlike transformers), and variable-length batching challenges where recurrent state must be padded or masked. Best practices: include guidance on choosing hybrid SSM-attention ratios based on task requirements (more attention for retrieval-heavy tasks, more SSM for long-context throughput), implementing custom CUDA kernels for parallel scan operations, leveraging MIMO (multi-input multi-output) SSMs from Mamba-3 for improved retrieval without increasing state size, and using RoPE projections for complex-valued SSM transitions. Testing: suggest evaluating with long-context benchmarks at various sequence lengths (1K, 16K, 100K+), measuring inference throughput versus transformer baselines, testing retrieval capabilities with needle-in-a-haystack, and validating that hybrid models maintain quality on both short and long sequences.