The old recurrent promise was beautiful:

read one token
update a fixed-size state
repeat

No growing key-value cache. No quadratic attention matrix. Just a state moving through time.

The old recurrent disappointment was also beautiful, in the tragic sense:

the state has to remember everything

If the model compresses the past too aggressively, a fact disappears before the query arrives. If it updates too eagerly, distractors overwrite the thing that mattered. Transformers avoided that bargain by storing many past vectors and letting the query look back. That is powerful, but the stored context grows with sequence length.

Mamba-style state-space models reopen the recurrent bargain with a small but important change:

the memory chooses its decay

The coefficients of the recurrence are allowed to depend on the current token. That is the “selective” in selective state-space model.

The Recurrence

A simple discrete state-space model looks like:

\[h_t = A h_{t-1} + B x_t,\] \[y_t = C h_t.\]

If \(A\), \(B\), and \(C\) are fixed through time, the model is linear time-invariant. That case has a useful second life: it can be evaluated as a convolution. This is one reason structured state-space models such as S4 became interesting for long sequences.1

But language is not time-invariant.

The token the should usually not overwrite a name. A closing parenthesis may need to interact with a previous opening parenthesis. A user ID might deserve to survive hundreds of tokens; a filler adjective might not. A fixed decay schedule is too polite. It treats the stream as if importance were stationary.

Mamba’s key move is to make the SSM parameters input-dependent. In the original Mamba paper, Gu and Dao describe selectivity as letting the model filter out irrelevant information and remember relevant information, while still using a hardware-aware scan rather than materializing a huge state tensor.2

For a scalar toy version, write:

\[h_t = a_t h_{t-1} + b_t x_t.\]

When \(a_t\) is near 1 and \(b_t\) is near 0, the model mostly carries memory forward. When \(a_t\) is small and \(b_t\) is large, the model overwrites memory with the current token.

That is the whole personality change.

A Selective Scan Lab

The lab below creates a toy copy task. There is a marked value in the sequence, a long run of distractor tokens, and a query near the end. A fixed-decay recurrence updates at the same rate for nearly every token. A selective recurrence uses a large update gate on marked tokens and a tiny update gate on distractors.

The default run has 128 tokens. The target is at position 70; the query is at position 122; the gap is 52 tokens. The fixed recurrence misses the target by about 0.951. The selective recurrence misses by about 0.040. Both paths are also recomputed through an associative scan of affine maps, and the default scan mismatch is exactly 0.

Deterministic browser experiment. The toy recurrence is scalar so the gates and scan are inspectable; real selective SSMs use learned projections, higher-dimensional states, fused kernels, and trained readouts.

Move Selectivity down.

The green trajectory starts behaving like the purple one. It writes too weakly when the marked value appears and lets distractors leak into the state. Move Query gap up. A fixed half-life becomes painful: the value is not wrong because the query is hard; it is wrong because the state followed a clock.

The bottom-right panel is the computation trick. Each step

\[h \mapsto a_t h + b_t x_t\]

is an affine map. Represent it as a pair:

\[(a_t, u_t), \quad u_t=b_t x_t.\]

Composing two maps is associative:

\[(a_2,u_2) \circ (a_1,u_1) = (a_2 a_1,\; a_2 u_1 + u_2).\]

That means the recurrence can be parallelized with a scan. The model is recurrent in meaning, but the training-time computation does not have to be a one-token-at-a-time loop. Blelloch’s prefix-sum work is the classical reference for this kind of all-prefix operation.3

The lab’s scan is deliberately tiny. Mamba’s implementation problem is the serious version: avoid writing expanded states through slow memory, fuse the discretization, scan, and output projection, and recompute intermediate states when useful. The public selective_scan_ref in the Mamba repository shows the plain recurrence; the optimized path is built to make that recurrence fast on modern accelerators.4

The Mask Hidden Inside The Recurrence

Unroll the scalar recurrence:

\[h_t = \sum_{j \le t} b_j x_j \left(\prod_{k=j+1}^{t} a_k\right).\]

The formatting hides a simple object: every earlier token contributes to the current state with a product of all the intervening decays. If the gates after a marked token stay near 1 for carry and near 0 for overwrite, that token can survive. If a later token opens the write gate, the old token is erased.

This is an attention-like story without a stored attention matrix.

Attention says:

for this query, compare against stored keys and mix stored values

A selective recurrence says:

while reading the stream, decide what enters the state and how long it persists

The Mamba-2 paper makes this connection formal through structured state-space duality. Dao and Gu connect SSMs and variants of attention through structured semiseparable matrices, and use that view to design a faster Mamba-2 core layer.5 In the toy scalar recurrence, the lower-triangular influence matrix is already visible: entry \((t,j)\) is the product of the decays between \(j\) and \(t\), times the write strength at \(j\).

That is not softmax attention. It is not trying to be. It is a compressed, structured memory map.

The Bottleneck Is The Feature

A Transformer context grows.

A recurrent state does not.

That fixed-size state is the advantage and the wound. It makes autoregressive inference cheap per token because there is no KV cache growing with the sequence. It also forces the model to compress the past. If a task needs exact access to many independent old facts, the recurrence has to store enough statistics to answer later. Sometimes it can. Sometimes it cannot.

This caveat is not only a practical shrug. Merrill, Petty, and Sabharwal’s Illusion of State paper argues that SSMs share important state-tracking limitations with transformers; their formal result places these models inside the same broad circuit-complexity family and their experiments show Mamba-style models struggling on some exact tracking tasks.6 That does not make selective state useless. It names the boundary: a compressed state is powerful when the right sufficient statistics exist, and brittle when the task keeps demanding many independent facts.

This is why “linear-time model beats Transformer” is the wrong headline.

The real question is:

what kind of memory does the task need?

State-space models are attractive when the past can be summarized into a moderate state that evolves predictably. Attention is attractive when the model benefits from direct pairwise access to many previous positions. Hybrids are attractive because real language asks for both.

The 2026 Turn

Mamba-3 is interesting because it does not merely say “make the recurrence bigger.” The March 2026 paper frames the next step as state-space engineering: an exponential-trapezoidal discretization, complex-valued state updates for richer state tracking, and a multi-input multi-output formulation aimed at better inference efficiency.7 The official repository now exposes Mamba-3 modules alongside Mamba and Mamba-2.8

That matters because it moves the discussion away from architecture slogans.

The hard questions are becoming more specific:

  • Which discretization gives the state the right dynamics?
  • When should the transition be scalar, diagonal, complex, or richer?
  • How much state is enough for retrieval and state tracking?
  • Which parts should use recurrence and which parts still need attention?
  • Can the kernel keep arithmetic units busy instead of just moving memory?

That last question is not cosmetic. Mamba’s original paper emphasized that a theoretically linear scan still has to respect the GPU memory hierarchy. Mamba-2 then used the SSD view to get more matrix-multiplication-friendly algorithms. Mamba-3 pushes from an inference-first angle.

The math and the machine are negotiating.

What The Lab Is Not

The lab is not a Mamba implementation.

It hand-codes gates. Mamba learns them. It uses one scalar state. Real models use many channels and learned projections. It uses a toy copy task. Language models need syntax, semantics, retrieval, calibration, and robustness. It draws the scan in JavaScript. Production kernels are written to avoid expensive traffic between memory levels.

The lab is only trying to isolate the idea:

fixed recurrence: forget by schedule
selective recurrence: forget by content
scan: compute recurrence prefixes in parallel because affine maps compose

That is enough to make the architecture feel less mysterious.

The model is not storing the past as a shelf of old tokens. It is continuously renegotiating a state. Every token asks:

do I write?
do I preserve?
do I erase?

The answer is the memory.

Audit Trail

The executable artifact is assets/js/selective-scan-lab.js. It exports an audit() function that checks the toy recurrence rather than only drawing it.

The audit verifies:

affine-map composition is associative
serial recurrence and prefix scan produce the same states
query-time influence weights reconstruct the query state
raising selectivity increases mark writes and suppresses noise writes
the selective path beats fixed decay across 180 generated copy tasks

In the current run, the grid covers five seeds, three sequence lengths, four query gaps, and three fixed half-lives. The worst scan mismatch is 0, and the smallest fixed-minus-selective error advantage at 96 percent selectivity is 0.662.

Paper Trail

  1. Albert Gu, Karan Goel, and Christopher Re, “Efficiently Modeling Long Sequences with Structured State Spaces”, ICLR 2022. ↩

  2. Albert Gu and Tri Dao, “Mamba: Linear-Time Sequence Modeling with Selective State Spaces”, ICLR 2024. The paper describes input-dependent SSM parameters, a hardware-aware scan, linear scaling in sequence length, and constant-time autoregressive inference state. ↩

  3. Guy E. Blelloch, “Prefix Sums and Their Applications”, CMU-CS-90-190, 1990. PDF: CMU mirror. ↩

  4. State Spaces Mamba repository, selective_scan_interface.py. The reference path updates x = deltaA * x + deltaB_u in a time loop; optimized kernels avoid the naive materialization pattern. ↩

  5. Tri Dao and Albert Gu, “Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality”, ICML 2024. ↩

  6. William Merrill, Jackson Petty, and Ashish Sabharwal, “The Illusion of State in State-Space Models”, ICML 2024. ↩

  7. Aakash Lahoti, Kevin Y. Li, Berlin Chen, Caitlin Wang, Aviv Bick, J. Zico Kolter, Tri Dao, and Albert Gu, “Mamba-3: Improved Sequence Modeling using State Space Principles”, ICLR 2026. OpenReview PDF: Mamba-3. ↩

  8. State Spaces, “Mamba SSM architecture”. The repository documents Mamba, Mamba-2, SSD minimal code, and Mamba-3 usage. ↩