Hybrid models interleave state-space model (SSM) layers with a few attention layers. NVIDIA Nemotron 3 , Qwen3.5 through Qwen3.8 , Kimi-K3 , and GLM-5.3-Flash are all built this way, with Mamba-2 , Gated DeltaNet (GDN) , or Kimi Delta Attention (KDA) as the SSM layer.

SSM layers are designed to make decoding fast: they use a fixed-size state in place of a key-value (KV) cache that grows with context length. At serving batch sizes, however, the SSM layers, not attention, can be the latency bottleneck up to 100K tokens of context. SSM layers also complicate two techniques that reliably speed up attention decoding: speculative decoding and quantization.

ReplaySSM makes SSM layers the fast part again. It represents the SSM state as a checkpoint plus a buffer of recent inputs.…