When an LLM engine process fails, the standard recovery path involves a cold restart. This requires loading weights into HBM from storage, compiling kernels, and capturing NVIDIA CUDA graphs. For large models, initialization can take several minutes, during which surviving workers must absorb the displaced traffic.

Shadow engine recovery, available as a preview feature in NVIDIA Dynamo, moves most of this recovery work off the serving path. It keeps a fully initialized shadow engine idle on the same GPUs as the active engine. The GPU Memory Service (GMS) shares the existing weights between the engines without creating another copy in HBM. If the active process fails, the shadow takes over within seconds. Re-initialization occurs in the background entirely off the serving path.

We measured the impact by deliberately terminating one worker in a two-worker GLM-5.2 deployment.…