Executive Summary
NVIDIA published Snapshot, an open source checkpoint and restore system for GPU workloads on Kubernetes. The measured result is a replica back in service in 3.6 to 42.7 seconds, against 6 to 19 minutes for a cold start on the largest mixture-of-experts models, a speedup of 8x to 36x. Checkpoints run to 1 TB across eight GPUs on a single node.
The useful finding is in the breakdown rather than the headline. Restoring GPU memory takes 18 to 29 seconds and dominates a large restore, while CRIU restores the CPU process in 4.2 to 10.8 seconds. What the restore removes is weight loading, kernel autotune and CUDA graph capture, the fixed cost of initialisation. It does not remove node provisioning or image pull, both of which sit outside the measured path. So the win is real and bounded, and the bound is your platform rather than your model.
A replica comes back in under 43 seconds. The same replica started cold takes 6 to 19 minutes when the model is a large mixture-of-experts. NVIDIA published that measurement on Tuesday next to an open source tool, NVIDIA Snapshot, that produces it.
The gap is not about faster hardware. It is about what a cold start actually spends its time on, and the tool removes exactly that part.
A cold start is three waits stacked on each other
Bringing an inference replica up has three waits in series. Weights load into GPU memory. Kernels compile and autotune for that hardware and that batch shape. CUDA graphs get captured. None of those steps depend on the request that will arrive later, and all of them repeat for every replica you start.
Snapshot checkpoints the finished state instead. It captures CPU and GPU memory of a fully initialised workload and restores that state on compatible infrastructure. The workload needs a small handshake, pausing before the checkpoint and resuming after the restore. Two custom resources carry it. PodSnapshot checkpoints a running pod, SnapshotJob starts a pod from a template and checkpoints it as soon as it is ready. A restore is an annotation, nvidia.com/restore-from, that names the snapshot to come back from.
The measured path starts after the pod exists
The published numbers are careful about their own boundary, and the boundary is the useful part. Infrastructure provisioning and container image loading sit outside the measured restore path. So 43 seconds is the time to rehydrate a workload, not the time for a pod to appear on a node that still has to be provisioned and to pull a multi-gigabyte image.
On a fleet where node readiness takes minutes and images are large, the end to end figure is longer than the headline. That does not make the number wrong. It makes the win local to the layer that was actually slow, which for large models is initialisation.
The breakdown supports that reading. On B200 with VAST NFS storage, GPU state restore dominates at 18 to 29 seconds while CRIU restores the CPU process in 4.2 to 10.8 seconds. Checkpoint creation costs 15 seconds to 5.5 minutes depending on model size, and checkpoint plus restore still came in under one cold start in every configuration NVIDIA tested. The first restored replica already pays for its own checkpoint.
A bounded restore changes how you hold capacity
Warm capacity exists to survive a start penalty. When the penalty is ten minutes, you keep replicas running and idle because a scale up that late misses the burst it was built for. When the penalty is under a minute, holding idle replicas becomes the worse trade.
Recovery changes the same way. A failed replica no longer means a cold start somewhere else. It means a restore onto whichever node is free, with the memory and filesystem state the original had.
The limit sits underneath. Snapshot is a component of the NVIDIA DSX AI Factory Platform and it restores onto compatible infrastructure, so node, driver and image still have to match what the checkpoint expects.
Time a replica from scheduling to its first token and split the number by phase, because only one phase just got cheap. Measure what fraction of your fleet is idle warmth waiting for a burst. Then check whether your platform can provision a node and pull an image inside the time the restore saves, because if it cannot, the checkpoint is no longer the bottleneck.
Related reading. Our look at the NVIDIA Kubernetes control plane and its tenant defaults, and at how NVIDIA put GPU cluster configuration under version control.
Sources. The NVIDIA technical writeup with the benchmark tables, the project on GitHub, VAST Data for the storage under test, and the Kubernetes documentation on the pod lifecycle.
Get the next one before it is old news
Independent analysis of cloud-native infrastructure, Kubernetes and data center economics. No vendor spin.