Platforms and drivers
A snapshot is a running engine, not a model file. It comes back only where the GPU state it holds still makes sense: the same GPUs, the same driver branch, the same CPU features.
What a platform is
| GPU model and count | device memory is restored byte for byte at the same addresses, and tensor-parallel ranks map one to one |
| driver branch | the CUDA driver checks its kernel module's version when a process reconnects to the GPU |
| topology class | NVLink and NVSwitch or PCIe decide which collectives the engine built |
| CPU features | compiled code in the dump may use any instruction the donor's CPU had |
The platform tag names the first two and the exact driver of the build: 8xh100-80gb-sxm-580.173.02. The rest is in manifest.json and checked by up before anything is downloaded.
One repository, many platforms
A model's repository holds one snapshot per platform, each under its tag, and lists them in latest, an OCI image index whose entries carry the GPU model and count, driver, profile and size. up NAME reads the index like docker pull reads a multi-arch image: it takes the entry this host can restore, or says for each one why it does not fit. Naming a tag skips the choice.
Driver branches
The CUDA user-mode driver remembers the kernel module's version when it starts, and the module refuses a reconnect from a different version. A plain snapshot therefore restores only under the exact driver it was built on, which would make every routine driver update a rebuild.
warmctl builds with the driver's version check off and carries the few host driver libraries the dump maps. A snapshot then restores under any release of its driver branch: built on 580.173.02, restored on 580.178.04, with the canary matching bitwise. Across branches, 580 to 595 or 610, the GPU state itself is incompatible, so up refuses another branch in its host check, before any download.
That is why the catalog publishes some models twice per GPU: a 580.x snapshot and a 610.x one, and up picks the one for the host's driver. A few older snapshots still need their exact driver; the catalog marks them.
Which libcuda the engine runs on
The vLLM image ships CUDA's forward-compatibility driver library, newer than the host's driver. On the 580 branch the engine uses it. From branch 595 on, multi-GPU engines run on the host's own libcuda instead, because the newer library on such a host breaks checkpointing GPU memory that processes share; single-GPU engines are unaffected.
In practice
- Check a host with
nvidia-smiorwarmctl doctor, and what it can restore withwarmctl catalog. - Updating the driver within its branch keeps every snapshot. Moving to another branch needs that branch's snapshots: pull them, or build them on the new driver.
- In Kubernetes, pin pods to nodes of the platform with GPU Feature Discovery labels, and keep a node pool on one driver branch.