How a restore works
A cold vLLM start spends minutes on work whose result never changes: compiling the model, autotuning kernels, capturing CUDA graphs. warmctl does that work once and keeps its result, the running engine itself, so every later start skips it.
Why not just faster storage
Fast storage shortens only the first step of a cold start, loading weights. The compilers and autotuners that follow do not care where the bytes came from: GLM-5.3-Flash on 8×H100 spends 7% of its 650 s cold start loading weights and most of the rest compiling, tuning and capturing. To skip those steps, the result has to be kept, and the result lives in process memory and GPU memory. So warmctl keeps the processes.
What a snapshot holds
| process tree | every process of the engine, dumped by CRIU: the API server, the workers, their memory, open files and loopback sockets |
| GPU state | each worker's CUDA context and device memory except the weights, checkpointed by NVIDIA cuda-checkpoint |
| shared GPU memory | what processes share for fast collectives, saved through NVIDIA cuInterpose on multi-GPU engines |
| weights | exported from vLLM's sleep-mode allocator in parts of up to 1 GiB, each with its SHA-256 and a fingerprint taken on the GPU |
| trees | /workspace, /tmp and /dev/shm of the engine's container: JIT and compile caches, logs |
manifest.json | every file with its size and SHA-256, the resolved profile, the hardware it ran on and the canary it answered |
Weights travel apart from the warm state, so two snapshots of one model share them, and a restore puts them back with a fast parallel copy instead of reloading a checkpoint.
Build
- Preflight. The GPUs, their hardware row, io_uring off, GPU leases.
- Start. vLLM in a donor container with sleep mode and warmctl's worker extension; an agent inside drives the steps.
- Warm up. Requests over every shape in the profile, so every kernel is compiled and tuned and every graph captured.
- Canary. One prompt, sent twice: the answers a restore must reproduce.
- Export and sleep. The weights out to disk, then
sleep(level=2)frees them from the GPU. - Checkpoint. Shared GPU memory released with addresses kept, cuda-checkpoint on every CUDA process, then CRIU dumps the frozen tree.
- Publish. Every file into the content-addressed store,
manifest.jsonlast.
Restore
- Preflight. The host must match the donor before anything is touched: GPU model, memory, driver branch, topology class, every CPU flag the donor had.
- Receiver. A container with the snapshot mounted read-only, or the pod itself in Kubernetes.
- Processes. The trees installed, CRIU restores the process tree stopped, then it continues.
- GPUs. cuda-checkpoint restores each worker's GPU state, at the same device addresses; cuInterpose re-creates shared allocations where they were.
- Wake. The weights reloaded to the same addresses and each allocation's fingerprint compared, then the KV cache.
- Canary. The donor's prompt twice; the answers must match under the profile's policy, bitwise for large models.
- Admit. The gateway opens. Until now every request got 503.
Same addresses matter because CUDA graphs replay raw device pointers: a graph captured at build time is only valid if every buffer is back where it was.
What is checked, and what is not
Checked: the SHA-256 of every file as it is written and pulled, and of every weights part while reloading with --verify-weights; the list of allocations after wake; every allocation's fingerprint; the canary; host compatibility.
Not checked: tensors one by one, graphs one by one. Anything the export does not hold stays in the CRIU dump, so an unknown allocation costs dump size, never correctness.
Where the seconds go
GLM-5.3-Flash on 8×H100 serves 37.3 s after up: CRIU 8.2 s, GPU state 10.5 s, weights 13.8 s, the rest staging, wake and the canary. That assumes the snapshot is on local disk and in the page cache; a cold page cache or a network share adds the time to read it. Afterwards the engine serves at stock vLLM speed or faster, because it carries its warmup.