Status and limits
warmctl is in beta. It restores vLLM engines of one to eight GPUs on one node, on the platform each snapshot was built for, and checks every restore before it serves. Here is what works today, what does not yet, and what is checked.
What beta means
Releases are tagged v2.0.0-beta.N and published as pre-releases on GitHub Releases; the install script picks the newest one. Every user-facing change is in the CHANGELOG. To keep a host or a cluster on one release, pin it: WARMCTL_VERSION for the install script, a release tag of the runtime image in Kubernetes.
Works today
| Engine | vLLM with sleep mode: every model in the catalog, and any other model you build a snapshot of |
| GPUs per engine | 1 to 8 on one node, tensor and expert parallel; NCCL and shared GPU memory are kept through NVIDIA cuInterpose |
| GPU models | NVIDIA H100 SXM, H200, B200, B300, L40S, RTX PRO 6000 |
| Drivers | The branch each snapshot was built on (580.x, 610.x, one catalog snapshot 615.x); any release of that branch restores it: why |
| Where it runs | Linux x86-64 hosts with Docker and the NVIDIA Container Toolkit; Kubernetes in privileged pods; KServe |
| Distribution | Any OCI registry, such as registry.warmctl.io, GHCR, ECR or Harbor (guide), or a shared file system (guide, RDMA NFS) |
| Without a GPU | catalog, images, mirror, push and pull also run on macOS and linux-arm64 |
Not supported yet
| Other engines | SGLang, TensorRT-LLM, TGI and LMDeploy are planned. Today a snapshot is a vLLM engine. |
| Several nodes | An engine that spans nodes cannot be snapshotted. One replica runs on one node; scale out with more replicas. |
| Another platform | A snapshot does not restore on another GPU model or count, another driver branch or a CPU without the donor's features. Build one per platform; the catalog does this for its models. |
| Confidential computing | cuda-checkpoint returns CUDA_ERROR_NOT_SUPPORTED on GPUs in CC mode, so every start there is cold: measured. |
| GPU hosts other than Linux x86-64 | The linux-arm64 build has no cuInterpose yet and is not proven on a GPU. |
| io_uring | CRIU cannot checkpoint it, so GPU hosts run with kernel.io_uring_disabled=2. |
What every restore checks
- Before anything is downloaded, the host must match the snapshot: GPU model and memory, driver branch, topology class and every CPU flag the donor had.
- Every file carries its SHA-256 in
manifest.json, checked as it is written and pulled;warmctl verify --deephashes a local image again. - The weights go back to the same GPU addresses, and each allocation's fingerprint is compared with the donor's.
- A canary: the donor's prompt, sent twice. The answers must match under the profile's policy, bitwise for large models. Until then the gateway answers 503.
Not checked: tensors one by one and CUDA graphs one by one. The details are in How a restore works.
Report a problem
Open an issue on GitHub with the output of warmctl doctor and the snapshot's platform tag, or write to hello@warmctl.io.