Tutorial

Build and publish a snapshot

Bake Qwen3-8B into a snapshot on one H100, push it to GitHub Container Registry and restore it on another host. Then do the same for a model of your own.

You need a host prepared as in Install with one free H100, about 40 GiB of disk for the weights and the snapshot, and a GitHub account. A second H100 host with the same driver branch shows the restore elsewhere; the first host works too.

Download the weights

Snapshots are built offline from weights on the host, pinned to a revision:

bash
$ pip install -U huggingface_hub
$ hf download Qwen/Qwen3-8B --revision b968826d9c46dd6066d109eabc6255188de91218 --local-dir /models/qwen3-8b

Look at the profile

A profile says how the engine is launched and warmed. The catalog's profiles ship inside the binary; profile show prints one with every default applied, profile argv the exact vllm serve command it runs:

bash
$ warmctl profile list | grep qwen3-8b
$ warmctl profile argv qwen3-8b-h100-tp1
qwen3-8b-h100-tp1
schema: warmctl.profile/v1
id: qwen3-8b-h100-tp1
model: {id: Qwen/Qwen3-8B, revision: b968826d…, served_names: [qwen3-8b]}
parallel: {tp: 1}
hardware: [h100-80gb-sxm]
vllm:
  max-model-len: 32768
  max-num-seqs: 64
  kv-cache-memory-bytes: 47244640256
graphs:
  mode: FULL_AND_PIECEWISE
  decode_batches: [1, 2, 4, 8, 16, 24, 32, 48, 64]
warmup:
  prefill_tokens: [512, 2048, 8192]

Build

bash
$ warmctl build --profile qwen3-8b-h100-tp1 --model-dir /models/qwen3-8b --gpus 0 --output /snaps/qwen3-8b

This is the one slow step, a few minutes: build starts vLLM in a container on GPU 0, lets it load, compile and capture CUDA graphs, warms it over every shape in the profile, records a canary, then checkpoints the processes, the GPU and the weights. It ends with snapshot completed.

Verify and try it

bash
$ warmctl verify /snaps/qwen3-8b --deep
$ warmctl up /snaps/qwen3-8b

verify --deep hashes every file against manifest.json. up with a directory restores it right here, with no registry involved. Stop it with Ctrl-C.

Push it to GitHub Container Registry

warmctl uses Docker's credentials. Log in with a token that has write:packages:

bash
$ echo $GITHUB_TOKEN | docker login ghcr.io -u YOUR_USER --password-stdin
$ warmctl push /snaps/qwen3-8b ghcr.io/YOUR_USER/qwen3-8b

push stores the snapshot as one OCI artifact under its platform tag, here 1xh100-80gb-sxm- plus your driver, and adds it to the repository's latest. Snapshots of the same model for other GPUs go to the same repository, each under its own tag.

Restore it on another host

bash
$ docker login ghcr.io -u YOUR_USER       # if the package is private
$ warmctl up ghcr.io/YOUR_USER/qwen3-8b

From latest, up takes the snapshot this host can restore, downloads it and serves it, exactly as with the public catalog.

Now a model of your own

Any model vLLM serves with sleep mode gets a profile in a few lines. Write one next to your weights:

llama31-8b-tp1.yaml
schema: warmctl.profile/v1
id: llama31-8b-tp1
model: {id: meta-llama/Llama-3.1-8B-Instruct, revision: 0e9e39f2}
parallel: {tp: 1}
hardware: [h100-80gb-sxm]
vllm: {max-model-len: 8192, kv-cache-memory-bytes: 21474836480}
bash
$ warmctl build --profile ./llama31-8b-tp1.yaml --model-dir /models/llama31-8b --gpus 0 --output /snaps/llama31-8b

vllm: takes any vllm serve flag; the image, warmup grid and canary policy come from the defaults. Try a flag without editing the file with --set vllm.max-model-len=16384. Every key: Profiles.

What you did

Next: use your own registry for a team, or run it in Kubernetes.