Serve over HTTPS

Give warmctl up a certificate and its key, and the gateway serves HTTPS from the first probe on. The pair is read again when it changes, so a rotated certificate needs no restart.

What changes

• Two flags. --tls-cert FILE and --tls-key FILE, PEM, always together. up loads the pair before anything else, so a missing or mismatched pair stops it at once.
• TLS 1.2 or later, HTTP/1.1. The same routes as over HTTP: /v1/models, /v1/completions, /v1/chat/completions, /metrics, /health. Plain HTTP to the port gets 400.
• From the first probe. In a pod the gateway opens before the snapshot is pulled; /health answers 503 over HTTPS until the canary passed, never a failed handshake.
• Rotation. On each new connection the gateway checks whether either file was replaced or rewritten and loads the new pair. A pair that does not load, say a certificate written before its key, keeps the last good one until it does. Open keep-alive connections finish on the certificate they started with.
• Only the gateway. The restored engine stays plain HTTP on the loopback, reached over a unix socket; nothing else listens on the network.

On a host

With a certificate from your CA, or a self-signed one to try it:

$ openssl req -x509 -newkey ec -pkeyopt ec_paramgen_curve:P-256 -nodes -days 30 \
    -subj /CN=gpu-1 -addext subjectAltName=DNS:gpu-1,IP:10.0.0.5 -keyout tls.key -out tls.crt
$ warmctl up qwen3-0.6b --listen 0.0.0.0:8443 --tls-cert tls.crt --tls-key tls.key

$ curl --cacert tls.crt https://10.0.0.5:8443/v1/models

up prints serving qwen3-0.6b on https://…. Replace both files, keeping their names, and the next connection gets the new certificate.

In Kubernetes

Mount a TLS secret, from cert-manager or your own, pass its files to up and probe over HTTPS. On the Deployment from Serve in Kubernetes:

# spec.template.spec
containers:
  - name: warmctl
    image: registry.warmctl.io/warmctl/runtime:latest
    args: [up, glm53-flash, --tls-cert, /tls/tls.crt, --tls-key, /tls/tls.key]
    readinessProbe:
      httpGet: {path: /health, port: 8080, scheme: HTTPS}
      periodSeconds: 1
    volumeMounts:
      - {name: tls, mountPath: /tls, readOnly: true}
volumes:
  - name: tls
    secret: {secretName: glm53-flash-tls}

The kubelet does not check a probe's certificate, so a self-signed one is fine for readiness. When the secret changes, the kubelet swaps the mounted files within about a minute and the gateway serves the new pair on the next connection. Mount the whole secret as a directory: a file mounted with subPath is never updated.

On KServe

KServe's enableLLMInferenceServiceTLS makes every LLMInferenceService pod an HTTPS server. With the warmctl preset from Serve on KServe, turning it on is all there is to do:

$ kubectl -n kserve edit configmap inferenceservice-config   # in "ingress": "enableLLMInferenceServiceTLS": true
$ kubectl -n kserve rollout restart deploy/llmisvc-controller-manager

For each service KServe then issues a self-signed certificate into the secret NAME-kserve-self-signed-certs (ca.crt, tls.crt, tls.key; names for localhost, the pod's IP and the workload Service), mounts it at /var/run/kserve/tls, switches the probes to HTTPS, has its scheduler read /metrics over HTTPS and names the workload Service port https. The preset passes the certificate to warmctl:

exec warmctl up --listen ${POD_IP}:8000{{ if .GlobalConfig.EnableTLS }} --tls-cert /var/run/kserve/tls/tls.crt --tls-key /var/run/kserve/tls/tls.key{{ end }} …

Check it through the workload Service, against KServe's CA:

$ kubectl -n llm get secret qwen3-8b-kserve-self-signed-certs -o jsonpath='{.data.ca\.crt}' | base64 -d > ca.crt
$ kubectl -n llm port-forward svc/qwen3-8b-kserve-workload-svc 8000
$ curl --cacert ca.crt https://localhost:8000/v1/models   # in a second terminal

When KServe reissues the secret, the pod presents the new certificate about a minute later, without a restart. On one L40S under KServe v0.21.0 that took 75 s, and the pod stayed Ready throughout.