Nemotron 3.5 Super is in the warmctl catalog

NVIDIAOpenMDW-1.1

Nemotron 3.5 Super VL 120B-A12B

120B · 12B active · Mamba MoE · BF16 · vision

• 4×H100·24 s·273 GiB·580
• 2×B200·23 s·270 GiB·580

restore24 s cold start208 s

nemotron35-superRun

NVIDIA’s Nemotron 3.5 Super VL 120B-A12B is a hybrid of Mamba and attention layers with a mixture of experts: 120B parameters, 12B active per token, multi-token prediction (MTP) for speculative decoding, and a vision encoder for images and video. This snapshot is the BF16 release: 249 GB of weights, so it needs four H100s or two B200s. NVIDIA also ships an NVFP4 version of about 75 GB that fits on a single B200; we are building that one next.

That also makes it slow to start. vLLM spends 208 s loading the weights, compiling and capturing CUDA graphs before it serves on 4×H100. The warm snapshot skips all of it: warmctl up serves in 24 s and on 2×B200 in 23 s (cold start 227 s), with the first token checked bit for bit against the engine that was snapshotted.

RUN ITTwo commands

On a host with 4×H100 or 2×B200 and an NVIDIA 580 driver:

$ curl -fsSL https://warmctl.io/install.sh | sh
$ warmctl up nemotron35-super

up picks the snapshot for the host’s GPUs and driver and serves an OpenAI-compatible API on port 8000, with reasoning, tool calls and image input enabled:

$ curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
    "model": "nemotron35-super", "temperature": 0, "chat_template_kwargs": {"enable_thinking": false},
    "messages": [{"role": "user", "content": "What is the capital of France? One word."}]}' | jq -r '.choices[0].message.content'
Paris

SPEEDAfter the restore

We ran vllm bench serve against the restored engines with the same load grid as our GLM-5.3-Flash benchmark: random prompts, fixed output lengths.

One request on 4×H100 decodes at 8.7 ms per token, about 124 tokens/s with MTP, and 32 concurrent long generations produce 1,445 tok/s. The hybrid layers keep long prompts cheap: a 96k-token prompt gets its first token in 1.32 s and decodes at the same 9.0 ms per token as a short one.

The profile is tuned for latency: it serves up to 32 requests at once with a 128K context. Beyond 32, requests wait in the queue, so throughput levels off at about 1,310 tok/s on 4×H100 while time to first token grows. The profiles are in the repository if you want a batch-heavy variant.