---
title: "Nemotron 3.5 Super is in the warmctl catalog"
description: "NVIDIA Nemotron 3.5 Super VL 120B-A12B is in the warmctl catalog: a warm vLLM snapshot for 4×H100 that serves in 24 s instead of a 208 s cold start. Benchmarks and two commands to run it."
author: Kirill Bespalov
date: 2026-10-15
url: https://warmctl.io/blog/nemotron-3-5-super/
---

# Nemotron 3.5 Super is in the warmctl catalog

> NVIDIA’s 120B hybrid Mamba MoE with vision is now a warm vLLM snapshot. One command, and it serves in seconds instead of minutes.

**Key results**

- Nemotron 3.5 Super VL 120B-A12B (BF16, TP4) on 4×H100: a vLLM cold start takes 208 s to engine ready; warmctl up restores the 273 GiB snapshot to serving in 23.6 s with a bitwise canary.
- After the restore, one request decodes at 8.7 ms per token on 4×H100 (about 124 tokens/s with MTP); 32 concurrent 2k-token generations produce 1,445 output tok/s.
- A 96k-token prompt gets its first token in 1.3 s on 4×H100 and decodes at 9.0 ms per token.

[
      NVIDIAOpenMDW-1.1
      Nemotron 3.5 Super VL 120B-A12B

120B · 12B active · Mamba MoE · BF16 · vision

      4×H100·**24 s**·273 GiB·**580**2×B200·**23 s**·270 GiB·**580**

        restore24 s
        cold start**208 s**

      nemotron35-super
Run
    ](https://warmctl.io/catalog/#nemotron35-super)

  NVIDIA’s **Nemotron 3.5 Super VL 120B-A12B** is a hybrid of Mamba and attention layers with a mixture of experts: 120B parameters, 12B active per token, multi-token prediction (MTP) for speculative decoding, and a vision encoder for images and video. This snapshot is the [BF16 release](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16): 249 GB of weights, so it needs four H100s or two B200s. NVIDIA also ships an [NVFP4 version](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-NVFP4) of about 75 GB that fits on a single B200; we are building that one next.

That also makes it slow to start. vLLM spends **208 s** loading the weights, compiling and capturing CUDA graphs before it serves on 4×H100. The warm snapshot skips all of it: `warmctl up` serves in **24 s** and on 2×B200 in **23 s** (cold start 227 s), with the first token checked bit for bit against the engine that was snapshotted.

| GPUs | vLLM cold start | warmctl up | Snapshot | Faster |
|---|---|---|---|---|
| 4×H100 | 208 s | 23.6 s | 273 GiB | 9× |

## Two commands

On a host with 4×H100 or 2×B200 and an NVIDIA 580 driver:

```
$ curl -fsSL https://warmctl.io/install.sh | sh
$ warmctl up nemotron35-super
```

`up` picks the snapshot for the host’s GPUs and driver and serves an OpenAI-compatible API on port 8000, with reasoning, tool calls and image input enabled:

```
$ curl -s localhost:8000/v1/chat/completions -H 'Content-Type: application/json' -d '{
    "model": "nemotron35-super", "temperature": 0, "chat_template_kwargs": {"enable_thinking": false},
    "messages": [{"role": "user", "content": "What is the capital of France? One word."}]}' | jq -r '.choices[0].message.content'
Paris
```

## After the restore

We ran `vllm bench serve` against the restored engines with the same load grid as our [GLM-5.3-Flash benchmark](https://warmctl.io/blog/boosting-dedicated-inference/): random prompts, fixed output lengths.

| Load | 4×H100 out tok/s | 4×H100 TTFT |
|---|---|---|
| 1 request · 1k in, 512 out | 134 | 0.11 s |
| 32 requests · 1k in, 2k out | 1,445 | 0.21 s |
| 64 requests · 1k in, 512 out | 1,278 | 13 s |
| 1 request · 32k in | 110 | 0.78 s |
| 1 request · 96k in | 93 | 1.32 s |
| 8 requests · 32k in | 315 | 0.97 s |

One request on 4×H100 decodes at **8.7 ms per token**, about 124 tokens/s with MTP, and 32 concurrent long generations produce **1,445 tok/s**. The hybrid layers keep long prompts cheap: a 96k-token prompt gets its first token in **1.32 s** and decodes at the same 9.0 ms per token as a short one.

The profile is tuned for latency: it serves up to 32 requests at once with a 128K context. Beyond 32, requests wait in the queue, so throughput levels off at about 1,310 tok/s on 4×H100 while time to first token grows. The [profiles](https://github.com/TensorRails/warmctl/tree/main/profiles) are in the repository if you want a batch-heavy variant.

### Browse the catalog

Nemotron 3.5 Super joins GLM-5.3-Flash, DeepSeek-V4.1-Flash, Qwen 3.6, Gemma 4, gpt-oss and more, each a warm vLLM snapshot that serves in seconds.

    [Open in the catalog](https://warmctl.io/catalog/#nemotron35-super)[Install warmctl](https://warmctl.io/docs/install/)[GitHub](https://github.com/TensorRails/warmctl)