Nemotron 3 Super Deployment: NVIDIA's Open Model on Your Own GPUs

NVIDIA's Nemotron 3 family made real noise in 2026 as the leading NVIDIA open model line: Nemotron 3 Ultra (550B total, 55B active) is the largest open-weight model NVIDIA has ever shipped, and it's currently the strongest US open-weight model on independent benchmarks. But Ultra needs a multi-node cluster to run. Nemotron 3 Super, the mid-sized member of the family at 120B total and 12B active, is the one that actually fits on GPUs you can rent today, and it introduces a genuinely novel hybrid Mamba-Transformer architecture that changes how memory scales with context length. This post covers what Super needs to self-host, why NVFP4 matters specifically for this model, and a working vLLM deployment.

This post assumes you already know why active parameters don't determine VRAM footprint in an MoE model; that ground is covered in the packet.ai DeepSeek V4 GPU requirements guide and the Mixtral 8x22B GPU requirements guide. What's genuinely new here is the Mamba-hybrid piece: Nemotron 3 Super isn't just sparse, it also replaces most attention layers with state-space model (SSM) layers, which changes how memory scales with context in a way pure-transformer MoE models don't share. For general VRAM sizing formulas, see the VRAM requirements guide.

What Nemotron 3 Super Actually Is

Nemotron 3 Super, sometimes shortened to Nemotron 120B given its total parameter count, is a 120-billion-parameter model with 12B active per token, released by NVIDIA in the first half of 2026 as the mid-sized member of the three-tier Nemotron 3 family (Nano, Super, Ultra). It's the first model to use LatentMoE, a new mixture-of-experts variant NVIDIA designed specifically to improve accuracy per parameter and per FLOP compared to standard MoE routing. It also incorporates Multi-Token Prediction (MTP) layers, which accelerate inference through built-in speculative decoding without needing a separate draft model.

The architectural piece that matters most for Nemotron 3 Super GPU requirements: Super is a hybrid Mamba-Attention model, not a pure transformer. Most of its layers use Mamba-2, a state-space model architecture, with standard attention layers mixed in at select points rather than throughout. This Nemotron Mamba design is a meaningfully different approach from Mixtral or DeepSeek, which are both sparse but built entirely on standard transformer attention.

Why the Mamba-Hybrid Design Changes the Memory Math

In a standard transformer, KV cache grows linearly with context length: every additional token adds a proportional amount of cached key and value data across every attention layer, which is why pure-transformer models can hit serious memory pressure at long context. Mamba-2 layers don't work this way. Instead of caching every past token, an SSM layer compresses the sequence history into a fixed-size recurrent state that doesn't grow as context length increases.

Since most of Nemotron 3 Super's layers are Mamba rather than attention, the model's memory footprint at long context looks very different from a same-size pure transformer. At the model's supported 1M token context window, a pure transformer's KV cache would be enormous; Nemotron 3 Super's SSM layers add essentially no additional cache pressure as context grows, since attention layers are a minority of the architecture. The practical result: Super's context length is far cheaper to actually use than the raw token count suggests, which matters directly for agentic workloads that accumulate long conversation or tool-use histories.

Nemotron 3 Super VRAM Requirements

These figures come directly from NVIDIA's own vLLM cookbook for the model, and they're specifically the minimums needed to load the checkpoint with recent drivers and CUDA 12.x; production headroom for KV cache and concurrent requests adds to each number. The gap between precisions here is unusually large for a 120B model precisely because of NVFP4's 4-bit footprint, roughly a third of the BF16 figure, and because Nemotron 3 Super was pretrained natively in NVFP4 rather than trained in higher precision and quantized afterward, so the low-precision checkpoint isn't a lossy compression of something else, it's the model's native format.

Nemotron 3 Super vLLM Deployment

vLLM has day-0 support for the Nemotron 3 family, developed directly with NVIDIA specifically because agentic workloads are the model's primary use case, and Nemotron agentic deployments depend on fast, reliable tool calling. The NVFP4 checkpoint on a single B200 is the simplest deployment:

vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 \
  --tensor-parallel-size 1 \
  --trust-remote-code \
  --max-model-len 32768 \
  --gpu-memory-utilization 0.85 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml \
  --reasoning-parser nemotron_v3

For FP8 on 2x H100 or H200, or BF16 on 4x A100, the same command works with the corresponding checkpoint and --tensor-parallel-size adjusted:

vllm serve nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-FP8 \
  --tensor-parallel-size 2 \
  --trust-remote-code \
  --max-model-len 32768 \
  --enable-auto-tool-choice \
  --tool-call-parser qwen3_xml

A few flags worth understanding rather than copying blind. --reasoning-parser nemotron_v3 is specific to this model's thinking-mode output format. --max-model-len 32768 is a practical default for production, not the model's ceiling; the full 1M context is available given sufficient VRAM headroom, but most deployments don't need it and setting it lower frees memory for concurrency. To use the model's native speculative decoding via MTP instead of a separate draft model, add:

  --speculative-config '{"method":"mtp","num_speculative_tokens":3}'

Since MTP is built into the checkpoint itself, this doesn't require downloading or managing a separate assistant model the way speculative decoding normally does.

Running Nemotron 3 Super on packet.ai

The three deployment tiers map cleanly onto hardware that's actually available today. A100 80GB is live on packet.ai at $1.43/GPU-hour, and 4x A100 covers the BF16 deployment. B200 is live at $3.75/hr and is the only path to NVFP4, the fastest and smallest option, since NVFP4 acceleration is Blackwell-exclusive. H100 is currently listed as coming soon on packet.ai rather than live; once available, 2x H100 would cover the FP8 tier, but B200 is the option to reach for today if NVFP4's smaller footprint and faster throughput matter more than waiting.

Given how differently NVFP4 and BF16 perform on this specific model, and how much cheaper NVFP4 is to run, testing both against your actual agentic workload before committing to a production configuration is worth the relatively small extra cost of running them briefly on Dynamic pricing.

Frequently asked questions

Last reviewed: August 12, 2026. For the active-versus-total parameter distinction shared by all MoE models, see the DeepSeek V4 guide and the Mixtral 8x22B guide. For general VRAM sizing across model families, see the VRAM requirements guide.