What Is NVfp4? Blackwell's 4-Bit Precision Format Explained

NVfp4 is NVIDIA's proprietary 4-bit floating-point format for Blackwell GPUs, running at 9,000 TFLOPS on the B200, exactly double its FP8 throughput, while shrinking a 70B model from 140 GB to roughly 40 GB in memory.

FP8 set the efficiency standard for LLM inference on Hopper. NVfp4 raises it again on Blackwell. Where FP8 halved memory versus FP16, NVfp4 halves it again. Where FP8 doubled throughput over BF16, NVfp4 doubles it over FP8.

That math compounds fast. A single B200 running NVfp4 can serve a 70B model that previously required two H100s at FP8, and it fits the model with memory to spare for a larger KV cache. At $3.75/hr on packet.ai's B200 SXM cluster, the per-token cost drops proportionally.

This post covers exactly what NVfp4 is, how its two-level block scaling works, how it compares to FP8 and INT4, which GPUs support it natively, and how to run it today with vLLM, TensorRT-LLM, or SGLang. If you want the full picture on FP8 before diving in, the FP8 vs FP16 vs BF16 guide covers the bit-level differences.

What Is NVfp4 and How Does It Work?

NVfp4 is NVIDIA's proprietary 4-bit floating-point quantization format, introduced with the Blackwell GPU architecture in 2025. It stores model weights in 4 bits instead of the 8, 16, or 32 bits used by FP8, BF16, or FP32, cutting storage in half relative to FP8 and enabling Blackwell's fifth-generation Tensor Cores to execute matrix operations at double the FP8 throughput.

The underlying encoding is E2M1: 1 sign bit, 2 exponent bits, and 1 mantissa bit. With only 4 representable exponents, the raw dynamic range is narrow, roughly −6.0 to +6.0. Stored alone, this would cause serious accuracy problems for transformer weight distributions. NVfp4 solves this with a two-level block scaling system that adapts the precision window locally across the tensor.

The two-level block scaling system

Without scaling, E2M1 would clip any weight outside −6.0 to +6.0 and lose all outlier information. Block scaling solves this by normalizing different parts of the tensor independently.

Level 1: every 16 consecutive weights share a single FP8 E4M3 scale factor. This is the key design choice that distinguishes NVfp4 from MXFP4 and basic INT4. The 16-element block is small enough to capture local variation in weight distributions, preventing outliers in one region of the tensor from degrading precision everywhere else.

Level 2: each full tensor gets one additional FP32 scale factor for global normalization.

The overhead is small: one FP8 value per 16 weights adds 0.5 bits per value on average, bringing effective precision to 4.5 bits rather than 4.0. That is a favorable tradeoff: meaningfully more accuracy than basic INT4 at nearly the same storage cost.

NVfp4 vs FP8 vs BF16: Throughput, Memory, and Accuracy on B200

The B200 is the only current production datacenter GPU where all three formats run natively on hardware. The numbers below come from the NVIDIA B200 HGX datasheet and NVIDIA's published NVfp4 benchmarks.

The B200 delivers 9,000 TFLOPS at FP4 dense, versus 4,500 TFLOPS at FP8 and 2,250 TFLOPS at BF16, per the NVIDIA B200 HGX datasheet.

On accuracy: DeepSeek-R1-0528 quantized from FP8 to NVfp4 shows 1% or less accuracy degradation on MMLU-Pro, GPQA Diamond, and LiveCodeBench, according to NVIDIA's published benchmarks. For most chat and instruction-following workloads, that margin is within acceptable tolerance.

Three Blackwell FP4 Formats: FP4, MXFP4, and NVfp4 Explained

"FP4" is not a single format. Blackwell's fifth-generation Tensor Cores support three distinct 4-bit floating-point variants, each with different block sizes, scale precision, and accuracy characteristics. The choice of format affects which toolchain you use, which checkpoints you can load, and how much accuracy you preserve.

NVfp4 outperforms MXFP4 on accuracy for two reasons: smaller block size (16 vs 32 elements) and more precise scale encoding (FP8 E4M3 vs E8M0). Both reduce quantization error at the cost of NVIDIA-only compatibility. MXFP4 uses the Open Compute Project's MX specification and runs on AMD Instinct MI355X under ROCm 7.x, relevant when cross-vendor portability matters. For NVIDIA-only deployments, NVfp4 is the right default.

Which GPUs Support NVfp4? Full Hardware Compatibility List

NVfp4 requires dedicated FP4 Tensor Cores. Only NVIDIA's Blackwell architecture includes them. On pre-Blackwell GPUs (Hopper H100/H200 and Ampere A100), loading an NVfp4 checkpoint forces dequantization of the weights to BF16 before inference, removing all throughput and memory benefit. The model still runs, but you get none of the FP4 advantages.

If you are currently running inference on H100 or H200, the highest-efficiency native quantization format available to you is FP8. NVfp4 becomes relevant only on migration to Blackwell. On packet.ai, both the H100 SXM (from Launching Soon) and H200 SXM (from Launching Soon) are available for FP8 workloads while waiting for Blackwell capacity.

The NVLink interconnect in an 8× B200 system also benefits from FP4. At 1.8 TB/s NVLink bandwidth, multi-GPU KV cache sharing at FP4 precision carries meaningfully lower data movement overhead than at BF16. For a detailed breakdown of how interconnect bandwidth affects inference throughput, the NVLink vs PCIe guide covers the tradeoffs.

NVfp4 vs INT4: Why Floating-Point Wins for LLM Accuracy

INT4 and NVfp4 both use 4 bits per value. That is where the similarity ends. The choice between them affects accuracy in ways that matter at scale, particularly on reasoning and coding benchmarks.

Dynamic range: the core difference

INT4 is a fixed-point format. It maps evenly spaced integers across a fixed range, typically 0 to 15 (unsigned) or −8 to 7 (signed). Every value gets the same step size. This handles uniformly distributed weights reasonably well but struggles with outliers, which are common in transformer weight matrices.

NVfp4 is a floating-point format. Like FP32 or BF16, its 2-exponent E2M1 encoding shifts the precision window, with more representable values near zero, fewer near the maximum. For the roughly Gaussian weight distributions in transformers, this logarithmic spacing preserves more signal per bit where it matters.

The block scaling amplifies this further. NVfp4's 16-element FP8 scale factor normalizes each local block independently. An outlier in one region of a weight matrix does not force the scale for the entire layer to expand and sacrifice precision elsewhere. INT4 quantization methods like GPTQ and AWQ can implement similar per-group scaling, but their group sizes (typically 128 elements) are 8× larger than NVfp4's 16-element blocks, giving coarser local normalization.

In practice, for standard chat and instruction-following tasks, NVfp4 and GPTQ INT4 produce similar output quality. The gap widens on reasoning and coding benchmarks. DeepSeek-R1-0528 at NVfp4 stays within 1% of its FP8 score on GPQA Diamond and LiveCodeBench. GPTQ INT4 on the same model typically shows 2-4% degradation on the same tests.

Running NVfp4 in vLLM, TensorRT-LLM, and SGLang

All three major LLM inference frameworks support NVfp4 as of August 2026. The fastest path is loading a pre-quantized checkpoint from Hugging Face directly, with no quantization calibration required.

Step 1: Get an NVfp4 checkpoint

NVIDIA publishes official NVfp4 checkpoints under the nvidia/ namespace on Hugging Face. Current examples include:

• nvidia/Llama-3.1-8B-Instruct-NVFP4
• nvidia/Llama-3.3-70B-Instruct-NVFP4
• nvidia/DeepSeek-R1-NVFP4
• nvidia/Llama-4-Scout-17B-16E-Instruct-FP4 (MoE)

If no pre-quantized checkpoint exists for your model, use NVIDIA Model Optimizer (nvidia-modelopt) with NVFP4_DEFAULT_CFG to quantize from BF16 or FP16 weights using a calibration dataset.

Step 2: Choose your serving framework

vLLM: serve a pre-quantized NVfp4 model

On a Blackwell GPU, serve any NVfp4 checkpoint the same way you would any other vLLM model:

vllm serve nvidia/Llama-3.3-70B-Instruct-NVFP4 \
  --tensor-parallel-size 1 \
  --max-model-len 8192

For MoE architectures (Llama 4 Scout), enable the FlashInfer MoE FP4 kernel:

VLLM_USE_FLASHINFER_MOE_FP4=1 vllm serve \
  nvidia/Llama-4-Scout-17B-16E-Instruct-FP4 \
  --tensor-parallel-size 1

Running NVfp4 Inference on packet.ai B200 Clusters

packet.ai's B200 SXM clusters are the on-demand path to NVfp4 inference today, starting at $3.75/hr per GPU. That is 3.8× below AWS p6 B200 instances at $14.24/hr for the same GPU silicon, and 1.57× below Lambda's B200 at $5.89/hr.

At NVfp4, a single B200's 192 GB HBM3e holds a 70B model in roughly 40 GB, leaving 152 GB for KV cache. That is enough KV cache for tens of thousands of simultaneous context tokens without tensor parallelism overhead across GPUs.

For teams that want managed NVfp4 inference without provisioning clusters manually, the packet.ai Token Factory (currently in private preview) provides a serverless LLM API endpoint backed by Blackwell hardware. If you are still on FP8 workloads, H100 SXM and H200 SXM clusters start at Launching Soon and Launching Soon respectively (currently on waitlist). Check packet.ai pricing for live availability across all GPU models.

Frequently asked questions

Last reviewed: August 2026. Run NVfp4 inference on packet.ai B200 SXM clusters from $3.75/hr. Check current availability.