NVIDIA Blackwell Architecture Explained: What's New in the B200 and RTX 50 Series

NVIDIA Blackwell is the first GPU architecture to use a dual-die chiplet design, connecting two reticle-limited dies via a 10 TB/s chip-to-chip interconnect on a single package, delivering 208 billion transistors, 192 GB HBM3e, FP4 precision inference, and NVLink 5 bandwidth that Hopper-class GPUs cannot match.

NVIDIA announced the Blackwell GPU architecture at GTC 2025 and began shipping B200 GPUs to data centers in early 2025. Two years later, Blackwell is now the default GPU generation for large-scale AI training and inference infrastructure. Most GPU cloud providers including packet.ai run Blackwell GPU clusters today, and demand is still outpacing supply: analyst estimates put the B200 backlog at roughly 3.6 million units through mid-2026.

The Blackwell generation covers three distinct entities worth distinguishing: the B200 GPU (the individual data center chip), the GB200 Grace Blackwell Superchip (one Grace CPU + two B200 GPUs in a single module), and the GB200 NVL72 (36 Superchips in one liquid-cooled rack). The RTX 50 series (GB202 die) is a separate consumer Blackwell line.

NVIDIA Blackwell Architecture: The Dual-Die Design Behind Every B200 GPU

Every previous NVIDIA data center GPU, from Volta through Hopper, used a monolithic die: one continuous piece of silicon. The B200 breaks that constraint. It uses a chiplet design: two compute dies, each fabricated on TSMC's 4NP (4nm-class) process node, mounted on a single package and connected by a custom 10 TB/s chip-to-chip interconnect.

The reason for the shift is physics. At 4nm feature sizes, reticle limits cap the maximum die area a fab can produce. A single Blackwell compute die at roughly 104 billion transistors is already at or near that reticle limit. Combining two dies on one package allows NVIDIA to reach 208 billion transistors per GPU while maintaining manageable yields. You only need each die to yield correctly, not a single die twice the size.

The 10 TB/s die-to-die interconnect is faster than the GPU's own memory bandwidth (8 TB/s), which is the design requirement that makes the dual-die approach work. Both dies share a unified memory space and operate as a single cache-coherent GPU from the software perspective. CUDA does not require any changes to use a B200.

The architecture is named after David Blackwell, an American mathematician and statistician known for contributions to game theory, probability theory, and mathematical statistics. NVIDIA has maintained a scientist naming convention across generations: Volta, Turing, Ampere, Hopper, Blackwell, and now Vera Rubin.

Fifth-Generation Tensor Cores and FP4 Precision

Blackwell introduces fifth-generation Tensor Cores with a new capability: native FP4 (4-bit floating point) support. Prior NVIDIA architectures supported FP8 (Hopper) and FP16/BF16 but could not execute FP4 matrix operations in hardware. Blackwell adds FP4 with microscaling formats that maintain accuracy across different precision levels automatically.

In practice: FP4 inference halves memory consumption relative to FP8. A model that required 160 GB at FP8 fits in 80 GB at FP4, which means a single B200 (192 GB) can serve models that previously required a two-GPU configuration. At scale, this is a meaningful cost reduction for inference serving.

Second-Generation Transformer Engine

Hopper introduced the first Transformer Engine, which automatically managed FP8 precision for transformer model layers. Blackwell's second-generation version extends this to FP4, with per-tensor and per-block microscaling that applies the tightest precision each layer can sustain without accuracy loss. NVIDIA positions this as the primary driver of Blackwell's energy efficiency gains: 25x better inference energy efficiency per operation compared to Hopper when running at FP4.

B200 Full Specs: What 208 Billion Transistors and 192 GB HBM3e Actually Mean

The B200 SXM is the flagship data center variant. The key specifications, with H100 and H200 figures for comparison:

The 1,000W TDP is the sharpest operational difference from Hopper. A standard air-cooled HGX H100 system runs at 700W per GPU. The B200 requires liquid cooling in most data center deployments, a meaningful infrastructure investment that GPU cloud providers absorb as part of building Blackwell clusters. When renting B200 from packet.ai, you get access to liquid-cooled infrastructure without managing the cooling overhead yourself.

B200 vs H100 vs H200: Benchmark Performance and Memory Comparison

The performance gap between Blackwell and Hopper is wide, but it is not uniform across workload types. Here is where the B200 is decisively faster, and where the delta narrows. For a full cost-and-throughput breakdown, see the H100 vs B200 comparison guide.

FP8 Compute Throughput

The B200 delivers 9,000 TFLOPS of sparse FP8, 2.3x the H100 and H200. On top of that, the B200 adds up to 20,000 TFLOPS of FP4 (20 PFLOPS peak), a precision tier Hopper cannot access at all. For inference-optimised workloads running at FP4, the effective throughput advantage over H100 is larger than the FP8 number suggests.

The B200 delivers up to 2.5x faster LLM throughput than the H100 per GPU, per packet.ai's published benchmarks. On specific large-scale workloads such as GPT-MoE 1.8T at peak FP4 utilisation, NVIDIA cites up to 4x.

Memory Capacity and Model Fit

For inference, the most important Blackwell advantage is often not compute throughput; it is memory capacity. Running a 70B parameter model at FP16 requires approximately 140 GB of VRAM. An H100 (80 GB) cannot fit that model without quantization. An H200 (141 GB) fits it barely. A B200 (192 GB) fits it comfortably and leaves 52 GB for KV cache, which directly raises maximum batch size and throughput per GPU.

With FP4, the B200's effective model capacity roughly doubles again: a 140B model at FP4 requires approximately 70 GB, fitting inside a single B200 with 122 GB available for KV cache. This changes the economics of large-model serving more than any raw TFLOPS number.

GB200 Grace Blackwell Superchip and the NVL72 Rack-Scale System Explained

The B200 GPU is also available in a more integrated form: the GB200 Grace Blackwell Superchip. Each Superchip pairs one NVIDIA Grace ARM CPU with two Blackwell GPUs, connected via NVLink-C2C at 900 GB/s, approximately 7x faster than PCIe Gen 5 (128 GB/s). This CPU-GPU connection bypasses the PCIe bottleneck that limits conventional HGX-style server designs.

The NVL72 design treats the entire rack as a single GPU. All 72 Blackwell GPUs share one NVLink domain with 1.8 TB/s per-GPU interconnect bandwidth, eliminating the inter-node communication latency that limits model-parallel inference on conventional InfiniBand clusters. For a trillion-parameter MoE model, this means the NVL72 can run tensor parallelism across all 72 GPUs with latency comparable to single-GPU operations. For teams considering multi-node scale, see the multi-node GPU cluster training guide.

One GB200 NVL72 rack delivers equivalent LLM inference capacity to approximately 30 HGX H100 servers, while consuming 25x less power per token according to NVIDIA's published TCO comparisons against air-cooled H100 infrastructure.

The NVL72 requires liquid cooling as a baseline. The per-rack thermal load at full utilisation makes air cooling impractical. Major cloud providers including AWS, Azure, Google Cloud, and CoreWeave have been deploying NVL72 racks since early 2026, primarily for frontier model training and high-throughput inference contracts.

RTX 50 Series Architecture: How Blackwell Powers Consumer GPUs

The RTX 50 series (RTX 5090, RTX 5080, RTX 5070) shares the Blackwell architecture name with the B200 but uses a different die, the GB202, designed for consumer workloads, GDDR7 memory, and gaming use cases. See the full RTX 5090 cloud benchmark guide for inference performance numbers.

Both share fifth-generation Tensor Cores with FP4 support. For local AI inference workloads, running smaller models (sub-14B at FP16 or sub-30B at FP4) on a developer machine, the RTX 5090 is a capable Blackwell GPU. It is not a B200 substitute: GDDR7 at 1.79 TB/s is a factor of 4.5 below HBM3e at 8 TB/s, and 32 GB VRAM caps the models it can serve.

For production inference or training at scale, the RTX 5090's absence of NVLink is the critical constraint. Multi-GPU tensor parallelism over PCIe is bandwidth-limited in ways that make it impractical for large model serving. The consumer Blackwell line is designed for single-GPU workloads, DLSS 4 multi-frame generation, and workstation AI, not cluster inference.

Blackwell vs Vera Rubin: The Architecture Coming After B200 in H2 2026

NVIDIA announced the Vera Rubin platform at CES 2026 and confirmed production timelines at GTC 2026 in March. The Rubin R100 GPU is the B200 successor. Key architectural changes:

Vera Rubin also introduces the Vera CPU (88-core ARM) as a replacement for the Grace CPU used in GB200, alongside NVLink 6 at higher bandwidth and HBM4 memory. The token cost for agentic AI workloads is projected at one-tenth of Blackwell's cost per token at equivalent utilisation, according to NVIDIA's GTC 2026 disclosures.

For teams planning GPU procurement today: Blackwell (B200 or B300) remains the rational choice for workloads deployable in 2026. Vera Rubin cloud instances will not reach broad availability before early 2027 based on historical hyperscaler deployment cadence. The B300 (288 GB HBM3e, 15 PFLOPS FP4) is the Blackwell Ultra variant currently shipping to data centers if you need more memory than the B200's 192 GB.

Rent Blackwell GPUs on packet.ai: B200 SXM from $3.75/hr

packet.ai runs B200 SXM clusters with liquid-cooled infrastructure. On-demand pricing starts at $3.75/hr per GPU (Dynamic tier) and $5.90/hr (Dedicated, single-tenant with 99.99% SLA), approximately 63% below hyperscaler on-demand pricing for equivalent Blackwell capacity, per packet.ai's published rate comparison. H200 SXM and H100 SXM are launching soon on packet.ai; the A100 80GB at $1.43/hr is available now for sub-70B workloads.

* H200 SXM and H100 SXM are coming soon to packet.ai. Join the waitlist for early access. AWS list price sourced from packet.ai's AWS pricing analysis.

B200 vs A100: Cost Per Workload on packet.ai

The per-workload cost shifts significantly depending on model size and whether the workload fills the GPU's memory. These estimates use packet.ai on-demand rates against typical run durations from published benchmarks:

Rates from packet.ai/pricing. Run cost estimates based on published benchmark run times. Fine-tune costs vary by dataset size and batch configuration.

For teams running inference on 70B or larger models, the B200's memory capacity and FP4 support make it the lowest cost-per-token option despite the higher hourly rate. The B200 vs H200 ROI framework walks through the break-even math in detail. A B200 serving a 70B model at full utilisation at $3.75/hr typically delivers better economics than two A100s with split-model overhead. See current B200 cluster configurations on packet.ai.

Frequently asked questions

Last reviewed: July 28, 2026. Ready to run workloads on Blackwell? Browse packet.ai cluster options for B200 and reserved pricing.