Running a 7B model? You don't need NVSwitch. Running 70B+ tensor-parallel? You do. Here is exactly where the line is, what the bandwidth gap costs you, and what to rent on packet.ai.
NVSwitch is NVIDIA's dedicated crossbar switch chip that turns point-to-point NVLink lanes into a fully connected GPU fabric, giving every GPU in a node full-bandwidth access to every other GPU at the same time, without routing through the CPU or PCIe.
Key takeaways
TL;DR for solo builders
Most GPU specs conversations focus on VRAM, TFLOPS, and memory bandwidth. NVSwitch almost never comes up. That is a problem, because for any workload that needs GPUs to exchange data constantly, the interconnect is as important as the compute chip itself.
This guide covers what NVSwitch actually is, how it differs from NVLink, why it matters for LLM inference and distributed training, and which GPU configurations include it. It is part of the GPU hardware concepts series; read alongside the deep dive on NVLink vs PCIe for AI workloads and the guide to SXM vs PCIe GPU form factors.
NVLink is a point-to-point lane between two GPUs. Think of it as a very fast private road. NVSwitch is the interchange that connects all those roads simultaneously, letting every GPU talk to every other at full speed, with no waiting and no detours through the CPU.
Without NVSwitch, connecting 8 GPUs over NVLink means some pairs still route through intermediate GPUs. You get latency, you lose bandwidth, and your AllReduce operations slow down. With NVSwitch between every GPU, all traffic goes through a non-blocking crossbar. Any GPU reaches any other at full NVLink bandwidth in a single hop.
What non-blocking actually means
Non-blocking means simultaneous traffic between all GPU pairs does not cause any pair to wait for another. Every pair gets its full allocated bandwidth at the same time. In an 8-GPU HGX node, that is 8 GPUs each writing to 7 others concurrently, all at 900 GB/s, without any of them slowing down.
NVSwitch is a physically separate chip from the GPU. NVIDIA manufactures it on its own process node, mounts it on the server baseboard or dedicated switch trays, and connects it to GPUs via NVLink cables or baseboard traces. Each GPU connects to multiple NVSwitch chips in parallel, distributing traffic across switching planes so no single switch chip becomes the bottleneck.
NVIDIA has shipped four NVSwitch generations. Each one doubled per-GPU bandwidth and expanded how many GPUs could join a single all-to-all fabric.
The DGX H100 uses four NVSwitch3 chips per baseboard. Each GPU connects to all four switches via 18 NVLink 4 lanes, split 5-4-4-5 across the switching planes. Every H100 in an 8-GPU node gets 900 GB/s bidirectional bandwidth to every other GPU, for a total of 3.6 TB/s of NVLink fabric bandwidth across the node.
The GB200 NVL72 pushed this much further. Nine NVSwitch4 trays wire 72 GPUs all-to-all via a cabled NVLink backplane. Total fabric bandwidth across the rack hits 130 TB/s, the largest single-domain NVLink fabric available anywhere in 2026.
Most explainers stop at "it switches NVLink traffic." Two other functions matter for production deployments.
All-to-all routing at full bandwidth. Any GPU can send data to any other at full NVLink speed, with all pairs running simultaneously. PCIe cannot do this. PCIe paths share a root complex through the CPU, so as more GPU pairs try to communicate concurrently, effective bandwidth per pair drops. NVSwitch does not share. Every pair gets its full allocation.
In-network compute via SHARP. Starting with NVSwitch3, NVIDIA embedded SHARP (Scalable Hierarchical Aggregation and Reduction Protocol) engines into the switch silicon. SHARP lets AllReduce operations run partly inside the switch fabric, instead of sending all gradient data out to GPU cores for aggregation. For distributed training using PyTorch FSDP or Megatron-LM, this directly cuts the bandwidth required for gradient synchronisation and reduces AllReduce latency. NVSwitch3 implements SHARP at 400 GFLOPS FP32 per switch chip.
NVLink multicast. NVSwitch supports one-to-many data distribution. One GPU can broadcast updated weights or activations to all others in a single operation. This matters for pipeline parallelism where the same tensor needs to reach multiple workers at the same time, and for NCCL broadcast collectives used by training frameworks.
PCIe Gen 5 x16 delivers 64 GB/s bidirectional. NVLink 4 on an H100 or H200 delivers 900 GB/s per GPU. NVLink 5 on a B200 delivers 1.8 TB/s per GPU. The numbers alone do not convey what this means for a real training job.
On a PCIe-only setup, AllReduce across 8 GPUs during full fine-tuning can eat 30-40% of step time on large models. On an NVSwitch-equipped node, the same AllReduce completes in under 5% of step time. For tensor-parallel inference of a 70B model, activations cross GPU boundaries on every forward pass. At 900 GB/s NVLink, that transfer takes microseconds. Over PCIe Gen 5, it takes roughly 14x longer and becomes a hard latency floor you cannot optimise around.
Benchmarks across transformer workloads with large gradient tensors show NVLink delivering 2-3x training throughput versus PCIe equivalents. The 64 GB/s vs 900 GB/s gap maps directly onto GPU idle time waiting for data.
When the gap does not matter
If GPUs are not exchanging data on every iteration: data-parallel serving where each GPU handles its own requests, or QLoRA fine-tuning where gradient tensors are small. PCIe is often sufficient in these cases. NVSwitch earns its premium on tensor-parallel and pipeline-parallel workloads where cross-GPU communication is constant and latency-sensitive.
Here is the practical answer, skipping the abstraction. It comes down to model size and whether you are splitting one model across multiple GPUs or running multiple independent jobs.
The honest summary: most solo builders and indie hackers running open models do not need NVSwitch at all. A 7B Llama or 13B Mistral inference server runs on a single RTX 4090 at $0.39/hr. You only cross into NVSwitch territory when a single model no longer fits on one card and you need real tensor parallelism across cards, which starts to matter around 70B parameters at FP16.
Check the packet.ai pricing page to compare all available GPUs side by side before committing to SXM hardware.
Understanding the physical layout helps when comparing cloud GPU configurations.
In a standard 8-GPU HGX H100 baseboard, four NVSwitch3 chips sit on the same PCB as the eight GPUs. Each GPU's 18 NVLink 4 ports are distributed 5-4-4-5 across the four switch chips, creating four independent switching planes. Any GPU reaches any other through any plane. Total bidirectional fabric bandwidth across the baseboard: 3.6 TB/s.
The GB200 NVL72 takes a different approach. Seventy-two GPUs cannot be wired all-to-all on a single PCB. Instead, nine NVSwitch4 trays hold the switch silicon. Copper cables connect every GPU to every NVSwitch tray. Each NVSwitch4 chip has 72 NVLink 5 ports, enough to reach all 72 GPUs at once. Nine trays, two chips per tray: 18 switch chips total. Every GPU connects to all 18 chips, each at 200 GB/s bidirectional per port, adding up to 1.8 TB/s per GPU.
For multi-node deployments, NVSwitch handles the intra-node fast fabric. Between nodes, InfiniBand takes over. A production cluster on packet.ai runs NVLink within each node and NVIDIA Quantum-2 InfiniBand at 400 Gb/s between nodes. The two interconnects operate in parallel: NVSwitch for tensor-parallel communication, InfiniBand for pipeline and data-parallel collective operations across nodes.
For a deeper look at how multi-node GPU clusters are architected, read the packet.ai guide to multi-node training on GPU clusters.
NVSwitch is exclusive to SXM-form-factor datacenter GPUs. Consumer cards and PCIe datacenter variants do not have it, regardless of how much VRAM they carry.
On packet.ai today, the B200 SXM is the only NVSwitch-equipped GPU available for immediate deployment. It ships NVSwitch4 as part of the HGX baseboard: 1.8 TB/s per GPU, available Dynamic from $3.75/hr. H100 SXM and H200 SXM are on the coming-soon list; join the notify list on the pricing page if you need them.
The RTX 6000 Pro at $0.66/hr and RTX 4090 at $0.39/hr are the smart picks for solo builders running models under 70B. Both are available now, significantly cheaper, and perfectly capable for single-GPU inference and LoRA fine-tuning.
A packet.ai 8-GPU B200 SXM node delivers 14.4 TB/s of total NVLink fabric bandwidth across the baseboard, more interconnect bandwidth than most entire clusters from five years ago.
Match your use case, pick the cheapest GPU that fits, and do not pay for NVSwitch you will not use.
Last reviewed: August 25, 2026. The B200 SXM on packet.ai is the only NVSwitch-equipped GPU available for immediate deployment, from $3.75/hr Dynamic, no commit. Deploy a B200 on packet.ai or get a wholesale cluster quote.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
