17B active parameters sounds small enough for any GPU. It isn't the number that actually decides your hardware. Here's the real VRAM math for every Llama model, 8B to 405B.
GPU sizing for Llama depends on which model you mean: dense Llama 3 models (8B, 70B, 405B) scale VRAM directly with parameter count, while Llama 4's Scout and Maverick use a mixture-of-experts design that separates active parameters from total parameters, and the total is what sets the memory floor.
Key takeaways
This guide covers every Llama model variant that matters for cloud GPU sizing in 2026: Llama 3.1/3.2/3.3 in the dense family, and Llama 4 Scout and Maverick in the MoE family. For each model you get the VRAM floor at FP16, FP8, and the most common quantised precisions, plus the packet.ai GPU that hits that floor with room to spare.
The Llama 3 family (3.1, 3.2, 3.3) is a dense transformer: every parameter is active on every forward pass, and the VRAM floor scales roughly linearly with parameter count. The rule of thumb is 2 bytes per parameter at FP16 and 1 byte per parameter at FP8.
At FP16, Llama 3.1 8B needs approximately 16 GB of VRAM, which fits on an RTX 4090 (24 GB) or L40S (48 GB) with headroom for KV cache and a moderate batch size. At FP8 the footprint drops to roughly 8 GB, fitting comfortably on any GPU with 16 GB or more. At Q4 quantisation the model sits under 5 GB, and can run on consumer GPUs with 8 GB VRAM. For production inference at scale, the 8B model on a single RTX 4090 or L40S handles moderate concurrency; an H100 or A100 provides higher throughput for heavy traffic.
At FP16, the 70B models need approximately 140 GB of VRAM, which exceeds every single GPU except the H200 (141 GB). In practice, most teams run 70B at FP8 (~70 GB) on a single H100 SXM (80 GB) with tight headroom, or at Q4 (~40 GB) on an A100 (80 GB) or L40S (48 GB). For full FP16 on a single card, the H200 is the only current option. For 70B at FP8 with comfortable headroom, the H200 or a dual-H100 setup is the practical choice in 2026.
At FP16, 405B needs roughly 810 GB, which requires a multi-GPU setup: a full 8-GPU H100 node (640 GB) handles it only at FP8 (~405 GB), or needs two 8-GPU nodes at FP16. At Q4 (~225 GB), three or four H100s cover it. The 405B is a multi-node training and large-scale inference model; it is not a single-GPU workload at any precision without aggressive quantisation.
Llama 4 uses a mixture-of-experts architecture. The active parameter count (what runs on each token) is much smaller than the total parameter count (all experts combined). The VRAM floor is set by the total parameters, not the active ones, because all experts must be loaded into memory even though only a fraction are used per forward pass.
Scout's 109B total parameters set the VRAM floor at approximately 109 GB at FP16, 55 GB at FP8, and around 30 GB at Q4. At FP8, Scout fits on the RTX Pro 6000 Blackwell (96 GB) with reasonable headroom, or on a single H100 SXM (80 GB) with tight margins. At Q4, Scout fits on an L40S (48 GB) or RTX Pro 6000 (96 GB) with plenty of headroom for KV cache and batching. Scout is the practical single-GPU Llama 4 option: at FP8 or Q4, it runs on available hardware without multi-node setup.
Maverick's 400B total parameters require approximately 400 GB at FP8 and around 200 GB at Q4. At FP8, Maverick needs multiple GPUs: a full 8-GPU H100 SXM node (640 GB aggregate) handles it, or an equivalent multi-card A100 setup. At Q4, Maverick fits in roughly 55 GB, which means a single H100 or two A100s. Teams running Maverick at Q4 on a single GPU trade quality for hardware cost; the FP8 version with a multi-GPU cluster is the production-quality path. It's roughly 200-224GB at Q4. At FP16, Maverick's footprint grows well beyond what any single current GPU offers, and needs a multi-GPU setup instead.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
