39B active parameters does not mean 39B of VRAM. Here is the real memory math for Mixtral 8x22B, and why 8x7B is not a live recommendation anymore.
Mixtral 8x22B is a 141-billion-parameter sparse mixture-of-experts model from Mistral AI. At FP8 it needs roughly 162GB of VRAM before KV cache; at BF16, closer to 280GB. Neither fits on a single GPU. This post covers what Mixtral 8x22B actually needs to self-host, why 8x7B isn't part of that answer anymore, and a working vLLM deployment across multiple A100s.
Key takeaways
The reason active parameters don't determine memory footprint is the same for every sparse MoE model, not something specific to Mixtral, and it's covered in more general depth in the packet.ai DeepSeek V4 GPU requirements guide, which walks through the same active-versus-total distinction on a different model family. This post assumes that context and focuses on what's specific to Mixtral: its actual routing mechanism, real numbers for the 8x22B model, and a working deployment. For general VRAM sizing formulas across model families, see the VRAM requirements guide.
Mixtral 8x22B is a decoder-only transformer where each feedforward block contains 8 distinct expert networks instead of one, the core Mixtral MoE design. A router network selects the top 2 of those 8 experts for every token, and only the selected experts' weights do any computation on that token. This is where the "8x22B" naming comes from, and it's also where it becomes misleading: the model is not eight separate 22B models, and it doesn't have 8 × 22B = 176B parameters in the naive sense either, since attention layers and other components are shared across all experts. The actual total is 141B parameters, with roughly 39B active on any given forward pass.
Released by Mistral AI in April 2024 under Apache 2.0, Mixtral 8x22B remains open-weight and self-hostable today, and it's still the Mistral MoE model to reach for if the older 8x7B isn't the right fit. It handles English, French, German, Italian, and Spanish, includes native function calling, and supports a 64K token context window. On benchmarks at the time of release, it outperformed the earlier Mixtral 8x7B and approached GPT-4-class performance on several standard evaluations, while costing meaningfully less to run per token than a dense model of comparable quality, since only 39B parameters worth of compute happens per token rather than the full 141B.
⚡ A note on Mixtral 8x7B
Mixtral 8x7B, the model that first brought sparse MoE to open-weight LLMs in December 2023, was officially retired by Mistral in March 2025. Their own documentation points to Mistral Small 4 as the replacement. If you're comparing "Mistral and Mixtral" options today, 8x7B is a historically important model, not a current self-hosting recommendation. This post covers 8x22B, the MoE model Mistral still supports.
The mental model to leave with: MoE reduces compute per token, not the amount of weight that must be resident for inference. All 141B total parameters have to sit in GPU memory before generation starts, since there's no way to know in advance which experts a given request will need. The 39B active count only tells you how many of those parameters participate in producing any one token, not how much memory the model occupies.
The router in Mixtral 8x22B chooses which 2 of 8 experts activate independently for every single token, and that choice can land on any expert in each layer. Across a full generation of any meaningful length, it's a near-certainty that all 8 get used somewhere in the sequence. There's no way to predict in advance which experts a given request will need, so nothing can be safely left out of memory.
This is precisely why community discussions around Mixtral 8x22B consistently include people trying to run it on a single 40GB or 48GB GPU and hitting an out-of-memory error immediately, sometimes after already seeing the model report a deceptively small active-parameter figure.
Mixtral VRAM planning comes down to two precision options, and neither fits a single card:
The FP8 figure comes from the straightforward calculation: 141B parameters at 1 byte each is roughly 141GB, plus the standard ~15% framework overhead vLLM and similar servers need for CUDA kernels, activation buffers, and allocator fragmentation, landing around 162GB. That leaves meaningful headroom on 4x A100 80GB (320GB combined) for KV cache and concurrent requests. BF16 at 2 bytes per parameter roughly doubles that to 280GB, which technically fits on 4x A100 but leaves uncomfortably little room for KV cache once real traffic hits the server; 5x A100 80GB (400GB) gives that configuration actual working headroom.
⚡ On the "80GB in 4-bit" figures floating around
Some sources cite much lower Mixtral 8x22B VRAM figures, in the 80GB range, for aggressive 4-bit quantization. That number is plausible for weight storage alone at INT4 (roughly 70-75GB), but it doesn't include KV cache, framework overhead, or activation memory, and 4-bit quantization on a model this size carries a real quality tradeoff worth testing against your actual task before committing to it. Treat low weight-only figures as a floor, not a deployment budget.
A100 is worth flagging specifically here: it lacks native FP8 Tensor Cores, so FP8 inference on A100 runs through software emulation rather than hardware acceleration. It still fits the memory budget and still works, just at lower throughput than the same FP8 configuration would get on H100 or newer hardware. If raw BF16 throughput matters more than cost, that's the tradeoff to weigh against A100's lower hourly price.
The detail most Mixtral tutorials miss: tensor parallelism alone is not the right configuration for a sparse MoE model. vLLM supports expert parallelism specifically for MoE architectures, and combining it with tensor parallelism is what actually makes a multi-GPU Mixtral deployment work well, rather than just technically fit.
vllm serve mistralai/Mixtral-8x22B-Instruct-v0.1 \
--tensor-parallel-size 4 \
--enable-expert-parallel \
--dtype fp8 \
--max-model-len 16384 \
--gpu-memory-utilization 0.92--enable-expert-parallel is the flag that matters most here. Without it, vLLM tensor-parallelizes every layer uniformly, including the expert layers, which is a workable but inefficient way to split an MoE model across GPUs. With expert parallelism enabled, individual experts can be distributed across GPUs more directly, which is a better fit for how the model's sparsity actually works. --dtype fp8 is what makes the 4x A100 80GB configuration fit at all; dropping to BF16 on the same 4-card setup leaves too little room for KV cache to be practical.
If BF16 throughput is a hard requirement rather than FP8, the same command with --dtype bfloat16 needs 5x A100 80GB instead of 4x to leave workable KV cache headroom.
Mixtral 8x22B's real deployment cost comes from needing multiple GPUs regardless of precision, which makes the per-GPU rate matter more than it would for a single-card model. packet.ai's A100 80GB is available now at $1.43/GPU-hour, dedicated, with both PCIe and SXM configurations. SXM is the one to choose here: Mixtral's tensor-parallel and expert-parallel communication between GPUs needs NVLink bandwidth that PCIe's 64 GB/s ceiling doesn't provide, and 4-5 PCIe A100s tensor-parallelizing a 141B model would bottleneck on inter-GPU communication badly enough to undercut the whole point of renting multiple GPUs.
Since this is a multi-GPU-from-the-start deployment rather than a single-card decision, testing FP8 against BF16 on your own workload before committing to a production configuration is worth the relatively small extra hourly cost of running both configurations briefly on Dynamic pricing rather than guessing which one your traffic actually needs. That direct comparison settles a Mistral self host decision faster than any spec sheet can.
Last reviewed: August 11, 2026. For the general active-versus-total parameter distinction that applies to every MoE model, see the DeepSeek V4 GPU requirements guide. For general VRAM sizing across model families, see the VRAM requirements guide.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
