RTX 5090 for AI in the Cloud: Pricing, Benchmarks vs RTX 4090

The RTX 5090 rents from Launching Soon on packet.ai in 2026, with 32 GB GDDR7 and 1,792 GB/s bandwidth. It is not a datacenter GPU. It is the ceiling of what consumer-tier hardware can do, and in 2026 that ceiling is high enough for production inference on models up to 30B parameters without quantisation.

The RTX 5090 occupies a specific and narrow sweet spot in 2026: it is the cheapest GPU that runs 13B and 30B models at full FP16 precision with usable throughput. For teams serving those model sizes on a per-project or bursty basis, it is the correct GPU. For teams running 70B+ or needing NVLink multi-GPU, it is not. This post covers the exact VRAM limits, vLLM throughput benchmarks, and cost comparison against the RTX 4090 and L40S so you can make that call quickly. For GPU cluster options including H100 and H200, see packet.ai GPU cluster.

RTX 5090 Specifications: Blackwell Architecture, 32 GB GDDR7, FP8 Tensor Cores

The RTX 5090 is NVIDIA's Blackwell-architecture consumer flagship, released in January 2025. Key differences from the RTX 4090 (Ada Lovelace):

The 1,792 GB/s memory bandwidth is the most important number for LLM inference. Memory bandwidth determines how fast the GPU can load model weights during the decode phase. The RTX 5090's bandwidth is 1.78x the RTX 4090 (1,008 GB/s) and 2.07x the L40S (864 GB/s). On memory-bandwidth-bound workloads, which includes most inference at small batch sizes, this directly translates to throughput improvement.

Native FP8 Tensor Cores on Blackwell enable proper hardware-accelerated FP8 inference, not the software emulation fallback on Ada Lovelace. For FP8-quantised models, this is meaningful: on the RTX 5090, FP8 decode runs at hardware speed, versus partial acceleration on the RTX 4090.

RTX 5090 VRAM Limits: Which Models Fit at 32 GB

VRAM capacity determines which models run at all. The 32 GB ceiling means the RTX 5090 fits models that cannot fit on the RTX 4090 (24 GB) but fails on models that require the L40S (48 GB) or H100 (80 GB).

The 32 GB sweet spot unlocks the 13B to 22B FP16 tier that the RTX 4090 cannot reach. For Llama 4 13B or Mistral 22B at FP16, the RTX 5090 is the cheapest single-GPU option. For 70B at any precision, you need an L40S or H100. The RTX 5090 does not change that calculus.

RTX 5090 vLLM Benchmarks: Throughput vs RTX 4090 and L40S

Throughput benchmarks below were run with vLLM v0.25.1 on CUDA 12.4, using the default PagedAttention and continuous batching configuration. Input/output ratio is 512 input tokens, 256 output tokens, at batch=8 unless noted.

The key finding: on 13B FP16, the RTX 5090 is 1.3x faster than the L40S despite costing 1.6x less per hour. The RTX 5090's 1,792 GB/s bandwidth significantly outpaces the L40S's 864 GB/s, which makes the RTX 5090 the cost-optimal GPU for 13B FP16 inference in 2026 if you can tolerate the PCIe interconnect limitation (single GPU only).

RTX 5090 vs RTX 4090 for AI: When to Upgrade

The RTX 5090 and RTX 4090 are not the same tier with a speed bump. The 32 GB GDDR7 and Blackwell FP8 Tensor Cores make the RTX 5090 a categorically different GPU for specific workloads.

The upgrade from RTX 4090 to RTX 5090 is worth it primarily for the 13B to 22B FP16 model tier. If your workload is entirely 8B models, the RTX 4090 at $0.39/hr is already cost-optimal. If you are running 13B to 22B at FP16, the RTX 5090 is the only sub-$1/hr option.

Running vLLM on RTX 5090: Setup and Configuration

vLLM on RTX 5090 works identically to any NVIDIA GPU. The RTX 5090 (sm_120 on Blackwell) is supported from vLLM v0.23+ with CUDA 12.4+. On packet.ai instances, CUDA 12.4 is pre-installed.

# Install vLLM on RTX 5090 instance
pip install vllm

# Serve Llama 4 13B at FP16 (fits in 32 GB with KV cache headroom)
vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct \
  --dtype bfloat16 \
  --max-model-len 16384 \
  --gpu-memory-utilization 0.92

# Serve at FP8 for Blackwell hardware-accelerated throughput
vllm serve meta-llama/Meta-Llama-3.1-8B-Instruct \
  --dtype fp8 \
  --max-model-len 32768

RTX 5090 vs L40S: Which GPU for 13B-30B Inference

The L40S has 48 GB GDDR6 but only 864 GB/s bandwidth, versus the RTX 5090's 32 GB GDDR7 at 1,792 GB/s. This makes the RTX 5090 faster for memory-bandwidth-bound inference on models that fit in 32 GB, despite having less VRAM. The L40S wins only when you need 33 to 48 GB, which means 70B at FP8 or large batch KV cache on 30B models.

Frequently asked questions

Last reviewed: July 23, 2026. Rent an RTX 5090 on packet.ai from Launching Soon. For larger models, see L40S (48 GB, from $0.92/hr), H100 SXM (80 GB, from Launching Soon), or browse GPU cluster options.