The L40S runs 13B models at FP16 and Flux images at near-H100 throughput for $0.92/hr. Here is the cost-per-token math that actually changes your GPU bill.
The NVIDIA L40S price on packet.ai is $0.92/GPU-hour dedicated: 48 GB of Ada Lovelace GDDR6 that runs Llama 3.1 13B at FP16 without quantisation and generates Flux and SDXL images at near-H100 throughput for a fraction of the cost.
Key takeaways
The nvidia l40s price on packet.ai sits at $0.92/GPU-hour dedicated, a figure that changes the cost-per-token math for production LLM serving and image generation compared to H100-class hardware. Most teams buying GPU cloud time are not running frontier-scale training. They are serving a 7B or 13B model behind an API, generating images for a product pipeline, or fine-tuning on domain data once a week. For those workloads, an H100 at $2.50/hr is the wrong answer, not because H100 is a bad GPU, but because you are paying for bandwidth and compute you will never saturate at moderate concurrency. The L40S at $0.92/hr is where that gap closes.
This post covers what workloads the L40S on packet.ai fits, real throughput numbers for LLM inference and image generation, and an honest comparison with the H100 so you can match the right GPU to your bill.
The L40S is built on NVIDIA's Ada Lovelace architecture, the same generation as the RTX 4090, but in a data-centre form factor with ECC memory, passive cooling, and a 350W TDP designed for 24/7 rack operation. The 18,176 CUDA cores and 568 fourth-generation Tensor Cores give it 91.6 TFLOPS of FP32 compute and 1,466 TFLOPS of FP8 with sparsity enabled.
The 48 GB GDDR6 at 864 GB/s is the defining spec for most inference workloads. It is enough to hold a 13B model at FP16 with room for KV cache, or a 34B model at 4-bit quantisation. That puts common open-source models (Llama 3.1 8B, Mistral NeMo 12B, Qwen3 14B, Mixtral 8x7B at INT4) comfortably within a single card without tricks.
What the L40S does not have: NVLink interconnect and MIG partitioning. Both are found on H100 SXM and H200. For tensor-parallel inference on 70B+ models or full fine-tuning runs that need fast all-reduce gradient sync, those omissions matter. For single-card or PCIe multi-card inference on sub-34B models, they do not.
LLM inference on the L40S is memory-bandwidth-bound at low batch sizes and compute-bound at higher batch sizes. The 864 GB/s GDDR6 bandwidth is sufficient for 7B to 13B models at batch sizes up to roughly 8 to 16 simultaneous requests, which covers most production chatbot and API endpoints that are not running at hyperscale concurrency.
A single L40S running Llama 3.1 8B via vLLM in FP16 delivers approximately 46 tok/s at batch size 1 and 336 tok/s at batch size 8. At $0.92/hr on packet.ai, that is approximately 1.32 million tokens per dollar at batch 8, a cost figure that holds up well against H100 pricing for endpoints running below sustained high concurrency.
Enabling FP8 via vLLM's --quantization fp8 flag roughly doubles effective throughput at memory-bound regimes on the L40S, since the 4th-gen Tensor Cores support native FP8 computation through the Transformer Engine. The A100, by comparison, lacks native FP8 and requires INT8 quantisation workarounds that add complexity without matching L40S FP8 efficiency gains.
For API serving and RAG pipelines where predictable p99 latency matters as much as raw throughput, the dedicated single-tenant card on packet.ai removes noisy-neighbour interference, a real problem on shared infrastructure that shows up as tail latency spikes under load.
Diffusion model inference (SDXL, Flux.1, Stable Diffusion 3) is compute-bound on the UNet and DiT backbone, not memory-bandwidth-bound the way autoregressive LLM decoding is. That changes the hardware calculus. The L40S's 91.6 TFLOPS FP32 compute is approximately 11% higher than the RTX 4090's 82.6 TFLOPS, while the 48 GB GDDR6 is double the RTX 4090's 24 GB VRAM.
For single-image SDXL generation at 1024x1024, the L40S delivers per-image throughput comparable to the H100, with H100 holding a modest advantage from higher compute density that only becomes meaningful at sustained high-concurrency batch generation. At the L40S's $0.92/hr versus an H100's $2.50/hr on packet.ai, the L40S generates images at roughly 2.7x lower cost per image for most production workloads.
The 48 GB VRAM removes the ceiling for Flux.1 dev and schnell at BF16, SDXL with ControlNet stacks and LoRA compositing, and video diffusion models like Wan Video, workloads that frequently OOM on 24 GB cards. ComfyUI workflows that chain multiple models (upscaler, IP-Adapter, ControlNet) on a single card run without the memory management overhead that 24 GB deployments require. For image generation at scale, Pixel Factory on packet.ai offers a managed API alternative for teams that prefer pay-per-image over managing GPU infrastructure.
Teams running a daily image generation pipeline (product renders, synthetic data, creative assets) find the L40S monthly rate of $604 on packet.ai significantly more predictable than per-image API pricing, which compounds fast above a few thousand images per month.
The L40S versus H100 comparison comes down to one number: your P50 batch size in production. Below batch 16, the L40S's lower hourly cost per GPU typically wins on cost-per-token even though the H100 generates tokens faster. Above batch 16, H100's 3.35 TB/s HBM3 bandwidth advantage compounds and the H100 pulls ahead on tokens per dollar.
Price per GPU per hour, dedicated single-tenant (packet.ai, July 2026)
packet.ai L40S dedicated at $0.92/hr is 63% cheaper per hour than the H100 SXM at $2.50/hr on the same platform, for workloads at moderate concurrency where the H100's throughput advantage does not close that cost gap.
packet.ai offers two billing models for the L40S. Dedicated hourly at $0.92/GPU-hour gives you a full, single-tenant card reserved exclusively for your workload with a 99.99% SLA and zero noisy-neighbour risk. Monthly flat rate at $604/month locks in the same single-tenant card at a fixed cost, approximately 35% below the hourly rate at continuous utilisation.
For teams running a production LLM inference endpoint continuously, the monthly rate at $604 is the right starting point. For teams experimenting with model serving or running batch jobs a few hours a day, the hourly rate keeps costs proportional to actual usage. Both plans provision in under 5 minutes and run on single-tenant dedicated hardware.
For multi-node L40S cluster deployments, packet.ai supports InfiniBand interconnect from 8 to 512 GPUs at wholesale pricing.
The L40S is a capable fine-tuning GPU for models up to 13B parameters at FP16 using QLoRA or LoRA. A 13B QLoRA run that takes 4 hours costs approximately $3.68 on an L40S at $0.92/hr, versus roughly $10 on an H100 at $2.50/hr. For teams running weekly or daily fine-tunes on domain data, that cost difference adds up to thousands of dollars per year.
The PCIe Gen4 interconnect limits multi-GPU gradient synchronisation throughput compared to NVLink-connected H100 SXM or H200 clusters. For full fine-tuning of 30B+ models with tensor parallelism and all-reduce synchronisation, the H100's NVLink fabric is the right choice. For QLoRA on 7B to 13B models on a single card, the L40S is sufficient and significantly cheaper.
For larger fine-tuning workloads (70B parameter models, RLHF, or full-parameter SFT at scale), see H100 SXM pricing on packet.ai or B200 SXM from $3.75/hr for the highest throughput option.
Last reviewed: 15 July 2026. Deploy an L40S on packet.ai from $0.92/hr, or browse available GPU clusters for multi-node inference at scale.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
