Deep dives on GPU infrastructure, model performance, and the economics of compute, from the team building software-defined GPUs at packet.ai.
All
Announcement
Engineering
Guide
Industry
Infrastructure
Product
Technical
packet.ai raised $19M to build the scheduling layer that treats GPU compute as a multi-dimensional resource, not a reserved card. Here's what the round is for, who led it, and what we're building.
Latest
DeepSeek raised API prices on August 16, 2026. Before you budget your next project, here's exactly what V4 Flash costs now — and where solo devs are finding better rates.
DeepSeek, Kimi K3, and Qwen all ship OpenAI-compatible APIs. Here is what each actually costs right now, and when self-hosting beats paying per token.
Z.ai shipped GLM-5.3 four days ago. It scores 66.9 on DeepSWE v1.1 and runs on an OpenAI-compatible endpoint you can point at with one line of code. Here is exactly how.
TTFT is the wait before the first word appears, not how fast the rest streams. Here is what causes high TTFT and how it differs from tokens per second.
Kimi K3 hit #1 on the open-source leaderboard July 2026. Here is what it actually costs to run it, the fastest way to call the API, and how solo devs can access it without a cluster.
Every time you message a chatbot, that is inference, not training. Here is what LLM inference actually means, how it works, and why it costs what it does.
You are not charged per request. You are charged per token. Here is what an AI token actually is, why output tokens cost 4-10x more, and how to stop paying too much.
KV cache is why your second token is fast. Here is how it actually works, why it eats GPU memory at scale, and new research on sharing it across models.
Six open models. Cheaper than Together AI, Groq, and Fireworks on Llama 3.3 70B. $0.02 embeddings, $0.06 classification, $0.85 for DeepSeek-V3. Here is everything on Token Factory pricing and setup.
Nemotron 3 Ultra grabbed the headlines. Super is the one you can actually self-host. Here is the real VRAM math and why its Mamba layers change everything.
DALL-E 3 charges $0.080 per image. Running FLUX.1 Dev on an L40S costs under $0.01 at scale. Here is the full cost math, GPU comparison, and a 10-minute setup guide.
39B active parameters does not mean 39B of VRAM. Here is the real memory math for Mixtral 8x22B, and why 8x7B is not a live recommendation anymore.
4 bits, twice the throughput: NVfp4 is what happens when NVIDIA bakes quantization into Blackwell Tensor Cores. Here is everything an ML engineer needs to know.
Google says 16GB. vLLM says 40GB. Both are right, for different questions. Here is what it actually takes to run Gemma 4 12B on a GPU in the cloud.
AWQ protects the weights that matter before quantizing. GPTQ quantizes everything, then corrects the damage. FP8 skips the algorithm. Here is when each one wins.
A small draft model guesses ahead, the big model checks its work in one pass. Here is why that is lossless, what makes it fast, and how to set it up in vLLM.
Your 70B model needs 70 GB at FP8 versus 140 GB at BF16. That difference is one GPU or two. And on packet.ai, one B200 at $3.75/hr runs it cheaper than two H100s anywhere else.
Prefix caching is already on in your vLLM deployment. Here is what it actually does, how to tune it, and how to tell if it is earning its keep.
Your model needs NVLink but your budget does not allow CoreWeave's 8-GPU minimum or AWS rates. This guide shows when PCIe works, when it does not, and how to access NVLink from $3.75/hr.
Same GPU silicon, two very different packages. Here is what SXM and PCIe actually require in power, cooling, and server hardware before you buy either.
Static batching makes every request wait for the slowest one. Here is the scheduling trick that fixed it, where it came from, and the tradeoff nobody mentions.
TGI is dead. SGLang beats vLLM by 29% on prefix-heavy workloads. TensorRT-LLM needs 28 minutes before its first request. Here is the decision framework that tells you which one to actually run.
Your GPU's TFLOPS rating does not decide how fast it generates tokens. Memory bandwidth does. Here's the spec that actually separates A100-era inference from H200-era inference.
Two models, no cluster required. Here is exactly how much VRAM each Qwen 3.6 model needs, and why the MoE variant is not as light as its 3B active count suggests.
GLM-5.2's 40B active parameters describe speed, not memory. Here's the real VRAM math Zhipu's own deployment recipe confirms, across every precision tier.
Blackwell is the first NVIDIA GPU with a chiplet design. Two dies, one package, 10 TB/s between them. Here is what that actually changes for training, inference, and the cost of running large models.
Every shared GPU platform claims 'seamless' migration. Here's what actually happens under the hood, and why vGPU migration and moving a live CUDA workload are not the same problem.
Most teams rent a single GPU and call it a cluster. Real clusters have two layers of interconnect, three parallelism strategies, and one rule that breaks most training jobs: never cross the node boundary with tensor parallelism.
Kimi K3 shipped less than a day ago, and hardware figures are still settling. Here's what's actually confirmed about VRAM and deployment, sourced directly from Moonshot's own repository.
Flux.1 Dev in FP8 needs 18–23 GB VRAM and generates in 9–10 seconds on an RTX 4090 at $0.39/hr. Full VRAM table for Flux.1 Schnell, Dev, FLUX.2 [klein] 4B and 9B, and every precision level. Real cost per image at packet.ai rates.
DeepSeek V4 Pro and Flash need very different hardware. Here's the real VRAM math for both, and why active parameters won't tell you what actually fits.
Your LLM API bill is not fixed. At 1B output tokens/month, Claude Sonnet 4.6 costs $15,000. Self-hosted Llama 4 70B on an H200 costs $130. Here is the math.
QLoRA fine-tuning a 70B model on a single A100 80GB costs $34–$51 and takes 24–36 hours. H100 at $2.50/hr cuts that to 8–12 hours and $20–$30. Here is the full cost math by method and model size.
A single server's NVLink tops out at 8 GPUs. Here's what changes, technically and practically, once a training run needs InfiniBand-connected servers instead.
32 GB GDDR7 at 1,792 GB/s, Blackwell architecture, from $0.59/hr on packet.ai. The RTX 5090 nearly doubles 30B inference throughput over the RTX 4090 and is the cheapest single GPU that fits 13B-22B models at FP16.
Most teams deploy vLLM and watch GPU utilisation sit at 30%. Here is what PagedAttention and continuous batching actually do to fix that, and how to run it on GPU cloud for a fraction of hyperscaler cost.
HunyuanVideo 1.5 needs a fraction of what the original HunyuanVideo requires, despite sharing the same name. Here's the real VRAM math for every major video model.
24 GB GDDR6X, Ada Lovelace, from $0.39/hr on packet.ai. The RTX 4090 is the cheapest capable GPU for AI. Here is exactly what it runs, what it cannot, and when to upgrade.
Langflow barely needs a GPU at all. The model behind your flow is the real hardware decision, and here's how to size for both without touching your own machine.
AWS raised GPU prices again: $14.24/hr per B200, $6.88/hr per H100. Here is what the same NVIDIA silicon costs on packet.ai, and the math behind the gap.
Most GPU comparisons start with the biggest card and work backward to justify it. L40S starts with what inference actually needs, and prices infrastructure accordingly.
Most teams hit OOM errors before their first successful vLLM request. Working Docker commands, exact VRAM math per model size, and GPU pricing from $0.66/hr.
17B active parameters sounds small enough for any GPU. It isn't the number that actually decides your hardware. Here's the real VRAM math for every Llama model, 8B to 405B.
Bare metal vs virtual machine GPU performance: the benchmark numbers cloud providers quote are single-workload, ideal-condition results. Here is what the data actually shows across inference, training, and multi-node distributed workloads.
Noisy neighbor problem on GPU cloud: why your p99 doubles when a co-located tenant saturates the PCIe fabric, and the four checks that prove it.
The A100 is $1.43/hr. The H100 is $2.50/hr and delivers 2.88x more LLM inference throughput at high concurrency. For QLoRA fine-tuning, total run cost is within 12.6% at 70B. Here is the full benchmark and cost breakdown.
Most people hit a wall running Ollama locally. Here's when renting a cloud GPU solves it, and why renting often beats buying hardware you'll rarely max out.
Your cloud provider lists '1x H100' but doesn't tell you if you're getting the whole card or a MIG slice. Here's how to tell the difference, and why it changes everything about your workload.
You picked the model. Now your deploy keeps OOM-crashing. The fix is knowing exactly which GPU your workload needs before you spin it up and what it costs per hour.
The desktop installer that works on your laptop won't run headless on a cloud server. Here's the setup that actually works, plus which GPU your model needs.
The GPU virtualization penalty is under 4 percent. So why do teams still pay a premium for bare metal? The answer hides in the CPU, the scheduler, and your compliance checklist.
One GPU handles Flux in full precision without breaking a sweat. Another runs out of memory halfway through. Here's exactly which card your ComfyUI workflow needs.
The B200 costs more per hour. Sometimes it costs less overall. Here's the framework for knowing which is true for your job.
The L40S runs 13B models at FP16 and Flux images at near-H100 throughput for $0.92/hr. Here is the cost-per-token math that actually changes your GPU bill.
The H100 vs B200 decision comes down to model size and budget. B200 delivers up to 4x the inference throughput of H100 for AI workloads, but H100 wins on cost per token for models under 70B parameters.
RTX Pro 6000 renters are paying up to 3.6x more than they need to. Here is the real price-per-GPU math, plus what 96GB actually unlocks for LLM inference.
Your GPU isn't always the bottleneck. In agentic AI pipelines, tool calls and orchestration leave GPUs idle for the majority of each request - here's what's actually happening and how to fix it.
B200 GPU pricing spans $3.75 to $27.04 per hour for identical silicon. Here is what drives the 7x spread across clouds, and how to pick a tier without overpaying.
8 LLMs, 8 GPUs, one server. From Qwen 3 30B MoE at 33.4 tok/s to Mistral 123B at 18.5 tok/s - and every model available via Token Factory at $0.10/M tokens.
Deep Blue beat Kasparov with 11.38 GFLOPS in 1996. A B200 GPU today is 4,500,000x more powerful - and rents for $3.75 an hour. Thirty years on, here is what that inflection point actually meant.
GPU pod snapshots save configuration, not disk images. Here’s exactly what survives termination and how to architect persistent storage so you stop losing pip packages.
The GPU utilization % on your monitoring dashboard measures whether any kernel is executing - not how efficiently the GPU is working. A GPU at 99% utilization can be performing at 10% of its theoretical throughput. Here's what to measure instead.
Native SkyPilot integration for packet.ai: <code>sky launch --cloud packet --gpus H100:1 train.yaml</code>. Same YAML, multi-cloud fallback, auto-stop, H100 from $0.65/hr.
Token Factory is packet.ai's OpenAI-compatible inference API - swap the base URL, get open model inference without managing GPU infrastructure. Here's why we built it and when to use it instead of a raw GPU instance.
OpenAI-compatible, $0.10/M real-time, $0.05/M batch, LoRA fine-tuning from $5. Here's exactly how Token Factory works under the hood.
Persistent Workspaces mount a PVC at /workspace so files, packages, and models survive pod restarts. Detach the GPU to stop paying; reattach right where you left off.
packet.ai GPU instances come pre-loaded with CUDA, PyTorch, TensorFlow, and Jupyter - no driver installation, no environment configuration. Here's how to connect VS Code, run JupyterLab, and use Docker with NGC containers on any packet.ai GPU.
H100 from $0.65/hr, H200 from $2.25/hr, B200 from $3.75/hr. No contracts, no spot/reserved maze, no vendor lock-in. Here’s what packet.ai is and how it works.
GPU pricing assumes 30% utilisation - you pay for the idle 70%. Here’s how dynamic placement achieves 5× better utilisation and why that drives packet.ai’s pricing.
Every GPU on packet.ai runs on hosted·ai-powered infrastructure. Not some. All of them. Here’s why that matters for reliability and what it means when something goes wrong.
GPU pricing models compared: on-demand, reserved, and spot explained with real break-even math. Know which model fits your workload before committing to hardware.
July 29, 2026
July 28, 2026
July 27, 2026
.webp)
