Every LLM deployment decision in 2026 starts with one question: open-weight or closed? For solo developers, the answer determines your cost structure and ops overhead. Here is what each path actually costs.
Open-weight models release their weights publicly so you can download, run, and fine-tune them on your own hardware. Closed-weight models keep parameters private and route every request through a vendor API, and you pay per token with no way out.
Key takeaways
The open vs closed question trips up more AI builds than almost anything else. Not because it's hard to answer, but because most people answer it too early, before they know what their traffic looks like or how much ops overhead they're willing to carry.
This guide is for developers weighing self-hosted open-weight models against managed closed APIs. If you're new to inference altogether, start with the LLM inference primer, then come back.
An open-weight model publishes its trained parameters. You download the weights from Hugging Face, load them into vLLM or SGLang on a GPU you rent or own, and run inference without calling anyone's API.
What you don't get: training data, training code, or any insight into how those weights were produced. You get the finished model, not the recipe that made it.
The distinction has legal teeth. The Open Source Initiative's OSAID 1.0 (October 2024) requires training data, training code, and weights, all under a permissive license. By that bar, almost nothing popular qualifies. Llama 4, Qwen3, DeepSeek R1, Mistral's recent releases all withhold training data. Pythia and OLMo satisfy the full definition, but they're research models, not the ones anyone deploys in production in 2026.
For most builders, the open-source/open-weight gap is academic. The real question: can you run it commercially, and does the license box you in? That varies by family, which is why the licensing section below matters more than the Hugging Face model card.
Closed-weight models are services. OpenAI, Anthropic, and Google run inference on their own hardware. You send a prompt, pay per token, get a response. You cannot download the model, you cannot run it locally, and there is no workaround for that.
This isn't a technical limitation. It's a business decision. The weights are the product. That works fine until the API doesn't offer what you need: data that can't leave your environment, custom fine-tuning on proprietary data, air-gapped deployment, or a bill that doesn't run at 3am while your app sits idle.
Read the actual license file before you deploy anything commercially. Not the README, not the model card, the license.
Apache 2.0 and MIT (Mistral, Qwen3, DeepSeek) are the lowest-friction licenses for commercial shipping. No user caps, no AUP that can change under you. If you're building on Llama 4, the 700M MAU threshold is fine for most teams. The clause letting Meta rewrite the AUP without notice is the one worth running past legal before you're dependent on it.
This decision isn't mainly about model quality. It's about which constraints you're willing to own. Here's how the two approaches actually compare across the things that affect a real deployment.
Control
Open-weight wins
Your hardware, your config. Run in a VPC, air-gapped, or on-prem. Fine-tune on data that never leaves your environment. Quantize to fit your VRAM budget.
Cost at Scale
Open-weight wins above ~10M tokens/day
GPT-4-equivalent API cost went from $20/M tokens in late 2022 to ~$0.40/M in 2026. Open-weight on GPU cloud still runs 5-10x cheaper at volume once hardware is amortized.
Compliance
Open-weight is often the only option
HIPAA, GDPR, classified environments, and sending data to a third-party API may not be an option. Air-gapped self-hosting is the only path for some workloads.
Capability
Closed-weight leads at the frontier
GPT-5, Claude Opus 5, Gemini still lead on general benchmarks. Open-weight closes roughly 70-90% of that gap. Fine-tune on a specific domain and that gap often disappears.
Latency
Depends entirely on setup
Closed APIs share rate limits across all tenants and p99 gets ugly under load. A dedicated GPU with your model loaded gives predictable latency, if it's sized right.
Ops Overhead
Closed-weight wins on day one
Closed API: one SDK call, done. Open-weight: model selection, VRAM sizing, serving framework, quantization, hardware, maintenance. Real cost that doesn't show up on a per-token comparison.
The crossover where self-hosting beats closed API cost lands somewhere between 10M and 30M tokens/day, depending on model size, utilization, and your input/output ratio. Below that: ops cost often wipes out the per-token savings. Above it: the math flips hard.
Hourly GPU prices don't mean much if you're building a side project. Here's what the numbers actually look like when you work in sessions, not SLAs.
A Qwen3-7B model at INT4 on an RTX 4090 at $0.39/hr costs $1.95 for a 5-hour build session. A full weekend of testing comes to under $8. A closed API charges you whether your app is shipping or sleeping.
Monthly billing saves up to 20% on top of that. An RTX 6000 Pro on a monthly commit drops from $0.66/hr to roughly $0.53/hr. No contracts, no minimum node counts. Full rate card here.
Egress is $0.04/GB vs $0.09/GB on AWS. 2TB NVMe local storage included per node, which matters when you're storing model weights.
Open-weight isn't the default right answer. It's the right answer in specific situations. These are the ones that actually matter.
Most teams in 2026 do both. Start on a closed API to ship fast. Once you know your traffic shape, move the high-volume workloads to open-weight. See the cheapest LLM API options in 2026 for where that crossover actually lands by model size.
Get the VRAM wrong and you're either OOM-crashed on launch or running at utilization so low the economics don't work. The rough rule for 4-bit quantized inference: half the parameter count in GB. A 14B model needs roughly 8GB; a 70B needs roughly 40GB. Add headroom for KV cache, which grows with batch size and context length.
Default serving stack in 2026: vLLM for batch throughput, SGLang for prefix-heavy RAG and multi-turn. Hugging Face moved TGI to maintenance mode in December 2025, so skip it for anything new. Full framework comparison: SGLang vs vLLM vs TensorRT-LLM.
A 70B Qwen3 model at INT4 on a single A100 80GB via vLLM on packet.ai costs $1.43/hr. At 50% utilization across concurrent requests, that's roughly $0.0003 per 1K output tokens.
Every open-weight guide covers VRAM math. Almost none of them cover what comes after: you still need to run the thing reliably in production. That's the cost most per-token comparisons skip.
Running open-weight in production means hardware provisioning, vLLM config (quantization, tensor parallelism), model loading, autoscaling, health checks, GPU memory monitoring, and keeping the serving framework patched. None of it is impossible. All of it takes time that most solo developers don't have to spare.
The utilization trap
A GPU at 30% utilization means 70% of your hourly cost is waste. The economics only work when you run the hardware hard: high concurrent volume or batched jobs. Solo developers with bursty, low-volume traffic often end up paying more per token than they would have on the closed API they left.
The typical path: build on a closed API, hit the cost wall, decide to self-host, then hit the ops wall. At that point there are two exits: hire someone to run it, or move to a managed inference layer that handles the GPU side. Token Factory is built for that second exit.
packet.ai is GPU cloud built for open-weight models. You can't run GPT-4o or Claude on your own hardware. But you can run Llama 4, Qwen3, DeepSeek V3, Mistral Large 3, Kimi K3, and GLM-5 from $0.39/hr, no credit card required, no minimum commitment.
Egress is $0.04/GB vs $0.09/GB on AWS. 2TB NVMe per node included. No ingress fees. For a fine-tuning cost breakdown by GPU and model size, see the best GPU for fine-tuning guide.
Two paths. If you want to skip the serving stack entirely, Token Factory is managed inference on packet.ai hardware: OpenAI-compatible, pre-loaded models, zero GPU config. If you want full control (SSH, custom vLLM, your own quantization), browse cluster options or spin up a Dedicated POD with a 99.99% SLA.
Last reviewed: September 10, 2026. No credit card required to start on packet.ai. Token Factory handles open-weight inference so you don't have to touch a GPU. Or browse GPU options from $0.39/hr.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →