A rented GPU costs the same idle or busy. Here is the real formula for when self-hosting actually beats a managed API, and when it does not.
Self hosting llm infrastructure and calling a managed API aren't different tiers of the same thing, they're different cost structures entirely: one is largely a fixed cost regardless of usage, the other scales with usage but carries no separate idle cost. Whichever is cheaper for you depends on how many requests you're actually sending and how continuously you send them, not on which option sounds more sophisticated. This guide covers the actual break-even math, why it depends heavily on which GPU and which model tier you're comparing, and what other than raw cost should factor into the decision.
Key takeaways
Self-hosting means renting or owning GPU capacity and paying for it by the hour (or amortizing a purchase), largely regardless of how much of that capacity you actually use. A managed api bills per token processed, with no separate charge for idle time between requests. These aren't two prices for the same thing; they're two different ways of being charged, and which one is cheaper depends entirely on how continuously you're actually using the capacity.
This distinction is why the question "is self-hosting llm infrastructure cheaper than an API" doesn't have a fixed answer. It depends on your specific request volume, your specific GPU choice, and the specific model tier you'd otherwise call through an API. Anyone giving you a flat answer without asking about your utilization is skipping the part of the analysis that actually determines it.
The comparison comes down to one relationship: self-hosting is cheaper once your effective cost per token, GPU hourly rate divided by tokens actually generated per hour at your real utilization, drops below the managed API's per-token rate for a comparable model. Written out:
self_hosted_cost_per_token = gpu_hourly_rate / (tokens_per_hour_at_your_utilization)
break_even_when: self_hosted_cost_per_token < managed_api_rate_per_tokenThe critical variable most rough estimates get wrong is "tokens per hour at your utilization," not "tokens per hour at maximum throughput." A GPU's theoretical maximum tokens-per-second, the number in a benchmark, only applies if the GPU is continuously busy. A GPU serving requests only 20% of the time still costs close to the full hourly rate for that entire hour, but only generated tokens during a fifth of it, meaning its effective cost per token is roughly five times higher than the benchmark number would suggest.
A concrete illustration makes this easier to apply. Suppose a GPU rents for $2/hr and can sustain 100,000 tokens/hour at full utilization, an effective cost of $0.02 per 1,000 tokens if kept continuously busy. At 20% utilization, that same GPU still costs $2 for the hour but only produces 20,000 tokens, pushing the effective cost to $0.10 per 1,000 tokens, five times higher purely from the utilization drop. If a comparable managed API charges $0.05 per 1,000 tokens, this GPU only wins the comparison above roughly 40% utilization; below that, the API is cheaper despite the GPU's lower headline hourly rate. The exact break-even utilization shifts with your own rate and throughput, but the shape of the math doesn't.
⚡ GPU rental rates vary too much to cite one number honestly
As of current market pricing, an A100 80GB rents anywhere from roughly $0.80/hr on marketplace platforms to over $5/hr on major hyperscalers; an H100 ranges from roughly $1.49/hr to over $14/hr depending on provider, commitment level, and whether it's spot or on-demand capacity. Purchasing hardware outright shifts the same tradeoff into an amortization question, a large upfront cost spread over the card's useful life, rather than an hourly rate, but the underlying utilization logic is the same either way. This is not a small spread. Any break-even calculation built around a single assumed GPU rate should be treated as illustrative, not authoritative, until you substitute your own actual quoted rate.
The same caution applies to the managed API side of the comparison, and it's easy to miss since a single headline "LLM API pricing" figure implies more consistency than actually exists. Efficient, smaller open-weight models served through hosted providers can run under $1 per million tokens combined; frontier-tier models can run $30 to $75 per million output tokens alone. That's a 50 to 75x spread within the same market, which means "self-hosting vs API" is really a different comparison depending on which model tier you'd otherwise be calling.
Self-hosting a small, efficient model that would only cost pennies per million tokens through an API rarely clears the break-even bar except at very high, consistent volume, since the API side of that comparison is already cheap. Self-hosting a model that would otherwise cost $30+ per million tokens through a frontier API clears that bar at meaningfully lower volume, since the API alternative is expensive enough that even modest, non-ideal GPU utilization can beat it.
Request volume and utilization sound similar but aren't the same thing, and conflating them is the most common mistake in this analysis. A high volume of requests concentrated in bursts, with long idle gaps between them, still produces low utilization if the GPU sits idle, and still largely billed, during those gaps. Steady, predictable, round-the-clock traffic at a moderate volume can produce high utilization even without an enormous total request count.
This is why "requests per day" alone is an incomplete way to frame the question, even though it's the natural first number people reach for. The same daily request count spread evenly across 24 hours produces dramatically better GPU utilization, and therefore a dramatically better self-hosted cost per token, than the same count concentrated into a few peak hours with the GPU otherwise idle. Bursty, unpredictable, or low-to-moderate volume traffic tends to favor a managed API for exactly this reason: you're not paying full price for the idle hours a self-hosted GPU would still largely bill you for.
Cost is the variable this post focuses on, but it isn't the only one that should factor into the decision, and treating it as the sole input risks a technically correct but practically wrong conclusion. Self-hosting gives you access to any model you can run, not just what a provider's catalog includes, which matters if you need a specific fine-tune or an open-weight model no managed provider currently serves. Self hosted inference also means the operational work, serving engine configuration, scaling, uptime, security patching, sits with your own team rather than a provider's.
A managed API removes that operational burden entirely in exchange for being limited to the provider's catalog and their infrastructure choices. Neither tradeoff is universally correct; it depends on whether your team has the capacity and expertise to take on serving infrastructure, and whether the specific model you need is even available through a managed provider in the first place.
Since the break-even point depends entirely on your own utilization and model choice, the only reliable way to answer this for a specific workload is running the actual math against your own traffic pattern, not a generic industry number. If you're still gathering that data, a managed inference api lets you observe real request volume and shape before committing to self-hosted infrastructure sized for a guess. packet.ai's Token Factory is being built as this kind of starting point, an OpenAI-compatible endpoint that bills per token so you can measure real usage patterns, though the exact per-token rates for specific models are still being finalized ahead of general availability.
Join the waitlist for early access once it opens.
Last reviewed: September 11, 2026. GPU rental rates and managed API pricing both change frequently; verify current rates against your specific providers before finalizing a decision. For the fundamentals of managed inference access, see the packet.ai What Is an LLM API guide.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
