The headline discount is not your real savings figure. Here is how to calculate what prompt caching is actually worth for your traffic.
Prompt caching, sometimes called context caching or prefix caching depending on the provider, is the single highest-leverage cost optimization available on nearly every major LLM API today, and most teams running one aren't measuring what it's actually saving them. This guide covers how the discount actually works, prompt caching anthropic, prompt caching openai, and Gemini and DeepSeek's own versions, how to calculate what it's worth for your own traffic, and the structural mistakes that quietly kill your hit rate.
Key takeaways
This post is the cost-and-savings companion to the mechanism itself. For exactly how vLLM's own automatic prefix caching works under the hood, block-level hashing, cache_salt isolation, hit-rate metrics, see the packet.ai vLLM prefix caching guide. This post stays on what the discount is actually worth in dollars and how to calculate it for your own workload, across both self-hosted and hosted-API caching.
Every request to an LLM starts with prefill: the model processing your entire prompt before it generates the first token of a response. If two requests share the same starting content, a system prompt, a set of tool definitions, a retrieved document, a growing conversation history, the first request already did that computation. Prompt caching means the second request doesn't pay for it again.
Mechanically, this is the same idea whether you're self-hosting with vLLM or calling a hosted prompt caching api, and prefix caching llm implementations all share the same core mechanic: the provider stores the KV cache tensors for a prompt prefix, and a later request that starts with an identical prefix gets those tensors reused instead of recomputed. What differs between self-hosted and hosted setups is the billing: on a self-hosted deployment, the savings show up as more available GPU throughput, since less compute is spent per request. On a hosted API, the savings show up directly as a discounted per-token rate on the cached portion of your bill.
Worth being direct about this before comparing numbers: these five rows are not five prices for the same thing. Anthropic cache pricing, context caching gemini, and OpenAI's own automatic version are genuinely different mechanisms wearing similar names, not one discount presented five ways. Anthropic requires you to manually mark cacheable blocks and charges a write premium; Gemini splits into two separate products with different guarantees; OpenAI needs no setup at all but won't commit to one fixed percentage. Reading the discount column alone, without the setup column next to it, will give you the wrong picture of what each provider actually offers.
Figures reflect each provider's published rates as of August 2026, verified directly against each provider's own current documentation. Providers change pricing and caching structure without much notice, sometimes by model generation rather than across the board; check current documentation for the specific model you're calling before budgeting off any single number here.
The genuinely important thing this table doesn't fully capture on its own: Gemini's implicit caching isn't a lower-guarantee version of explicit caching, it's a different risk entirely. Google's own documentation is explicit that implicit caching passes on the discount only when a cache hit actually occurs, with no guarantee that it will for any given request; explicit caching is the only path that guarantees the discount, at the cost of manual setup and a storage fee. OpenAI's variation is generational, not random: pre-GPT-5.6 models use a flat 50% discount with no write cost, while GPT-5.6 and later models use a different structure entirely, with a 1.25x write cost and a discount that varies by the specific model within that generation. Anthropic's structure has stayed the most consistent across model generations of the four, though its own documentation notes minimum cacheable prompt lengths differ by model, so even Anthropic's numbers should be checked against the specific model you're calling rather than assumed to be identical across the lineup.
The real prompt caching cost math is the same regardless of provider, once you know your discount rate and the cache hit rate llm teams should be tracking on their own traffic:
effective_input_cost = (uncached_fraction × full_rate) + (cached_fraction × discounted_rate)
savings_percent = 1 − (effective_input_cost / full_rate)A worked example using DeepSeek V4 Flash's current off-peak rates ($0.22 per million tokens on a cache miss, $0.007 on a cache hit): a workload with a stable system prompt achieving a 90% hit rate on its input tokens pays an effective input rate of (0.10 × $0.22) + (0.90 × $0.007) = $0.028 per million tokens, an 87% reduction from the uncached rate, even though the cache discount itself is roughly 97%. That eleven-point gap between the discount rate and the actual savings is your hit rate showing up directly in the bill: a 90% discount at a 20% hit rate would save far less than this example, and a lower-discount provider at a higher hit rate can easily beat a higher-discount provider at a lower one. Ask what hit rate is realistic for your actual traffic before comparing headline discount numbers against each other.
Caching, on every provider, depends on an exact prefix match: the cached and new requests need to be identical from the very first token up to wherever the shared content ends. The single most common way teams accidentally defeat this: putting variable, per-request content, a user ID, a timestamp, a session token, ahead of the static system prompt or shared context in the prompt template.
If that variable content sits at the front, the prefix match breaks immediately, and none of the static content behind it gets any caching credit, no matter how much of it is genuinely identical across requests. The fix is structural, not a configuration flag: put static, shared content first (system instructions, tool definitions, shared documents), and put variable, per-request content last. This single reordering is frequently the difference between a near-zero hit rate and a high one, on any provider, with no other changes required.
⚡ Where else hit rate quietly leaks
Beyond prompt ordering: some providers' caches expire after a period of inactivity, so a workload with long gaps between repeated requests may see lower hit rates than the same traffic arriving more densely. Multi-workspace or multi-tenant setups can also unintentionally fragment what would otherwise be a shared cache into several smaller, less-effective ones. Both are worth checking directly against your traffic pattern rather than assuming a stable prefix alone guarantees a high hit rate.
Whether caching is earning its keep for a specific workload is an empirical question, not something to assume from having it enabled. Every major provider's API response includes the token breakdown needed to calculate a real hit rate: separate fields for cached versus uncached input tokens on a given request. Logging those two numbers over a representative traffic window, not a single request, gives you the actual ratio to plug into the savings formula above, rather than an assumed or hoped-for number.
For a self-hosted vLLM deployment specifically, the packet.ai vLLM prefix caching guide covers the exact Prometheus metrics to track and how to calculate hit rate from them.
Whether you're self-hosting or calling a provider's API, the underlying economics are the same: caching is close to a free win whenever your traffic has real, repeated structure, and something worth actively checking for rather than assuming. packet.ai's Token Factory is being built around per-token billing with these efficiency mechanisms applied at the serving layer, so the savings from patterns like prompt caching apply without requiring separate configuration on your end. Token Factory is currently in private preview, with its model catalog and pricing still being finalized.
Join the waitlist for early access once it opens.
Last reviewed: August 27, 2026. Provider pricing changes without much notice; verify current rates against each provider's own documentation before budgeting. For the mechanism itself in a self-hosted vLLM deployment, see the vLLM prefix caching guide.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →