You are not charged per request. You are charged per token. Here is what an AI token actually is, why output tokens cost 4-10x more, and how to stop paying too much.
An AI token is the smallest unit of text an LLM processes, roughly 4 characters or 0.75 words in English, and every token you send or receive is what you pay for on any inference API.
Key takeaways
Here is the thing most people miss when they get their first AI API bill: you are not charged per request. You are not charged per second. You are charged per token, and tokens add up faster than you expect.
For solo builders and indie developers, this is the most important number to understand. A sloppy prompt structure can double your costs overnight without changing your output quality at all. This post explains what tokens actually are, why they cost what they cost, and how to use packet.ai Token Factory to keep your inference bill small from day one.
A token is not a word. It is not a character. It sits somewhere between the two, and the exact split depends on the model.
LLMs use subword tokenization, typically Byte-Pair Encoding (BPE), to break text into chunks the model can process numerically. Common short words usually become a single token. Rare or long words get split into pieces. Punctuation, whitespace, and numbers each get their own token slots.
These numbers are averages for English prose. Switch to Python code, JSON payloads, or Japanese text and the token count per word climbs fast. Two prompts that look similar in length can cost very different amounts depending on what is inside them.
Before an LLM can process your prompt, every token gets mapped to an integer ID from the model's vocabulary. GPT-4 uses a vocabulary of roughly 100,000 tokens; Llama 3 uses 128,000. That integer ID becomes an embedding vector, which is what the model actually runs attention over.
If a word is not in the training vocabulary, it gets split into subword fragments, each one using a separate token slot. The word "tokenization" might be one token on a model trained heavily on technical text, or three tokens on a smaller-vocabulary model. This is why technical jargon, brand names, and variable names inflate token counts beyond what a simple word count would suggest.
Watch out
Different models use different tokenizers. A prompt that costs 800 tokens on GPT-4 may cost 950 on a Llama-based API. Always test your actual prompts before locking in a cost estimate.
Not all tokens cost the same. Here is what each category means for your bill.
Why are output tokens so much more expensive? Because generation is autoregressive. The model runs a full forward pass through all its parameters for every single token it writes, one at a time. Reading your input is one parallel pass. Writing the response is thousands of sequential passes. That compute gap is what the price gap reflects.
In most real apps, input tokens already outnumber the user's actual message by 10-50x once you add in system prompts, conversation history, and RAG context. For solo builders watching a tight budget, trimming that invisible overhead is usually where the biggest savings are hiding.
The 1 token = 0.75 words estimate holds for clean English prose. It falls apart quickly for everything else.
If you are building a product for Japanese users and benchmarked costs on English prompts, expect a nasty surprise. The tokenizer does not care about your budget. Always test with real production data, not a clean English demo.
The billing formula is simple: cost = (input_tokens x input_rate) + (output_tokens x output_rate). Providers publish rates per million tokens. Where you host the model is often more important than which model you pick.
Together AI rates from together.ai official model pages, August 2026. packet.ai rates from packet.ai/pricing.
packet.ai Token Factory runs Kimi K3 at $1.50/$7.50 per million input/output tokens, half the Together AI list price. Llama 3.3 70B is $0.30 flat, versus $1.04 on Together AI. That 71% gap is not a sale. It is the permanent result of running on owned B200 GPU infrastructure at 80-100% utilisation instead of 20-40%.
Free tier. No card required.
packet.ai Token Factory gives you 1 million free tokens per day. No credit card. No trial period that auto-bills you. Just an API key that works. Start building free.
Want to understand how Token Factory handles inference end to end? The Token Factory deep dive covers the full stack.
You do not need a new model or a new provider to reduce costs. These four changes are all on your side of the API call.
Trim your system prompt
Your system prompt runs on every single request. A 2,000-token system prompt on 1M daily requests costs $3.00 per day at $1.50/1M input tokens, and the user has not even typed anything yet. Cut it to 500 tokens and you save $2.25/day, $67/month, from one edit. Start there before touching anything else.
Set max_tokens on every call
Output tokens cost 4-10x more than input. If you leave max_tokens unset, the model will write until it decides it is done. For classification, 10-20 tokens is enough. For summaries, try 150. Set the limit explicitly and you will be surprised how rarely you actually need the default.
Pass fewer RAG chunks
RAG pipelines are quiet budget killers. Teams routinely pass 10 retrieved chunks when 2-3 would do. That is 5,000-20,000 extra input tokens per request. Get your retrieval precision right and you will cut input costs faster than any other single change.
Use Llama 3.3 70B for the boring tasks
Llama 3.3 70B at $0.30/$0.30 per million tokens handles most classification, summarisation, and instruction-following tasks that teams default to frontier models for at 5-10x the cost. Save Kimi K3 for the hard jobs. Use Llama for everything else. And if you want to self-host rather than use an API, packet.ai Dynamic GPU gets you a live RTX 6000 Pro in under 5 minutes at $0.66/hr.
Last reviewed: August 2026. Build without the bill shock. packet.ai Token Factory is OpenAI-compatible, runs Kimi K3, DeepSeek V4, and Llama 3.3, and starts free. Or explore GPU clusters when you need dedicated multi-node capacity.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →
