Token Factory: packet.ai's OpenAI-Compatible LLM Inference API

Token Factory is packet.ai's managed LLM inference API: six open models from $0.02 to $0.85 per million tokens, OpenAI-compatible, scale-to-zero, no cold-start tax, no GPU management, and cheaper than every major managed inference provider for equivalent open models.

Most developers running LLM workloads are paying one of two prices: frontier API rates from OpenAI or Anthropic, or the operational cost of managing their own vLLM server on a GPU pod. Token Factory is the third option. You get managed infrastructure with per-token billing, open model quality, and an API shape your existing code already knows.

This post covers every model, every price, how billing actually works, who should use Token Factory versus a raw GPU pod, and how it compares to other managed inference providers. For the full API specification including batch processing and LoRA fine-tune internals, read the Token Factory API deep dive. For the origin story and engineering decisions behind it, see why we built Token Factory. For the full self-hosted vs managed cost comparison, see LLM inference cost in 2026.

What Is Token Factory and How Does It Work?

Token Factory is a hosted inference API. It runs a curated set of open models behind an OpenAI-compatible endpoint. You point your existing OpenAI SDK at api.packet.ai/v1, swap your API key, and your application runs unchanged. packet.ai handles GPU provisioning, model loading, autoscaling, and cold-start management. You handle the application.

The four design priorities from the product page:

Token Factory Model Catalog: All Six Models, Prices, and Context Windows

Token Factory launches with six models covering every major open-weight use case. All prices are indicative for launch, with input and output metered separately at the same per-model rate. Prices sourced from packet.ai/token-factory.

Prices indicative for launch. Input and output metered separately at the same per-model rate. Source: packet.ai/token-factory. Your own fine-tunes are hostable on request at the same billing rate.

How to Pick the Right Token Factory Model for Your Workload

Llama 3.1 8B at $0.06/1M is the default for high-volume, lower-complexity tasks: classification, routing, document summarisation, structured JSON extraction. 128K context handles long inputs without chunking. For any pipeline step that runs millions of tokens per day at simple tasks, start here.

Llama 3.3 70B at $0.59/1M is the most popular model in the catalog for a reason: it handles complex instruction following, multi-turn RAG, agentic tool use, and nuanced writing. The $0.06 to $0.59 jump buys 62 billion additional parameters. For most chatbot and assistant products, this is the right call.

DeepSeek-V3 at $0.85/1M is the catalog's strongest reasoning model. It is a Mixture-of-Experts architecture with a 64K context window, purpose-built for code generation, debugging, and multi-step reasoning chains. If you are building a code review tool or a technical assistant where accuracy matters more than latency, DeepSeek-V3 is the pick.

BGE-M3 at $0.02/1M is the cheapest model in the catalog and the only embedding model. Use it for the retrieval leg of any RAG pipeline. Embedding 100,000 documents at an average of 500 tokens each costs approximately $1.00.

Token Factory vs Together AI, Groq, and Fireworks: Managed Inference Pricing Compared

Token Factory is priced below every comparable managed inference provider for open models. The table below compares rates verified from each provider's official pricing page for Llama 3.3 70B, the most common benchmark for open model inference pricing.

Rates verified from each provider's official pricing page, August 2026. Together AI raised Llama 3.3 70B Turbo from $0.88 to $1.04 in July 2026. Groq input matches Token Factory but output is 34% higher. Token Factory is the only provider in this list that supports LoRA adapter hosting at launch-catalog pricing.

At $0.59/1M for both input and output on Llama 3.3 70B, Token Factory is 43% cheaper than Together AI ($1.04), 34% cheaper than Fireworks ($0.90), and undercuts Groq's output rate by 25%. The output gap against GPT-4o ($10.00/1M) is 16x.

Who Should Use Token Factory: A Practical Decision Guide

Token Factory is the right default for developers, indie builders, and small teams who want LLM inference without infrastructure management. The deciding factor is not company size: it is whether idle cost and operational overhead are acceptable to your project.

The crossover point where a dedicated GPU pod or a dynamic GPU pod becomes cheaper than Token Factory depends on utilisation. At 100% GPU utilisation on an RTX 6000 Pro Dynamic pod ($0.66/hr), you get approximately 2.4 million output tokens per hour from Llama 3.3 70B with vLLM, which costs $1.42/1M at the GPU-hour rate versus $0.59/1M on Token Factory. For most teams running below 40% utilisation (which covers almost every development workload and early-stage product), Token Factory wins on cost every time.

For the full break-even analysis across five GPU billing models, the GPU pricing models comparison covers the exact utilisation threshold where per-token beats per-hour. To understand the self-hosted vLLM path before comparing, the vLLM deployment tutorial covers the full setup on packet.ai GPU pods.

Cost Calculator: What Your Workload Actually Costs on Token Factory

All numbers use verified launch prices from packet.ai/token-factory. Output tokens drive most inference cost, so the table uses output token estimates. Input costs apply at the same per-model rate.

Output token estimates only. Input costs apply at the same per-model rate. Prices verified from packet.ai/token-factory. Highlighted cells show the lowest-cost model for each workload type.

Six Features Token Factory Includes on Every Model

Every model in the catalog ships with the same complete feature set. No tier restrictions, no add-on fees for streaming or function calling.

Use Case Matrix: Which Token Factory Model to Use for Each Job

Token Factory covers the three production categories shown on the product page: chatbots and assistants, RAG and search, and agents and batch. Here is the full model-to-use-case mapping.

Three Steps to Your First Token on Token Factory

Token Factory is launching soon. Setup takes under five minutes once you have an API key.

# pip install openai, then point it at packet.ai
from openai import OpenAI

client = OpenAI(
    base_url="https://api.packet.ai/v1",
    api_key="$PACKET_API_KEY",
)

resp = client.chat.completions.create(
    model="llama-3.3-70b",
    messages=[{"role": "user", "content": "Hi!"}],
    stream=True,
)

curl https://api.packet.ai/v1/chat/completions \
  -H "Authorization: Bearer $PACKET_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"model":"llama-3.3-70b","messages":[{"role":"user","content":"Hi!"}],"stream":true}'

// npm i openai
import OpenAI from "openai";
const client = new OpenAI({ baseURL: "https://api.packet.ai/v1", apiKey: process.env.PACKET_API_KEY });
const stream = await client.chat.completions.create({
  model: "llama-3.3-70b", stream: true,
  messages: [{ role: "user", content: "Hi" }],
});

How Token Factory Billing Works: Scale-to-Zero, No Minimums, No Egress

Token Factory bills per token. Input and output are metered separately at the same per-model rate. There is no platform fee, no minimum monthly spend, no egress charge, and no cold-start tax. You pay for exactly the tokens your application processes and nothing else.

The scale-to-zero property changes the economics for any workload with uneven traffic. A Dynamic GPU pod at $0.66/hr running Llama 3.3 70B with vLLM costs $475 per month at 100% uptime, whether or not it receives a single request. Token Factory costs nothing during idle hours.

Hosting Your Own Fine-Tune on Token Factory

Token Factory supports LoRA adapters and full model checkpoints behind the same OpenAI-compatible endpoint. You fine-tune a base model (Llama 3.3 70B, Llama 3.1 8B, or a compatible base) and upload either the LoRA adapter weights or the full checkpoint, then reference it by model ID in your API calls. packet.ai handles the serving. Your application code does not change. Reference your fine-tune by model ID instead of a catalog model ID.

This covers the most common production deployment pattern for open models: a domain-fine-tuned checkpoint that outperforms the base model on your specific task, served without GPU operations overhead. The same per-token billing applies as for catalog models.

For GPU sizing during the fine-tuning run itself before you hand the checkpoint to Token Factory, the fine-tuning GPU sizing guide covers QLoRA memory math and cost per run on A100 versus H100. For the inference serving side, if you are evaluating self-hosted vLLM versus Token Factory, the SGLang vs vLLM vs TensorRT-LLM guide covers inference engine trade-offs at different scales.

Frequently asked questions

Prices verified from packet.ai/token-factory, August 2026. Indicative for launch; input and output metered separately. Competitor rates verified from official provider pricing pages, August 2026. For GPU infrastructure to self-host LLM inference at sustained high volume, browse available GPU clusters on packet.ai.