Together AI is an inference company first. Its serverless API serves hundreds of open models per token with a fast proprietary engine (speculative decoding, FlashAttention-3, custom kernels), and it layers dedicated endpoints, fine-tuning and on-demand GPU clusters on top. For a team that wants an OpenAI-shaped API for open models, it is one of the two or three obvious choices.
packet.ai is a GPU cloud whose inference product, Token Factory, is on a waitlist. Today the comparison is between buying tokens and buying GPU-hours; once Token Factory ships it becomes tokens versus tokens. This article covers both, with the numbers that exist now.
Platform overview: packet.ai vs Together AI
What Together AI offers
Together's pricing has four layers. Serverless per-token: Llama 3.3 70B at $1.04 per million tokens in and out, smaller models from $0.03/M, frontier open models to about $4.50/M, batch at 50% off. Dedicated endpoints: H100 $5.49–6.49/hr, B200 $8.99–11.95/hr depending on tier. On-demand GPU clusters: H100 $3.99–4.79/hr, H200 $5.99/hr, B200 $8.19/hr, with 7–30 day reservations bringing the H100 to about $4.19/hr and 91–180 days to $3.29/hr. Fine-tuning: per processed token, then hosted on a dedicated endpoint.
- One of the fastest open-model inference engines available; throughput per GPU is the core pitch
- No GPU below the H100 on any tier
- Dedicated endpoints bill by the hour whether or not traffic arrives
- Fine-tuned weights are downloadable; hosting them costs the dedicated-endpoint rate
What packet.ai delivers
Together sells inference three ways above the GPU itself: serverless per-token, a dedicated endpoint, or an on-demand cluster you run your own stack on. The last one is the direct comparison to packet.ai, and it is where the gap is largest. Together's cheapest B200 (on-demand cluster) is $8.19/hr; packet.ai's B200 starts at $3.75/hr Dynamic, a 54% gap, before you even get to the dedicated-endpoint tier at $8.99–11.95/hr.
- Dynamic: scheduler-enforced multi-tenant, hourly, run your own inference engine. B200 starting at $3.75/hr, 54% below Together's cheapest B200 cluster rate.
- Dedicated: a whole card, single-tenant, 99% SLA. RTX 4090 at $0.39/hr, L40S at $0.92/hr, A100 80GB at $1.43/hr, B200 at $6.99/hr, still below Together's cluster rate for the same card.
What Together has that packet.ai does not yet: a live per-token API with hundreds of models and a proprietary inference engine tuned for throughput. Token Factory, packet.ai's answer, is on a waitlist with early published pricing for Kimi K3 at $1.50/M input and $7.50/M output. Until it ships, the honest comparison is Together's tokens against packet.ai's GPU-hour plus your own serving stack, which is cheaper at steady volume but takes more setup.
Billing is hourly with monthly commits up to 20% off. There are no platform fees and no ingress charges; egress is $0.04/GB. Up to 2 TB of local NVMe per node is included. Dynamic instances are SSH-ready in under five minutes, Dedicated in five to ten. Capacity is live in the US (California, Virginia, Texas, Oregon) and Europe (Frankfurt, Amsterdam, Paris, London, Dublin). H100 SXM, H200 and RTX 5090 are on the notify list.
Against Together, packet.ai's GPU-hour is a fraction of the price: a B200 at $3.75/hr Dynamic against $8.19/hr for Together's cheapest B200 (on-demand cluster) or $8.99/hr and up for a dedicated endpoint. You run vLLM or SGLang yourself, which means you also keep the throughput gains Together sells back to you. For pay-per-token, Token Factory is on a waitlist with early pricing published for Kimi K3 at $1.50/M input and $7.50/M output.
packet.ai vs Together AI at a glance
Published starting rates in USD per GPU-hour, on-demand unless noted. Percentages compare the first price in each cell.
| Category | packet.aiYOU | Together AI |
|---|---|---|
| B200 GPU-hour | $3.75/hr Dynamic · $6.99/hr Dedicated−54% | $8.19/hr cluster · $8.99–11.95/hr dedicated endpoint |
| H100 GPU-hour | Launching soon (notify list) | $3.99–4.79/hr cluster · $5.49–6.49/hr endpoint |
| Cheapest GPU on the rate card | RTX 4090 $0.39/hr | H100 $3.99/hr (cluster, on-demand) |
| Per-token LLM API | Token Factory (waitlist); Kimi K3 preview $1.50/M in · $7.50/M out | Live; Llama 3.3 70B $1.04/M; batch 50% off |
| Model choice on API | Curated open models at launch; bring-your-own on GPU tier | Hundreds of open models, serverless |
| Inference engine | Yours (vLLM, SGLang, TensorRT-LLM) on your GPU | Together's proprietary engine |
| Fine-tuning | Run any framework on Dynamic/Dedicated GPUs, hourly | Managed, per processed token; hosting at endpoint rate |
| Idle cost | Hourly while instance is up | Serverless: none · Dedicated endpoint: full hourly rate |
| Multi-node training | InfiniBand clusters to 1,024+ GPUs, ~30% below retail on term | On-demand clusters to 256 GPUs; reserved larger |
| Data transfer | No ingress fee; egress $0.04/GB | Not itemised for API; cluster storage billed separately |
Detailed comparison
Tokens or GPU-hours: doing the maths
Together's serverless Llama 3.3 70B at $1.04 per million tokens is a great price for low or unpredictable volume. At high, steady volume the maths flips. As an illustrative example: a single B200 on packet.ai at $3.75/hr running a 70B model in FP8 with vLLM can, with well-tuned batching, produce well over a million output tokens an hour, which pushes the effective cost per million tokens toward the low single dollars and keeps falling as you batch harder. Actual throughput depends heavily on your serving stack, batch size and sequence length, so treat this as a planning estimate rather than a benchmark.
Together's answer to that is its dedicated endpoint, but at $8.99–11.95/hr for a B200 you are paying two to three times packet.ai's rate for the same card plus Together's engine. Whether the engine's throughput advantage is worth 2–3× is the whole decision, and it depends on how good your own vLLM tuning is.
What Together does that packet.ai does not (yet)
A live per-token API with hundreds of models, zero idle cost, and a managed fine-tuning pipeline whose output you can download. If you are shipping a product this month and do not want to run an inference server, Together is the finished article.
packet.ai's Token Factory is the direct competitor and it is on a waitlist. Early published pricing for Kimi K3 is $1.50/M input and $7.50/M output with a free tier of a million tokens a day. Until it ships, packet.ai's inference story is GPU-hours plus your own serving stack.
What packet.ai does that Together does not
Anything below the H100. Together's rate card starts at $3.99/hr; packet.ai's starts at $0.39/hr. A 7B or 13B model on an RTX 6000 Pro at $0.66/hr, an image pipeline on an L40S at $0.92/hr, an embedding service on an RTX 4090 at $0.39/hr: none of these have a home on Together.
Also: control. Your GPU, your engine, your quantisation, your logs, your data path. For teams with data-residency or model-IP concerns, a Dedicated card in a known US or EU data centre with a 99% SLA is a different proposition from a shared inference API.
Fine-tuning economics
Together charges per processed token for fine-tuning and then bills the resulting model at dedicated-endpoint rates ($5.49/hr H100 and up) for as long as it is deployed. On packet.ai a LoRA fine-tune of a 70B model on a Dynamic B200 is $3.75 an hour for however many hours it takes, and serving it afterwards is the same $3.75/hr card. For iterative fine-tuning the hourly model is usually cheaper and always more predictable.
Who should choose which
Choose packet.ai if
- You have steady inference volume and want cost per million tokens to fall as utilisation rises
- Your models fit RTX 6000 Pro, L40S or A100 - cards Together AI has no raw GPU offering for
- You want to run your own serving engine and keep the throughput and latency gains
- You fine-tune iteratively and want hourly GPU costs rather than per-token pricing
Choose Together AI if
- You want a live per-token API today with hundreds of open-source models and no infrastructure
- Traffic is low or spiky and zero idle cost matters more than the per-unit rate
- You want a managed fine-tuning pipeline with downloadable weights and no GPU management
- You need Together's inference engine performance without building or operating your own stack
Conclusion
Together AI is inference as a product. packet.ai is inference as a GPU you own for the hour, at a fraction of Together's per-GPU price, with a per-token product (Token Factory) on the way. Low, spiky volume: use Together's serverless API. Steady volume or anything below an H100: rent the card on packet.ai. When Token Factory ships, revisit.
Same silicon. Smarter economics.
Every rate on this page is a published starting price you can deploy against today. Dynamic launches in under five minutes; no credit card to start.
Additional resources
- packet.ai GPU pricing: the full Dynamic, Dedicated and Clusters rate card.
- packet.ai documentation: API, CLI and platform guides.
- How intelligent scheduling works: why Dynamic keeps peak performance at a lower price.
- Clusters and wholesale quotes: multi-node InfiniBand capacity at ~30% below retail.
- Rent GPUs on packet.ai: specs, pricing and workload fit for every card.