Streaming does not make a model faster. Here is how SSE actually works, why TTFT is a separate question, and what good streaming UX requires.
Waiting for a complete response before showing anything is the single biggest avoidable source of perceived slowness in an LLM product. Token streaming fixes that: the server sends output incrementally as the model generates it, so a user sees words appear as they're produced instead of staring at a blank screen until the whole answer is ready. This guide covers how LLM response streaming actually works over the wire, why Server-Sent Events is used for it, and what a genuinely good streaming UX requires beyond just turning streaming on.
Key takeaways
data: , but a single chunk's delta can contain more or less than one tokenWithout streaming, an API call waits for the model to finish generating the entire response, then returns it as one complete block. For a short answer this barely matters. For a longer response, the user stares at nothing for however long the full generation takes, even though the first few words were ready long before the last ones.
Streaming doesn't make the model reach its first token any faster. It changes what happens after that first token exists: instead of the client waiting for the complete response, it can render each subsequent increment as it arrives. The underlying generation speed, and the time to produce that first token, are identical either way; the packet.ai What Is TTFT guide covers that metric directly. What streaming changes is purely when the client starts receiving and displaying output, which is a real, separate lever from how fast the first token is actually produced.
Server-Sent Events is a web standard for a server to push data to a client over a single, long-lived HTTP connection, without the client needing to poll or re-request. Under the hood, it relies on the same chunked response mechanism HTTP uses generally for sending data before its total size is known, applied specifically to the text/event-stream content type. It's a common transport for llm streaming specifically because the data only needs to flow one direction, server to client. A chat completion doesn't need the client sending data back mid-stream, which is exactly the case SSE is built for; a full-duplex protocol like WebSockets solves a different problem and adds complexity streaming inference doesn't actually need.
An SSE stream is plain text over an ordinary HTTP response, formatted as a sequence of events. In the OpenAI-compatible shape that most providers use, each event is a line starting with data: , followed by a JSON object, followed by a blank line separating it from the next event:
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":"Hello"},"index":0}]}
data: {"id":"chatcmpl-abc","choices":[{"delta":{"content":" world"},"index":0}]}
data: [DONE]Each chunk's delta field contains only the new content for that event, not the full response so far; the client is responsible for accumulating deltas in order to reconstruct the complete message as it streams in. OpenAI-compatible APIs commonly end the stream with a literal data: [DONE] line. Worth being precise here: SSE itself has no requirement for a terminal marker like this; it's an API convention some providers adopt, not a feature of the protocol. Anthropic's API, for instance, doesn't use it at all, ending its stream with a distinct final event instead, covered below.
⚡ Streaming is incremental, not necessarily one-token-per-event
It's common shorthand to describe llm streaming response delivery as token by token, and it's close enough for everyday use, but it isn't a strict guarantee. A single delta can contain a fragment of a token, exactly one token, or several, depending on the model's tokenizer, how the serving engine batches output, and network conditions between server and client. Building a UI that assumes exactly one token per chunk is a fragile assumption; building one that just appends whatever text arrives in each delta is not.
Setting stream: true, or openai stream true in Python SDK terms (stream=True), is the common trigger across OpenAI-compatible APIs, and self-hosted engines like vLLM and Ollama produce the same OpenAI-format SSE chunks by default, which is part of why the OpenAI shape has become a genuine industry convention rather than one vendor's proprietary format. But it isn't universal, and compatibility with that shape doesn't guarantee identical streaming behavior across every provider that claims it. Anthropic's Messages API is the clearest example: rather than a single generic delta.content shape terminated by [DONE], its documented event flow is a specific, named sequence: message_start, then for each content block a content_block_start, one or more content_block_delta events, and a content_block_stop, then one or more message_delta events, and finally a message_stop event that ends the stream. Anything that needs to work across multiple providers, an aggregator, a gateway, or an app switching between providers, needs to normalize these formats rather than assume one shape fits all of them.
Turning streaming on at the API level is necessary but not sufficient for a good real time llm experience. A few practical details separate a streaming implementation that feels responsive from one that technically streams but doesn't feel any faster to use.
Worth stating plainly, since it's easy to conflate the two: a fast time to first token and a good streaming experience are related but separate outcomes. A fast TTFT can still produce a poor streaming experience if the remaining output arrives slowly or the client buffers it before rendering, and conversely, a well-built client can make a middling TTFT feel less noticeable by handling the wait gracefully. Techniques that specifically target getting that first token out faster, like prefix caching, are a separate lever from streaming itself; the packet.ai guide to reducing LLM inference latency covers those techniques directly. On the implementation side, building a fastapi stream response endpoint or its equivalent in another backend framework typically means proxying the provider's own SSE stream through rather than buffering it server-side first, since buffering on your own backend reintroduces the exact delay streaming exists to remove. That backend, in turn, is usually talking to an LLM inference API, and understanding what that layer is and isn't responsible for helps clarify where streaming logic actually belongs.
Since OpenAI-compatible SSE streaming has become a genuine industry convention, a managed inference API that speaks that format works as a drop-in for existing streaming client code. packet.ai's Token Factory is being built as an OpenAI-compatible endpoint with streaming support, so switching to it doesn't mean rewriting how a client consumes responses. It's currently in private preview, with the specific model catalog and pricing still being finalized.
Join the waitlist for early access once it opens.
Last reviewed: September 2, 2026. For the metric streaming most directly improves the perception of, see the packet.ai What Is TTFT guide. For techniques that reduce the wait before the first token specifically, see the guide to reducing LLM inference latency.
Same models. Same API. Fraction of the cost. Start free — no credit card required.
Start Building →