How LLM Streaming Responses Work (Server-Sent Events)

When a chatbot’s reply appears word by word instead of all at once, that’s streaming — the server sends generated tokens to the client as soon as the model produces them, rather than waiting for the complete response before sending anything. The mechanism behind most of these interfaces is Server-Sent Events (SSE), a decade-old, purpose-built web standard for exactly this kind of one-way, incremental delivery from server to browser.

Why LLMs need streaming in the first place

A large language model doesn’t produce its output all at once — it generates one token at a time, each one conditioned on everything generated before it, in a loop that continues until the response is complete or hits a stop condition. A long response can take several seconds to fully generate. Without streaming, the client sends a request and then simply waits — no output at all — until every token exists, at which point the entire response arrives in one burst.

Streaming closes that gap by sending each token to the client the moment it’s generated, rather than making the user wait for the whole batch. The total generation time doesn’t change, but the perceived latency drops dramatically — the user sees the first word appear in a few hundred milliseconds instead of staring at a blank space for the full duration of generation. This is the same instinct behind Core Web Vitals prioritizing when something visible happens over pure total load time: perceived responsiveness and actual completion time are different things, and streaming optimizes for the one users actually feel.

Why SSE, specifically

A streaming LLM response is naturally one-directional: the server has something to say, continuously, until it’s done, and the client doesn’t need to send anything back mid-stream. That’s exactly the shape Server-Sent Events was designed for — a long-lived HTTP connection where the server pushes a sequence of text-based events, and the browser’s built-in EventSource API (or a simple fetch with a readable stream reader, which is what most LLM API clients actually use for finer control) consumes them as they arrive.

A raw SSE response for an LLM API typically looks like a sequence of small, incrementally sent chunks:

data: {"delta": "The"}

data: {"delta": " capital"}

data: {"delta": " of"}

data: {"delta": " France"}

data: {"delta": " is"}

data: {"delta": " Paris"}

data: [DONE]

Each data: line is a discrete event, typically carrying a small JSON payload with the next chunk of text — a delta — that the client appends to what it has displayed so far. A final sentinel event signals the stream is complete.

Why not WebSockets?

WebSockets support full two-way communication over a single persistent connection, which is more capability than a chat completion actually needs mid-response — the client isn’t sending anything back while tokens are streaming down. SSE is a simpler protocol that runs over plain HTTP, works through the same proxies and load balancers as any other HTTP request without special handling, and includes automatic reconnection behavior built into the browser’s EventSource API. For a strictly server-to-client stream like LLM token generation, SSE is the leaner tool for the job; WebSockets earn their complexity when the client also needs to push data back in real time mid-conversation, such as live voice input.

	Server-Sent Events	WebSockets
Direction	Server → client only	Bidirectional
Protocol	Plain HTTP	Own protocol, upgraded from HTTP
Built-in reconnection	Yes, via EventSource	No, must be implemented manually
Typical LLM use	Streaming chat completions	Real-time voice, live collaborative input
Proxy/firewall friendliness	High — it’s just HTTP	Can require special handling

What happens on the client

The client’s job is simple but has to happen fast enough to feel instantaneous: read each incoming delta, append it to a growing string, and re-render. In practice this means the UI is re-rendering many times per second for a fast-generating model, which is why chat interfaces typically render the accumulated text as it arrives rather than waiting to parse and format a complete, final response — full markdown parsing and syntax highlighting are often applied progressively, or finalized once the stream ends, precisely because reformatting the entire visible output on every single token would be wasteful.

Streaming and cancellation

Because a stream is a long-lived connection rather than a single request-response cycle, it can be interrupted cleanly — a user clicking “stop generating” closes the underlying connection (commonly via an AbortController on the client side), and a well-built server stops generating further tokens for that request rather than continuing to compute output nobody will see. This is a meaningful cost consideration for anyone billing by generated tokens: a canceled stream that stops generation immediately, rather than finishing the full response server-side regardless, is the difference between paying for exactly what was actually shown to the user and paying for output that was thrown away.

Streaming and reasoning models

Streaming gets more nuanced with reasoning models, which may generate a substantial internal reasoning trace before producing the final answer. Some APIs stream the reasoning tokens too, so users watching a chat interface can see the model “thinking” through the problem; others withhold the reasoning trace entirely and only stream the final answer, either for product design reasons or because the reasoning content isn’t meant to be user-facing at all.

The takeaway

LLM chat interfaces feel responsive because they stream generated tokens to the browser as they’re produced, almost always over Server-Sent Events — a simple, one-directional, plain-HTTP protocol that’s a better fit for this job than the added complexity of a WebSocket connection the client doesn’t actually need to write back over. The total time to generate a full response doesn’t change, but streaming turns a long silent wait into immediate, incremental output, which is most of what makes an LLM interface feel fast in practice.