Server-Sent Events (SSE) is a strong default for logs, AI token streams, job progress, and other flows where the server sends most of the data. FastAPI now provides native EventSourceResponse and ServerSentEvent primitives. The hard production work is not the yield; it is defining event identity, recovery, proxy behavior, cancellation, backpressure, authorization, and observability.

When should you choose SSE?

SSE is a one-way event stream carried over HTTP. Browsers parse text/event-stream and can reconnect with the last accepted event ID. That makes it simpler than WebSocket when commands can use ordinary HTTP and updates flow mainly from server to client.

Requirement SSE WebSocket Polling
One-way server updates Excellent fit Works, but heavier Higher latency and request overhead
Continuous two-way traffic Poor fit Best fit Poor fit
Existing HTTP auth and proxies Usually simple Requires upgrade-aware infrastructure Simple
Resume hint Last-Event-ID Application-defined Cursor in each request
Browser API EventSource WebSocket fetch

Use WebSocket for continuous audio upload, collaborative cursors, or game state. Use SSE when the product needs to deliver generation output, build status, or state changes. The same state-first principle appears in ZoyTown's revision-safe bilingual publishing architecture: transport success must not be confused with business-state commitment.

What does a recoverable FastAPI endpoint look like?

The official FastAPI SSE guide documents EventSourceResponse, ServerSentEvent, and Last-Event-ID. The following is an engineering skeleton, not a claim about a measured deployment:

from collections.abc import AsyncIterable
from typing import Annotated

from fastapi import FastAPI, Header, Request
from fastapi.sse import EventSourceResponse, ServerSentEvent

app = FastAPI()


async def events_after(cursor: int) -> AsyncIterable[dict]:
    # Example boundary: use an append-only log, outbox, or durable stream.
    for item in await event_store.list_after(cursor, limit=100):
        yield item


@app.get("/runs/{run_id}/events", response_class=EventSourceResponse)
async def run_events(
    run_id: str,
    request: Request,
    last_event_id: Annotated[str | None, Header()] = None,
) -> AsyncIterable[ServerSentEvent]:
    cursor = int(last_event_id) if last_event_id else 0
    async for item in events_after(cursor):
        if await request.is_disconnected():
            break
        yield ServerSentEvent(
            data={"runId": run_id, "status": item["status"]},
            event="run.status",
            id=str(item["sequence"]),
        )

An event id should be a stable resume cursor, not a decorative random value. Persist the business event before exposing it to subscribers. If the service sends first and stores later, a failure in that gap creates an event the client saw but can never replay.

How should Last-Event-ID recovery work?

On reconnect, the browser sends the last accepted id in Last-Event-ID. Define five invariants:

  1. IDs are monotonic within a stream partition, or otherwise provide a total order.
  2. The read contract means “strictly after this cursor.”
  3. Consumers are idempotent because duplicates can still occur at network boundaries.
  4. Retention exceeds the normal offline window.
  5. An expired cursor produces an explicit reset path rather than silently jumping to “now.”
event: stream.reset
data: {"reason":"cursor_expired","snapshotUrl":"/runs/42"}

Do not encode access tokens, email addresses, or sensitive compound database keys in the cursor. A valid cursor is not authorization. Every read still needs tenant and resource filters derived from the authenticated principal.

Why does Nginx deliver events in bursts?

A reverse proxy may buffer upstream output. The Nginx proxy_buffering documentation explains that an upstream X-Accel-Buffering response header can control buffering. FastAPI's native SSE response sets X-Accel-Buffering: no, uses Cache-Control: no-cache, and emits keep-alive pings, but deployment configuration still deserves an explicit test.

location /api/streams/ {
    proxy_pass http://api;
    proxy_http_version 1.1;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;

    proxy_buffering off;
    proxy_cache off;
    proxy_read_timeout 75s;
}

Do not copy an extremely long timeout without reasoning. The proxy timeout should be longer than the heartbeat interval, and the same relationship must hold at the load balancer and CDN. Validate time-to-first-event, not merely a final 200 response.

How should heartbeats, timeouts, and cancellation interact?

The HTML Living Standard SSE section defines stream parsing and reconnection behavior. A heartbeat is normally a comment line, not a fake business event. FastAPI emits pings during idle periods, but application work must remain cancellable:

  • Put cancel-aware timeouts around queue and database waits.
  • Stop cursors, model streams, and downstream requests when a subscriber leaves.
  • Avoid broad exception handlers that swallow CancelledError.
  • Separate job execution from subscription. A disconnected browser should not cancel a task that was already committed unless that is an explicit product action.

This separation mirrors dynamic SSR content delivery: the response path and the underlying content state are independently verifiable. An open SSE connection is likewise not proof that a job succeeded.

How do you control backpressure?

An unbounded per-client queue turns one slow tab into a memory leak. Use a bounded buffer and choose a policy by event type:

  • State updates may coalesce intermediate values and keep the latest state.
  • Audit events must not be dropped; let the client catch up from a durable log.
  • Token streams can batch small chunks while preserving order.
  • High-frequency metrics can be sampled by time window.

When the buffer is full, record a low-cardinality reason, close the connection, and require a cursor-based rebuild. Do not log full prompts, generated tokens, or sensitive event bodies as error context.

What are the authorization traps?

Native browser EventSource does not offer the same arbitrary-header control as fetch. Same-origin cookie authentication is straightforward, but it requires CSRF defenses, SameSite policy, origin checks, and short sessions. If you mint a subscription ticket:

  1. Scope it to one resource and one action.
  2. Give it a very short lifetime and preferably one-time use.
  3. Do not put a durable secret in a URL that proxies, analytics, or referrers may retain.
  4. Re-evaluate resource authorization on the server.

For cross-origin streams, avoid wildcard CORS with credentials. First challenge whether cross-origin delivery is necessary; a same-origin reverse proxy is often the simpler security boundary.

What should a production test prove?

curl -Nsv https://example.com/api/streams/42 \
  -H 'Accept: text/event-stream'

Test the following behaviors:

  • Content-Type is text/event-stream and the CDN does not cache the stream.
  • The first event arrives within the expected interval instead of at connection close.
  • Heartbeats arrive before the shortest proxy idle timeout.
  • A forced disconnect reconnects with Last-Event-ID and resumes after that event.
  • Duplicate delivery cannot duplicate a charge, write, or notification.
  • Closing the client releases downstream generators and cursors as designed.
  • Connection count, duration, disconnect reason, and resume lag are observable.
  • Logs exclude credentials, URL tickets, and full sensitive payloads.

Common failure modes

The endpoint returns 200, but the UI updates every few seconds

That is usually buffering in Nginx, a CDN, compression middleware, or an application wrapper. Compare timestamps at each layer, bypass proxies one at a time, and inspect X-Accel-Buffering plus proxy_buffering.

Reconnects lose events

Likely causes are send-before-store behavior, an incorrect >= versus > cursor query, or insufficient retention. Inject failures between store, send, and acknowledgement to prove the contract.

Memory grows while connection count stays flat

Inspect bounded queue sizes, orphaned generator tasks, and database cursors. Stable connection count does not imply stable resource use.

What should the browser client implement?

EventSource reconnects automatically, but the product still needs a visible state machine. This example demonstrates state handling only; it is not an authentication design:

const stream = new EventSource("/api/streams/42");

stream.addEventListener("run.status", (event) => {
  const payload = JSON.parse(event.data);
  renderStatus(payload.status);
});

stream.addEventListener("stream.reset", async (event) => {
  stream.close();
  const { snapshotUrl } = JSON.parse(event.data);
  await reloadSnapshot(snapshotUrl);
});

stream.onerror = () => {
  renderConnectionState("reconnecting");
};

Do not create a new EventSource immediately inside every onerror callback. The browser already has a reconnection algorithm; layering another uncontrolled loop can create a connection storm. Decide whether background tabs remain subscribed based on event value, mobile energy cost, and the server connection budget.

How does SSE scale across workers?

An in-process asyncio.Queue can serve only the worker holding that connection. If producers and subscribers may run in different processes or hosts, use a shared event source such as Redis Streams, a message broker, a database change stream, or an outbox. Evaluate:

  • Does the system retain a resumable cursor, or only broadcast transient pub/sub messages?
  • Can catching-up consumers be paged and rate-limited?
  • Does the partition key preserve order for one resource?
  • When the broker fails, does the endpoint pause, return 503, or read from a database?
  • Can retention expire a valid Last-Event-ID earlier than the product promises?

Do not introduce a broker merely because the transport is SSE. A bounded in-process queue may be enough for an internal single-process tool. Add a durable event layer when cross-worker delivery, recovery, or auditability becomes a real requirement.

FAQ

Can an SSE response use POST?

FastAPI documents SSE over POST, but the browser's native EventSource constructor directly issues GET requests. Use a streaming fetch client for POST, or separate “create the job” as POST from “subscribe to the job” as GET.

Should SSE responses use gzip?

Compression middleware can buffer small chunks and increase time-to-first-event. Compression is not universally forbidden, but flush behavior must be measured through the real proxy chain. If it cannot be proven, disable response transformation for the event-stream route.

Should every generated token be one event?

Not necessarily. Tiny events increase framing, scheduling, and rendering overhead. Batch by a small character threshold or short time window while preserving order, cancellation, and a clear final event.

A practical decision rule

The smallest reliable SSE loop is: durable event, stable ID, bounded read, disabled proxy buffering, heartbeat before timeout, cursor-based recovery, idempotent consumption, and auditable resource cleanup. Prove that loop before adding multiplexing or another messaging layer. For evidence and claim boundaries in technical publishing, see the official-source GEO/AEO guide.