Skip to content

feat(zig): chunked transfer encoding for StreamingResponse / SSE end-to-end #163

@justrach

Description

@justrach

Sub-task of #146.

Current state (verified)

  • `python/turboapi/sse.py` defines `EventSourceResponse(StreamingResponse)` with all the right SSE wire-format helpers (`ServerSentEvent`, `format_sse_event`, ping wrapper, etc.). test: re-enable TestWebSocket + add SSE wire-format coverage (#146) #162 added 18 unit tests locking the format down.
  • `python/turboapi/responses.py` defines `StreamingResponse` with a `body_iterator()` async generator and `body = b""` placeholder.
  • `python/turboapi/request_handler.py` reads `response.body` directly in every dispatch path (fast handler, async fast handler, enhanced handler, pos handler) and ships that as the body. Result: `StreamingResponse` / SSE responses currently send empty bodies through the Zig server.
  • `zig/src/server.zig` has zero references to `StreamingResponse`, `text/event-stream`, `Transfer-Encoding`, or chunk-based body sending.

Gap

To make SSE actually stream, the Zig HTTP path needs to:

  1. Detect `StreamingResponse` in dispatch — request_handler returns a sentinel tuple shape (e.g. `(status, content_type, _STREAM_SENTINEL, py_iterator)`) instead of `(status, content_type, body)`.
  2. Send headers + chunked transfer encoding — `Transfer-Encoding: chunked` in the response header block; suppress the `Content-Length` synthesis path.
  3. Loop pulling chunks — call back into Python via a callback (or vectored Python C-API) to pull the next chunk from `body_iterator()`. Format each as `\r\n\r\n` and write to socket.
  4. Honor client disconnect — short-circuit if peer closes (currently we'd churn forever feeding a dead socket).
  5. Async generator integration — `StreamingResponse.body_iterator` is an `async def` — need a way to drive the event loop one step at a time from Zig, or move the iteration to the Python async pool that the server already uses for async route handlers.
  6. Keep-alive vs close — chunked responses end with the `0\r\n\r\n` zero-length chunk; connection can stay alive afterward.

Suggested implementation order

  1. Sync iterator path first — `StreamingResponse(content=iter([...]))` with a plain Python iterator. No event-loop integration needed; just a Zig-side "pull next chunk" callback that calls Python's `next(it)`. Gets chunked transfer encoding through end-to-end with the simplest test surface.
  2. Async iterator path second — wire the async pool already used by `create_fast_async_handler` to drive `body_iterator()`'s `anext`. Reuse the pool; don't spawn a new event loop per request.
  3. Disconnect handling — `writev`/`send` failure → close the iterator (`gen.aclose()`), free Python refs.
  4. Test plan — un-skip `test_event_source_response_end_to_end_over_zig_server` in `tests/test_sse.py` and add: bounded SSE event count, infinite SSE with client cancel, large per-chunk body (10MB), chunk slicing across socket boundaries, gzip middleware interaction (must NOT accidentally buffer-and-compress chunked responses).

Why this matters

A user (in #146) is building a ChatGPT-style backend on TurboAPI — token-stream SSE is the primary use case. Currently the framework's SSE story is non-functional. Implementing this unlocks LLM streaming, server push notifications, log tailing, and real-time analytics endpoints.

Risk: medium. The Zig changes are localized to the response-write path and can be gated behind the `StreamingResponse` sentinel — non-streaming responses are unaffected. Async-iterator integration is the trickiest part and may need its own design pass.

Out of scope for this issue: WebSocket. Tracked separately (extension of #114).

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions