Skip to content

feat(patch): opt-in SSE streaming for /v1/patch_sweep grids#236

Merged
RhizoNymph merged 2 commits into
feat/patch-sweep-spansfrom
feat/patch-sweep-stream
Jul 5, 2026
Merged

feat(patch): opt-in SSE streaming for /v1/patch_sweep grids#236
RhizoNymph merged 2 commits into
feat/patch-sweep-spansfrom
feat/patch-sweep-stream

Conversation

@RhizoNymph

Copy link
Copy Markdown
Owner

Stacked on #235 (feat/patch-sweep-spans).

What

Adds opt-in stream: bool = False to PatchSweepRequest. When true, /v1/patch_sweep returns a text/event-stream (SSE) response that emits results as the grid runs instead of holding one JSON response open until the whole sweep finishes:

  • A start event (grid shape / resolved axes / auto_captured) so a consumer can size a live heatmap up front.
  • A cell event per cell as it completes (completion order, no ordering promise), each carrying hook (forward-compatible with multi-hook sweeps), layer, position, value. A cell that voids mid-sweep re-emits with value: null + error, mirroring how voided cells land in skipped.
  • A terminal summary event carrying the exact non-streaming PatchSweepResponse payload, then [DONE].

The fan-out + response assembly are factored into a single async generator both paths consume: streaming serializes each event to SSE; non-streaming drains it and returns the summary. The non-streaming contract is unchanged. Cells still run concurrently (asyncio.as_completed); on client disconnect the outstanding cell tasks are cancelled, aborting their engine requests best-effort.

Pre-fan-out errors (bad hook/layers, span/alignment failure, missing source without clean_prompt, source validation) still return a plain JSON 400 — the stream only starts once the fan-out begins.

Client: PatchStudy.sweep_layers_positions(..., server_side=True, on_cell=fn) sends stream: true, invokes on_cell per cell event, and builds the returned SweepResult from the summary event (identical to the non-streaming result). Without on_cell, behavior is unchanged.

Why

A large grid (all_prompt × many layers on a real prompt = 1000+ cells) holds a single HTTP response open for minutes with zero progress signal; a proxy/client timeout loses the whole nearly-done sweep. Streaming surfaces incremental progress and lets clients drive a live heatmap.

Expected conflicts

Overlaps with sibling branch feat/patch-multihook-lifecycle (multi-hook hooks field + source-run lifecycle) in api_router.py's fan-out / response-assembly sections, protocol.py, and docs/features/activation_patching.md. The cell-event schema already includes hook so the event contract does not need a breaking change when multi-hook lands.

@RhizoNymph RhizoNymph merged commit ae47dbc into feat/patch-sweep-spans Jul 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant