EventHandler return values: stream yielded/returned values to the frontend #7608

Summary

Let a frontend caller dispatch a backend event and consume every value the handler yields or returns as an async iterator:

const stream = addEvents([ReflexEvent("state.search", { query })]);
for await (const hit of stream) {
  render(hit);
}
// or, for a handler that simply returns one value:
const rows = await addEvents([ReflexEvent("state.load_rows", { page })]);

Async-iterator style is used for every dispatch because it is the superset of the Python handler shapes Reflex already supports: a plain return x handler is a stream of one value, a sync generator yields N values (plus an optional return), an async generator yields N values.

Each dispatched event carries a client-generated request id so that several in-flight requests on one connection can tell their results apart.

Motivation

Today the only value round-trip is in the opposite direction: rx.call_script / rx.call_function evaluate JS and feed the result to a Python handler via a callback. There is no way for JS to call a Python handler and get a value back. That blocks a whole class of React integrations whose props expect an async callback that resolves with data, for example:

• async option loaders (loadOptions on select/autocomplete components)
• server-side row models / infinite scroll datasources (getRows)
• validation callbacks that must resolve to an error message
• upload pre-flight hooks that must resolve with a URL or a reject reason
• enterprise helpers that today fake it with rx.call_script + fetch() and a global promise chain (see need a way to serialize frontend events #6766)

It also gives JS a clean way to await completion of a backend handler (await addEvents(...) resolves when the handler finishes), which is the missing primitive behind the frontend serialization problem in #6766.

#6223 already anticipates this ("Return values are sent back in the final delta — eventually allowing event calls to return a promise on the frontend"); this issue is the design for that part.

Current behavior

• addEvents(events, args, event_actions) (useEventLoop, packages/reflex-base/src/reflex_base/.templates/web/utils/state.js) returns the promise from queueEvents, which resolves once the event has been emitted on the socket, not when the backend has processed it. When the dispatch is debounced or throttled away, applyEventActions returns undefined.
• Wire shape frontend -> backend: {name, payload?, event_actions?, handler?, router_data}. EventNamespace.on_event (reflex/app.py) keeps only _EVENT_FIELDS (name, router_data, payload) when building the Event, so unknown wire fields are ignored by the backend (reflex-otel relies on that for traceparent).
• Backend -> frontend: one socket message type, "event", carrying a serialized StateUpdate {delta?, events?} (reflex/state.py). StateUpdate.final is deprecated and unused.
• process_event (packages/reflex-base/src/reflex_base/event/processor/base_state_processor.py) drives coroutine / sync / generator / async-generator handlers. Every yielded value (and a sync generator's return value) goes through chain_updates, which flushes the delta and then _check_valid_yield raises TypeError for anything that is not None, an Event, EventHandler, EventSpec, or a sequence of those (tests/units/test_state.py::test_state_with_invalid_yield).
• The backend already has a per-event id: EventContext.txid (packages/reflex-base/src/reflex_base/event/context.py), generated server-side, used as the key for EventProcessor._futures / _tasks and inherited as parent_txid by chained events. EventFuture resolves when the handler task finishes.
• default_backend_exception_handler (reflex/app.py) already defines what a handler failure may tell the browser: "{Type}: {message}" outside prod mode, "Contact the website administrator." in prod mode.

Proposed design

Frontend API

addEvents(...) returns a result stream object:

• implements [Symbol.asyncIterator]: yields each value the handler yielded/returned, in order, interleaved with (and ordered after) the state deltas that preceded each value on the backend;
• is thenable: await stream resolves with the last value the handler produced (undefined when it produced none) and rejects when the handler raised. "Last value" is the one rule that fits every handler shape: a Python async generator cannot return a value, and a sync generator's return x is by construction its last value. Promise resolution auto-unwraps thenables, so a compiled event chain can be passed straight to a React prop expecting (args) => Promise<T>;
• completes when the invoked handler finishes (success, exception, or cancellation). It is scoped to the invoked handler only: values produced by events the handler chained (yield State.other) are not included, and the stream does not wait for them. No wait_all variant in this item.

When addEvents is called with several events, the stream merges their values in arrival order and completes when all of them have completed. Call addEvents once per event to get separate streams.

Debounce and throttle. A debounced call attaches the caller to the dispatch that eventually fires: every call that lands in the same debounce window gets the same stream object, which is bound to the real stream when the timer fires, so an autocomplete that debounces keystrokes can await from any keystroke and receive the result of the one that was sent. A throttled-out call attaches to the most recent dispatch that fired for the same action key while it is still open; if none is open it gets an already-completed empty stream. Frontend-only events (_redirect, _call_script, ...) and REST-dispatched uploads complete immediately with no values.

Wire protocol

Frontend -> backend: add one field to the event object, id (string), generated per dispatched event in applyEvent right before socket.emit. Cheap and unique per tab: a random per-page-load prefix plus an incrementing counter (a reload wipes the pending map, so a late result from a pre-reload background handler must not collide with a new id).

Backend -> frontend: add a results field to StateUpdate, a list of per-request records, so results ride the existing "event" socket message and inherit its ordering, its lost-and-found routing under RedisTokenManager, and its disconnect handling:

{"results": [{"id": "ab12:7", "value": 42}]}
{"results": [{"id": "ab12:7", "done": true}]}
{"results": [{"id": "ab12:7", "error": "ValueError: bad page"}]}

Where possible the done record is merged into the StateUpdate that carries the handler's final delta flush, so the common case (a handler that changes state and returns nothing) costs no extra socket message. A no-op handler costs one small done message it did not send before.

The error text follows the same rule as default_backend_exception_handler: "{Type}: {message}" outside prod mode, a fixed generic message in prod mode. The stream rejects either way; the backend exception handler (and its toast) still runs as today.

Backend

• Event gains an optional id: str | None wire field (added to _EVENT_FIELDS; validated as a bounded-length str in on_event). It is never set on events the backend creates, so chained events report nothing.

• EventContext gains request_id: str | None, copied from event.id by EventProcessor.enqueue and not propagated by fork(). The client-chosen id is deliberately kept separate from txid: _futures and _tasks are keyed globally by txid, so a client must not be able to pick it.

• EventContext gains an emit_result_impl channel next to emit_delta_impl / emit_event_impl, wired in EventProcessor.configure to emit_update(StateUpdate(results=[...])). Handlers never call it directly; chain_updates does.

• chain_updates / _check_valid_yield classify each yielded object:

  • None: delta flush only (unchanged);
  • Event / EventHandler / EventSpec / lambda, or a sequence of those: events (unchanged);
  • anything else: a result value, emitted after the delta flush through ctx.emit_result(value) when request_id is set, dropped otherwise;
  • a sequence mixing events and non-events stays a TypeError.

  This is a strict relaxation: every input that becomes a result today raises TypeError.

• A sync generator's return value and a coroutine's return value go through the same path (they already reach chain_updates).

• Completion is emitted from the future-done path (EventProcessor._finish_task / _resolve_future), so every request with an id terminates exactly once: done on success, error on exception (after the backend exception handler ran), done with cancelled: true when superseded or cancelled. For background handlers that is when the background task ends.

• Values are serialized with the same serializer registry as deltas, so dataclasses, pydantic models, datetimes etc. work; a non-serializable value raises on the backend and surfaces as an error record.

Interaction with #6223 (awaitable EventHandler). Backend consumption of results is out of scope here: nothing outside the processor can obtain an EventFuture today, and #6223 is where self-queuing handlers will return one. To keep that work cheap, results are produced through the context channel (ctx.emit_result) rather than written straight to the socket, so a future can later subscribe to the same channel; request_id is the only thing that ties a result to a socket. The two items should agree on await semantics: await handler in #6223 resolving with the last value (and iteration giving each value) mirrors the frontend thenable defined above.

Frontend runtime

• pending = Map<id, stream> in state.js; applyEvent registers the stream before emitting.
• socket.on("event"): after dispatching update.delta and before queueing update.events, route each update.results record to its stream (value -> push, done -> close, error -> fail). Unknown ids (stale after reload) are ignored.
• socket.on("disconnect"): fail every pending stream with a connection error; a reconnect re-hydrates state anyway, and results emitted while disconnected are dropped by emit_update today.
• queueEvents / addEvents return the stream (merged for batches).
• applyEventActions: the debounce path keeps one deferred stream per debounce key (the helper's ${name}__${delay} key) and hands it to every call in the window; invokeTarget binds it to the real stream when the timer fires. The throttle path records the last open stream per key and returns it to throttled-out calls.

Compatibility

• Old frontend + new backend: no id on the wire, so no results are emitted; handlers that yield values just run (previously they raised).
• New frontend + old backend: id is filtered out by _EVENT_FIELDS; streams never complete. Acceptable only during dev; the frontend is compiled against its backend, so this pairing does not happen in a built app.
• No change to the compiled JS of ordinary event triggers beyond the id.
• No component changes, so no .pyi regeneration.

Out of scope (follow-ups)

• Typed results: honor a handler's return annotation (-> int, -> AsyncIterator[Row]) so the compiler can type the stream / Var.
• A Python-side helper to pass a handler as an async callback prop with the result type declared, once typed results exist.
• Forwarding the eval result of _call_script / _call_function into the stream of a frontend-only dispatch.
• Backend consumption of results (EventHandler redesign: awaitable semantics #6223).
• wait_all semantics (stream completes when the whole chain settles).
• Using the completion promise to serialize frontend events (need a way to serialize frontend events #6766).

Implementation plan

1. Backend protocol (reflex-base, reflex): Event.id, EventContext.request_id + emit_result_impl, StateUpdate.results, classification in chain_updates, completion/error records from the future-done path. Unit tests in tests/units/reflex_base/event/processor/test_base_state_processor.py, tests/units/test_state.py (update test_state_with_invalid_yield), tests/units/test_app.py (on_event accepts/validates id).
2. Frontend runtime (state.js): id generation, pending map, stream object (async iterator + thenable), routing in socket.on("event"), disconnect handling, return value of addEvents, debounce/throttle attachment in applyEventActions.
3. Integration test (Playwright, extend an existing app under tests/integration/tests_playwright/): a handler that yields three values then returns one; JS driven via page.evaluate consumes the stream with for await and await, asserts ordering against a state delta emitted between yields, asserts rejection on a raising handler, asserts two concurrent requests do not cross results, and asserts two calls inside one debounce window share one result.
4. Docs + news: new section in docs/events/yield_events.md ("Yielding values to the frontend"), a note in docs/api-reference/browser_javascript.md, a wrapping-react recipe for async callback props; news/+event-results.feature.md and packages/reflex-base/news/+event-results.feature.md.