You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Enterprise deployments need predictable memory use and explicit completion semantics for Messages requests that execute built-in tools. Extend the existing native Messages loop with a documented lifecycle and resource contract across all inference rounds.
Proposed solution
Validate message and content-block transitions through one Messages ingestion path, keeping transport framing and client delivery separate from semantic validation.
Bound retained text, thinking, signatures, tool arguments, tool call outputs, pending calls, and reconstructed history. Distinguish retained bytes from wire bytes so smaller deltas do not consume disproportionate response capacity.
Reuse bounded scheduling contracts for built-in tools, with defined timeout, overload, disconnect, cancellation, and task-cleanup behavior.
Preserve native Messages content, client-executed function tools, rebased indexes, and one public message lifecycle.
Acceptance criteria
Invalid ordering, duplicate indexes/completions, missing identifiers, malformed event data, and open blocks at termination have explicit tested outcomes.
Premature EOF cannot dispatch an incomplete tool call or become an apparently successful message. Document any intentional engine-specific tolerance; retain forward compatibility for harmless unknown events.
Limits cover accumulated content and pending work across rounds, with deterministic errors and bounded resource release.
Tests cover fragmented UTF-8/SSE, many small deltas, large arguments/outputs, slow readers, disconnects during inference/tools, and round-limit exhaustion.
Enterprise Readiness enhancement. Coordinate with #241/#244 and #288 for shared boundary and budget contracts; this issue owns Messages behavior. No worker-placement change is required without measurements.
Problem statement / motivation
Enterprise deployments need predictable memory use and explicit completion semantics for Messages requests that execute built-in tools. Extend the existing native Messages loop with a documented lifecycle and resource contract across all inference rounds.
Proposed solution
Acceptance criteria
Additional context
Enterprise Readiness enhancement. Coordinate with #241/#244 and #288 for shared boundary and budget contracts; this issue owns Messages behavior. No worker-placement change is required without measurements.
Source review at
afa6c14, not a new runtime reproduction: Messages streaming and Messages JSON loop.Enterprise Readiness tracker: #316