You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Operators need the full inference cost of a Messages turn, including rounds used by built-in tools. Provide consistent, documented accounting for JSON and SSE clients without counting repeated usage snapshots more than once.
Proposed solution
Add a typed per-round usage accumulator and a turn aggregate shared by the JSON and streaming Messages paths.
Define input, output, cache-read, and cache-creation accounting for supported upstream fields. Distinguish missing usage from zero and retain documented provider semantics.
Specify how complete turn usage is exposed while preserving the native Messages streaming schema. If the wire protocol cannot represent every total, document a separate typed telemetry surface rather than inventing wire fields.
Separate per-round records from the aggregate so downstream metering counts each inference contribution once.
Acceptance criteria
Deterministic one-, two-, and multi-tool-round fixtures verify totals for JSON and SSE.
JSON and SSE expose equivalent complete-turn accounting through the documented surfaces.
Errors, cancellation, client-tool handoff, and round-limit exhaustion report available usage as partial when appropriate; they never fabricate a completed total.
Usage from hidden built-in-tool rounds is included without exposing their internal content or double-counting the public aggregate.
Document units, accounting boundaries, and examples for operators.
Additional context
Enterprise Readiness enhancement. Coordinate with #279 for OpenTelemetry export; this issue owns usage semantics and correctness, not a billing platform.
Problem statement / motivation
Operators need the full inference cost of a Messages turn, including rounds used by built-in tools. Provide consistent, documented accounting for JSON and SSE clients without counting repeated usage snapshots more than once.
Proposed solution
Acceptance criteria
Additional context
Enterprise Readiness enhancement. Coordinate with #279 for OpenTelemetry export; this issue owns usage semantics and correctness, not a billing platform.
Source review at
afa6c14, not a new runtime reproduction: Messages streaming and Messages JSON loop.Enterprise Readiness tracker: #316