docs(proposals): add How? section to #210 response-based token counting#643
Open
mkoushni wants to merge 2 commits into
Open
docs(proposals): add How? section to #210 response-based token counting#643mkoushni wants to merge 2 commits into
mkoushni wants to merge 2 commits into
Conversation
praxis-bot
reviewed
Jun 22, 2026
praxis-bot
left a comment
Collaborator
There was a problem hiding this comment.
PR Review
Summary: Adds the How? section to the #210 proposal, transitioning status from proposed to accepted. Well-structured design that correctly addresses all four open questions and aligns with the existing token_usage library and FilterContext APIs.
| Severity | Count |
|---|---|
| Medium | 3 |
No critical or large issues found. The design decisions are sound -- StreamBuffer for body aggregation, explicit provider: key over auto-detection, and header-only path for Bedrock InvokeModel are all correct choices.
praxis-bot
reviewed
Jun 22, 2026
praxis-bot
reviewed
Jun 22, 2026
praxis-bot
reviewed
Jun 22, 2026
…token counting Signed-off-by: mkoushni <mkoushni@redhat.com>
- Add listeners block to YAML configuration example - Align ProviderKind::OpenAi casing with TokenUsageProvider::OpenAi - Update response_body_mode hook table row to reflect Stream/StreamBuffer split Signed-off-by: mkoushni <mkoushni@redhat.com>
b7bf13b to
39631ec
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
docs(proposals): add How? section to #210 response-based token counting
Summary
Completes the graduation criteria for proposal #210 by adding the How? section — the design and implementation plan for the
token_countfilter that extracts token usage from AI provider response bodies and headers.What changed
docs/proposals/00210_response-based-token-counting.md— 193 lines added, status updated fromproposed→accepted.Open questions resolved
All four open questions from the What?/Why? section are answered in the new How? section:
provider:YAML key — no auto-detection. Azure and OpenAI share the same JSON schema so auto-detection would be ambiguous.BodyMode::StreamBuffer— proxy buffers all response body bytes and delivers them once withend_of_stream: true. Stream close is the authoritative trigger, covering providers that omit[DONE](Google Gemini).message_start+message_delta) for Anthropic; header-only for Bedrock InvokeModel.Design content added
TokenCountConfig,ProviderKindenum,TokenCountFilterstruct with YAML snippet.HttpFilterhook table — behaviour of each hook (on_request,on_response,response_body_access,response_body_mode,on_response_body).on_response;BodyAccess::Noneprevents unnecessary buffering.FilterContextmetadata keys —token.input,token.output,token.totalwritten viactx.set_token_usage.ai/mod.rs,http/mod.rs,builtins/mod.rs, andregistry.rs.token_countwithprovider: openai.Related