Repository navigation
fix: anthropic OpenAI usage - #33
Merged
TriDefender merged 2 commits intoAug 29, 2026
Merged
Conversation
…path translateResponseAnthropicToOpenAI copied input_tokens straight into prompt_tokens. That is wrong whenever the upstream serves any of the prompt from cache, which z.ai now does implicitly on every repeated prefix: a request whose real prompt was 1224 tokens reported 6, because 1216 of them had landed in cache_read_input_tokens and 2 in the fresh bucket. The two APIs do not agree on what the input number means. Anthropic keeps three mutually exclusive buckets — input_tokens is only the uncached remainder — while OpenAI's prompt_tokens is inclusive of cache hits. So the conversion is a sum, not a copy: prompt_tokens = input + cache_read + cache_creation prompt_tokens_details.cached_tokens = cache_read total_tokens = prompt_tokens + completion_tokens That is the same arithmetic the official ZCode client uses in its own normalizers (convertAnthropicMessagesUsage sums all three buckets; convertOpenAIChatUsage treats prompt_tokens as the total and derives the uncached part by subtracting cached_tokens), and it makes the new helper the counterpart of the openaiUsageToAnthropic that already existed for the reverse direction. Two deliberate omissions. The Anthropic-style cache fields are not mirrored onto the OpenAI usage object — they are vendor extensions, and a client validating with z.strict() or serde(deny_unknown_fields) would reject them. And prompt_tokens_details is emitted only when the upstream actually reported a cache bucket, so "explicitly zero cache reads" stays distinguishable from "no cache accounting at all". Totals round-trip exactly; buckets do not, because OpenAI has no cache-creation member and those tokens come back classified as fresh input. Both directions still agree on prompt_tokens and total_tokens. /v1/responses needed no change: chat-to-responses already read prompt_tokens_details.cached_tokens correctly and was only being starved by the zeros upstream of it. Claude-Session: https://claude.ai/code/session_01FVzUVu95tXXTuW1bkwSs1D
Every streamed response through the OpenAI exit reported
prompt_tokens: 0. The upstream z.ai Anthropic endpoint now always sends
message_start.usage as {input_tokens: 0, output_tokens: 0} and puts the
real counts in the terminal message_delta — the same event that carries
stop_reason:
data: {"type":"message_delta","delta":{"stop_reason":"end_turn"},
"usage":{"input_tokens":30,"output_tokens":16,
"cache_read_input_tokens":64, ...}}
The state machine read input_tokens once, from message_start, and read
only output_tokens out of message_delta, so the input and both cache
buckets were discarded. This is not a regression here — the last change
to that code was 494bbad in June; the upstream moved.
Replace the two scalars on TranslationState with a single AnthropicUsage
snapshot and fold each event into it field by field. The merge overwrites
rather than accumulates, because the upstream re-reports absolute totals:
a later 0 has to win over an earlier non-zero, while a field the event
omits keeps its previous value. message_start still merges, so an
upstream that does front-load its usage keeps working.
Ordering matters: usage is folded in before the stop_reason branch is
evaluated, since both arrive in the same event and the finish chunk is
where the usage is published. The stop_reason branch also grew a
finishReasonSent guard so a second stop-bearing delta cannot emit a
duplicate finish chunk.
Usage that arrives only after the finish chunk was emitted updates the
snapshot but is not re-announced. Publishing it would mean a trailing
chunk with an empty choices array, which some clients index into
blindly; no observed upstream orders the events that way.
The message_delta event type was widened to match what the wire actually
carries (Partial<AnthropicUsage>, optional delta and stop_reason).
server_tool_use and service_tier are left unmodelled and ignored.
Verified against the live upstream after warming the implicit cache: raw
Anthropic passthrough reported input 30 + cache_read 64, and both the
streaming and non-streaming OpenAI exits reported prompt_tokens 94 /
total 110 / cached_tokens 64. /v1/responses agreed. glm-5.3 matched.
Claude-Session: https://claude.ai/code/session_01FVzUVu95tXXTuW1bkwSs1D
demian-bot
added a commit
to demian-bot/zcode-api
that referenced
this pull request
Aug 29, 2026
… the PR branch per maintainer's own merge of 8b273aa
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
参考 zcode 实现，修复当前 prompt token 恒为 0 的问题。