The usage-divergence flag was removed from PR #48 because its baseline was wrong: it compared provider-reported cost against the pre-call estimate, which prices output at the full max_tokens cap — so it flagged flagged: true on essentially every normal short completion (e.g. 1000in/300out under an 8192 cap → ratio ~0.06 < 1/4 → flagged) and spammed the audit chain with usage_divergence events.
The right design (do this here)
The flag's purpose is to catch a server under-reporting usage. Compare provider-reported usage against usertrust's OWN independent token count of the ACTUAL content:
independentCost = estimateCost(model, estimateInputTokens(actualInputMessages), countTokens(actualOutputText))
- flag when
actualCost / independentCost < 1/factor (reported materially below what we independently counted) — optionally also the over direction.
- Acceptance: a normal 1000in/300out call → reported ≈ independent count → ratio ≈ 1 → NOT flagged; a server claiming ~50 tokens for ~2000 tokens of content we counted → ratio ≈ 0.03 → flagged.
Why it's non-trivial (the blast radius that made it too large for #48)
No output TEXT is accumulated anywhere today — every stream path accumulates provider-reported usage NUMBERS only. This needs:
- Per-provider output-text accumulation across THREE paths: generic
createGovernedStream, the settleViaMessageStream streamEvent tap, and non-stream response bodies (OpenAI choices[].message.content, Anthropic content[].text, Responses output_text/output).
- A text→token counter (reuse/extend estimateInputTokens' tokenizer).
- Thread the independent count into the divergence computation at both settle sites.
- Re-introduce receipt.divergence + config.divergence + the usage_divergence audit event with the corrected semantics.
This closes the 'blind trust in provider-reported usage' limitation properly (it is intentionally left open in LIMITATIONS.md until this lands).
The usage-divergence flag was removed from PR #48 because its baseline was wrong: it compared provider-reported cost against the pre-call estimate, which prices output at the full max_tokens cap — so it flagged
flagged: trueon essentially every normal short completion (e.g. 1000in/300out under an 8192 cap → ratio ~0.06 < 1/4 → flagged) and spammed the audit chain with usage_divergence events.The right design (do this here)
The flag's purpose is to catch a server under-reporting usage. Compare provider-reported usage against usertrust's OWN independent token count of the ACTUAL content:
independentCost = estimateCost(model, estimateInputTokens(actualInputMessages), countTokens(actualOutputText))actualCost / independentCost < 1/factor(reported materially below what we independently counted) — optionally also the over direction.Why it's non-trivial (the blast radius that made it too large for #48)
No output TEXT is accumulated anywhere today — every stream path accumulates provider-reported usage NUMBERS only. This needs:
createGovernedStream, thesettleViaMessageStreamstreamEvent tap, and non-stream response bodies (OpenAIchoices[].message.content, Anthropiccontent[].text, Responsesoutput_text/output).This closes the 'blind trust in provider-reported usage' limitation properly (it is intentionally left open in LIMITATIONS.md until this lands).