fix(analytics): codex tmux turns priced correctly — provider attribution, usage normalization, history heal (#860) - #861
Merged
Conversation
…ion, usage normalization, gpt-5.6-sol rates (#860) Source changes (tests follow in next commit): - TmuxSession._ANALYTICS_PROVIDER class attr (anthropic) used at the log_turn_usage call instead of a hardcoded "anthropic"; CodexTmuxSession overrides with "codex_cli" (SDK-path default, aliased onto openai rates by _provider_alias). - _normalize_turn_usage hook: identity on the Claude transport; CodexTmuxSession converts the codex schema (inclusive input_tokens + cached_input_tokens) to the daemon's disjoint convention ONCE, before any consumer reads the dict — mirrors CodexSession.uncached_input_tokens. - pricing.RATE_TABLE: OpenAI family (gpt-5.6-sol, gpt-5.5 at 5/30/0.50; gpt-5.3-codex at 1.75/14/0.175); no cache-write billing. - analytics seeds: gpt-5.6-sol row; _migrate_codex_provider_attribution heals deployed anthropic/gpt-% rows to codex_cli (idempotent, narrow). - agent_registry: gpt-5.6-sol catalog seed; gpt-5.5 catalog price realigned to official 5/30/0.50 via _PRICE_CORRECTIONS. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ched rate Tests for every #860 seam: - tmux_session: provider comes from _ANALYTICS_PROVIDER (subclass-driven), base _normalize_turn_usage is identity, hook runs BEFORE all consumers (accumulation, pricing, analytics see the same normalized dict). - codex_tmux_session: _ANALYTICS_PROVIDER=codex_cli; normalization unit matrix (convert/pass-through/malformed/clamp/no-mutate); end-to-end turn-complete pricing a codex-schema usage dict at openai rates with the disjoint split. - pricing: gpt-5.6-sol/gpt-5.5 ($5/$30/$0.50), gpt-5.3-codex ($1.75/$14/$0.175), cache writes bill $0. - analytics_store: seed/RATE_TABLE parity guard restructured to be provider-aware (gpt rows must sit under openai); migration tests (heal, reprice, narrow, idempotent). - agent_registry: gpt-5.6-sol catalog seed, openai seed/RATE_TABLE parity, gpt-5.5 stale-price correction + operator-override safety. Migration amendment (src): the provider flip alone would over-bill history ~7x — pre-#860 rows logged codex's INCLUSIVE input with the cached span unrecorded (fleet rollouts measure 97.5% cached). Rows carrying that signature (cached_input_tokens=0) move the whole count to the cached column: error bounded ~-20%, failing toward the visible undercount, same direction as _normalize_turn_usage. Verified against a copy of the prod DB: 779 mislogged turns, $0.00 -> $33.43 (naive flip would have shown ~$320). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…861 P2) The sol model page (developers.openai.com, verified 2026-07-10) documents cache writes at 1.25x the uncached input rate ($6.25/Mtok); _GPT_FRONTIER hardcoded $0 for the whole family and the regression locked it. Split gpt-5.6-sol into its own rate dict with 6.25 in both write positions (OpenAI has no TTL split); gpt-5.5 and gpt-5.3-codex keep $0 — their pages list no write tariff. Regressions updated: sol write-billing pinned through both compute entry points; the shared-dict identity assertion dropped. Latent-only today (codex rollout usage carries no cache-creation fields), so the #860 migration evidence is unaffected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #860.
The bug
TmuxSession._log_turn_cost_and_analyticshardcodedprovider="anthropic", so every CodexTmuxSession turn landed in analytics asanthropic/gpt-*— a (provider, model) pair the pricing lookup can never match (_provider_aliasnever crosses providers). All codex tmux burn priced at $0 and was invisible in every cost view. On prod today: 779 turns, ~64M input tokens, $0.00.A second, latent half: codex reports
input_tokensINCLUSIVE of the cached prefix (cached span undercached_input_tokens), while every daemon consumer follows the Anthropic disjoint convention. Even with a rate row, codex turns would have over-billed the cached span at the full input rate — ~7x on real fleet traffic (measured across the fleet's 2026-07 rollouts: 97.5% of codex input is cached reads).The fix
_ANALYTICS_PROVIDERclass attr on TmuxSession (anthropic); CodexTmuxSession overrides withcodex_cli(the SDK path's value, aliased onto openai rate rows at read time)._normalize_turn_usagehook: identity on the Claude transport; CodexTmuxSession converts the codex schema to the disjoint convention ONCE, before any consumer (accumulation, pricing, analytics, context gauge) reads the dict. MirrorsCodexSession.uncached_input_tokenson the SDK path. Missing/malformed cached counts pass through untouched — fail toward the visible undercount, never a silent double-bill.pricing.RATE_TABLE: OpenAI family —gpt-5.6-sol/gpt-5.5at $5/$30 (cached $0.50),gpt-5.3-codexat $1.75/$14 ($0.175). No cache-write billing.gpt-5.6-solrow; seed/RATE_TABLE parity guard restructured to be provider-aware._migrate_codex_provider_attribution: heals deployedanthropic/gpt-%rows tocodex_cli(turn usage + session facts; idempotent, narrow — no honest row can match). Rows carrying the pre-Codex tmux agents priced at $0: provider misattribution + missing gpt-5.6-sol rates + dropped cached-token split #860 signature (cached_input_tokens=0) also move their inclusive input count to the cached column: repricing it at the full rate would show ~$320 (~7x overstated); the shift bounds the error at roughly -20%, failing toward the undercount.gpt-5.6-solcatalog seed;gpt-5.5catalog price realigned to the official $5/$30 via_PRICE_CORRECTIONS(exact-stale-triple gated, operator overrides survive).Evidence — prod DB copy through the migration
Tests
31 new/updated across the five seams (attribution via class attr, normalization matrix + call-site ordering, gpt rates, provider-aware parity guard, migration heal/reprice/narrow/idempotent, catalog seed + price correction). Full suite: 4302 passed, 2 skipped.
Deploy note
Merge is HELD for tonight's release + daemon bounce (batched with #831). The migration runs on daemon init; the dashboard's codex history moves $0 → ~$33 notional.
🤖 Generated with Claude Code