Repository navigation
fix(ai): count prompt cache reads and writes in Anthropic and Bedrock input tokens - #3662
Conversation
… input tokens The Anthropic API and Claude on Bedrock report cache reads and writes outside input_tokens (total input = input_tokens + cache_read_input_tokens + cache_creation_input_tokens). The OpenAI-compatible and Gemini clients already put cached tokens inside InputTokens, so the same field meant different things per provider, and with caching on the Anthropic and Bedrock prompt counts in the chat footer and `atmos ai exec` JSON showed only the uncached tail. Add base.UsageFromAnthropic, used by both clients, so InputTokens and TotalTokens include cache reads and writes, and keep usage when a reply reports only cached input. Document that the cache counts are part of the prompt count.
|
Tip Atmos Pro
No affected stacks workflow was detected for this pull request. |
CI timing summaryLatest completed GitHub Actions runs for
Wall-clock time spans the earliest included workflow creation through the latest completion. Aggregate runner time adds each job's execution time, so concurrent jobs are counted separately.
Longest jobs (top 10)
Updated automatically when a PR workflow finishes. |
Quality Gate
Readiness✅ Fewer than 150 changed files Reviews✅ CodeRabbit approval (current head) ✅ Review threads resolved or dismissed MeasuresCurrent commit: Quality Gate details
🤖 Next steps for this PRAtmos performanceNo significant regressions detected against the newest compatible snapshot.
Describe affected
All other benchmarks
Full results are available in the comparison artifacts. |
|
No actionable comments were generated in the recent review. Walkthrough
Merge Risk:
|
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #3662 +/- ##
==========================================
- Coverage 87.30% 87.28% -0.02%
==========================================
Files 3041 3042 +1
Lines 276938 276946 +8
==========================================
- Hits 241769 241745 -24
- Misses 24956 24976 +20
- Partials 10213 10225 +12
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
|
Tip Atmos Pro
No affected stacks workflow was detected for this pull request. |
|
Tip Atmos Pro
No affected stacks workflow was detected for this pull request. |
|
These changes were released in v1.235.0-nightly.43989067298. |
what
InputTokens(andTotalTokens) now include prompt cache reads and writes, as they already do for the OpenAI-compatible and Gemini providers. The conversion is in one shared helper,base.UsageFromAnthropic, used by both clients.input_tokens: 0), instead of being dropped.types.Usageandformatter.TokenUsage, and theatmos ai execdocs, say thatcachedandcache_creationare parts ofprompt.why
input_tokens: total input =input_tokens+cache_read_input_tokens+cache_creation_input_tokens. OpenAI'sprompt_tokensand Gemini'spromptTokenCountalready include cached tokens.input_tokens90,output_tokens12 and 5,526 tokens read from the cache, the footer showed↑ 90 · ↓ 12 · cache: 5.5kand now shows↑ 5.6k · ↓ 12 · cache: 5.5k;tokens.promptinatmos ai exec --format jsonis 5,616 instead of 90. That matches what the docs example already implies (cached890 withinprompt1,234).CacheReadTokens/CacheCreationTokensare unchanged, and nothing else readsInputTokens(checked: chat footer and session totals,atmos ai execJSON, session export).Tests:
pkg/ai/agent/base/usage_test.go: no cache, cache read only, cache write only, both, whole prompt from cache, output only, no tokens (nil); cache counts never exceedInputTokens.TestUsageCalculations"With cache tokens" case now expects input 900 / total 1,100.go vet,gofumptandgolangci-lint(v2.13.2, repo config) are clean on the changed packages,go build ./...passes, andgo testpasses for every package that imports the changed code.references
promptTokenCountincludes cached content)Summary by CodeRabbit
Bug Fixes
Documentation