Skip to content

Add a prompt-cache report and per-day cache diagnostics - #42

Merged
starkdmi merged 1 commit into
mainfrom
feat/cache-context-stats
Oct 7, 2026
Merged

starkdmi merged 1 commit into
mainfrom
feat/cache-context-stats

Conversation

@starkdmi

@starkdmi starkdmi commented Oct 7, 2026

Copy link
Copy Markdown
Owner

Add statsai report cache, which shows how much logical input the prompt
cache served, how much context each model call processed, and when reuse
dropped. For Claude Code and Codex it walks each agent's calls in order and
flags a suspected cache loss when cached reads fall at least 50% and 4,096
tokens below what the previous call left reusable. Each loss records the
reusable context it missed and its cost: the call's price minus its price
had that context been read from the cache, priced as one request by the
call's own model. Losses are grouped by the gap since the previous call.
Model changes, compactions, and calls that cannot be ordered or split are
boundaries, not losses. A Claude Code call that neither read nor wrote the
cache, as when a custom endpoint serves it, is counted apart and not judged.

The report takes --from/--to, --all, --provider, --account, --session,
--timeline for local 10-minute buckets, --details for per-call evidence,
and --json. The daemon serves the same report at /reports/cache, and
statsai schema cache-report prints its schema.

  • Codex turn events list each model call with its request start, model,
    and compaction boundary. Rows too large to read still mark compaction.
  • Claude Code events record their request start, sub-agent, and compaction
    boundary, following file order because compaction rewrites history.
  • Daily summaries carry an optional, compact cache_health object. It is
    sent only to receivers that advertise support, and an existing store
    gains it once, rewriting only summaries whose content changes. Fields
    that are zero on most days are left out then.
  • The cache hit ratio now counts cache writes in its denominator.
  • Migration 30 orders the session index by start time, so sessions read
    in order and the calls after a changed one are an index range.

Add `statsai report cache`, which shows how much logical input the prompt
cache served, how much context each model call processed, and when reuse
dropped. For Claude Code and Codex it walks each agent's calls in order and
flags a suspected cache loss when cached reads fall at least 50% and 4,096
tokens below what the previous call left reusable. Each loss records the
reusable context it missed and its cost: the call's price minus its price
had that context been read from the cache, priced as one request by the
call's own model. Losses are grouped by the gap since the previous call.
Model changes, compactions, and calls that cannot be ordered or split are
boundaries, not losses. A Claude Code call that neither read nor wrote the
cache, as when a custom endpoint serves it, is counted apart and not judged.

The report takes --from/--to, --all, --provider, --account, --session,
--timeline for local 10-minute buckets, --details for per-call evidence,
and --json. The daemon serves the same report at /reports/cache, and
`statsai schema cache-report` prints its schema.

- Codex turn events list each model call with its request start, model,
  and compaction boundary. Rows too large to read still mark compaction.
- Claude Code events record their request start, sub-agent, and compaction
  boundary, following file order because compaction rewrites history.
- Daily summaries carry an optional, compact `cache_health` object. It is
  sent only to receivers that advertise support, and an existing store
  gains it once, rewriting only summaries whose content changes. Fields
  that are zero on most days are left out then.
- The cache hit ratio now counts cache writes in its denominator.
- Migration 30 orders the session index by start time, so sessions read
  in order and the calls after a changed one are an index range.
@starkdmi
starkdmi merged commit 01b112f into main Oct 7, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant