Compaction awareness for context coverage - #24
Merged
Conversation
Claude Code v2.1.132 emits compactions as a `type:"user"` record with `isCompactSummary:true` (summary text in `message.content`), paired with a `type:"system" subtype:"compact_boundary"` boundary marker. The spike doc's assumed `type:"compaction"` content-block format never matched real data, so every live compaction was silently dropped — the TUI stats panel, activity feed, overlay, and `--coverage` output all showed no compaction info. This commit: - Detects `isCompactSummary:true` on user records (taking precedence over the `<command-name>/clear</command-name>` heuristic). - Plumbs Compacted events through `TailerOutput` so live tails dispatch them to `App::process_compaction` instead of discarding them. - Wires up the stats panel summary, activity feed marker, popup overlay, and coverage report (text + JSON, schema v2) for compactions. - Replaces the misleading parser test with fixtures from the real JSONL shape. The `compactMetadata` block on the system boundary record (preTokens, postTokens, trigger, durationMs) is still ignored — a follow-up will surface that token delta in the UI and coverage output. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pairs the `type:"system" subtype:"compact_boundary"` record (with `compactMetadata.trigger/preTokens/postTokens/durationMs`) with the following `isCompactSummary` user record. The parser emits a new `ParsedLine::CompactBoundary` variant that batch parsing and the live tailer buffer and fold into the next `Compacted` event. Across-poll buffering is held on `LogTailer` so a boundary that lands alone in one poll still pairs with the summary in a later poll. Surfaces the metadata in: - Stats panel: `195.7k → 6.4k tok · manual` line under the last compaction's timestamp. - Overlay title: `Compaction #N · 195.7k → 6.4k tok · manual` when metadata is present (falls back to bare timestamp when absent). - Coverage text: `Tokens: 195684 → 6416 (-96.7%) · trigger: manual · 64.7s` line between header and files. - Coverage JSON: optional `metadata` object on each compaction entry (omitted via `skip_serializing_if` when absent — additive, schema version stays at 2). `CompactionEvent.metadata` and `CompactionSummary.metadata` are `Option<...>` so older Claude Code versions without the boundary record still produce a Compacted event (just without the token info). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Compaction destroys most of the model's verbatim file content; the prose summary is too fuzzy to reliably reconstruct what survived. The previous behavior left pre-compaction reads in the ledger, which over-reported coverage in the TUI and the --coverage output. Now treat a compaction like /clear for the ledger only: snapshot the pre-compaction state into compaction_history (as before), then wipe `ledger` and `compaction_call_count`. `compaction_history`, `activity`, agent tracking, and session metadata are preserved. Mirrored in `coverage::run_report` so the replayed coverage report reflects post-compaction context only — `compactions` is preserved. Cross-file ordering: `session_log_files()` now returns the main session JSONL last so subagent events accumulate before the main session's compaction wipes the ledger. Residual risk (a subagent finishing *after* the main session's compaction will have its post-compaction reads erroneously wiped) is documented in a function doc comment. Strictly correct ordering would need a cross-file timestamp merge; deferred. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #24 +/- ##
==========================================
- Coverage 76.88% 74.68% -2.21%
==========================================
Files 22 23 +1
Lines 5823 6640 +817
Branches 5823 6640 +817
==========================================
+ Hits 4477 4959 +482
- Misses 1250 1573 +323
- Partials 96 108 +12 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
Forced compactions in Claude Code v2.1.132 were silently dropped — the TUI stats panel, activity feed, overlay, and
--coverageoutput all showed no compaction info. The spike doc had assumed compactions arrived as atype:"compaction"content block, but the real shape is two records: atype:"system" subtype:"compact_boundary"marker (with token/trigger metadata) followed by atype:"user" isCompactSummary:truerecord carrying the summary text. The parser never matched real data, and the live ledger continued reporting pre-compaction reads as still-in-context — over-reporting coverage.What changed
Three commits, each independently reviewable:
c87c6da— detect real compaction records. Parser recognizesisCompactSummary:trueuser records (taking precedence over the/clearheuristic) and plumbs Compacted events throughTailerOutput. Stats panel, activity feed, overlay, and coverage report (text + JSON, schema v2) all surface compactions.a83ffe0— surfacecompact_boundarytoken metadata. Pairs the boundary record with the following summary so we can showpre → post tok · trigger · duration. Buffering lives onLogTailerso a boundary that arrives alone in one poll still pairs with the summary in a later poll. JSONmetadatafield is optional and additive (schema v2 unchanged).eb18adc— clear live ledger on compaction. Compaction destroys most verbatim file content; treat it like/clearfor the ledger.App::process_compactionnow snapshots the pre-state intocompaction_history, then wipesledgerandcompaction_call_count.coverage::run_reportmirrors this.compaction_history, activity, agents, and session metadata are preserved.To keep cross-file event ordering sane,
session_log_files()now returns the main session JSONL last so subagent reads accumulate before any compaction in the main file wipes the ledger.Known limitation
A subagent finishing after the main session's compaction will still have its post-compaction reads erroneously wiped. The main-last ordering covers the typical case; a full cross-file timestamp merge is deferred and documented in
session_log_files().Verification
End-to-end against a real session JSONL with a manual compaction:
Tokens: 195684 → 6416 (-96.7%) · trigger: manual · 64.7smetadata: { trigger, pre_tokens, post_tokens, duration_ms }on each compaction entrystate_before)