Skip to content

perf: skip JSONL files outside the aggregation window by mtime - #38

Open
long-910 wants to merge 1 commit into
claude/fix-dedupe-readers-y9ahrwfrom
claude/perf-mtime-skip-y9ahrw
Open

long-910 wants to merge 1 commit into
claude/fix-dedupe-readers-y9ahrwfrom
claude/perf-mtime-skip-y9ahrw

Conversation

@long-910

@long-910 long-910 commented Jul 7, 2026

Copy link
Copy Markdown
Owner

Stacked on #37 — this PR is based on claude/fix-dedupe-readers-y9ahrw because both touch the same reader functions. Merge #37 first; GitHub will retarget this PR to main automatically.

Summary

The 60-second status bar poll (readAllUsage) and the burn-rate prediction (readRecentCosts) read and parsed every session file under ~/.claude/projects/ on each pass — including months-old history that cannot affect any aggregation window. On installations with a long Claude Code history this is the dominant cost of the poll loop.

Approach

findAllJsonlFiles() gains an optional newerThanMs cutoff that skips files whose mtime is older. This is safe for time-windowed aggregation because session JSONL files are append-only: a file not modified since time T cannot contain entries with timestamps after T. Every cutoff keeps a safety margin.

Caller Cutoff
readAllUsage (60s poll, widest window 7d) now − 7d − 1h
readRecentCosts (burn rate, 30-min window) window start − 60s
wasJsonlUpdatedRecently rewritten on top of the same filter

The heatmap already had an equivalent mtime filter (0.4.x); this brings the remaining readers in line.

Tests / Docs

  • New unit suite for the filter: temp HOME with a fresh file and a 10-day-backdated file (fs.utimes), asserting unfiltered vs. filtered results and wasJsonlUpdatedRecently behaviour.
  • docs/DATA.md: "mtime-Based File Skipping" section with the caller/cutoff table and the always-keep-a-margin rule.
  • CHANGELOG.md: Unreleased → Performance entry.
  • npm run lint ✅, npm run compile ✅, unit suites (37 tests) ✅ via mocha. The @vscode/test-electron runner cannot download VS Code in this sandbox; please rely on CI for the extension-host suites.

🤖 Generated with Claude Code

https://claude.ai/code/session_012ZLCLQeudf1LQBnBSyyr4f


Generated by Claude Code

The 60-second status bar poll (readAllUsage) and the burn-rate
prediction (readRecentCosts) read and parsed every session file under
~/.claude/projects/ on each pass, including months-old history that
cannot affect any aggregation window.

- findAllJsonlFiles() now takes an optional mtime cutoff (epoch ms);
  session files are append-only, so a file not modified since T cannot
  contain entries with timestamps after T
- readAllUsage skips files untouched for over 7 days (+1h margin)
- readRecentCosts skips files untouched for over its 30-min window
  (+60s margin)
- wasJsonlUpdatedRecently rewritten on top of the same filter
- Unit tests for the filter (temp HOME with fresh + backdated files)
- docs/DATA.md: new mtime-based file skipping section

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012ZLCLQeudf1LQBnBSyyr4f
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants