fix: Fix Metrics History Destruction & Self-Exclusion Blind Spot

docs/memory/cli/data-pipeline.md

@@ -15,6 +15,8 @@ Entry point: `src/node/core/cli.ts`. Data types: `src/node/core/types.ts`. Data
 - Global flags (parsed by `parseGlobalFlags` into `GlobalFlags`): `--json`, `--csv`, `--md`, `--sync`, `--fresh`/`-f`, `--watch`/`-w`, `--interval`/`-i <s>`, `--user`/`-u <user>`, `--by-machine`, `--no-color`, `--no-rain`, `--version`/`-V`/`-v`. Separately, `--skip-brew-update` is an `update`-scoped command-specific flag detected via raw-argv membership — it is NOT in `GlobalFlags`
 - `--by-machine` MUST show per-machine cost breakdown columns in tables; works with snapshot and single-tool history; incompatible with all-tools history pivot (warn on stderr and ignore); compatible with `--watch`, `--json`, and `-u`; in single mode, shows one machine (hostname); uses `fetchToolMergedWithMachines` which returns `{ entries, machineMap }` where machineMap is `Map<string, UsageEntry[]>` (machine → entries)
 - `--user`/`-u` MUST set a target user for data display (multi mode only); in single mode, warn on stderr and ignore; when targeting a different user, display only metrics repo data (no local ccusage data); when targeting the same user as `config.user`, behave identically to the default (no `-u`) path — perform a local fetch (using cache unless `--fresh`/`-f` is provided), write to the metrics repo, then merge with other machines
+- `maxMergeEntries(a: UsageEntry[], b: UsageEntry[])` (exported from `src/node/core/fetcher.ts`, alongside `mergeEntries`) MUST pick, per date label, whichever whole entry has the greater `totalCost` — no field mixing, no summing; on equal `totalCost` the entry from `a` wins (call sites pass the live local fetch as `a`); output MUST be sorted ascending by label and inputs MUST NOT be mutated (pure function, Constitution V)
+- In multi mode, the own-user merge pipeline (`fetchToolMerged` and `fetchToolMergedWithMachines` in `src/node/core/cli.ts`) MUST run: live local fetch → `writeMetrics` (never-shrink guarded — see `sync/multi-machine.md`) → `readRemoteEntriesByMachine(metricsDir, config.user, null, toolKey)` (all machines, one walk) → split the own machine out of the map → `effectiveLocal = maxMergeEntries(local, ownSnapshots)` → `mergeEntries(effectiveLocal, otherMachines)`. `fetchToolMergedWithMachines` MUST set the own-machine `machineMap` entry to the max-merged view so `--by-machine` shows the corrected own-machine column and the flattened entries reflect it. Single mode (live entries only, no repo reads) and the `-u <other-user>` path (repo-only, `excludeMachine = null`) are unaffected
 - `--json`, `--csv`, and `--md` MUST be mutually exclusive with each other; `--watch` MUST be incompatible with any of them. Violations print `Error: {flag-a} and {flag-b} are incompatible` to stderr and exit 1 (`parseGlobalFlags` in `src/node/core/cli.ts`)
 - `--interval` range: 5-3600 seconds, default 10
 - Non-data commands (`init-conf`, `init-metrics`, `sync`, `status`, `update`, `shell-init`, `help`) MUST be dispatched before grammar parsing
@@ -40,6 +42,7 @@ Entry point: `src/node/core/cli.ts`. Data types: `src/node/core/types.ts`. Data
 - **Output format as a single enum plumbed through dispatch** (260423-lx0g): `outputFormat: "table" | "json" | "csv" | "md"` is resolved once in `parseGlobalFlags` and each dispatch function switches on it in one place. Avoids multiplying `if (jsonFlag) ... else ...` branches as new formats are added.
 - **Static shell completion scripts** (260423-lx0g): Bash/zsh/fish completion scripts are hardcoded string constants in `src/node/core/completions.ts`. No dynamic lookup (the grammar is stable and runtime `tu --list-*` calls would add 50-200ms per tab-press); no `$SHELL` auto-detection (explicit arg avoids silent mismatches when `$SHELL` and the running shell diverge).
 - **`--skip-brew-update` detected via raw-argv membership, not `parseGlobalFlags`** (260531-e96v): The `update` dispatch passes `process.argv.includes("--skip-brew-update")` into `runUpdate` rather than threading the flag through `parseGlobalFlags`/`GlobalFlags`. It is command-specific to `update` (which ignores positional/data args); routing it through the shared data-flag path would broaden the blast radius and risk perturbing unrelated flag handling. Membership testing matches the existing `rawArgs.includes(...)` idiom (e.g. `--sync`). The cross-toolkit contract mandated the exact flag name `--skip-brew-update` (no alias).
+- **Own-machine snapshots max-merged into the live view** (260610-srmi): Claude Code purges transcripts older than ~30 days, so the live ccusage fetch under-reports old days; the previous pipeline excluded the machine's own repo dir from remote reads (`excludeMachine = config.machine`), leaving its own synced history invisible to itself once transcripts were purged. The multi-mode pipeline now reads all machines in one `readRemoteEntriesByMachine(..., null, ...)` walk, splits out the own machine, and applies per-day whole-entry max (`maxMergeEntries`) to the own share before the existing sum-merge with other machines. Max, not sum: a partially-purged day still yields a residual live entry, and summing residual + snapshot would double-count the surviving transcripts. Reuses the existing by-machine walk unchanged (minimum pathways) instead of adding a single-machine read helper; merged totals only ever increase relative to the pre-change pipeline. Watch mode shares the same fetch path and is corrected for free.
 
 ## Changelog
 
@@ -54,3 +57,4 @@ Entry point: `src/node/core/cli.ts`. Data types: `src/node/core/types.ts`. Data
 | 2026-04-01 | Added `-v` (lowercase) as version flag alias alongside `--version` and `-V` (260401-kuuh) |
 | 2026-04-23 | Migrated child process spawning from `exec` to `execFile` with argv arrays; `TOOLS` shape changed from `{name, command, needsFilter}` to `{name, binary, prefixArgs, needsFilter}`; added `--csv`/`--md` global flags and `outputFormat` enum dispatch; added `tu completions <shell>` non-data subcommand with static bash/zsh/fish scripts (260423-lx0g) |
 | 2026-05-31 | Added `--skip-brew-update` flag to `tu update` — skips only the internal `brew update --quiet` refresh; version check and `brew upgrade` unaffected (260531-e96v) |
+| 2026-06-10 | Self-view fix: added pure `maxMergeEntries` to `fetcher.ts`; multi-mode `fetchToolMerged`/`fetchToolMergedWithMachines` read all machines (`excludeMachine = null`) and max-merge own-machine snapshots into the live view before the sum-merge; single mode and `-u <other-user>` unchanged (260610-srmi) |

docs/memory/cli/index.md

@@ -2,4 +2,4 @@
 
 | File | Description | Last Updated |
 |------|-------------|--------------|
-| [data-pipeline.md](data-pipeline.md) | CLI argument parsing, data fetching, caching, tool registry | 2026-05-31 |
+| [data-pipeline.md](data-pipeline.md) | CLI argument parsing, data fetching, caching, tool registry | 2026-06-10 |

docs/memory/sync/index.md

@@ -2,4 +2,4 @@
 
 | File | Description | Last Updated |
 |------|-------------|--------------|
-| [multi-machine.md](multi-machine.md) | Git-based metrics sync, JSONL storage, remote merging, auto-clone | 2026-03-06 |
+| [multi-machine.md](multi-machine.md) | Git-based metrics sync, JSONL high-water-mark storage (never-shrink writes, self-view max-merge), remote merging, auto-clone, repair script | 2026-06-10 |

docs/memory/sync/multi-machine.md

@@ -2,17 +2,19 @@
 
 ## Overview
 
-Multi-machine sync (`src/node/sync/sync.ts`) enables aggregating AI usage costs across multiple machines via a shared git repository. Each machine writes per-day JSONL files to a structured directory hierarchy, then syncs via git commit/pull/push.
+Multi-machine sync (`src/node/sync/sync.ts`) enables aggregating AI usage costs across multiple machines via a shared git repository. Each machine writes per-day JSONL files to a structured directory hierarchy, then syncs via git commit/pull/push. Day-file snapshots are high-water marks: `writeMetrics` never shrinks them, and at display time each machine max-merges its own snapshots back into its live view (the live view under-reports old dates once Claude Code purges transcripts older than ~30 days).
 
 ## Requirements
 
 - Sync MUST require `mode=multi` in config; single-mode rejects sync operations
 - Metrics directory structure MUST follow: `{metricsDir}/{user}/{year}/{machine}/{toolKey}-{YYYY-MM-DD}.jsonl`
 - Each JSONL file MUST contain a single `UsageEntry` JSON object
 - `writeMetrics()` MUST write local entries to the metrics directory (creates dirs as needed)
+- `writeMetrics()` MUST NOT shrink a day-file: when the existing file parses as JSON with a finite numeric `totalCost` and the incoming entry's `totalCost` is strictly lower, the write is skipped silently (no stderr). Absent, empty, or unparseable files — including JSON without a finite `totalCost` — are treated as absent (write proceeds); equal or greater incoming `totalCost` overwrites as before, so today's file keeps refreshing as the day grows. The guard (`isShrinkingWrite`) lives inside `writeMetrics` itself, covering every call site (`fetchToolMerged` and `fetchToolMergedWithMachines` in `src/node/core/cli.ts`, and `fullSync`)
 - `readRemoteEntriesByMachine(metricsDir, targetUser, excludeMachine, toolKey)` MUST return `Map<string, UsageEntry[]>` grouped by machine directory name; when `excludeMachine` is non-null, that machine is excluded from the map
 - `readRemoteEntries(metricsDir, targetUser, excludeMachine, toolKey)` MUST delegate to `readRemoteEntriesByMachine` and flatten the result into a single `UsageEntry[]`; signature and behavior unchanged from callers' perspective
 - `mergeEntries()` MUST sum token counts and costs for entries with matching labels [INFERRED]
+- In multi mode, the own-user display path MUST merge the machine's own repo snapshots back into its live view: `fetchToolMerged`/`fetchToolMergedWithMachines` (`src/node/core/cli.ts`) read ALL machines via one `readRemoteEntriesByMachine(metricsDir, config.user, null, toolKey)` walk, split the own machine's entries out of the map, and use `effectiveLocal = maxMergeEntries(local, ownSnapshots)` (per-day whole-entry max on `totalCost`, defined in `src/node/core/fetcher.ts`) as the own-machine share before sum-merging the other machines; `-u <other-user>` (repo-only) and single mode are unchanged
 - `syncMetrics()` MUST: (1) git add user dir, (2) commit if changes, (3) pull --rebase, (4) push (with one retry). All git commands MUST be invoked via `execFile("git", [...argv])` (no shell subprocess) — the internal `git` helper is `(args: string[]) => execFileAsync("git", ["-C", metricsDir, ...args])`. Argv entries (including `metricsDir`) are passed as literal strings; no quote parsing is performed. Interrupted-rebase recovery (`execFile("git", ["-C", metricsDir, "rebase", "--abort"])`) and single-retry push preserve their pre-change semantics
 - `fullSync()` MUST: fetch all tools locally, write metrics, sync via git, touch `.last-sync` timestamp
 - `isStale()` MUST return true if `.last-sync` is older than 3 hours or missing
@@ -21,12 +23,15 @@ Multi-machine sync (`src/node/sync/sync.ts`) enables aggregating AI usage costs
 - Clone failures MUST write a marker file (`.clone-failed`) with ISO timestamp; retry suppressed for 3 hours
 - `init-metrics` MUST clone the metrics repo and clear any clone-failed marker
 - When metrics dir is missing and can't be cloned, MUST fall back to single mode with a warning
+- `scripts/repair-metrics.mjs` (one-time repair tooling; standalone `node` script, NOT bundled into `dist/tu.mjs`, no imports from `src/`) MUST restore shrunk day-files from the metrics repo's git history: one `git log --format=%H%x09%cs --name-only -- '*.jsonl'` walk builds per-file commit lists (no per-file `git log`), `git show <sha>:<path>` finds each tracked day-file's historical-max `totalCost`, and files below their max by more than `CENT_TOLERANCE` ($0.01) are reported with path, current, max, max-commit short SHA + date, delta, per-user subtotals, and a grand total. Dry-run is the default (no modification); `--write` restores the full historical-max blob byte-exact into the working tree only (no commit/push — left to the user; idempotent). `--repo` defaults to `~/.tu/metrics_repo`; missing/non-git repo and unknown args fail loud (stderr + exit 1); deleted-at-commit paths and unparseable historical blobs are skipped without crashing
 
 ## Design Decisions
 
 - **Git as sync transport**: Using a git repository avoids building a custom sync server. Commit/pull/push is simple and works over SSH. Rebase on pull avoids merge commits.
 - **Per-day JSONL files**: One file per tool per day enables efficient incremental writes and avoids conflicts when multiple machines sync.
-- **User-scoped remote reads**: `readRemoteEntries` reads only from the target user's directory (default: config user). The `excludeMachine` parameter (default: config machine) prevents double-counting with local data. When `-u` targets a different user, `excludeMachine` is `null` to include all of that user's machines. When `-u` targets the same user as `config.user`, the default path is used instead (local fetch, cached unless `--fresh`, plus merge), ensuring identical behavior to no `-u` flag.
+- **User-scoped remote reads**: `readRemoteEntries`/`readRemoteEntriesByMachine` read only from the target user's directory (default: config user). Since 260610-srmi every production call site passes `excludeMachine = null` (all machines in one walk): double-counting with local data is prevented by max-merging the own machine's snapshots into the live view instead of excluding the own dir — the old exclusion created a self-view blind spot where a machine could not see its own synced history once its local transcripts were purged. When `-u` targets a different user, the path is repo-only across all of that user's machines. When `-u` targets the same user as `config.user`, the default path is used instead (local fetch, cached unless `--fresh`, plus merge), ensuring identical behavior to no `-u` flag. The now test-only `excludeMachine` parameter is flagged as a deletion candidate for a follow-up signature cleanup (260610-srmi plan).
+- **Day-files are high-water marks** (260610-srmi): Claude Code purges session transcripts older than ~30 days, so a live ccusage fetch for an old date collapses toward zero. Treating each day-file as the fullest complete snapshot ever observed fixes two bugs with one principle: (1) writes — `writeMetrics` skips shrinking writes, where unconditional overwrites had silently destroyed correct history on every post-purge sync (measured before the guard: $5,160.63 across 21 day-files for one user); (2) display — each machine max-merges its own repo snapshots back into its live view so its synced history stays visible to itself after the purge. Whole entries only — never per-field max (fabricates chimera entries mixing token/cost fields from different snapshots, violating Constitution V) and never residual+snapshot sums (double-counts the surviving transcripts of partially-purged days).
+- **One-time repair from git history, manual and working-tree only** (260610-srmi): nothing was ever deleted from the metrics repo's history — only overwritten — so `scripts/repair-metrics.mjs` restores each shrunk day-file losslessly by writing back the full blob of the commit where its `totalCost` peaked (never patching just the cost field). Dry-run by default; `--write` touches the working tree only, leaving review/commit/push to the user. Standalone unbundled script (precedent: `scripts/help-dump.mjs`) keeps Constitution III intact. Sequencing constraint: run only after the guarded binary is installed on every actively-syncing machine, otherwise an old binary re-clobbers restored values at the rolling retention edge.
 - **Graceful degradation**: If multi mode is configured but the metrics repo is unavailable, the tool falls back to single mode rather than failing. This ensures the tool always works for local data.
 - **Clone-failed marker with cooldown**: Prevents repeated clone attempts on every invocation when the repo is unreachable (e.g., no network). 3-hour cooldown matches the staleness threshold.
 
@@ -40,3 +45,4 @@ Multi-machine sync (`src/node/sync/sync.ts`) enables aggregating AI usage costs
 | 2026-03-07 | Fixed `-u` same-user: falls through to fresh-fetch path instead of reading stale repo data |
 | 2026-03-07 | Added `readRemoteEntriesByMachine` returning grouped `Map<string, UsageEntry[]>`; refactored `readRemoteEntries` to delegate and flatten |
 | 2026-04-23 | Migrated git invocations from `exec("git -C ... ...")` to `execFile("git", [...argv])` — no shell fork, paths with spaces/quotes pass through as literal argv entries (260423-lx0g) |
+| 2026-06-10 | Never-shrink guard in `writeMetrics` (skip silently when incoming `totalCost` is lower than the existing day-file's); display path max-merges own-machine snapshots back into the live view (all production reads now pass `excludeMachine = null`); added standalone one-time repair script `scripts/repair-metrics.mjs` (dry-run default, `--write` working-tree only) (260610-srmi) |

fab/changes/260610-srmi-fix-metrics-history-destruction/.history.jsonl

@@ -0,0 +1,15 @@
+{"action":"enter","driver":"fab-new","event":"stage-transition","stage":"intake","ts":"2026-06-10T14:43:30Z"}
+{"args":"Fix metrics history destruction and self-exclusion blind spot in multi-machine sync, plus one-time repo repair. Three ordered parts (backlog items r01f, 87hw, gsji in main worktree backlog): (1) Guard writeMetrics so a sync can never shrink history — Claude Code purges transcripts \u003e30 days old, live ccusage data for old dates collapses toward zero, and every sync/merged-fetch overwrites correct historical per-day JSONL files with post-purge residue (~$5,160 destroyed across sahil machines, ongoing). (2) One-time repair script restoring shrunk day-files to historical max from git history, all users/machines, run after guard ships. (3) Fix self-exclusion blind spot: merge own-machine repo entries into local view via per-day max so purged local history doesn't vanish from a machine's own view.","cmd":"fab-new","event":"command","ts":"2026-06-10T14:43:30Z"}
+{"delta":"+3.5","event":"confidence","score":3.5,"trigger":"calc-score","ts":"2026-06-10T14:45:44Z"}
+{"delta":"+0.0","event":"confidence","score":3.5,"trigger":"calc-score","ts":"2026-06-10T14:45:55Z"}
+{"cmd":"fab-fff","event":"command","ts":"2026-06-10T14:58:49Z"}
+{"action":"enter","driver":"fab-fff","event":"stage-transition","stage":"apply","ts":"2026-06-10T14:59:05Z"}
+{"cmd":"fab-continue","event":"command","ts":"2026-06-10T15:00:54Z"}
+{"action":"enter","driver":"fab-continue","event":"stage-transition","stage":"review","ts":"2026-06-10T15:19:20Z"}
+{"cmd":"fab-continue","event":"command","ts":"2026-06-10T15:21:30Z"}
+{"action":"enter","driver":"fab-fff","event":"stage-transition","stage":"hydrate","ts":"2026-06-10T15:36:59Z"}
+{"event":"review","result":"passed","ts":"2026-06-10T15:36:59Z"}
+{"cmd":"fab-continue","event":"command","ts":"2026-06-10T15:38:21Z"}
+{"action":"enter","driver":"fab-fff","event":"stage-transition","stage":"ship","ts":"2026-06-10T15:44:08Z"}
+{"cmd":"git-pr","event":"command","ts":"2026-06-10T15:45:52Z"}
+{"action":"enter","driver":"git-pr","event":"stage-transition","stage":"review-pr","ts":"2026-06-10T15:49:11Z"}