feat(usage): monitor token usage across agent accounts and cezar runs

[sheeerth]

🎯 Goal

Give the cockpit a permanent, honest answer to "how many tokens have I burned, and how close am I to the ceiling?" — counting the agents' own sessions, not just the ones cezar started. A read-out sits in the sidebar footer at all times; /usage holds the breakdown.

What Changed

• Contract (packages/contract/src/usage.ts, exported from the index): GET /api/v1/workspace/usage → {generatedAt, accounts[], runs}. Two halves that are deliberately never summed — every cezar run appears in both, so adding them double-counts. accounts[] is what each agent CLI recorded in its own home; runs is what cezar's own runs spent.
• Readers (packages/cezar/src/usage/):
  • samples.ts — the one definition of a window (rolling5h plus today/last7d/last30d anchored on the host's LOCAL midnight), the cache weighting reused from core/usage.ts, and de-duplication. Three readers do not get three notions of "today".
  • jsonl-scan.ts — incremental reader for append-only logs: resume at the last complete line (a half-flushed tail is re-read, not dropped), discard the cache when a file shrank, never open a file untouched since the retention floor.
  • providers.ts — Claude (<home>/projects/**, subagent transcripts included) and Codex (<home>/sessions/**, plus the rate_limits tail read).
  • runs-usage.ts — per-STEP attribution over run records, because run.tokensUsed is by construction their sum.
  • snapshot.ts — assembly and the per-account degrade policy.
• Server (server.ts): a usageRoutes family chained into workspaceV1, plus a demand-driven usage WS topic, both behind the same stale-while-revalidate cache shape health uses (30 s TTL, 5 min staleness ceiling). Project runs are read the way runs-index reads them — straight off runs.json for an unowned project, never through a built context, so opening a usage panel cannot prune worktrees or resume agents.
• Cockpit: UsageChip in the sidebar footer (a shell SLOT, mounted by app-shell-container, so AppShell stays presentational), the /usage route and its nav item, lib/usage-view.ts for the presentation logic the chip and the page share, and one root-level useTokenUsageSubscription mirroring useHealthSubscription.
• Docs: spec .ai/specs/2026-08-14-token-usage-monitor.md, the route's entry in BACKWARD_COMPATIBILITY.md §2, and the README view table (7 → 8 views).

The details that decide whether the numbers are right

• Claude Code writes one assistant reply once per stream flush, and --resume copies earlier replies into the new session file. A measured transcript held 69 usage lines carrying 28 distinct replies, so samples de-duplicate on message.id + requestId; summing lines would roughly double every figure.
• Codex folds cached tokens into input_tokens and reports a per-event delta beside a cumulative total. The delta is summed (so a resumed session adds up) and the cache is split out (so the weighting can price it at ~10%).
• No percentage is ever invented. Codex publishes rate_limits to disk and those are shown verbatim, with the reset resolved; Claude publishes none locally, so its limits[] is empty rather than filled from a plan ceiling cezar does not know.
• Cost appears only where a backend reported real money — never derived from a price table.

🧪 Tests

• npm run typecheck — clean (contract, api-client, server, web).
• npm test — 5707 passed / 314 files, including 51 new cases.
• npm run test:unit — 36 passed. npm run build + check:pack — ok (478 files). npm run test:package — 12 passed.
• New coverage: window/dedup/weighting arithmetic (samples.test.ts), the incremental reader's append/shrink/partial-tail/cutoff behavior (jsonl-scan.test.ts), both vendor formats including the stream-flush and --resume duplicates and the rate_limits shape (providers.test.ts), step-vs-run attribution and grouping (runs-usage.test.ts), the shared view logic (usage-view.test.ts), and the chip/page including the hidden-metrics, hosted-mode, unreadable-project and empty states (usage.test.tsx).
• Updated the pinned nav/topic lists the new nav item and WS topic legitimately extend (nav-items, app-shell, project-groups, command-palette, health-topic).
• Verified against a real ~/.claude (63 MB, 66 projects, 30 days): 11 505 samples, 740 ms cold, 20 ms warm — the incremental cache does what it claims.

💥 Breaking Changes

None. The route is additive, no existing shape changed, no new env var, no new user-authored state. Hosted mode serves accounts: [] (those homes are on another machine), and CEZ_HIDE_TOKEN_METRICS=1 removes the surfaces entirely.

One caveat worth a reviewer's eye

Codex's rollout format could not be verified against a live ~/.codex on this machine — there is none. The parser handles both the nested info.last_token_usage shape and the older flat one, and an unrecognized line contributes nothing rather than a wrong number, but a Codex user's confirmation would be worth having.

🤖 Generated with Claude Code

[github-actions]

📦 npm preview dry run — 0.10.0-pr2.7

⚠️ Dry run — the NPM_TOKEN secret is not configured, so nothing was actually published. See docs/publishing.md for the one-time admin setup.

Try this PR build (exact pinned version — copy-paste as-is):

npx cezar-cli@0.10.0-pr2.7                                # cockpit at http://localhost:4321
npx cezar-cli@0.10.0-pr2.7 run "…"                        # headless run
npx cezar-cli@0.10.0-pr2.7 server-deploy --platform <id>  # roll a server to this exact build

Also tagged: npm install -g cezar-cli@pr-2 (moving tag for this PR).
Packages: cezar-cli@0.10.0-pr2.7 → @open-mercato/cezar@0.10.0-pr2.7 → @open-mercato/cezar-api-client@0.10.0-pr2.7.

[sheeerth]

🤖 Claiming this PR — starting om-auto-fix-pr run. Started: 2026-08-23T07:28:28Z.

Note: this repository carries only the default GitHub labels, so every pipeline label this run would normally apply (in-progress, review, merge-queue, needs-qa) is skipped by the apply_label guard rather than created. Label state is reported in the run summary instead.

[sheeerth]

🤖 om-auto-review-pr taking over the chain lock — review + autofix pass after the base merge. Started: 2026-08-23T07:37:02Z.

[sheeerth]

🔍 Code Review: feat(usage): monitor token usage across agent accounts and cezar runs

🎯 Summary

Re-review of PR #2 after two changes since the last pass: the review fixes in 5aef4a31, and the base merge in 12e476ef that brought 34 commits of main (including the 0.10.0 release bump, the global Tasks page, and the sidebar-footer fix open-mercato#879) onto this branch. The feature itself is one workspace-level read, GET /api/v1/workspace/usage, carrying two halves that are deliberately never summed — what each agent CLI recorded in its own home, and what cezar's own runs spent — plus a permanent read-out in the sidebar footer and a /usage page.

The scope reviewed is the full merged diff: 37 files, +3411/−20, of which 16 are new. Reviewed with particular attention to the three merge resolutions, since a textual auto-merge can compile and still be semantically wrong.

What this pass confirms as good: the two halves stay separate at the contract level rather than by convention, so nobody can accidentally add them; the incremental JSONL reader handles the three cases that matter (resume at the last complete line, discard on shrink, skip files untouched since the retention floor) and has tests that assert on parse counts rather than only on results, so the incrementality itself is pinned; the de-duplication key addresses a real property of Claude's transcript format that is easy to miss; the route inherits runs-index's side-effect-free contract, and the new usage-api.test.ts pins it by asserting a cold project's runs/ directory never appears.

Verdict

✅ approve — no blockers and no majors. Every finding from the previous pass is fixed and pinned by a test, the base merge is semantically clean, and CI is green on the merged head. The one caveat below is a disclosure, not a finding.

Submitted as a comment review rather than a formal approval only because GitHub does not permit approving one's own pull request. The verdict above is the review's real outcome.

🧪 Validation Gate

Run in configured order on the merged head (12e476ef):

Command	Status	Notes
npm run typecheck	✅ PASS	Clean across contract, api-client, server and web.
npm test	⚠️ PASS in CI, 6 environment-only failures locally	See the note below — the same six fail on unmodified origin/main in this sandbox, and CI runs the suite green on this exact commit.
npm run test:unit	✅ PASS	36 passed.
npm run build	✅ PASS	check:pack ok — 491 files, 85 under web/dist.
npm run test:package	✅ PASS	15 passed.
GitHub Actions — Unit, build, E2E, and package	✅ PASS	4m29s on the merged head.
GitHub Actions — Publish npm snapshot	✅ PASS	26s.

On the six local failures. They are getRepoInfo / worktreeShortstat / commitAll / health / projects / automations cases that assert behaviour outside a git repository. In this sandbox cezar points TMPDIR at .ai/cezar/tmp/<taskId> (the open-mercato#785 agent-tmpdir feature), which is inside the checkout — so mkdtempSync(tmpdir()) is not outside a repo at all and the assertions legitimately fail. Two pieces of evidence make this conclusive rather than convenient: the same six fail on an unmodified origin/main worktree in the same environment, and GitHub Actions runs the full suite green on this branch's merged head. Per the CI procedure, failures that also occur on the base branch are out of scope for this PR and are reported, not fixed from here.

Findings

No blockers, majors, minors or nits survived this pass. The previous review's six findings were all fixed in 5aef4a31 and are re-verified here:

• reconcile() now invalidates workspaceQueryKeys.usage, so a remotely served cockpit — which gets no usage WS topic, because a browser WebSocket cannot carry a reverse proxy's credentials — refreshes on reconnect and visibility instead of freezing on its first snapshot. Pinned by the reconcile-doctrine tests.
• The runs half now counts the boot project even when the registry does not list it. This was the worse defect the first review missed: registration is suppressed for task worktrees and for $HOME, so the panel printed a confident zero in exactly the setup cezar's own task worktrees run in.
• readUsage computes fresh when no socket hub is injected, matching readHealth.
• The Claude de-duplication key falls back to message.id alone when a line carries no requestId (measured: 23 of 10 267 real usage lines).
• scanJsonlTree returns {samples, newestPath, droppedFiles} and takes an onForget eviction signal — one tree walk instead of two, a bounded per-file model map, and a file-cap undercount that says so.
• The per-account failure reason is a fixed string, since Node's fs errors quote the path they failed on.

Merge resolution review (12e476ef)

Three files conflicted and each was checked for semantic — not merely textual — correctness:

• packages/web/src/routes.tsx — both sides added an import (UsageRoute here, GlobalTasksRoute on main); both kept. The <Route path="usage"> registration survived intact.
• packages/web/src/api/queries.ts — main changed useRunsIndex to take refetchIntervalMs; that signature is kept, this branch's useTokenUsage / useTokenUsageSubscription are kept, and the two doc blocks were re-separated so each again precedes its own function (the naive resolution left main's runs-index doc block orphaned above this branch's hook).
• BACKWARD_COMPATIBILITY.md — main added referenceStatuses to the runs-index shape; that line is taken from main and the new workspace/usage entry sits beside it.

Every mount point was then verified to survive the merge rather than assumed: the usage route, the usageChip slot in both app-shell and app-shell-container, the nav item, the reconcile invalidation and the root subscription. The merged NAV_ITEMS order is coherent, and the pinned nav/palette/shell tests pass against it.

One adjacent interaction was checked specifically because main had just touched it: open-mercato#879 fixed the sidebar footer's controls row overflowing at nightly-length version strings. This PR adds a separate footer row, so it does not re-enter that row's constraint; at the enforced MIN_SIDEBAR_WIDTH of 264px the new row's worst-case content (999.9k / 5h plus a 40px bar and 100%) measures roughly 166–176px inside a 236px content box, so it has margin and no elastic item is required.

💥 Breaking Changes

• No exported or public symbol removed or renamed without a deprecation path — the AppShell usageChip prop and NavProps addition are optional and additive.
• No function signature changed in a breaking way.
• No required type field removed or narrowed.
• No HTTP route removed or renamed; /api/v1/workspace/usage is new, single-mount, and inventoried in BACKWARD_COMPATIBILITY.md §2 together with its degrade and hosted-mode behaviour.
• No field removed or retyped in an existing response shape.
• No event or message name renamed; the WS topic usage is new and trusted-only by default, which is correct for a payload naming projects and accounts.
• No CLI command or flag changed.
• No database schema to change — state is plain files, and no file format was touched.
• No config key renamed, no default changed silently, no new CEZ_* variable and no new user-authored state.
• No contract had to change, so no deprecation window applies.

🧪 Test Coverage

The feature carries 96 test cases across seven files. The arithmetic (windows anchored on local midnight, retention floor, clock-skew tolerance, cache weighting, de-duplication) is covered in samples.test.ts; the incremental reader's append / shrink / partial-tail / cutoff / eviction behaviour in jsonl-scan.test.ts, with assertions on how many lines were actually parsed; both vendor formats — including the stream-flush duplicates, the --resume copies, the Codex cached-token split and the rate_limits shape — in providers.test.ts; step-versus-run attribution and grouping in runs-usage.test.ts; the shared presentation logic in usage-view.test.ts; the chip and page including hidden-metrics, hosted-mode, unreadable-project and empty states in usage.test.tsx; and the route itself — both halves, hosted-mode withholding, the boot-project fallback, the side-effect-free contract and the degrade-to-zeros path — in usage-api.test.ts.

Gaps worth naming, none of them blocking:

1. Codex's rollout format is still unverified against a live ~/.codex — there is none on this machine. The parser accepts both the nested info.last_token_usage shape and the older flat one, and an unrecognized line contributes nothing rather than a wrong number, so the failure mode is an undercount rather than a false figure. A Codex user's confirmation would close this.
2. No before/after bundle measurement. The page adds no browser dependency and draws with CSS bars rather than a chart library, so the growth is source only; the built main chunk is 280 kB / 75 kB gzip. Worth a number if the repo starts tracking one.

[sheeerth]

🔬 UI QA — /usage and the sidebar usage chip

Verified by driving a real Chrome (agent-browser) against a locally booted cezar
(.ai/scripts/test-env-up.sh, CEZ_DRY_RUN=1, production build). Read-only on source:
nothing was edited, pushed, or merged.

Verdict: ✅ PASS — every required step passed. One step is honestly marked not exercised.

Priority

P1 — a primary user-facing surface (a new page plus a permanent element in the app shell).
Not P0: the feature is read-only, touches no auth, money, or data scoping.

What was verified

#	Where	What was expected	Result	Evidence
1	/usage, local mode	Page renders both halves, labelled and never summed; account cards carry windows, a 30-day sparkline and a model breakdown	✅ PASS	step-01-usage-page.png
2	Sidebar footer, every route	The chip is present, shows the rolling-5h total, and falls back to the day's total when no vendor quota exists	✅ PASS	step-01-usage-page.png (bottom left: 21.4M / 5h · 21.4M today)
3	Tasks page → chip click	The chip navigates to the usage page	✅ PASS	step-03-chip-navigates.png
4	Codex account with no ~/.codex	Row is present and states why, rather than vanishing or showing a zero	✅ PASS	step-01-usage-page.png (right card)
5	Claude account	No quota percentage is shown — Claude publishes none locally	✅ PASS	step-01-usage-page.png (no % on the Claude card)
6	Hosted mode (CEZ_REMOTE=1)	Account rows withheld with an explanation; the cezar-tasks half still answers	✅ PASS	Text capture below
7	Live incrementality	Numbers advance as the running agent appends to its transcript	✅ PASS	21.4M → 21.8M across two captures ~4 min apart
8	Mobile viewport	Layout holds at phone width	⚠️ not exercised — the provider's device emulation did not take effect in this sandbox, so no mobile claim is made

Notes worth a reviewer's eye

Hosted mode was verified first, by accident and then on purpose. The sandbox this ran in exports
CEZ_REMOTE=1, so the first boot came up hosted and the page correctly rendered "This cockpit is
served remotely, so the agent homes it would read are on another machine. Only the cezar tasks below
can be counted from here." with capabilities.localHandoff: false and an empty accounts[], and the
sidebar chip correctly did not render. The environment was then rebooted with CEZ_REMOTE unset to
exercise the local-mode surface above. Both branches are therefore verified against a real server
rather than only in unit tests.

The numbers are real. GET /api/v1/workspace/usage answered from this machine's actual
~/.claude: 21.4M weighted tokens in the rolling 5 h window, 247.5M over 7 days, 369.7M over 30,
split across claude-opus-5 / claude-opus-4-8 / claude-haiku-4-5. The Codex row reports
available: false with no sessions recorded for this account yet, which is correct — there is no
~/.codex here. The cezar-tasks half reads zero because the test env boots against a throwaway
CEZ_HOME, so its registry holds no runs; that is the honest answer for that sandbox, not a defect.

One cosmetic nit, not blocking: the account card header reads Claude Code Default and then
default again at the right edge — the account's label and the isDefault badge say the same word
twice. Worth collapsing when the badge and the label agree.

Environment

• App: production build, http://127.0.0.1:53197, CEZ_DRY_RUN=1 (agent CLIs mocked; no login, no network).
• Browser: Chrome for Testing 152.0.7977.54 via agent-browser 0.33.2, launched with --no-sandbox
  (this host disables unprivileged user namespaces) and a short TMPDIR (cezar's own agent TMPDIR
  exceeds Chrome's singleton-socket path limit). Neither workaround touches the app under test.

step-01-usage-page.png

step-03-chip-navigates.png

[sheeerth]

🧪 Follow-up: browser-level test for the usage surface

This change ships thorough component tests (usage.test.tsx) and a route test (usage-api.test.ts), but no e2e spec — packages/web/e2e/ has no usage case. The QA pass above was driven by hand, so nothing yet stops the surface regressing silently. Ready-to-implement scenario for a follow-up om-integration-tests run, matching the conventions of the existing specs in that folder:

File: packages/web/e2e/usage.e2e.ts

1. The page renders both halves, labelled. Boot the fixture app, navigate to /usage, assert a region named Agent accounts and a region named cezar tasks both exist, and that the four window labels (Last 5h, Today, Last 7 days, `Last 30 days") appear under each. This is the invariant the whole feature rests on — the two halves are never merged into one total.
2. The chip is present in the shell and navigates. From /, assert [data-slot="usage-chip"] exists, click it, and assert the heading Token usage is reached. Pins the always-visible read-out and its link in one step.
3. An unavailable account states why. With a fixture whose Codex home does not exist, assert the Codex card renders and contains its reason text rather than being absent or showing a zero.
4. Hosted mode withholds the account half. Boot the fixture with CEZ_REMOTE=1 and assert the "served remotely" paragraph is shown, that no [data-slot="usage-account"] article exists, and that the cezar tasks region still renders. This is a contract clause in BACKWARD_COMPATIBILITY.md §2 and currently has unit coverage only.
5. Hidden metrics remove the surface. With CEZ_HIDE_TOKEN_METRICS=1, assert the page shows the "Token metrics are hidden" state and the sidebar chip is absent.

The fixture work is the only real cost: steps 3–5 need a server booted with a controlled CLAUDE_CONFIG_DIR/CODEX_HOME pointing at a small transcript fixture, so the numbers are deterministic. packages/web/e2e/fixtures is the place for it.

Evidence only — no labels were changed by this QA pass.

[sheeerth]

🤖 om-auto-fix-pr — run summary

Verdict: merge-ready. Mergeable, CI green, reviewed, and UI-verified with screenshots. This skill never merges — handing off to om-approve-merge-pr / a human.

Base merge (step 3)

main had moved 34 commits ahead and the PR was CONFLICTING. Merged origin/main into the branch (12e476ef) and resolved three conflicts:

• packages/web/src/routes.tsx — both sides added an import; kept both (UsageRoute and main's GlobalTasksRoute).
• packages/web/src/api/queries.ts — kept main's new useRunsIndex(enabled, refetchIntervalMs) signature alongside this branch's useTokenUsage/useTokenUsageSubscription, and re-separated the two doc blocks so each again precedes its own function (the naive resolution orphaned main's runs-index docblock above this branch's hook).
• BACKWARD_COMPATIBILITY.md — took main's runs-index line (it gained referenceStatuses) and kept the new workspace/usage entry beside it.

Every mount point was then re-verified rather than assumed: the usage route, the usageChip slot in both app-shell and app-shell-container, the nav item, the reconcile invalidation, and the root subscription. PR is now MERGEABLE / CLEAN.

Review (step 4.1)

Full re-review submitted above. No blockers, no majors — the six findings from the earlier pass are fixed and pinned by tests, and the merge resolutions check out semantically. Submitted as a comment review purely because GitHub does not allow approving one's own PR; the body states the real verdict.

CI (step 4.2)

Already green on the merged head — Unit, build, E2E, and package (4m29s) and Publish npm snapshot (26s). No stabilization iterations were needed.

One thing worth recording: six tests fail locally in this sandbox and none of them are this PR's doing. They are the "outside a git repository" cases, and cezar points TMPDIR at .ai/cezar/tmp/<taskId> — inside the checkout — so mkdtempSync(tmpdir()) is not outside a repo at all. Verified two ways: the same six fail on an unmodified origin/main worktree in the same environment, and CI runs the suite green on this exact commit. Per the CI procedure, base-branch failures are reported, never fixed from a feature branch.

UI verification (step 4.3)

Ran and passed — see the QA comment above with screenshots. Both deployment branches were exercised against a real server: hosted mode withholds the account rows and hides the chip, local mode renders the page and chip against this machine's actual ~/.claude (21.4M weighted tokens in the rolling 5 h window). The mobile-viewport smoke is marked not exercised rather than passed — the provider's device emulation did not take effect here.

Follow-ups (step 5)

One cosmetic nit found by QA, deliberately not fixed here to keep the PR's scope: on /usage, an account card whose label is "Default" prints the word twice — Claude Code Default … default — because account.label and the isDefault badge coincide on the zero-config machine. Fix would be to drop the badge when it only repeats the label (packages/web/src/routes/usage.tsx, AccountCard). Issues are disabled on this repository, so it is recorded here rather than filed; it needs a tracked home if issues are ever enabled.

A second, larger follow-up — the missing browser-level test for this surface — is written up as its own comment above, ready for om-integration-tests.

Labels (step 6)

None applied, by the guard, not by omission. This repository carries only the default GitHub labels (bug, documentation, enhancement, …); every pipeline label this run would normally set — in-progress, review, merge-queue, needs-qa, priority and risk — is absent, so apply_label logs a skip rather than creating labels the project never chose. The QA gate therefore cannot be expressed as a label here: treat this PR as needing a human QA sign-off before merge, which is what needs-qa would otherwise say.

Lock released; no CI follow-up is owed, since CI has already settled green.