Skip to content

Add a Grok usage collector for the agents panel - #13219

Closed
d4dgpt-netizen wants to merge 2 commits into
omacom:quattrofrom
d4dgpt-netizen:add-grok-agent-usage
Closed

d4dgpt-netizen wants to merge 2 commits into
omacom:quattrofrom
d4dgpt-netizen:add-grok-agent-usage

Conversation

@d4dgpt-netizen

Copy link
Copy Markdown

Why

Grok is already a first-class Omarchy default agent (omarchy default agent grok, Super+Shift+Ctrl+A, the agents menu). The top-bar agents widget never showed it, because that panel only draws usage records written by omarchy-agent-usage-<agent> collectors, and Omarchy shipped collectors for Claude, Codex, and Fireworks only.

What

  • Add omarchy-agent-usage-grok, which scans $GROK_HOME/sessions/**/usage.json (default ~/.grok/sessions) into the same display-ready JSON contract the panel already watches.
  • Count each turn once (session totals are a fallback when Grok has not written a turns array), keep input/cache/output buckets mutually exclusive, honor GROK_HOME, and cache like the Codex collector (--force / --limits-only).
  • Ship assets/grok.svg and assets/grok-light.svg from the official Grok mark.
  • Document Grok in the agents plugin README and manual/17-ai.md.

Grok CLI does not publish a rate-limit endpoint the way Claude Code and Codex do, so the tab shows local token history (today, last week, by model) without a limits meter.

Test plan

  • bash test/shell.d/agent-usage-grok-scanner-test.sh
  • Live scan of ~/.grok/sessions produced a ready record; writing it to ~/.local/state/omarchy/agents/usage/grok.json made a Grok chip appear in the agents panel next to Claude Code and Codex.

Grok is a first-class default agent but never appeared in the bar
widget, which only draws collectors that write a usage record.
Scan $GROK_HOME/sessions usage.json files (default ~/.grok/sessions)
into the same JSON contract Claude and Codex already use, and ship
the Grok mark so the panel can render a tab.

Grok CLI does not publish a rate-limit endpoint, so the record
carries local token history without a limits list.
SuperGrok is a shared weekly pool across Chat, Imagine, Voice, and
Build, not a 5-hour session window. Probe the same CLI billing
endpoint the Grok TUI uses, and draw that week plus Grok Build's
share of it in the same limits rows Claude and Codex already use.
@llstrk

llstrk commented Sep 27, 2026

Copy link
Copy Markdown

Automated AI review

Community review: Independent automated community review, unaffiliated with the Omarchy team, intended to help prepare PRs for their review.

Verified: the local-stats collector fits the agents panel contract. omarchy-agent-usage-update discovers it and writes grok.json with the same key set as the Claude record, and both new tests plus the existing agent-usage tests pass at head. The limits path has one significant issue: on an expired sign-in it redeems the Grok CLI's refresh token and discards the rotated replacement. There are also counting and parsing errors that inflate the numbers. Three other open PRs add the same collector file (see the end).

Grok CLI references below are to its public source, xai-org/grok-build at f0e3be1, whose version crate reads 1.0.41 (the version the collector sends).

Expired sign-in: the collector redeems the CLI's refresh token and discards the replacement

When the saved access token has expired (bin/omarchy-agent-usage-grok:656), refresh_access_token (line 460) POSTs the stored refresh_token to auth.x.ai/oauth2/token. It keeps only the new access_token and never writes auth.json. This happens before the 15-second reuse check (line 676), so every run while the token is expired makes a token request.

Synthetic probe (fake IdP that rotates refresh tokens and rejects reuse, no real endpoint contacted):

run 1  presents rt-original -> receives rt-rotated -> billing OK, limits shown
       auth.json afterwards: byte-identical, still rt-original
run 2  presents rt-original -> invalid_grant -> "Sign-in expired"
run 3  presents rt-original -> invalid_grant -> "Sign-in expired"

4 consecutive --limits-only runs: 4 token requests, 1 billing request

Grok's source treats these tokens as single-use:

  • xai-grok-login/src/storage.rs:23: "a refresh token can be used only once".
  • oidc/refresh.rs:29-35: Grok assumes, as a "conservative client-side bound", that an IdP accepts an already-rotated token for about 60 s, and says re-sending it after that "trips the IdP's reuse detection and revokes a successor".
  • On invalid_grant the CLI discards its stored tokens and asks the user to run grok login.

Impact: a user who leaves Grok closed past one access-token lifetime can find the Grok CLI signed out after the collector's next run. It then keeps re-presenting a spent token on every refresh and panel open. The collector is enabled by default. This consequence is inferred from Grok's source; the live IdP was not tested, and the token lifetime is not in source.

Smaller problems on the same path:

  • oauth_login (line 418) picks the first auth.json entry with a key that has not expired (or has no expiry), regardless of scope or auth_mode. For example, when an expired OAuth entry comes first, a following API-key entry or legacy web-login entry is sent as the Bearer token to the billing endpoint.
  • The token endpoint is hard-coded, so an enterprise OIDC entry's refresh token goes to auth.x.ai.
  • An entry without expires_at is never treated as expired. The CLI applies a 30-day TTL from create_time.
  • GROK_AUTH_PATH, which the CLI honours, is ignored.

Suggested change: drop refresh_access_token and follow omarchy-agent-usage-claude (lines 804-821), which deliberately does not refresh: report "Sign-in expired", serve open cached windows and let the Grok CLI refresh its own token. Alternatively, fetch billing through the CLI's ACP _x.ai/billing method, as #6485 and #10400 do. Select the entry by the xAI OAuth scope and skip api_key and web_login entries. No test covers the refresh path: a mutant with the refresh disabled passes both PR tests.

Forked and subagent sessions are counted twice

scan_sessions sums every sessions/**/usage.json (line 272) without de-duplication.

  • Forks: Grok copies the parent's usage.json turns into the fork with their original endedAt (xai-grok-shell/src/session/storage/jsonl/copy.rs:384-402, copy_usage defaults to true).
  • Subagents: the parent's usage folds in each subagent's spend (xai-chat-state/src/usage.rs:132-157), and the child is a separate session directory marked session_kind: "subagent" in summary.json.
Synthetic tree Actual Collector
Parent with 2 turns, fork with those 2 plus 1 new 3 prompts, 6600 tokens 5 prompts, 9900 tokens
Parent including one subagent's spend, plus the child session 5500 tokens, 1 session 6600 tokens, 2 sessions

The fork case follows Grok's copy code directly. The subagent case is modelled on Grok's source, not observed on a real ~/.grok.

Impact: inflated prompt, session, per-day and per-model counts, which also carry into synced snapshots.

Suggested change: skip session directories whose summary.json has a session_kind starting with subagent (#10400 does this; #12352 filters subagent-tagged events instead), and drop inherited fork turns, for example by reading parent_session_id and the fork point from summary.json, or by de-duplicating turns across files.

Credit usage below 1 % is read as a fraction

Grok defines creditUsagePercent as "a percentage of the allowance (0.0 to 100.0)" (xai-grok-shell/src/extensions/billing.rs:61), and its TUI always treats it that way. parse_billing reuses a scale heuristic: it divides by 100 only when some value is at least 1 (lines 570 and 500).

creditUsagePercent Grok meaning Panel shows
0.4 0.4 % 40 %
0.95 (Build 0.6) 0.95 % 95 %, bar alarm set
1.0 1 % 1 %

Impact: every weekly period starts at zero, so a nearly unused pool can show as almost exhausted and raise the bar alarm early in the week.

Related gaps in the same parser:

  • A payload without creditUsagePercent produces no rows, so when no cached window is still open (as right after a weekly reset) the section reports "Grok limits unavailable". The CLI treats a missing value as 0 %. Whether the live endpoint omits the field at zero usage is not confirmed.
  • The deprecated monthlyLimit/used shape also produces no rows, while the CLI derives a percentage from it. The limits test's "legacy" case includes creditUsagePercent: 25, so it does not exercise that shape.

Suggested change: always divide creditUsagePercent and usagePercent by 100, and treat a missing creditUsagePercent as 0 when currentPeriod is present. An always-divide variant still passes the PR's tests.

Cache-write tokens are counted twice

Grok's prompt_tokens includes cache reads and cache writes (xai-grok-sampling-types/src/conversation.rs:772-787), and the Codex collector subtracts both. split_tokens (line 144) subtracts only cache reads, then adds cache writes as their own bucket. A synthetic turn totals 130 tokens where Grok's own total is 110.

Impact: inflated token totals for Grok sessions on Anthropic Messages backends, the only backends that report cache writes.

Suggested change: subtract cache writes as well as cache reads from inputTokens in split_tokens. The scanner test currently expects the double-counted value (214 rather than 210), so its expected numbers need updating along with the fix.

Other notes

Open question, Build meter spelling: the collector matches only "product": "GrokBuild" (line 545 onward). Grok's only public fixture for this shape uses PRODUCT_GROK_BUILD (billing.rs:568-570) and notes the CLI does not read productUsage yet. In that fixture the Build figure (61.2) is higher than the pool (42.5), which does not fit a "share of the weekly pool" meter; shown as a limit, such a value can become the headline and set the alarm. A redacted real response would settle both points; accepting both spellings is a cheap hardening.

Smaller notes:

  • CLIENT_VERSION = "1.0.41" is sent as x-grok-client-version, which Grok's agent/proxy_headers.rs describes as required by the proxy's version gate. A raised minimum would make the billing probe fail while local stats continue.
  • The PR description still says the tab has no limits meter and lists only the scanner test; the second commit adds the billing probe, token refresh and limits test.
  • manual/17-ai.md says Grok's paid plans "share one weekly credit pool". Grok's source treats unified billing as a per-user setting with weekly or monthly periods, and the collector itself labels monthly periods.
  • AUTH_HELP ("sign in to start recording usage") is shown while local usage is already being recorded; sign-in only gates the limits.

Related open PRs

Verified: Grok's nanosecond RFC 3339 timestamps are attributed to the right local day, malformed usage.json files are skipped, GROK_HOME is honoured, cache writes are atomic under concurrent writers, the unwritable-cache fallback works, no token appears in stdout, stderr or the caches, and without auth.json the collector makes no network request (with no sessions either, the tab stays hidden). Scan cost was about 0.45 s per 1000 sessions of 20 turns on a warm in-memory filesystem, a lower bound.


Review information

Test scope: Source review of the PR head 72221e9a against the Omarchy collectors, update command and agents panel QML, and the Grok CLI's public Rust source. The PR tests, existing agent-usage tests, the real omarchy-agent-usage-update, mutants and behavioural probes ran in an isolated sandbox with no network and synthetic ~/.grok data and fake tokens. No xAI endpoint or real account was used, so live IdP rotation, token lifetime and the live billing response shape are untested. The panel was not rendered on a desktop.

AI process: Opus 5.5 Medium coordination and synthesis, Opus 5.5 Xhigh technical review and final fact check, GPT 6 Sol Xhigh search for related issues, Opus 5.5 Medium editorial check.

Opt out: To stop receiving these reviews, reply to this comment saying so.

@dhh

dhh commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Thanks! Grok's section in the agents panel landed with #13992 (plan, weekly credits meter, accounts, sessions). Closing this in favor of it.

@dhh dhh closed this Oct 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants