Grok Build: characterize logs/unified.jsonl and events.jsonl on a real install (follow-up to PR #298)

[willwashburn]

Status against main @ 5f9aeca (2026-10-01): the implementation landed in PR #298 (merged 2026-09-30 as part of the #293–#299 stack). What remains is the characterization the PR deliberately left open: every unified.jsonl field name and the session-id join were taken from tokscale's parser, not from a real Grok install, and the catalog doc marks them inferred. This issue is narrowed to that.

What shipped (PR #298)

• <GROK_HOME>/logs/unified.jsonl is a second Grok source: byte-cursor reads on every sweep and every hydration (before the stamp check), watched and fingerprinted. Rows are staged in grok_unified_usage (whole-row hash key) and attached as text-less assistant events with raw_kind = "unified_log_usage" and their own request span (crates/ai-hist/src/ingest.rs:13284-13920).
• No double count: a turn's turn_completed.usage is demoted to non-counted turn_usage only when a unified row falls inside that turn's window. Diagnostics GROK_USAGE_MIXED_SOURCES, GROK_USAGE_PARTIAL, GROK_UNIFIED_LOG_UNREADABLE.
• turn_completed.usage.{inputTokens, outputTokens, cachedReadTokens, reasoningTokens, costUsdTicks, modelUsage} is now read and normalized as per-request usage (docs/session-catalog.md, "Usage: a context proxy, plus per-turn usage when the build writes it").
• eventId reuse handled: first occurrence keeps ev:<id>, repeats get ev:<id>#n.
• Model/start fallbacks: _meta.modelId, single-key modelUsage, summary.json current_model_id/model_id (top level and under info, both tried), first 500 lines of events.jsonl when summary.json is absent. totalTokensBeforeCompaction is named in the signals marker and kept verbatim; it is deliberately not reconciled (accounting is burn's).
• grok_events_v3 → v4 re-reads Grok sessions once; HYDRATION_PARSER_VERSION 13 → 14. Fixture corpus unified-usage.

What remains: confirm against a real Grok Build install

The catalog doc's jq checklist (docs/session-catalog.md, "How each adapter works → grok") is the procedure. Someone with Grok Build installed should run it and record the answers in the doc's provenance column:

1. logs/unified.jsonl row shape. Field names for session id, pid, model, and the input/output/cache token counts; whether rows carry a timestamp (PR feat(grok): read per-inference usage from logs/unified.jsonl (#212) #298 treats a timestamp-less row as covering every turn of its session, which favours never double counting and could under-count). Which of event_id | eventId | id | uuid | ctx.* is present. Whether process-start and model-change rows exist as tokscale describes (crates/tokscale-core/src/sessions/grok.rs:1030-1070).
2. Session-id join. Whether the log's session id equals summary.json info.id / the session directory name.
3. events.jsonl. Whether it exists in a current session directory at all, and its model_id / session_id / ts keys.
4. turn_completed.usage semantics. tokscale fixes inputTokens as including cachedReadTokens and outputTokens as including reasoningTokens; confirm, since crate::usage normalization depends on it.
5. summary.json parent-session references for forked/restored sessions: the guide names them, field spelling unknown; still "not read" in the doc.

Each confirmed or corrected fact flips the matching unverified / inferred cell in the catalog's Grok record-shape table and, if the shape differs, becomes a fixture under tests/fixtures/grok/unified-usage/.

Acceptance

• The Grok record-shape table has no unverified cell for unified.jsonl or events.jsonl, and the ADR capture-matrix cell for Grok usage moves from ◐ to ●, or the doc records why it cannot.
• If a shape differs from tokscale's, the parser and fixture are corrected in the same change.

Related

• [G1] Grok CLI: event-level parity (chat_history + updates.jsonl join, tools, subagents, compaction, real timestamps) #167 (closed), PR feat(grok): read per-inference usage from logs/unified.jsonl (#212) #298 (implementation), [G1] Checked-in harness fixture corpus and parser characterization tests (port burn's 40 fixtures) #163 (fixture corpus), tokscale d8fd670 sessions/grok.rs (the inferred source).