Skip to content

feat(cli): refresh the statusline ctx segment per settled provider request (#4545) #4550

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
me2seeks merged 2 commits into from
Sep 3, 2026
+814 −0
Merged

feat(cli): refresh the statusline ctx segment per settled provider request (#4545) #4550

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1d07c14
docs(cli): design for live TUI ctx updates (#4545)
me2seeks Sep 2, 2026
4be9222
feat(cli): refresh the statusline ctx segment per settled provider re…
me2seeks Sep 2, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
144 changes: 144 additions & 0 deletions docs/tui-live-ctx-updates.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# TUI live ctx updates (#4545)

Status: design. Issue: https://github.com/apache/maka/issues/4545

## Problem

The TUI statusline `ctx used/window pct%` segment updates **once per turn**, when
the turn fully ends. During a long agentic turn — dozens of tool steps over
minutes, exactly when the context grows fastest — the indicator sits stale at
the previous turn's value, so the user loses the signal that says "time to
`/compact` or wrap up".

## Audit: how it works today

Every claim below was verified against `main` (`6c632b1339`).

### Current TUI path (push, once per turn)

| # | Claim | Evidence |
|---|-------|----------|
| 1 | The ctx segment renders `used = modelContextWindow - usage.contextRemaining`; window comes from the model catalog | `packages/cli/src/pi-transcript.ts` L1678–1693 (`renderMakaPiStatusLine`), window wired at `packages/cli/src/pi-tui-runner.ts` L618, L1266 |
| 2 | `usage.contextRemaining` is only written by `accumulateUsage`, reached from stored messages (transcript rebuild) or a live `token_usage` SessionEvent | `packages/cli/src/pi-transcript.ts` L248–267 (`accumulateUsage`), L472, L751, L1003 |
| 3 | The runtime emits `token_usage` with `contextRemaining` exactly once per send, in the *Final usage event* block after the agent loop breaks | `packages/runtime/src/ai-sdk-backend.ts` ~L2738–2800 |
| 4 | Mid-turn, every `step-finish` boundary already captures `stepUsage.inputTokens` into `lastStepInputTokens` — but it only feeds the end-of-turn computation and the durable `recordUsageCheckpoint` hook, which is fire-and-forget persistence, not a live event | `packages/runtime/src/ai-sdk-backend.ts` L2181–2202; hook contract L751–753 |
| 5 | `/context` is refused mid-turn, but the gate is the TUI's own `runControl` serial lock (exists to stop prompts racing session/model switches), not a protocol limit | `packages/cli/src/pi-tui-runner.ts` L3261–3265, L882–914 |

### Desktop prior art (pull, per settled request)

| # | Claim | Evidence |
|---|-------|----------|
| 6 | The Host commits a latest-context snapshot at **every provider request settlement** (each LLM step), carrying `inputTokens` and `contextWindow` | `packages/runtime/src/provider-request-telemetry.ts` `finalize` → `emitModelCallAttempt` → `accounting.record({ attempt, latestContext })` (~L469–640); `packages/runtime/src/latest-context-snapshot.ts` |
| 7 | The commit is awaited **before** the `finish` part is enqueued to the consumer, so any UI event that follows the step (e.g. `tool_start`) observes the snapshot already durable — no read race | `packages/runtime/src/provider-request-telemetry.ts` stream `pull` handler ~L368–390 |
| 8 | `context.diagnostics.query` is a plain read: header snapshot + run-store projection read; no execution authority, no busy gate | `packages/runtime-host/src/server/context-coordinator.ts` `#queryDiagnostics`; spec `mode: 'query'` in `packages/runtime-host/src/protocol/context.ts` L107–117 |
| 9 | The desktop inspector subscribes to the live session event stream and re-reads the diagnostics on trace-relevant events (`tool_start`, `tool_result`, `token_usage`, `provider_retry`, `error`, `complete`, `abort`), coalesced at 400 ms; a failed re-read leaves the last value standing | `apps/desktop/src/renderer/session-trace-refresh.ts` L21–37; `apps/desktop/src/renderer/features/workbar/tools/inspector/use-session-trace.ts` L59 (`TRACE_REFRESH_DEBOUNCE_MS = 400`), L255–274 |
| 10 | Desktop derives the bar as `used = inputTokens`, `ratio = used / contextWindow`, from the snapshot alone | `session-inspector-overview-model.ts` `contextBudget()` ~L210–241 |
| 11 | The TUI driver already exposes the same query; the TUI always talks to the Host | `packages/cli/src/runtime-host-session-driver.ts` L1117 (`getContextDiagnostics`); interface `packages/cli/src/session-driver.ts` L230 (optional) |
| 12 | The TUI runner's `onEvent` sees every live event mid-turn | `packages/cli/src/pi-tui-runner.ts` L1455–1483 |

Semantics line up: the statusline's `contextRemaining = window − lastStepInputTokens`
(#1067) and the snapshot's `inputTokens` describe the same settled request, so
`contextRemaining ≡ diagnostics.contextWindow − diagnostics.inputTokens`.

## Design: reuse the desktop pull model in the TUI

Add a live-refresh hook to the TUI runner. No protocol, runtime, or persistence
changes.

### New module: `packages/cli/src/tui-context-refresh.ts`

- `isCtxRefreshRelevantEvent(event: SessionEvent): boolean` — same event set as
desktop's `TRACE_RELEVANT_EVENT_TYPES` (audit #9). `tool_start`/`tool_result`
are the mid-turn step boundaries; the rest close or annotate the turn.
Keeping the set identical to desktop's keeps one answer to "when is the
context worth re-reading".
- `createCtxRefresher({ query, apply, delayMs, schedule, cancel })` — a
restart-on-call debounce with a monotonic revision counter, mirroring
desktop's `createRefreshCoalescer` plus the `contextRevisionRef` guard:
only the latest issued query may apply; a late or failed resolution leaves
the current value standing (audit #9). Clock and timer injected, following
the runner's existing `shellRunTicker` seam, so tests drive it
deterministically.

### Wiring in `pi-tui-runner.ts`

In `onEvent` (audit #12), after `applyMakaSessionEventToTranscript`:

1. `if (isCtxRefreshRelevantEvent(event)) ctxRefresher.request()`.
2. The refresher calls `input.driver.getContextDiagnostics?.()` directly —
deliberately **not** through `runControl`, whose serial lock exists for
mutations (audit #5).
3. On `status: 'available'` with both `inputTokens` and `contextWindow`
present, set `state.usage.contextRemaining = contextWindow − inputTokens`
and `requestRender()`. The statusline keeps its existing formula, color
thresholds, and degradation states untouched; the catalog window stays the
displayed denominator, matching what the `token_usage` path already does
(both windows derive from the selected model's metadata).
4. Stale-session guard: the query captures `driver.getSessionId()` at request
time and `apply` drops the result when it changed — session switches reset
`state.usage` (`replaceTranscript`), and a pre-switch value must not land
afterwards. This guard covers every switch path uniformly, so no per-switch
cancellation wiring is needed; a refresh scheduled across a switch simply
queries the adopted session, which is the value the statusline should show.
5. Lifecycle: `ctxRefresher.cancel()` on teardown (alongside the existing
ticker disposal), which also retires any in-flight query.
6. Event coverage: every live turn drains through `runMakaPiTuiTurn`'s
`onEvent` (user-submitted and Host-attached turns alike) and the
`resumeLatest` loop — both hooked. `/compact` is deliberately not hooked:
its own `token_usage` already writes the authoritative post-compact value.

The end-of-turn `token_usage` event stays the authoritative **persisted**
record; the pull only enriches the live turn. Both derive from the same
settled request, so they cannot disagree.

### Out of scope (recorded, not forgotten)

- Unlocking `/context` mid-turn over the same query path — a free follow-up,
kept out of this PR to stay small.
- Desktop needs nothing; it already has this granularity.
- Token-level updates during one streaming request: providers only report
input tokens at completion, so exact mid-request values do not exist; the
pre-dispatch `bytes/4` estimate is too rough (base64 attachments) to show.

### Why not a new push event

- Protocol surface: a new SessionEvent type touches the core schema, the
backend emission point, the host mapper, and rebuild/persistence semantics.
- It creates a second derivation of the same number; pull keeps TUI and
desktop on one source of truth (the snapshot row), so resume / backfill /
compact edge cases cannot drift between two paths.
- Reusing `token_usage` with partial fields was rejected: `accumulateUsage`
treats it as cumulative billing input, and "incomplete usage is no usage"
(#972).

## Test plan (`packages/cli/src/__tests__/`)

- Mid-turn `tool_start` with a diagnostics result → statusline ctx reflects
the new value before turn end.
- Debounce: a burst of relevant events within the window issues one query.
- Revision guard: two overlapping queries resolve out of order → the older
resolution is dropped.
- Query failure / `status: 'unavailable'` → previous value stands.
- Session switch between request and resolution → value not applied.
- Driver without `getContextDiagnostics` (optional method) → no-op, no crash.
- Turn-end `token_usage` still lands exactly as today (regression guard on
`accumulateUsage`).
96 changes: 96 additions & 0 deletions docs/tui-live-ctx-updates.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
<!--
Licensed to the Apache Software Foundation (ASF) under one
or more contributor license agreements. See the NOTICE file
distributed with this work for additional information
regarding copyright ownership. The ASF licenses this file
to you under the Apache License, Version 2.0 (the
"License"); you may not use this file except in compliance
with the License. You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing,
software distributed under the License is distributed on an
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
KIND, either express or implied. See the License for the
specific language governing permissions and limitations
under the License.
-->

# TUI ctx 实时更新：人话讲解（#4545）

英文正式设计见 [tui-live-ctx-updates.md](./tui-live-ctx-updates.md)。这篇用大白话讲清楚：
问题是什么、desktop 已经怎么做的、我们要抄什么、以及那些名词都是什么意思。

## 一句话版本

TUI 底部状态栏有个 `ctx 45k/200k 23%` 的指示器，告诉你"上下文装满多少了"。
现在它**每轮对话结束才刷新一次**；而 desktop 版 maka 是**模型每跑完一步就刷新**。
方案：把 desktop 的刷新方式原样搬到 TUI，只改 CLI 包，不动任何协议。

## 先搞懂名词

| 名词 | 人话解释 |
|------|----------|
| **token** | 模型读/写文字的最小计费单位，约等于一个词的碎片。你给它的和它回你的都按 token 算。 |
| **上下文窗口（context window）** | 模型一次能看到的最大 token 总量，比如 200k。系统提示、历史消息、工具定义、工具结果全塞在里面。装满了就必须压缩（compact），否则报错或降智。 |
| **ctx 指示器** | TUI 状态栏上的 `ctx 已用/总量 百分比`，告诉你窗口还剩多少。 |
| **turn（轮）** | 你按一次回车 → agent 完全停下来，这整个过程。agentic 场景下，一个 turn 里模型可能反复"想一步、调个工具、再想一步"，跑几分钟。 |
| **step（步）** | 一个 turn 内部的每一次"模型调用 + 工具执行"循环。一个 turn = 很多 step。ctx 就是在 step 之间涨上去的（工具结果塞进了上下文）。 |
| **请求结算（settle）** | 一次模型请求跑完，provider（模型厂商）上报"这次实际用了多少 token"。**只有结算时才能拿到精确数字**——流式输出途中谁也不知道这次请求的输入到底多少 token，所以"token 级实时"在原理上就不可能，谁也做不到。 |
| **token_usage 事件** | runtime 在**整个 turn 结束后**发的一条消息，里面有这次 turn 的用量账单。TUI 现在的 ctx 就靠它刷新——这就是"每轮才更新"的根源。 |
| **快照（latest-context snapshot）** | Host（后端进程）在**每次请求结算时**写的一行记录："最近一次请求，输入 X token，窗口 Y。" 每个 step 都会更新，不用等 turn 结束。desktop 的 ctx 条就是读这行记录。 |
| **pull（拉）vs push（推）** | push = 后端主动把数据塞给界面（token_usage 事件就是 push，但一轮只推一次）。pull = 界面自己开口问："现在上下文多满了？" desktop 用的是 pull。 |
| **防抖（debounce）** | 事件密集来时（一步里可能连发好几个事件），等 400ms 合并成一次查询，避免刷屏。desktop 就是这么做的，我们照抄。 |
| **busy gate（忙锁）** | TUI 自己的一把锁：turn 运行时禁止执行 `/model`、`/session` 这类会改状态的命令，防止打架。`/context` 命令现在也被它挡住——但注意，这是 TUI 自己的规定，**不是后端禁止查询**。我们的刷新钩子绕开这把锁直接问后端，合法。 |
| **竞态（race）** | "查询发出时数据还没写好"的风险。已排除：后端是先落库快照、再发事件给界面（顺序有 await 保证），所以界面收到事件时快照必然已就绪。 |

## 问题到底是怎么回事

```
你 → 发消息 ── turn 开始 ────────────────────────────── turn 结束
step1 step2 step3 ... step20 │
│ │ │ │ │
ctx 涨了 又涨了 又涨了 又涨了 token_usage 事件
│ │ │ │ │
TUI 状态栏: 【旧值】【旧值】【旧值】...【旧值】 【终于更新!】
desktop: 【更新】【更新】【更新】...【更新】 【更新】
```

最讽刺的是：**数据后端早就有**——每个 step 结束都记了账（审计 #4/#6），
只是没人告诉 TUI 的界面。desktop 会主动去问，TUI 不会。

## 方案（抄 desktop 的作业）

在 TUI 的事件处理入口（`onEvent`，每个 live 事件都经过这里）挂一个钩子：

1. 收到 `tool_start` / `tool_result` 等"上下文可能变了"的事件 → 触发防抖刷新；
2. 400ms 防抖后，调 `driver.getContextDiagnostics()`（现成的接口，desktop 同款）
问 Host："最新快照是啥？"；
3. 拿到 `inputTokens`（已用）和 `contextWindow`（总量）→ 更新状态栏数字；
4. 防护措施照抄 desktop：
- **版本号防乱序**：两次查询先后发出、后发先至时，丢弃过期的结果；
- **失败保留旧值**：查询失败不清空，原来的数字继续站着；
- **会话切换丢弃**：查询回来时如果用户已经换了会话，结果作废。

turn 结束时原本的 `token_usage` 事件照常到达——它和快照说的是同一次请求，
数字天然一致，不打架。

## 为什么不选别的路

- **新加 push 事件**：要动核心事件协议、runtime 发射点、host 转发、持久化语义，
还会造成"同一个数两条来源"的漂移风险。pull 方案零协议改动，且 TUI 和
desktop 读的是同一行记录，永远不会不一致。
- **复用 token_usage 事件发半成品**：它会累加计费字段，发半个会把账算重复，
违反仓库"用量不完整就当没有"（#972）的原则。
- **请求发出前用字节数/4 估一个值**：误差太大（图片附件的 base64 会严重失真），
不值得。

## 改动范围

只动 `packages/cli`：

- 新增 `tui-context-refresh.ts`：事件过滤器 + 防抖器（约几十行，仿 desktop 的
`session-trace-refresh.ts`）；
- `pi-tui-runner.ts`：`onEvent` 里挂钩子、写结果、渲染；
- 测试：中途刷新、防抖合并、乱序丢弃、失败保留、切会话丢弃、turn 末事件回归。
Loading