From 0fa4e247b79d25e788bd2c5263cd5031aef5ebeb Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 29 Aug 2026 00:59:07 +0800 Subject: [PATCH 1/3] fix: context budget guard + no-window warning (#347) A model with no declared context window (limit.context=0) disabled every percentage threshold in the plugin, so requests could grow past the backend's real window and be rejected with HTTP 400 - which opencode swallows as an empty exit-0 response (billion-context#317). - enforceContextBudget: deterministic prune-to-fit guard in messages.transform (truncate largest old tool outputs, then clear oldest) using the model window or an absolute maxContextLimit, reserving completionReserveTokens (default 32768) for completion - one-time WARN when the model reports no context window and the catalog has no entry, with actionable config guidance - new config: compress.completionReserveTokens (schema + docs) - 14 new unit tests --- CONFIGURATION.md | 6 + CONFIGURATION.zh-CN.md | 6 + dcp.schema.json | 9 +- devlog/2026-08-28_context-budget-guard/REQ.md | 131 +++++++ .../WORKLOG.md | 118 ++++++ lib/config.ts | 7 + lib/hooks.ts | 19 + lib/messages/enforce-budget.ts | 237 ++++++++++++ lib/state/state.ts | 2 + lib/state/types.ts | 6 + tests/enforce-budget.test.ts | 348 ++++++++++++++++++ 11 files changed, 888 insertions(+), 1 deletion(-) create mode 100644 devlog/2026-08-28_context-budget-guard/REQ.md create mode 100644 devlog/2026-08-28_context-budget-guard/WORKLOG.md create mode 100644 lib/messages/enforce-budget.ts create mode 100644 tests/enforce-budget.test.ts diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 8a6c6d5e..1c035c83 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -303,6 +303,12 @@ Core compression behavior. - **Status:** ACTIVE - **Description:** Always protect the most recent user message from compression, regardless of `preserveRecentMessages` or `preserveRecentTokens`. +#### `compress.completionReserveTokens` +- **Type:** `number` +- **Default:** `32768` +- **Status:** ACTIVE +- **Description:** Tokens reserved for the model's completion by the context-budget guard. The guard estimates the request's input size and, if it exceeds `window - completionReserveTokens`, deterministically truncates (then clears) old compressible tool outputs until it fits — summaries, protected tools, the first user message, and the last 3 messages are never touched. The default `32768` covers opencode's `32000` `max_tokens` fallback for models with no declared `limit.output`. The guard is a no-op unless a context window is known: the model's declared limit, or an absolute (number) `compress.maxContextLimit`. + --- ### `gc` (Generation & Cleanup) diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index 3fa5ad9e..a0d8705c 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -303,6 +303,12 @@ ACP 从最多三层配置文件中读取(后加载的覆盖先加载的): - **状态:** ACTIVE - **说明:** 始终保护最近一条用户消息不被压缩,无论 `preserveRecentMessages` 或 `preserveRecentTokens` 如何设置。 +#### `compress.completionReserveTokens` +- **类型:** `number` +- **默认值:** `32768` +- **状态:** ACTIVE +- **说明:** 上下文预算守卫为模型补全预留的 token 数。守卫估算请求输入大小,若超过 `window - completionReserveTokens`,则确定性地截断(随后清除)旧的可压缩工具输出直至达标——摘要、受保护工具、首条用户消息和最近 3 条消息永不被改动。默认 `32768` 覆盖 opencode 对未声明 `limit.output` 模型的 `32000` `max_tokens` 回退值。只有当上下文窗口已知时守卫才生效:模型声明的 limit,或绝对值(数字)`compress.maxContextLimit`。 + --- ### `gc`(生成与清理) diff --git a/dcp.schema.json b/dcp.schema.json index 5b6fd1db..179029ab 100644 --- a/dcp.schema.json +++ b/dcp.schema.json @@ -298,6 +298,12 @@ "type": "boolean", "default": true, "description": "Always protect the most recent user message from compression." + }, + "completionReserveTokens": { + "type": "number", + "default": 32768, + "minimum": 0, + "description": "Tokens reserved for the model's completion when the context-budget guard prunes tool outputs so the request fits the window. Default 32768 covers opencode's 32000 max_tokens fallback for models with no declared limit.output. The guard is a no-op unless a context window is known (model limit or an absolute maxContextLimit)." } }, "default": { @@ -322,7 +328,8 @@ "lastSegmentSoftBlock": true, "preserveRecentMessages": 20, "preserveRecentTokens": 20000, - "preserveLastUserMessage": true + "preserveLastUserMessage": true, + "completionReserveTokens": 32768 } }, "gc": { diff --git a/devlog/2026-08-28_context-budget-guard/REQ.md b/devlog/2026-08-28_context-budget-guard/REQ.md new file mode 100644 index 00000000..4f2ccb73 --- /dev/null +++ b/devlog/2026-08-28_context-budget-guard/REQ.md @@ -0,0 +1,131 @@ +# REQ - Context budget guard + no-window warning + +- Task ID: `2026-08-28_context-budget-guard` +- Home Repo: `opencode-acp` +- Created: 2026-08-28 +- Status: InProgress +- Priority: P1 +- Owner: ework-daemon (agent) +- References: ranxianglei/billion-context#317, ranxianglei/opencode-acp#347 + +## 1. Background & Problem Statement + +- **Context**: Production incident (billion-context#317): long-lived headless sessions + (per-message resume via `opencode run --session `) fail fast on resume — model + process exits in ~5s with code 0 and no output, every retry reproduces, fresh session + works. Two cases: 230,529 and 37,178 `totalPruneTokens`, both `lastCompaction: 0`. +- **Current behavior (symptom)**: For a custom model with no declared context window + (opencode `/config/providers` reports `limit: {context: 0, output: 0}`), the plugin + never learns `modelContextLimit`. All percentage thresholds (min/max/emergency, GC) + silently disable, the only surviving protection (advisory 50K-growth nudge) is + model-cooperative, and opencode's `max_tokens` fallback (32,000 when `limit.output` + is unknown) is not accounted for. The request grows until the model backend rejects + it with HTTP 400 ("Requested token count exceeds the model's maximum context length + of 262144 tokens. You requested a total of 262527 tokens: 230527 tokens from the + input messages and 32000 tokens for the completion"). opencode swallows the 400: + `session.error` bus event, idle, exit 0, no output — the session is permanently + stuck (context only grows across retries). +- **Expected behavior**: + 1. When the model reports no context window and the catalog has no entry for it, + surface a loud, actionable warning (once per session) instead of failing blind. + 2. When the model reports a context window, never send a request whose estimated + input exceeds `window - completionReserve` — deterministically prune compressible + tool outputs (truncate to prefix+suffix, then clear oldest) until the request + fits. The guard ONLY enforces the model-reported window: an absolute + `compress.maxContextLimit` is a soft nudge threshold, not the backend's real + limit, and pruning to it would destroy context the backend would accept. +- **Impact**: Any deployment with a custom/self-hosted model lacking a `limit` entry + (sglang/vLLM local backends are the common case) with long sessions. Permanent + session loss (only a fresh session id recovers), no error surface for the user. + +## 2. Reproduction (if applicable) + +- **Environment**: + - Node: 22+ (plugin runtime; opencode 1.14.x host) + - OS/Arch: linux-x64 +- **Minimal reproduction steps**: + 1) Configure a custom model with no `limit` (e.g. sglang qwen via `vllm-qwen` + provider, `options.maxTokens` set but no `limit.context`). + 2) Run a long session where the model calls `compress` repeatedly so + `totalPruneTokens` grows (advisory nudges only, no hard gate). + 3) Resume with `opencode run --session "..."` once estimated input + + max_tokens (32,000 fallback) exceeds the backend's real window (262,144). + 4) Observe: 400 from backend, opencode exits 0 with no output, retries loop. +- **Relevant configuration**: + - `~/.config/opencode/opencode.json`: model without `limit`; `compaction.auto: false`. + - `acp.jsonc`: no absolute `compress.maxContextLimit` (percentages disabled). +- **Workaround (verified)**: add `"limit": {"context": 262144, "output": 16384}` to + the model definition in opencode.json (this is what enables the guard — + `modelContextLimit` becomes known); optionally an absolute + `compress.maxContextLimit` in acp.jsonc to make the advisory nudges proactive + (soft threshold only — the guard does not use it). + +## 3. Constraints & Non-Goals + +- **Constraints**: + - Backward compatibility: fully additive. Default behavior with a known model window + is unchanged (guard budget = window - reserve; existing GC truncation still runs + first). No persisted-state schema change (`noContextLimitWarned` is transient). + - Performance requirements: estimation reuses the existing Anthropic tokenizer path + (`getCurrentTokenUsage` + `countAllMessageTokens`); pruning loop only runs when + over budget. + - Resource limits: must never touch summaries (compress tool outputs), protected + tools, the first user message, or the last 3 messages (same protections as + `truncateLargeToolOutputs`). +- **Non-Goals** (explicitly out of scope): + - Fixing opencode's exit-0-on-400 (upstream issue; body drafted in #317). + - Learning the window from a 400 response (no plugin hook exposes response errors; + tracked as design note in #347). + - Changing nudge/GC trigger semantics. + +## 4. Acceptance Criteria (must be testable) + +- **Correctness**: + - [ ] `resolveContextWindow` returns `state.modelContextLimit` when set, and + undefined otherwise — deliberately NOT falling back to an absolute + `compress.maxContextLimit` (soft nudge threshold, not a backend window; + pruning to it regressed e2e-blocks-nudges by starving the nudge of its + compressible targets). + - [ ] `estimateWireTokens` = last-assistant reported usage + tokens of messages + after it; falls back to full content estimate + systemPromptTokens when no + assistant token data exists. + - [ ] `enforceContextBudget` is a no-op when the window is unknown or the estimate + is within budget. + - [ ] Over budget: largest old compressible tool outputs are truncated + (prefix+suffix, same marker as `truncateLargeToolOutputs`) until the estimate + fits; if truncation alone cannot fit, oldest outputs are cleared to the + standard placeholder. + - [ ] Protections hold: first user message, last 3 messages, `protectedTools`, + compress-tool outputs (summaries), already-cleared outputs. + - [ ] Idempotent: a second run after pruning does not modify messages further. + - [ ] Once-per-session WARN when the model reports no window and the catalog has no + entry, with actionable guidance (opencode.json `limit` or absolute + `compress.maxContextLimit`). +- **Performance / Stability**: + - [ ] No measurable overhead on the under-budget path beyond one + `getCurrentTokenUsage` scan (already computed elsewhere in the pipeline). +- **Regression**: + - [ ] New test file `tests/enforce-budget.test.ts` added and passing. + - [ ] Full suite green: `npm run build`, `npm run typecheck`, `npm test`. + +## 5. Proposed Approach (optional) + +- **Affected modules & entry files**: + - `lib/messages/enforce-budget.ts` (new) — window resolution, wire estimation, + deterministic prune-to-fit. + - `lib/hooks.ts` — call the guard right after `truncateLargeToolOutputs` in + `messages.transform`; warn-once after model-limit reconciliation. + - `lib/config.ts` — new optional `compress.completionReserveTokens` (default + 32768, covering opencode's 32,000 `max_tokens` fallback). + - `lib/state/types.ts` + `lib/state/state.ts` — transient + `noContextLimitWarned` flag (not persisted). + - `dcp.schema.json`, `CONFIGURATION.md`, `CONFIGURATION.zh-CN.md` — docs. + - `tests/enforce-budget.test.ts` (new). +- **Risks**: + - Over-pruning if the estimate overshoots (tokenizer vs backend tokenizer drift): + mitigated by the reserve margin (default 32768 >> drift) and by only pruning + compressible tool outputs, never user text or summaries. + - Estimate undercount when a backend counts `max_tokens` differently: reserve is + configurable via `completionReserveTokens`. +- **Rollback strategy**: Revert the single commit; all changes are additive and the + guard is a no-op without an absolute/known window. diff --git a/devlog/2026-08-28_context-budget-guard/WORKLOG.md b/devlog/2026-08-28_context-budget-guard/WORKLOG.md new file mode 100644 index 00000000..936ff2c9 --- /dev/null +++ b/devlog/2026-08-28_context-budget-guard/WORKLOG.md @@ -0,0 +1,118 @@ +# WORKLOG - Context Budget Guard + No-Window Warning + +- Task ID: `2026-08-28_context-budget-guard` +- Home Repo: `opencode-acp` +- Status: Done +- Updated: 2026-08-28 23:50 + +## 1. Summary + +- **What was done** (1–3 sentences): + Added a deterministic prune-to-fit guard (`enforceContextBudget`) to the + `messages.transform` pipeline and a once-per-session loud warning for models + that report no context window. Fixes ranxianglei/opencode-acp#347 + (billion-context#317): sessions whose input grew past the backend's real + window were rejected with HTTP 400, which opencode swallows as an empty + exit-0 response, permanently stumping the session with no error surface. +- **Why** (1–3 sentences): + When a custom model has no `limit` entry, `modelContextLimit` stays + undefined and every percentage threshold (min/max/emergency, GC) is + disabled; only advisory nudges remain, so input grows unbounded until the + backend 400s (230,527 input + 32,000 completion > 262,144 window in the + production case). The guard makes the known-window path deterministic; the + warning makes the unknown-window path loud and actionable. +- **Behavior / compatibility changes**: Yes — additive. With a known model + window, requests estimated above `window - completionReserveTokens` (default + 32,768) now have old compressible tool outputs truncated/cleared until they + fit (after the existing GC truncation). With an unknown window, a one-time + WARN with config guidance is logged. No persisted-state schema change + (`noContextLimitWarned` is transient). +- **Risk level**: Low — guard only prunes tool outputs already classified + compressible (same protections as `truncateLargeToolOutputs`), never user + text or summaries; all changes additive; full suite green. + +## 2. Change Log + +### Commits + +| Commit | Description | +|--------|-------------| +| `62976d4` | fix: context budget guard + no-window warning (#347) | + +### Key Files + +- `lib/messages/enforce-budget.ts` — new: `resolveContextWindow` (model + window only), `estimateWireTokens`, `enforceContextBudget` (phase 1 + truncate largest old tool outputs, phase 2 clear oldest to placeholder). +- `lib/hooks.ts` — guard call after `truncateLargeToolOutputs` in + `messages.transform`; warn-once after model-limit reconciliation. +- `lib/config.ts` — new optional `compress.completionReserveTokens` + (default applied at the consumer, `DEFAULT_COMPLETION_RESERVE_TOKENS`). +- `lib/state/types.ts`, `lib/state/state.ts` — transient + `noContextLimitWarned` flag (default/reset false, not persisted). +- `dcp.schema.json`, `CONFIGURATION.md`, `CONFIGURATION.zh-CN.md` — docs for + `completionReserveTokens`. +- `tests/enforce-budget.test.ts` — new: 14 tests covering window resolution, + estimation, no-op paths, phase 1/2 pruning, protections, idempotency, + over-budget warning. + +## 3. Design & Implementation Notes + +- **Entry point / key function**: + `enforceContextBudget(state, config, logger, messages)` in + `lib/messages/enforce-budget.ts`, called from `createChatMessageTransformHandler` + in `lib/hooks.ts` right after `truncateLargeToolOutputs`. +- **Key configuration items**: + - `compress.completionReserveTokens` (number, default 32768) — reserved for + the completion; covers opencode's 32,000 `max_tokens` fallback when + `limit.output` is 0/unknown. +- **Key logic explanation**: + - Window resolution uses ONLY `state.modelContextLimit`. An absolute + `compress.maxContextLimit` is deliberately NOT a fallback: it is a soft + nudge/compression threshold, not the backend's real limit. Pruning to a + guessed threshold destroys context the backend would accept and starves + the nudge of its compressible targets (regressed + e2e-blocks-nudges "compressible ranges injected into suffix message when + shouldNudge fires" during development: guard pruned the one large tool + output below the recommendation floor → `nothingToCompress` → no suffix). + Users with an unknown window get the loud one-time warning instead. + - Estimation: last assistant's reported usage (input + cacheRead + + cacheWrite + output + reasoning) + tokens of messages after it; fallback + (no assistant token data) = full content estimate + cached system prompt + tokens. + - Protections (same as `truncateLargeToolOutputs` plus): first user message, + last 3 messages, `protectedTools`, compress-tool outputs (summaries), + already-cleared outputs. Truncation reuses the same marker + (`[truncated for context space`) so the two mechanisms are idempotent + together. + - Same-turn caveat: after pruning, the estimate stays stale until the next + turn (assistant-reported usage); a re-run finds no candidates and logs a + "still over budget" warning — self-corrects on the next turn when the + model reports the shrunken input. + +## 4. Testing & Verification + +### Build & Test Commands + +```sh +# Build +cd opencode-acp && npm run build + +# Run full test suite +node --import tsx --test tests/*.test.ts + +# Run specific test file +node --import tsx --test tests/enforce-budget.test.ts + +# Type check +npx tsc --noEmit +``` + +### Results + +- `npm run typecheck` — clean. +- `npm test` — 1043/1043 pass (14 new in `tests/enforce-budget.test.ts`). +- `npm run build` — success (dist/index.js 419.61 KB). +- Regression check: `tests/e2e-blocks-nudges.test.ts` 10/10 (was failing + during development while the guard used the absolute `maxContextLimit` as a + window; fixed by restricting the guard to the model-reported window). diff --git a/lib/config.ts b/lib/config.ts index 2a34c660..e1cfa8b1 100644 --- a/lib/config.ts +++ b/lib/config.ts @@ -40,6 +40,12 @@ export interface CompressConfig { preserveRecentTokens?: number /** Always protect the most recent user message (default: true). */ preserveLastUserMessage?: boolean + /** + * Tokens reserved for the model's completion when enforcing the context + * budget guard (default: 32768 — covers opencode's 32000 max_tokens + * fallback for models with no declared limit.output). + */ + completionReserveTokens?: number } export interface Commands { @@ -392,6 +398,7 @@ export function mergeCompress( preserveRecentMessages: override.preserveRecentMessages ?? base.preserveRecentMessages, preserveRecentTokens: override.preserveRecentTokens ?? base.preserveRecentTokens, preserveLastUserMessage: override.preserveLastUserMessage ?? base.preserveLastUserMessage, + completionReserveTokens: override.completionReserveTokens ?? base.completionReserveTokens, } } diff --git a/lib/hooks.ts b/lib/hooks.ts index 6c96e227..794e6755 100644 --- a/lib/hooks.ts +++ b/lib/hooks.ts @@ -25,6 +25,7 @@ import { import { filterMessages, filterMessagesInPlace } from "./messages/shape" import { getLastUserMessage } from "./messages/query" import { truncateLargeToolOutputs } from "./messages/truncate-tools" +import { enforceContextBudget } from "./messages/enforce-budget" import { handleContextCommand, handleStatsCommand, @@ -224,6 +225,23 @@ export function createChatMessageTransformHandler( }) } await updatePerTurnState(state, logger, messages) + + if ( + state.modelContextLimit === undefined && + !state.noContextLimitWarned && + requestModel?.providerID && + requestModel?.modelID && + registry.resolveModelLimit(requestModel.providerID, requestModel.modelID) === undefined + ) { + state.noContextLimitWarned = true + logger.warn( + 'Model reports no context window and the catalog has no entry for it; all percentage thresholds (min/max/emergency, GC) and the context-budget guard are disabled. Set the model limit in opencode.json (e.g. "limit": {"context": 262144, "output": 16384}) or set an absolute compress.maxContextLimit in acp.jsonc.', + { + session: state.sessionId, + model: `${requestModel.providerID}/${requestModel.modelID}`, + }, + ) + } } syncCompressPermissionState(state, config, hostPermissions, output.messages) @@ -255,6 +273,7 @@ export function createChatMessageTransformHandler( const prePruneTokens = getCurrentTokenUsage(state, output.messages) prune(state, logger, config, output.messages) truncateLargeToolOutputs(state, config, logger, output.messages) + enforceContextBudget(state, config, logger, output.messages) hideConsumedCompressCalls(state, output.messages) assignMessageRefs(state, output.messages) const compressionPriorities = buildPriorityMap(config, state, output.messages) diff --git a/lib/messages/enforce-budget.ts b/lib/messages/enforce-budget.ts new file mode 100644 index 00000000..e6dd25f2 --- /dev/null +++ b/lib/messages/enforce-budget.ts @@ -0,0 +1,237 @@ +import { SessionState, WithParts } from "../state" +import type { PluginConfig } from "../config" +import { Logger } from "../logger" +import { + COMPACTED_TOOL_OUTPUT_PLACEHOLDER, + countAllMessageTokens, + countTokens, + extractCompletedToolOutput, + getCurrentTokenUsage, +} from "../token-utils" + +/** + * Default completion reserve in tokens. opencode falls back to max_tokens=32000 + * when the model's limit.output is unknown/0, so the reserve must cover that + * worst case (billion-context#317: 230527 input + 32000 completion > 262144 + * window → 400 → silent exit 0). + */ +export const DEFAULT_COMPLETION_RESERVE_TOKENS = 32768 + +const TRUNCATION_MARKER = "[truncated for context space" +const KEEP_PREFIX_CHARS = 2000 +const KEEP_SUFFIX_CHARS = 2000 +const PROTECT_RECENT_MESSAGES = 3 +const MIN_CLEAR_TOKENS = 200 + +export interface EnforceBudgetResult { + applied: boolean + window: number + reserve: number + budget: number + estimatedTokens: number + finalEstimate: number + truncatedCount: number + clearedCount: number +} + +/** + * Resolve the context window the budget guard enforces against. + * + * ONLY the model-reported window (state.modelContextLimit) is used. An + * absolute compress.maxContextLimit is deliberately NOT a fallback: it is a + * soft nudge/compression threshold, not the backend's real limit. Pruning to + * a guessed threshold destroys context the backend would have accepted (and + * starves the nudge of its compressible targets — see + * e2e-blocks-nudges "compressible ranges injected" regression). Users with + * an unknown window get the loud one-time warning from hooks.ts instead. + */ +export function resolveContextWindow(state: SessionState): number | undefined { + if (typeof state.modelContextLimit === "number" && state.modelContextLimit > 0) { + return state.modelContextLimit + } + return undefined +} + +/** + * Estimate the input token count of the request about to be sent. + * + * Primary: the last assistant message's reported usage (input + cacheRead + + * cacheWrite + output + reasoning — exactly what the model saw last turn plus + * what it produced, which is now history) + the tokens of every message after + * it (the new user message of this turn). + * + * Fallback (no assistant token data yet): full content estimate of all + * messages + the cached system prompt estimate. + */ +export function estimateWireTokens(state: SessionState, messages: WithParts[]): number { + const base = getCurrentTokenUsage(state, messages) + if (base > 0) { + let lastAssistant = -1 + for (let i = messages.length - 1; i >= 0; i--) { + if (messages[i].info.role === "assistant") { + lastAssistant = i + break + } + } + let additions = 0 + for (let i = lastAssistant + 1; i < messages.length; i++) { + additions += countAllMessageTokens(messages[i]) + } + return base + additions + } + + let total = 0 + for (const m of messages) total += countAllMessageTokens(m) + return total + (state.systemPromptTokens ?? 0) +} + +/** + * Deterministic prune-to-fit: shrink compressible tool outputs until the + * estimated request fits `window - reserve`. Runs after + * truncateLargeToolOutputs (which only fires on a known modelContextLimit and + * only above the GC threshold), so it is the last line of defense against the + * model backend rejecting the request (HTTP 400 → opencode exit 0, no output). + * + * Phase 1: truncate the largest old tool outputs (prefix + suffix kept, same + * marker as truncateLargeToolOutputs for idempotency). + * Phase 2: if still over budget, clear the oldest remaining outputs to the + * standard cleared placeholder. + * + * Never touches: the first user message, the last 3 messages, protectedTools, + * compress-tool outputs (summaries), or already-cleared outputs. + */ +export function enforceContextBudget( + state: SessionState, + config: PluginConfig, + logger: Logger, + messages: WithParts[], +): EnforceBudgetResult | undefined { + const window = resolveContextWindow(state) + if (window === undefined) return undefined + + const reserve = config.compress?.completionReserveTokens ?? DEFAULT_COMPLETION_RESERVE_TOKENS + const budget = window - reserve + if (budget <= 0) return undefined + + const estimatedTokens = estimateWireTokens(state, messages) + if (estimatedTokens <= budget) { + return { + applied: false, + window, + reserve, + budget, + estimatedTokens, + finalEstimate: estimatedTokens, + truncatedCount: 0, + clearedCount: 0, + } + } + + const protectedIndex = messages.length - PROTECT_RECENT_MESSAGES + const protectedTools = new Set(config.compress?.protectedTools ?? []) + + const candidates: Array<{ part: any; content: string; tokens: number; index: number }> = [] + for (let mi = 0; mi < protectedIndex; mi++) { + if (mi === 0 && messages[mi].info.role === "user") continue + const msg = messages[mi] + const parts = Array.isArray(msg.parts) ? msg.parts : [] + for (const part of parts) { + if (part?.type !== "tool") continue + if (part.state?.status !== "completed") continue + if (part.tool === "compress") continue + if (protectedTools.has(part.tool)) continue + + const content = extractCompletedToolOutput(part) + if (content === undefined) continue + if (content === COMPACTED_TOOL_OUTPUT_PLACEHOLDER) continue + + const tokens = countTokens(content) + if (tokens <= 0) continue + candidates.push({ part, content, tokens, index: mi }) + } + } + + let saved = 0 + let truncatedCount = 0 + let clearedCount = 0 + + const truncatable = candidates + .filter( + (c) => + !c.content.includes(TRUNCATION_MARKER) && + c.content.length > KEEP_PREFIX_CHARS + KEEP_SUFFIX_CHARS, + ) + .sort((a, b) => b.tokens - a.tokens) + + for (const c of truncatable) { + if (estimatedTokens - saved <= budget) break + const prefix = c.content.slice(0, KEEP_PREFIX_CHARS) + const suffix = c.content.slice(-KEEP_SUFFIX_CHARS) + const truncated = + prefix + + `\n\n...${TRUNCATION_MARKER} — original ~${c.tokens} tokens]...\n\n` + + suffix + c.part.state.output = truncated + saved += c.tokens - countTokens(truncated) + truncatedCount++ + } + + if (estimatedTokens - saved > budget) { + const clearable = candidates + .filter((c) => { + const out = extractCompletedToolOutput(c.part) + return ( + out !== undefined && + out !== COMPACTED_TOOL_OUTPUT_PLACEHOLDER && + countTokens(out) > MIN_CLEAR_TOKENS + ) + }) + .sort((a, b) => a.index - b.index) + + for (const c of clearable) { + if (estimatedTokens - saved <= budget) break + const current = extractCompletedToolOutput(c.part) + if (current === undefined || current === COMPACTED_TOOL_OUTPUT_PLACEHOLDER) continue + c.part.state.output = COMPACTED_TOOL_OUTPUT_PLACEHOLDER + saved += countTokens(current) - countTokens(COMPACTED_TOOL_OUTPUT_PLACEHOLDER) + clearedCount++ + } + } + + const finalEstimate = Math.max(0, estimatedTokens - saved) + if (truncatedCount > 0 || clearedCount > 0) { + logger.warn("Context budget guard: pruned tool outputs to fit the request", { + session: state.sessionId, + estimatedTokens: Math.round(estimatedTokens), + budget, + window, + reserve, + truncatedCount, + clearedCount, + estimatedSavedTokens: Math.round(saved), + finalEstimate: Math.round(finalEstimate), + }) + } + if (finalEstimate > budget) { + logger.warn( + "Context budget guard: still over budget after pruning all compressible tool outputs; the request may be rejected by the model", + { + session: state.sessionId, + finalEstimate: Math.round(finalEstimate), + budget, + window, + }, + ) + } + + return { + applied: truncatedCount > 0 || clearedCount > 0, + window, + reserve, + budget, + estimatedTokens, + finalEstimate, + truncatedCount, + clearedCount, + } +} diff --git a/lib/state/state.ts b/lib/state/state.ts index dc449536..ca459a61 100644 --- a/lib/state/state.ts +++ b/lib/state/state.ts @@ -207,6 +207,7 @@ export function createSessionState(): SessionState { modelID: undefined, systemPromptTokens: undefined, qualityGateRetryPending: false, + noContextLimitWarned: false, } } @@ -249,6 +250,7 @@ export function resetSessionState(state: SessionState): void { state.modelID = undefined state.systemPromptTokens = undefined state.qualityGateRetryPending = false + state.noContextLimitWarned = false } export async function ensureSessionInitialized( diff --git a/lib/state/types.ts b/lib/state/types.ts index b9736f21..5fd42b6f 100644 --- a/lib/state/types.ts +++ b/lib/state/types.ts @@ -160,4 +160,10 @@ export interface SessionState { * - Normal call (no acknowledgeRisk) → quality runs normally */ qualityGateRetryPending: boolean + /** + * Transient flag (NOT persisted): set to true after the "model reports no + * context window" warning has been emitted for this session, so the + * warning fires at most once per session per process. + */ + noContextLimitWarned: boolean } diff --git a/tests/enforce-budget.test.ts b/tests/enforce-budget.test.ts new file mode 100644 index 00000000..78fc15c3 --- /dev/null +++ b/tests/enforce-budget.test.ts @@ -0,0 +1,348 @@ +import assert from "node:assert/strict" +import test from "node:test" + +import { createSessionState } from "../lib/state" +import type { SessionState, WithParts } from "../lib/state/types" +import type { PluginConfig } from "../lib/config" +import type { Logger } from "../lib/logger" +import { + DEFAULT_COMPLETION_RESERVE_TOKENS, + enforceContextBudget, + estimateWireTokens, + resolveContextWindow, +} from "../lib/messages/enforce-budget" + +const noopLogger: Logger = { + debug: () => {}, + info: () => {}, + warn: () => {}, + error: () => {}, + child: () => noopLogger, +} as unknown as Logger + +const warnState: { warnings: string[] } = { warnings: [] } +const warnLogger: Logger = { + debug: () => {}, + info: () => {}, + warn: (msg: string) => { + warnState.warnings.push(msg) + }, + error: () => {}, + child: () => noopLogger, +} as unknown as Logger + +// ~45 chars / ~12 tokens of mixed prose so token counts track chars/4 +// regardless of tokenizer run-length behavior on repeated characters. +const FILLER = "The quick brown fox jumps over the lazy dog. ".repeat(20) + +function makeConfig(overrides: { + modelContextLimit?: number + maxContextLimit?: number | `${number}%` + reserve?: number + protectedTools?: string[] +} = {}): { config: PluginConfig; state: SessionState } { + const state = createSessionState() + state.sessionId = "session-budget" + if (overrides.modelContextLimit !== undefined) { + state.modelContextLimit = overrides.modelContextLimit + } + const config = { + enabled: true, + autoUpdate: false, + debug: false, + pruneNotification: "off" as const, + pruneNotificationType: "chat" as const, + commands: { enabled: true, protectedTools: [] as string[] }, + experimental: { allowSubAgents: false, customPrompts: false }, + protectedFilePatterns: [] as string[], + compress: { + permission: "allow" as const, + showCompression: false, + summaryBuffer: true, + maxContextLimit: 150000 as number | `${number}%`, + minContextLimit: 50000 as number | `${number}%`, + nudgeFrequency: 5, + iterationNudgeThreshold: 15, + nudgeForce: "soft" as const, + protectedTools: overrides.protectedTools ?? ([] as string[]), + protectTags: false, + protectUserMessages: false, + }, + strategies: { + deduplication: { enabled: true, protectedTools: [] as string[] }, + purgeErrors: { enabled: true, turns: 4, protectedTools: [] as string[] }, + }, + gc: { + algorithm: "truncate" as const, + promotionThreshold: 5, + maxBlockAge: 15, + maxOldGenSummaryLength: 3000, + majorGcThresholdPercent: "100%" as const, + batchCleanup: { + lowThreshold: "60%" as const, + highThreshold: "75%" as const, + forceThreshold: "90%" as const, + }, + }, + } as unknown as PluginConfig + if (overrides.maxContextLimit !== undefined) { + config.compress.maxContextLimit = overrides.maxContextLimit + } + if (overrides.reserve !== undefined) { + config.compress.completionReserveTokens = overrides.reserve + } + return { config, state } +} + +let idCounter = 0 +function nextId(prefix: string): string { + idCounter++ + return `${prefix}-${idCounter}` +} + +function makeUserText(id: string, text: string): WithParts { + return { + info: { + id, + role: "user", + sessionID: "session-budget", + time: { created: Date.now() }, + } as any, + parts: [{ type: "text", text }] as any, + } +} + +function makeAssistantText(id: string, text: string): WithParts { + return { + info: { + id, + role: "assistant", + sessionID: "session-budget", + time: { created: Date.now() }, + } as any, + parts: [{ type: "text", text }] as any, + } +} + +function makeAssistantWithTokens(id: string, input: number, output = 100): WithParts { + return { + info: { + id, + role: "assistant", + sessionID: "session-budget", + time: { created: Date.now() }, + tokens: { input, output, reasoning: 0, cache: { read: 0, write: 0 } }, + } as any, + parts: [{ type: "text", text: "ok" }] as any, + } +} + +function makeToolMessage(id: string, output: string, tool = "bash"): WithParts { + return { + info: { + id, + role: "user", + sessionID: "session-budget", + time: { created: Date.now() }, + } as any, + parts: [ + { + type: "tool", + tool, + state: { status: "completed", output, input: {}, time: {} }, + }, + ] as any, + } +} + +test("resolveContextWindow: uses modelContextLimit", () => { + const { state } = makeConfig({ modelContextLimit: 200000 }) + assert.equal(resolveContextWindow(state), 200000) +}) + +test("resolveContextWindow: no window without modelContextLimit (absolute maxContextLimit is a soft threshold, not a window)", () => { + const { state } = makeConfig({ maxContextLimit: 100000 }) + assert.equal(resolveContextWindow(state), undefined) +}) + +test("resolveContextWindow: no window with percentage maxContextLimit", () => { + const { state } = makeConfig({ maxContextLimit: "70%" }) + assert.equal(resolveContextWindow(state), undefined) +}) + +test("estimateWireTokens: base usage plus additions after last assistant", () => { + const { state } = makeConfig({ modelContextLimit: 200000 }) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + makeAssistantWithTokens(nextId("a"), 9000, 100), + makeUserText(nextId("u"), "follow-up question"), + ] + const est = estimateWireTokens(state, messages) + assert.ok(est >= 9100, `expected >= 9100, got ${est}`) + assert.ok(est < 9300, `expected < 9300, got ${est}`) +}) + +test("estimateWireTokens: fallback sums content plus system prompt", () => { + const { state } = makeConfig({ modelContextLimit: 200000 }) + state.systemPromptTokens = 500 + const messages: WithParts[] = [ + makeUserText(nextId("u"), FILLER.repeat(2)), + makeAssistantText(nextId("a"), FILLER.repeat(2)), + ] + const est = estimateWireTokens(state, messages) + assert.ok(est >= 500, `expected >= 500, got ${est}`) +}) + +test("enforceContextBudget: no-op when window unknown", () => { + const { config, state } = makeConfig({ maxContextLimit: "70%" }) + const messages: WithParts[] = [makeUserText(nextId("u"), FILLER.repeat(100))] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.equal(result, undefined) +}) + +test("enforceContextBudget: no-op when under budget", () => { + const { config, state } = makeConfig({ modelContextLimit: 200000, reserve: 32768 }) + const big = makeToolMessage(nextId("t"), FILLER.repeat(100)) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + big, + makeAssistantWithTokens(nextId("a"), 50000), + makeUserText(nextId("u"), "next"), + ] + const before = big.parts[0].state.output + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.applied, false) + assert.equal(big.parts[0].state.output, before) +}) + +test("enforceContextBudget: truncates largest old tool outputs until under budget", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const t1 = makeToolMessage(nextId("t"), FILLER.repeat(130)) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + t1, + makeAssistantWithTokens(nextId("a"), 5000), + makeUserText(nextId("u"), "mid"), + makeAssistantWithTokens(nextId("a2"), 99500), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.applied, true) + assert.ok(result!.truncatedCount >= 1, `expected truncation, got ${JSON.stringify(result)}`) + assert.ok(result!.finalEstimate <= result!.budget, "final estimate must fit budget") + assert.ok(String(t1.parts[0].state.output).includes("[truncated for context space")) +}) + +test("enforceContextBudget: never touches the last 3 messages or the first user message", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const firstUser = makeUserText(nextId("u"), "original task " + FILLER.repeat(60)) + const old = makeToolMessage(nextId("t"), FILLER.repeat(130)) + const recent = makeToolMessage(nextId("t"), FILLER.repeat(30)) + const messages: WithParts[] = [ + firstUser, + old, + makeAssistantWithTokens(nextId("a"), 5000), + makeUserText(nextId("u"), "mid"), + makeAssistantWithTokens(nextId("a2"), 99500), + makeUserText(nextId("u"), "next"), + recent, + ] + const firstUserBefore = firstUser.parts[0].text + const recentBefore = recent.parts[0].state.output + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(firstUser.parts[0].text, firstUserBefore) + assert.equal(recent.parts[0].state.output, recentBefore) + assert.ok(String(old.parts[0].state.output).includes("[truncated for context space")) +}) + +test("enforceContextBudget: skips protected tools and compress summaries", () => { + const { config, state } = makeConfig({ + modelContextLimit: 100000, + reserve: 1000, + protectedTools: ["bash"], + }) + const other = makeToolMessage(nextId("t"), FILLER.repeat(130), "grep") + const protectedTool = makeToolMessage(nextId("t"), FILLER.repeat(130), "bash") + const summary = makeToolMessage(nextId("t"), FILLER.repeat(130), "compress") + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + other, + protectedTool, + summary, + makeAssistantWithTokens(nextId("a"), 99500), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(protectedTool.parts[0].state.output, FILLER.repeat(130)) + assert.equal(summary.parts[0].state.output, FILLER.repeat(130)) + assert.ok(String(other.parts[0].state.output).includes("[truncated for context space")) +}) + +test("enforceContextBudget: clears oldest outputs when truncation alone cannot fit", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const outputs: WithParts[] = [] + for (let i = 0; i < 10; i++) { + outputs.push(makeToolMessage(nextId("t"), FILLER.repeat(20))) + } + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + ...outputs, + makeAssistantWithTokens(nextId("a"), 130000), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.applied, true) + assert.ok(result!.clearedCount >= 1, `expected clearing, got ${JSON.stringify(result)}`) + assert.ok(result!.finalEstimate <= result!.budget, "final estimate must fit budget") + assert.equal(outputs[0].parts[0].state.output, "[Old tool result content cleared]") +}) + +test("enforceContextBudget: idempotent on second run", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const t1 = makeToolMessage(nextId("t"), FILLER.repeat(130)) + const bigAssistant = makeAssistantWithTokens(nextId("a2"), 99500) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + t1, + makeAssistantWithTokens(nextId("a"), 5000), + makeUserText(nextId("u"), "mid"), + bigAssistant, + makeUserText(nextId("u"), "next"), + ] + const first = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(first!.applied) + const afterFirst = String(t1.parts[0].state.output) + ;(bigAssistant.info as any).tokens.input = 70000 + const second = enforceContextBudget(state, config, noopLogger, messages) + assert.equal(afterFirst, String(t1.parts[0].state.output)) + assert.ok(second) + assert.equal(second!.applied, false) + assert.equal(second!.truncatedCount, 0) +}) + +test("enforceContextBudget: warns when still over budget after all pruning", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const hugeUser = makeUserText(nextId("u"), FILLER.repeat(400)) + const messages: WithParts[] = [ + hugeUser, + makeAssistantWithTokens(nextId("a"), 99500), + makeUserText(nextId("u"), "next"), + ] + warnState.warnings.length = 0 + const result = enforceContextBudget(state, config, warnLogger, messages) + assert.ok(result) + assert.equal(result!.applied, false) + assert.ok( + warnState.warnings.some((w) => w.includes("still over budget")), + `expected over-budget warning, got ${JSON.stringify(warnState.warnings)}`, + ) +}) + +test("enforceContextBudget: default reserve covers opencode's 32000 max_tokens fallback", () => { + assert.ok(DEFAULT_COMPLETION_RESERVE_TOKENS >= 32000) +}) From 318c94de5eb9b4570614ca8ee66137571d4aa544 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 29 Aug 2026 03:09:33 +0800 Subject: [PATCH 2/3] =?UTF-8?q?fix:=20review=20fixes=20=E2=80=94=20reserve?= =?UTF-8?q?=20validation,=20estimate=20alignment,=20shrink=20guard,=20docs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Dual-agent review of the context budget guard found: - BLOCKER: compress.completionReserveTokens missing from VALID_CONFIG_KEYS → spurious "Unknown keys" toast for users setting the documented key - NaN reserve hazard: non-numeric config value made budget NaN, disabling every early-exit (guard would truncate all candidates) - estimateWireTokens undercount: anchored on last assistant by role while getCurrentTokenUsage anchors on last assistant WITH token data; also dropped the system prompt estimate when getCurrentTokenUsage fell back to content estimation - truncation could GROW content just over the 4000-char threshold (prefix/suffix overlap + marker line) - docs (schema, EN/CN, warning text) claimed an absolute compress.maxContextLimit enables the guard; it deliberately does not Tests: +9 (23 in enforce-budget.test.ts, 9 new in config-validation.test.ts); full suite 1052/1052. --- CONFIGURATION.md | 2 +- CONFIGURATION.zh-CN.md | 2 +- dcp.schema.json | 2 +- lib/config-validation.ts | 23 ++++++ lib/hooks.ts | 2 +- lib/messages/enforce-budget.ts | 51 +++++++++++--- tests/config-validation.test.ts | 33 +++++++++ tests/enforce-budget.test.ts | 119 ++++++++++++++++++++++++++++++-- 8 files changed, 213 insertions(+), 21 deletions(-) diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 1c035c83..ba4bac33 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -307,7 +307,7 @@ Core compression behavior. - **Type:** `number` - **Default:** `32768` - **Status:** ACTIVE -- **Description:** Tokens reserved for the model's completion by the context-budget guard. The guard estimates the request's input size and, if it exceeds `window - completionReserveTokens`, deterministically truncates (then clears) old compressible tool outputs until it fits — summaries, protected tools, the first user message, and the last 3 messages are never touched. The default `32768` covers opencode's `32000` `max_tokens` fallback for models with no declared `limit.output`. The guard is a no-op unless a context window is known: the model's declared limit, or an absolute (number) `compress.maxContextLimit`. +- **Description:** Tokens reserved for the model's completion by the context-budget guard. The guard estimates the request's input size and, if it exceeds `window - completionReserveTokens`, deterministically truncates (then clears) old compressible tool outputs until it fits — summaries, protected tools, the first user message, and the last 3 messages are never touched. The default `32768` covers opencode's `32000` `max_tokens` fallback for models with no declared `limit.output`. The guard is a no-op unless the model's context window is known (declared `limit.context` in opencode.json, or a catalog entry). An absolute (number) `compress.maxContextLimit` does **not** enable the guard — it is a soft nudge threshold, not the backend's real limit. --- diff --git a/CONFIGURATION.zh-CN.md b/CONFIGURATION.zh-CN.md index a0d8705c..b8596a65 100644 --- a/CONFIGURATION.zh-CN.md +++ b/CONFIGURATION.zh-CN.md @@ -307,7 +307,7 @@ ACP 从最多三层配置文件中读取(后加载的覆盖先加载的): - **类型:** `number` - **默认值:** `32768` - **状态:** ACTIVE -- **说明:** 上下文预算守卫为模型补全预留的 token 数。守卫估算请求输入大小,若超过 `window - completionReserveTokens`,则确定性地截断(随后清除)旧的可压缩工具输出直至达标——摘要、受保护工具、首条用户消息和最近 3 条消息永不被改动。默认 `32768` 覆盖 opencode 对未声明 `limit.output` 模型的 `32000` `max_tokens` 回退值。只有当上下文窗口已知时守卫才生效:模型声明的 limit,或绝对值(数字)`compress.maxContextLimit`。 +- **说明:** 上下文预算守卫为模型补全预留的 token 数。守卫估算请求输入大小,若超过 `window - completionReserveTokens`,则确定性地截断(随后清除)旧的可压缩工具输出直至达标——摘要、受保护工具、首条用户消息和最近 3 条消息永不被改动。默认 `32768` 覆盖 opencode 对未声明 `limit.output` 模型的 `32000` `max_tokens` 回退值。只有当模型的上下文窗口已知时守卫才生效(opencode.json 中声明的 `limit.context`,或目录条目)。绝对值(数字)`compress.maxContextLimit` **不会**启用守卫——它是软性的提示阈值,而非后端的真实限制。 --- diff --git a/dcp.schema.json b/dcp.schema.json index 179029ab..6398847b 100644 --- a/dcp.schema.json +++ b/dcp.schema.json @@ -303,7 +303,7 @@ "type": "number", "default": 32768, "minimum": 0, - "description": "Tokens reserved for the model's completion when the context-budget guard prunes tool outputs so the request fits the window. Default 32768 covers opencode's 32000 max_tokens fallback for models with no declared limit.output. The guard is a no-op unless a context window is known (model limit or an absolute maxContextLimit)." + "description": "Tokens reserved for the model's completion when the context-budget guard prunes tool outputs so the request fits the window. Default 32768 covers opencode's 32000 max_tokens fallback for models with no declared limit.output. The guard is a no-op unless the model's context window is known (declared limit or catalog entry); an absolute maxContextLimit is a soft nudge threshold and does not enable the guard." } }, "default": { diff --git a/lib/config-validation.ts b/lib/config-validation.ts index 27ba9016..99b80e6a 100644 --- a/lib/config-validation.ts +++ b/lib/config-validation.ts @@ -48,6 +48,7 @@ export const VALID_CONFIG_KEYS = new Set([ "compress.preserveRecentMessages", "compress.preserveRecentTokens", "compress.preserveLastUserMessage", + "compress.completionReserveTokens", "gc", "gc.algorithm", "gc.promotionThreshold", @@ -537,6 +538,28 @@ export function validateConfigTypes(config: Record): ValidationErro }) } + if ( + compress.completionReserveTokens !== undefined && + typeof compress.completionReserveTokens !== "number" + ) { + errors.push({ + key: "compress.completionReserveTokens", + expected: "number", + actual: typeof compress.completionReserveTokens, + }) + } + + if ( + typeof compress.completionReserveTokens === "number" && + compress.completionReserveTokens < 0 + ) { + errors.push({ + key: "compress.completionReserveTokens", + expected: "non-negative number (>= 0)", + actual: `${compress.completionReserveTokens}`, + }) + } + if ( typeof compress.iterationNudgeThreshold === "number" && compress.iterationNudgeThreshold < 1 diff --git a/lib/hooks.ts b/lib/hooks.ts index 794e6755..222e8403 100644 --- a/lib/hooks.ts +++ b/lib/hooks.ts @@ -235,7 +235,7 @@ export function createChatMessageTransformHandler( ) { state.noContextLimitWarned = true logger.warn( - 'Model reports no context window and the catalog has no entry for it; all percentage thresholds (min/max/emergency, GC) and the context-budget guard are disabled. Set the model limit in opencode.json (e.g. "limit": {"context": 262144, "output": 16384}) or set an absolute compress.maxContextLimit in acp.jsonc.', + 'Model reports no context window and the catalog has no entry for it; all percentage thresholds (min/max/emergency, GC) and the context-budget guard are disabled. Set the model limit in opencode.json (e.g. "limit": {"context": 262144, "output": 16384}) to enable them (also fixes the 32000 max_tokens fallback); an absolute compress.maxContextLimit in acp.jsonc only enables proactive nudges, not the guard.', { session: state.sessionId, model: `${requestModel.providerID}/${requestModel.modelID}`, diff --git a/lib/messages/enforce-budget.ts b/lib/messages/enforce-budget.ts index e6dd25f2..bba6d21c 100644 --- a/lib/messages/enforce-budget.ts +++ b/lib/messages/enforce-budget.ts @@ -1,4 +1,5 @@ import { SessionState, WithParts } from "../state" +import type { AssistantMessage } from "@opencode-ai/sdk/v2" import type { PluginConfig } from "../config" import { Logger } from "../logger" import { @@ -66,18 +67,30 @@ export function resolveContextWindow(state: SessionState): number | undefined { export function estimateWireTokens(state: SessionState, messages: WithParts[]): number { const base = getCurrentTokenUsage(state, messages) if (base > 0) { - let lastAssistant = -1 + // Align with getCurrentTokenUsage: base is the usage of the LAST + // assistant WITH token data (it skips tokenless aborted requests). + // Count additions after that same assistant — not after the last + // assistant by role. If the role-last assistant has no token data, + // the gap between the two (user message + aborted output) would + // otherwise be undercounted. + let baseAssistant = -1 for (let i = messages.length - 1; i >= 0; i--) { - if (messages[i].info.role === "assistant") { - lastAssistant = i - break - } + if (messages[i].info.role !== "assistant") continue + const tokens = (messages[i].info as AssistantMessage).tokens + if ((tokens?.input || 0) <= 0 && (tokens?.output || 0) <= 0) continue + baseAssistant = i + break } - let additions = 0 - for (let i = lastAssistant + 1; i < messages.length; i++) { - additions += countAllMessageTokens(messages[i]) + if (baseAssistant >= 0) { + let additions = 0 + for (let i = baseAssistant + 1; i < messages.length; i++) { + additions += countAllMessageTokens(messages[i]) + } + return base + additions } - return base + additions + // base > 0 without a token-data assistant means getCurrentTokenUsage + // itself fell back to content estimation — redo it here with the + // system prompt estimate included. } let total = 0 @@ -99,6 +112,10 @@ export function estimateWireTokens(state: SessionState, messages: WithParts[]): * * Never touches: the first user message, the last 3 messages, protectedTools, * compress-tool outputs (summaries), or already-cleared outputs. + * + * Note: the estimate is computed before message-ID tags and nudge text are + * injected later in the pipeline; those additions (a few hundred tokens) are + * absorbed by the completion reserve. */ export function enforceContextBudget( state: SessionState, @@ -109,7 +126,17 @@ export function enforceContextBudget( const window = resolveContextWindow(state) if (window === undefined) return undefined - const reserve = config.compress?.completionReserveTokens ?? DEFAULT_COMPLETION_RESERVE_TOKENS + // Config validation warns on bad values but does not block loading, so + // defend here: a non-numeric reserve would make `budget` NaN and every + // `<= budget` break condition false → the guard would truncate ALL + // candidates with no early exit. + const configuredReserve = config.compress?.completionReserveTokens + const reserve = + typeof configuredReserve === "number" && + Number.isFinite(configuredReserve) && + configuredReserve >= 0 + ? configuredReserve + : DEFAULT_COMPLETION_RESERVE_TOKENS const budget = window - reserve if (budget <= 0) return undefined @@ -171,6 +198,10 @@ export function enforceContextBudget( prefix + `\n\n...${TRUNCATION_MARKER} — original ~${c.tokens} tokens]...\n\n` + suffix + // Content just over the 4000-char threshold: prefix and suffix + // overlap and the marker line makes the "truncated" form LONGER. + // Skip it (phase 2 may still clear it) instead of growing it. + if (truncated.length >= c.content.length) continue c.part.state.output = truncated saved += c.tokens - countTokens(truncated) truncatedCount++ diff --git a/tests/config-validation.test.ts b/tests/config-validation.test.ts index 57275e70..1dd74f67 100644 --- a/tests/config-validation.test.ts +++ b/tests/config-validation.test.ts @@ -232,6 +232,39 @@ test("validateConfigTypes catches wrong type for compress.preserveLastUserMessag assert.equal(result[0].expected, "boolean") }) +test("getInvalidConfigKeys accepts compress.completionReserveTokens", () => { + const result = getInvalidConfigKeys({ + compress: { completionReserveTokens: 32768 }, + }) + assert.deepEqual(result, []) +}) + +test("validateConfigTypes accepts numeric compress.completionReserveTokens", () => { + const result = validateConfigTypes({ + compress: { completionReserveTokens: 32768 }, + }) + assert.deepEqual(result, []) +}) + +test("validateConfigTypes catches wrong type for compress.completionReserveTokens", () => { + const result = validateConfigTypes({ + compress: { completionReserveTokens: "32768" }, + }) + assert.equal(result.length, 1) + assert.equal(result[0].key, "compress.completionReserveTokens") + assert.equal(result[0].expected, "number") + assert.equal(result[0].actual, "string") +}) + +test("validateConfigTypes rejects negative compress.completionReserveTokens", () => { + const result = validateConfigTypes({ + compress: { completionReserveTokens: -1 }, + }) + assert.equal(result.length, 1) + assert.equal(result[0].key, "compress.completionReserveTokens") + assert.equal(result[0].expected, "non-negative number (>= 0)") +}) + test("getInvalidConfigKeys accepts new preserveRecent* keys", () => { const result = getInvalidConfigKeys({ compress: { diff --git a/tests/enforce-budget.test.ts b/tests/enforce-budget.test.ts index 78fc15c3..89df10f2 100644 --- a/tests/enforce-budget.test.ts +++ b/tests/enforce-budget.test.ts @@ -11,6 +11,7 @@ import { estimateWireTokens, resolveContextWindow, } from "../lib/messages/enforce-budget" +import { countTokens } from "../lib/token-utils" const noopLogger: Logger = { debug: () => {}, @@ -31,8 +32,8 @@ const warnLogger: Logger = { child: () => noopLogger, } as unknown as Logger -// ~45 chars / ~12 tokens of mixed prose so token counts track chars/4 -// regardless of tokenizer run-length behavior on repeated characters. +// ~45 chars / ~10 tokens per sentence of mixed prose so token counts track +// chars/4 regardless of tokenizer run-length behavior on repeated characters. const FILLER = "The quick brown fox jumps over the lazy dog. ".repeat(20) function makeConfig(overrides: { @@ -185,12 +186,18 @@ test("estimateWireTokens: base usage plus additions after last assistant", () => test("estimateWireTokens: fallback sums content plus system prompt", () => { const { state } = makeConfig({ modelContextLimit: 200000 }) state.systemPromptTokens = 500 + const content = FILLER.repeat(2) const messages: WithParts[] = [ - makeUserText(nextId("u"), FILLER.repeat(2)), - makeAssistantText(nextId("a"), FILLER.repeat(2)), + makeUserText(nextId("u"), content), + makeAssistantText(nextId("a"), content), ] + // Single text-part messages count exactly countTokens(text), so the + // fallback must equal 2x content + system prompt (tiny tolerance for + // join overhead). + const expected = 500 + countTokens(content) * 2 const est = estimateWireTokens(state, messages) - assert.ok(est >= 500, `expected >= 500, got ${est}`) + assert.ok(est >= expected, `expected >= ${expected}, got ${est}`) + assert.ok(est <= expected + 5, `expected <= ${expected + 5}, got ${est}`) }) test("enforceContextBudget: no-op when window unknown", () => { @@ -284,19 +291,25 @@ test("enforceContextBudget: skips protected tools and compress summaries", () => test("enforceContextBudget: clears oldest outputs when truncation alone cannot fit", () => { const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + // 4050 chars: just over the 4000-char truncation threshold, so the + // truncated form (2000 + marker + 2000 ≈ 4062 chars) would be LONGER — + // phase 1 must skip every candidate (pure char-count fact, no BPE + // margin), forcing phase 2 (clearing) to do all the work. + const content = FILLER.repeat(4) + FILLER.slice(0, 450) const outputs: WithParts[] = [] for (let i = 0; i < 10; i++) { - outputs.push(makeToolMessage(nextId("t"), FILLER.repeat(20))) + outputs.push(makeToolMessage(nextId("t"), content)) } const messages: WithParts[] = [ makeUserText(nextId("u"), "hello"), ...outputs, - makeAssistantWithTokens(nextId("a"), 130000), + makeAssistantWithTokens(nextId("a"), 103000), makeUserText(nextId("u"), "next"), ] const result = enforceContextBudget(state, config, noopLogger, messages) assert.ok(result) assert.equal(result!.applied, true) + assert.equal(result!.truncatedCount, 0, `truncation must skip non-shrinking content: ${JSON.stringify(result)}`) assert.ok(result!.clearedCount >= 1, `expected clearing, got ${JSON.stringify(result)}`) assert.ok(result!.finalEstimate <= result!.budget, "final estimate must fit budget") assert.equal(outputs[0].parts[0].state.output, "[Old tool result content cleared]") @@ -346,3 +359,95 @@ test("enforceContextBudget: warns when still over budget after all pruning", () test("enforceContextBudget: default reserve covers opencode's 32000 max_tokens fallback", () => { assert.ok(DEFAULT_COMPLETION_RESERVE_TOKENS >= 32000) }) + +test("enforceContextBudget: no-op when window minus reserve is not positive", () => { + const { config, state } = makeConfig({ modelContextLimit: 10000, reserve: 32768 }) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + makeToolMessage(nextId("t"), FILLER.repeat(130)), + makeAssistantWithTokens(nextId("a"), 99500), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.equal(result, undefined) +}) + +test("enforceContextBudget: no crash with fewer than 3 messages", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + const t1 = makeToolMessage(nextId("t"), FILLER.repeat(130)) + const messages: WithParts[] = [t1, makeAssistantWithTokens(nextId("a"), 99500)] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.applied, false) + assert.equal(result!.truncatedCount, 0) + assert.equal(result!.clearedCount, 0) + assert.equal(t1.parts[0].state.output, FILLER.repeat(130)) +}) + +test("enforceContextBudget: non-numeric reserve falls back to default", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000 }) + ;(config.compress as any).completionReserveTokens = "oops" + const outputs: WithParts[] = [] + for (let i = 0; i < 3; i++) { + outputs.push(makeToolMessage(nextId("t"), FILLER.repeat(130))) + } + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + ...outputs, + makeAssistantWithTokens(nextId("a"), 69000), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.reserve, DEFAULT_COMPLETION_RESERVE_TOKENS) + assert.equal(result!.budget, 100000 - DEFAULT_COMPLETION_RESERVE_TOKENS) + assert.equal(result!.applied, true) + // With a NaN budget (old code) every `<= budget` break was false and the + // guard never fit; with the fallback reserve it must stop once it fits. + assert.ok( + result!.finalEstimate <= result!.budget, + `expected final <= budget, got ${result!.finalEstimate} vs ${result!.budget}`, + ) +}) + +test("estimateWireTokens: counts the gap when the last assistant has no token data", () => { + const { state } = makeConfig({ modelContextLimit: 200000 }) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + makeAssistantWithTokens(nextId("a"), 9000, 100), + makeUserText(nextId("u"), FILLER.repeat(2)), + makeAssistantText(nextId("a2"), "ok"), + makeUserText(nextId("u"), "next"), + ] + // base 9100 + gap user message (~400) + "ok" + "next". A role-based scan + // would anchor at the tokenless "a2" and miss the gap message entirely. + const est = estimateWireTokens(state, messages) + assert.ok(est >= 9300, `expected >= 9300, got ${est}`) + assert.ok(est < 9800, `expected < 9800, got ${est}`) +}) + +test("enforceContextBudget: skips truncation that would grow the output, clears instead", () => { + const { config, state } = makeConfig({ modelContextLimit: 100000, reserve: 1000 }) + // 4050 chars: just over the 4000-char threshold, so prefix + marker + + // suffix (~4066 chars) would be LONGER than the original. + const content = FILLER.repeat(4) + FILLER.slice(0, 450) + const small = makeToolMessage(nextId("t"), content) + const messages: WithParts[] = [ + makeUserText(nextId("u"), "hello"), + small, + makeAssistantWithTokens(nextId("a"), 5000), + makeUserText(nextId("u"), "mid"), + makeAssistantWithTokens(nextId("a2"), 99500), + makeUserText(nextId("u"), "next"), + ] + const result = enforceContextBudget(state, config, noopLogger, messages) + assert.ok(result) + assert.equal(result!.applied, true) + assert.equal(result!.truncatedCount, 0, `truncation must skip non-shrinking content: ${JSON.stringify(result)}`) + assert.equal(result!.clearedCount, 1) + assert.ok( + result!.finalEstimate <= result!.budget, + `expected final <= budget, got ${result!.finalEstimate} vs ${result!.budget}`, + ) + assert.equal(small.parts[0].state.output, "[Old tool result content cleared]") +}) From 18debdbbdb52e9b2d7db8caa3deee06ebc485205 Mon Sep 17 00:00:00 2001 From: ework-agent Date: Sat, 29 Aug 2026 03:10:01 +0800 Subject: [PATCH 3/3] =?UTF-8?q?docs:=20WORKLOG=20=E2=80=94=20dual-agent=20?= =?UTF-8?q?review=20findings=20and=20fixes?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit --- .../WORKLOG.md | 72 +++++++++++++++++-- 1 file changed, 65 insertions(+), 7 deletions(-) diff --git a/devlog/2026-08-28_context-budget-guard/WORKLOG.md b/devlog/2026-08-28_context-budget-guard/WORKLOG.md index 936ff2c9..1826f83f 100644 --- a/devlog/2026-08-28_context-budget-guard/WORKLOG.md +++ b/devlog/2026-08-28_context-budget-guard/WORKLOG.md @@ -3,7 +3,7 @@ - Task ID: `2026-08-28_context-budget-guard` - Home Repo: `opencode-acp` - Status: Done -- Updated: 2026-08-28 23:50 +- Updated: 2026-08-29 02:45 ## 1. Summary @@ -37,7 +37,8 @@ | Commit | Description | |--------|-------------| -| `62976d4` | fix: context budget guard + no-window warning (#347) | +| `0fa4e24` | fix: context budget guard + no-window warning (#347) | +| `318c94d` | fix: review fixes — reserve validation, estimate alignment, shrink guard, docs | ### Key Files @@ -52,9 +53,18 @@ `noContextLimitWarned` flag (default/reset false, not persisted). - `dcp.schema.json`, `CONFIGURATION.md`, `CONFIGURATION.zh-CN.md` — docs for `completionReserveTokens`. -- `tests/enforce-budget.test.ts` — new: 14 tests covering window resolution, - estimation, no-op paths, phase 1/2 pruning, protections, idempotency, - over-budget warning. +- `lib/config-validation.ts` — `compress.completionReserveTokens` added to + `VALID_CONFIG_KEYS` + type/value checks in `validateConfigTypes` (review + blocker: the key was documented but not whitelisted → spurious "Unknown + keys" toast). +- `tests/enforce-budget.test.ts` — 23 tests covering window resolution, + estimation (incl. tokenless-aborted-request gap), no-op paths (budget<=0, + <3 messages), non-numeric reserve fallback, phase 1/2 pruning, + non-shrinking truncation skip, protections, idempotency, over-budget + warning. +- `tests/config-validation.test.ts` — +4 tests for + `compress.completionReserveTokens` (key accepted, numeric accepted, wrong + type, negative). ## 3. Design & Implementation Notes @@ -111,8 +121,56 @@ npx tsc --noEmit ### Results - `npm run typecheck` — clean. -- `npm test` — 1043/1043 pass (14 new in `tests/enforce-budget.test.ts`). -- `npm run build` — success (dist/index.js 419.61 KB). +- `npm test` — 1052/1052 pass (23 in `tests/enforce-budget.test.ts`, +4 in + `tests/config-validation.test.ts`). +- `npm run build` — success (dist/index.js 420.69 KB). - Regression check: `tests/e2e-blocks-nudges.test.ts` 10/10 (was failing during development while the guard used the absolute `maxContextLimit` as a window; fixed by restricting the guard to the model-reported window). +- CI pre-flight: `./scripts/ci/check-pr.sh 2026-08-28_context-budget-guard + origin/master` — all checks pass. + +### Dual-Agent Review (2026-08-29) + +Two independent agent reviews (source + tests) of the PR branch. + +**Source review — REQUEST-CHANGES, 1 blocker + 7 minors.** Fixed: +- **BLOCKER**: `compress.completionReserveTokens` missing from + `VALID_CONFIG_KEYS` → spurious "Unknown keys" toast for anyone setting the + documented key. Added to the whitelist + `validateConfigTypes` + (number, >= 0). +- **NaN reserve hazard**: config validation is advisory-only (warns, doesn't + block load), so a non-numeric `completionReserveTokens` made `budget` NaN + and every `<= budget` early-exit false → the guard truncated ALL + candidates. Now falls back to `DEFAULT_COMPLETION_RESERVE_TOKENS`. +- **estimateWireTokens undercount**: anchored additions on the last assistant + *by role* while `getCurrentTokenUsage` anchors on the last assistant *with + token data* (it skips tokenless aborted requests) → the gap between the two + was undercounted. Also, when `getCurrentTokenUsage` itself fell back to + content estimation, the system prompt estimate was dropped. Both fixed. +- **Truncation could grow content**: for output just over the 4000-char + threshold, prefix + suffix overlap and the marker line made the "truncated" + form LONGER. Now skipped (phase 2 may still clear it). +- **Docs contradicted code**: schema + EN/CN docs + the warn-once text claimed + an absolute `compress.maxContextLimit` enables the guard; it deliberately + does not (soft nudge threshold, not the backend's real limit). Corrected. + +Left as known limitations (documented, low impact): +- The one-time no-window warning can fire one turn early for a catalog-unknown + model that declares its limit (messages.transform runs before + system.transform sets the limit). Harmless — once per session. +- First-user-message protection is index-0-only (matches + `truncateLargeToolOutputs`; post-compaction nuance). +- The "still over budget" WARN repeats each transform while the condition + persists (deliberate — a genuinely alarming state). + +**Test review — APPROVE-WITH-NITS.** No blockers. Nits addressed: +- Fragile premise in the clear-phase test (phase-1 shortfall had a ~10% BPE + margin) → rebuilt so phase 1 provably saves 0 (char-count fact) and phase 2 + does all the work; asserts `truncatedCount === 0`. +- Weak fallback-estimate assertion (`est >= 500`) → tightened to near-exact + equality (`2*countTokens(content) + systemPrompt`, ±5 tolerance). +- Coverage gaps added: budget<=0 no-op, <3 messages, non-numeric reserve + fallback, tokenless-aborted-request gap estimate, non-shrinking truncation + skip, +4 config-validation tests. +- Cosmetic: FILLER comment token estimate corrected (~10, not ~12).