Skip to content

fix: context-limit safety net for spawn+resume mode (#346) #349

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ranxianglei wants to merge 2 commits into
base: master
Choose a base branch
from
+1,377 −50
Open

fix: context-limit safety net for spawn+resume mode (#346) #349

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
efdaeca
fix: context-limit safety net for spawn+resume mode (#346)
Aug 28, 2026
e5f01d6
fix: address review findings for #346
Aug 28, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions CONFIGURATION.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,12 @@ Core compression behavior.
- **Status:** DEPRECATED
- **Description:** **Deprecated — scheduled for removal alongside `minContextLimit`.** Per-model override for `minContextLimit`. Still honored until removed.

#### `compress.contextLimitFallback`
- **Type:** `number`
- **Default:** `128000`
- **Status:** ACTIVE
- **Description:** Fallback context window (absolute tokens) used when the model's limit is unknown — e.g. custom providers with no declared limit, headless spawn+resume sessions where the limit was never learned, or the brief window after a model switch invalidates a stale limit. Drives all percentage thresholds (`maxContextLimit`/`minContextLimit`), the emergency nudge override, batch cleanup GC, and in-flight tool-output truncation. The real model limit always takes precedence when known, as do per-model overrides (`modelMaxLimits`/`modelMinLimits`). Set to `0` to disable the fallback (legacy behavior: no safety net until the limit is learned).

#### `compress.nudgeFrequency`
- **Type:** `number`
- **Default:** `5`
Expand Down
6 changes: 6 additions & 0 deletions CONFIGURATION.zh-CN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -190,6 +190,12 @@ ACP 从最多三层配置文件中读取（后加载的覆盖先加载的）：
- **状态：** DEPRECATED
- **说明：** **已废弃——将与 `minContextLimit` 一同移除。** 按模型覆盖 `minContextLimit`。在此之前仍然生效。

#### `compress.contextLimitFallback`
- **类型：** `number`
- **默认值：** `128000`
- **状态：** ACTIVE
- **说明：** 当模型上下文窗口未知时使用的回退窗口（绝对 token 数）——例如未声明 limit 的自定义 provider、从未学到 limit 的 headless spawn+resume 会话，或模型切换使旧 limit 失效后的短暂窗口。驱动所有百分比阈值（`maxContextLimit`/`minContextLimit`）、紧急 nudge 覆盖、批量清理 GC 与 in-flight 工具输出截断。已知真实模型 limit 时始终优先，按模型覆盖（`modelMaxLimits`/`modelMinLimits`）同样优先。设为 `0` 可禁用回退（旧行为：学到 limit 前无安全网）。

#### `compress.nudgeFrequency`
- **类型：** `number`
- **默认值：** `5`
Expand Down
6 changes: 6 additions & 0 deletions dcp.schema.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,6 +192,12 @@
]
}
},
"contextLimitFallback": {
"description": "Fallback context window (absolute tokens) used when the model's limit is unknown (e.g. custom providers with no declared limit, or the brief window after a model switch invalidates a stale limit). Drives nudge thresholds, emergency override, GC, and in-flight truncation. Per-model limits (modelMaxLimits/modelMinLimits) take precedence. Set to 0 to disable the fallback (legacy behavior: no safety net until the limit is learned).",
"type": "number",
"default": 128000,
"minimum": 0
},
"nudgeFrequency": {
"type": "number",
"default": 5,
Expand Down
126 changes: 126 additions & 0 deletions devlog/2026-08-28_spawn-resume-context-limit/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,126 @@
# DESIGN - Context-limit safety net for spawn+resume mode (issue #346)

- Task ID: `2026-08-28_spawn-resume-context-limit`
- Home Repo: `opencode-acp`
- Created: 2026-08-28
- Status: Accepted

## 1. Problem Statement

- **What problem are we solving?**
ACP's safety net (nudge anchors, emergency override, batch-cleanup GC,
in-flight tool-output truncation) is gated on `state.modelContextLimit`. In
headless per-message spawn+resume mode the limit was never known when the
messages-transform pipeline ran, so every percentage threshold resolved to
`undefined` and the safety net was silently disabled. Sessions grew to the
serving wall (~229K tokens on a 262144 window + ~17K system + 16K
`max_tokens`) and entered an infinite empty-response retry loop (exit 0, no
error).
- **Why now?** Two production sessions froze at the wall with plugin logs
proving `postTokens == prePruneTokens` and `nudged=false` at every size.

## 2. Goals & Non-Goals

- **Goals**:
- The limit is known on the first request of a spawned process.
- A configurable fallback window bounds the conversation when the limit is
genuinely unknown.
- In-flight truncation fires before the serving wall (overhead-aware).
- A loud ERROR when the post-transform context still exceeds the budget.
- **Non-Goals**:
- Changing opencode-core's exit-0 empty response (upstream issue).
- Persisting the model's max-output-tokens limit (constant reserve +
`gc.majorGcThresholdPercent` escape hatch instead).
- Changing the messages-transform pipeline order.

## 3. Current Architecture (if applicable)

- **How it works today**:
- `state.modelContextLimit` is written only by the system-prompt hook
(`lib/hooks.ts`), which runs AFTER the messages-transform within one
request — and it is never persisted (saves happen inside the
messages-transform pipeline, before the system hook of the same request).
- The model-limit catalog (`lib/state/model-limits.ts`) is seeded at plugin
init via a fire-and-forget `client.config.providers()` call that races
server readiness in spawned processes.
- `resolveContextTokenLimit` (`lib/messages/inject/utils.ts`) returns
`undefined` for percentage thresholds when `state.modelContextLimit` is
`undefined`; `isContextOverLimits` then reports no limit crossed; anchor
sets stay empty; `runBatchCleanup` and `truncateLargeToolOutputs` early
return.
- **Pain points**: limit learned and lost per request; no retry on init-time
hydration failure; truncation threshold at 100% of the window ignores the
system prompt + `max_tokens` overhead.

## 4. Proposed Architecture

- **Overview**:
```
system.transform (per request)
└─ learn limit → if changed: saveSessionState (NEW)
messages.transform (per request)
├─ catalog resolve (existing #312 reconciliation)
│ └─ miss → registry.hydrateAndResolve: ONE lazy hydration/process (NEW)
├─ effective limit = model limit ?? compress.contextLimitFallback (NEW helper)
├─ nudge thresholds / emergency override / GC / truncation use effective limit
└─ post-transform: if postTokens > limit − systemPromptTokens − 16384
→ logger.error("ACP hard guard: ...") (NEW)
```
- **Key components**:
- `SessionStateRegistry.hydrateAndResolve(client, providerId, modelId)` —
resolve; on miss, hydrate from the client once per process, re-resolve.
- `resolveEffectiveContextLimit(state, config): {limit, source:
"model"|"fallback"} | undefined` — single source of truth for the window.
- `compress.contextLimitFallback` (default 128000, `0` disables).
- `OUTPUT_RESERVE_TOKENS = 16384` (`lib/messages/truncate-tools.ts`).
- **Data flow**:
- Limit lifecycle: system hook → state file (persisted on change) → next
process's `ensureSessionInitialized` (already restored) → threshold math.
Fallback chain: persisted/learned model limit → catalog (init seed or
lazy hydration) → `contextLimitFallback` → `undefined` (legacy no-op).
- **API / interface changes**:
- New config key `compress.contextLimitFallback` (validated, schema'd,
documented).
- New registry method `hydrateAndResolve`.
- New export `resolveEffectiveContextLimit` + `EffectiveContextLimit`.
- New export `OUTPUT_RESERVE_TOKENS`.
- Persisted state shape: unchanged (fields already existed).

## 5. Design Decisions & Rationale

| Decision | Options Considered | Chosen | Why |
| --------------------------- | ------------------------------------------------------------------------------------ | ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Where to persist the limit | (a) system hook saves on change; (b) messages transform re-derives from catalog only | (a) | The system hook is the only place the host tells us the real limit; saving on change is cheap (one file write per model/limit change, not per request). |
| Init-time hydration retry | (a) retry loop at init; (b) lazy one-shot hydration during a request | (b) | During a request the server is guaranteed up (we are inside its pipeline); one retry per process avoids repeated HTTP calls; the fallback covers the still-unknown case. |
| Unknown-limit behavior | (a) keep legacy no-op; (b) new fallback key, default on | (b) with `0` escape hatch | The issue's core complaint is "the safety net never engages"; a conservative 128K default bounds sessions while `0` restores legacy behavior for anyone who relied on it. |
| Truncation threshold | (a) keep 100% of window; (b) `min(configured, limit − systemPromptTokens − 16384)` | (b) | The serving wall includes system prompt + tool schemas + `max_tokens`; 100% of the window is already past it (the production failure). `min()` keeps `gc.majorGcThresholdPercent` as the user escape hatch for larger outputs. |
| Overflow signaling | (a) throw/abort from the plugin; (b) loud ERROR log | (b) | Plugins cannot set the process exit code or reject the request; an ERROR in the daily log gives the operator/orchestrator a greppable signal. Exit-0 empty response is filed upstream. |
| Internal-agent limit writes | (a) write before signature check (status quo); (b) skip internal agents | (b) | Title/summary/compaction agents run on their own small model; their limit must not corrupt the session's real limit (would shrink every threshold). |

## 6. Impact Analysis

- **Backward compatibility**:
- State file: additive only (fields pre-existed).
- Config: new key defaults on; `contextLimitFallback: 0` restores the exact
legacy behavior.
- Sessions with a known model limit: unchanged (model limit always
precedence).
- Sessions with an unknown limit: previously unbounded, now bounded by the
fallback window (intended behavior change).
- **Performance**: at most one extra `client.config.providers()` call per
process (and only on a catalog miss); one extra state save per limit change.
No new per-request work on the common path.
- **Security**: none (no new external input handling).
- **Dependencies**: none.

## 7. Migration Plan (if applicable)

- **Steps**:
1. Ship with the fallback default ON (128000).
2. Operators of very large-window custom providers who want the legacy
"no safety net until learned" behavior set
`"compress": { "contextLimitFallback": 0 }`.
3. Operators whose `max_tokens` exceeds 16K lower
`gc.majorGcThresholdPercent` (e.g. `"85%"`).
- **Feature flags / gradual rollout**: `contextLimitFallback: 0` is the
rollback switch; no flag needed beyond it.
144 changes: 144 additions & 0 deletions devlog/2026-08-28_spawn-resume-context-limit/REQ.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
# REQ - Context-limit safety net never engages in spawn+resume mode (issue #346)

- Task ID: `2026-08-28_spawn-resume-context-limit`
- Home Repo: `opencode-acp`
- Created: 2026-08-28
- Status: InProgress
- Priority: P0
- Owner: ranxianglei
- References: [issue #346](https://github.com/ranxianglei/opencode-acp/issues/346), [issue #312](https://github.com/ranxianglei/opencode-acp/issues/312) (preceding fix: model-limit catalog)

## 1. Background & Problem Statement

- **Context**: ACP's entire safety net (nudge anchors, emergency override,
in-flight tool-output truncation, batch cleanup GC) is gated on
`state.modelContextLimit`. Every percentage threshold
(`compress.maxContextLimit`/`minContextLimit` = "80%",
`compress.emergencyThresholdPercent` = "98%",
`gc.majorGcThresholdPercent` = "100%") resolves to `undefined` when the limit
is unknown, and every consumer treats `undefined` as "do nothing".
- **Root cause** (proven from two production sessions, plugin daily logs):
in headless per-message spawn+resume mode (orchestrator spawns a fresh
opencode process per message, resumes by session ID) the limit is never
known at the time the messages-transform pipeline runs:
1. Within one request the host fires `messages.transform` BEFORE
`system.transform` (sst/opencode, confirmed in #312). The only writer of
`state.modelContextLimit` is the system hook — so on the (only) request a
spawned process handles, the limit is still `undefined` during all
threshold math.
2. The system hook never persists the limit it learns (`saveSessionState`
only runs inside the messages-transform pipeline), so the next spawned
process starts from `modelContextLimit: undefined` again. The limit is
learned and lost, every message, forever.
3. The catalog seed at plugin init (`hydrateModelLimitsFromClient`,
fire-and-forget) races server readiness: the provider-config HTTP call
can fail before the server is up, and nothing retries.
- **Current behavior (symptom)**: at 229,479 / 229,535 tokens (sglang qwen 27B,
`max_model_len: 262144`) every transform logs
`prePruneTokens == postTokens`, `nudged=false`; proactive compaction never
fired (`lastCompaction: 0`). Total request size (context + ~15-20K system &
tool schema + `max_tokens: 16384`) exceeds the serving window → immediate
length rejection → opencode exits 0 with zero output → orchestrator retries
the identical failing request forever.
- **Expected behavior**:
1. The model context limit is known on the first request of a spawned
process (persisted from the previous request's system hook, or lazily
hydrated from the server during the request — the server is guaranteed up
by then).
2. When the limit is genuinely unknown, a configurable fallback limit bounds
the conversation instead of disabling the safety net.
3. In-flight tool-output truncation fires before the request hits the
serving wall (limit minus system-prompt + output-token overhead), not at
100% of the window.
4. When the post-transform context still exceeds the model budget, ACP logs
an ERROR (the exit-0 empty response is opencode-core behavior; the plugin
cannot set the exit code).
- **Impact**: any headless spawn+resume deployment with a custom/direct
provider whose limit is not resolvable from the init-time catalog is
unbounded: guaranteed infinite retry loop at the length-rejection wall, no
user-visible error.

## 2. Reproduction

- **Environment**: opencode 1.14.46 + opencode-acp v1.14.25, direct-to-sglang
(qwen 27B, `max_model_len: 262144`, `max_tokens: 16384`), headless
per-message spawn+resume, `acp.jsonc` containing only the `$schema` ref
(all defaults).
- **Minimal reproduction steps**:
1. Spawn opencode per message, resume by session ID; grow the conversation
past 80% of the model window (or past the fallback limit).
2. Observe `Chat transform complete` logs: `postTokens == prePruneTokens`,
`nudged=false` at every size; no truncation, no batch cleanup.
3. At ~229K tokens the request exceeds `max_model_len` → rejected → empty
run, exit 0 → retry loop.
- **Relevant configuration**: all defaults. `gc.majorGcThresholdPercent:
"100%"` means in-flight truncation only starts at the full window — already
past the serving wall once system prompt + tool schema + `max_tokens` are
added.

## 3. Constraints & Non-Goals

- **Constraints**:
- Backward compatibility: persisted state shape is additive only
(modelContextLimit/identity already persisted); new config key
`compress.contextLimitFallback` defaults to on; `0` restores legacy
behavior. Internal `dcp` naming untouched.
- No new dependencies. No `any`. Tests pass under
`node --import tsx --test tests/*.test.ts`.
- The messages-transform pipeline order is unchanged.
- **Non-Goals**:
- Fixing opencode-core's exit-0 empty response (upstream issue candidate,
reported separately).
- Time-based nudge cadence / emergency-notification cadence changes.
- Persisting the model's max-output-tokens limit (the 16K reserve constant
covers the reported deployment; users with larger `max_tokens` can lower
`gc.majorGcThresholdPercent`).

## 4. Acceptance Criteria

- **Correctness**:
- [ ] The system hook persists `modelContextLimit` + model identity on
change, so a freshly spawned process resumes with the limit known.
- [ ] Internal-agent requests (title/summary/compaction) no longer overwrite
the session limit with a different model's limit.
- [ ] On a catalog miss during messages.transform, the catalog is hydrated
once lazily (server is up during a request) before threshold math.
- [ ] With an unknown limit, `compress.contextLimitFallback` (default 128000) drives nudge thresholds, emergency override, batch cleanup,
and in-flight truncation; `0` disables the fallback.
- [ ] In-flight truncation threshold = `min(gc.majorGcThresholdPercent ×
limit, limit − systemPromptTokens − 16384)`; production numbers
(limit 262144, system ~17K, 229479 tokens) now trigger truncation.
- [ ] Post-transform tokens above the model budget log an ERROR with the
budget breakdown.
- **Performance / Stability**:
- [ ] No new per-request HTTP calls on the common path (lazy hydration at
most once per process; persistence only on change).
- **Regression**:
- [ ] New tests cover: limit persistence, lazy hydration, fallback
thresholds, overhead-aware truncation (production repro), hard-guard
error log. Each verified to FAIL against the pre-fix code.
- [ ] Full suite passes.

## 5. Proposed Approach

- **Affected modules & entry files**:
- `lib/hooks.ts` — system hook: move limit write after internal-agent
check, persist on change; messages transform: lazy hydration, effective
limit in filter ctx + logs, hard-guard ERROR.
- `lib/state/state.ts` — `SessionStateRegistry.hydrateAndResolve()` (lazy
one-shot hydration).
- `lib/state/utils.ts` — `resolveEffectiveContextLimit(state, config)`
helper (`model` | `fallback` source).
- `lib/config.ts` + `lib/config-validation.ts` + `dcp.schema.json` — new
`compress.contextLimitFallback` (number, default 128000, 0 = off).
- `lib/messages/inject/utils.ts` — `resolveContextTokenLimit` /
`isContextOverLimits` use the effective limit.
- `lib/gc/merge.ts` — `runBatchCleanup` uses the effective limit.
- `lib/messages/truncate-tools.ts` — `OUTPUT_RESERVE_TOKENS = 16384`,
overhead-aware threshold.
- Docs: CONFIGURATION.md (+ zh-CN), devlog.
- **Risks**: fallback default (128K) makes unknown-limit sessions compress
earlier than "never" — intended; disable with `contextLimitFallback: 0`.
Persisting the limit adds one small write on model/limit change only.
- **Rollback strategy**: revert the PR; state files remain compatible (new
fields are ignored by older versions).
Loading
Loading