[Bug] In-flight prune is a no-op (postTokens == prePruneTokens) at 229k tokens → request exceeds server context → empty-response resume loop

[ranxianglei]

Mechanism (proven from plugin daily logs)

Two production sessions (headless opencode, resumed per-message by an orchestrator) grew until every request failed. The plugin's own log shows the root cause:

2026-08-27T15:59:26Z Chat transform complete | session=ses_7fb333d7… messages=145 prePruneTokens=229479 postTokens=229479 nudged=false
2026-08-28T14:28:31Z Chat transform complete | session=ses_7fb2e8b9… messages=99  prePruneTokens=229535 postTokens=229535 nudged=false

postTokens == prePruneTokens on every transform — even at 229k tokens. The in-flight prune never removes anything, nudged=false throughout, and the persisted state shows lastCompaction: 0 (proactive compaction also never fired; stats.totalPruneTokens 37k/230k respectively, all content parked in prune.messages).

Failure chain

1. Conversation reaches ~229k tokens (plus ~15-20k system+tool schema + max_tokens: 16384)
2. Total exceeds the serving backend's context window (sglang qwen 27B, max_model_len: 262144) → immediate request rejection
3. opencode run ends with exit 0 and zero output (no error surfaced)
4. Orchestrator retries → identical ~4.6s failure, forever (3× consecutive observed per session)

Both sessions froze at almost exactly the same size (229,479 / 229,535) — the length-rejection wall.

Environment

• opencode 1.14.46 + opencode-acp@latest (plugin log says v=1.14.25), direct-to-sglang (no bili in this path)
• Headless per-message spawn+resume (ACP path), daemon-driven
• acp.jsonc contains only the $schema ref (all defaults)

Expected

The plugin's core contract is bounding the conversation. At minimum:

• In-flight prune must actually reduce tokens before the request when near the model context limit (or prune thresholds are misconfigured/never fire in this mode)
• Proactive compaction (lastCompaction) should trigger long before 229k
• On unrecoverable context overflow, surface an error instead of a silent empty run with exit 0 (the orchestrator can't distinguish this from success)

Workaround in use

Orchestrator detects the empty-response loop and starts a fresh session (loses in-session continuity; conversation history only survives via the issue thread).

Cross-ref: initially misfiled as ranxianglei/billion-context#318 (environment there corrected — no bili proxy in this path; the state model investigation may still be relevant to acp-kernel).

State files and full logs available on request.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Acknowledged — investigating. The log signature (postTokens == prePruneTokens at 229k, nudged=false, lastCompaction: 0, content parked in prune.messages) points at one of: (a) the model context limit is unresolved on this path so all %-based thresholds (nudge 45/55%, compaction, GC) never fire, or (b) the prune map is populated but the replacement step is a no-op (e.g. everything is protected by preserveRecentMessages/protected tools). Pulling the state file analysis next; will report root cause + fix plan in a follow-up comment.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Root cause identified (code-level, v1.14.25). Two structural defects compound in your per-message spawn+resume mode:

1. The model context limit never reaches the messages-transform pipeline in this mode

state.modelContextLimit is written only by the system.transform hook (lib/hooks.ts:100), which fires after messages.transform within each request. Every %-based safety net in ACP reads that value during messages.transform:

• nudge thresholds (min 45% / max 55%) → resolve to undefined when the limit is unknown (lib/messages/inject/utils.ts:121)
• emergency threshold (98%), batch cleanup (100%, lib/gc/merge.ts:201), emergency tool-output truncation (100%, lib/messages/truncate-tools.ts:34)

In a long-lived process the limit is available from the 2nd LLM request onward (in memory). But with per-message spawn+resume each process handles exactly one user message: messages.transform runs with the limit unknown, then system.transform sets it in memory — and no save ever happens after that, so the value is never persisted (all saveSessionState calls occur during messages.transform, i.e. before the system hook writes it). The only rescue is the fire-and-forget init-time catalog hydration (index.ts:69, client.config.providers()), which degrades silently on failure (server not ready at init, or the model absent from the catalog — e.g. a custom sglang model with no limit.context declared in opencode.json). Net effect: limit unknown forever → overMinLimit/overMaxLimit always false → no anchor nudges, no emergency path, no in-flight reduction. That is exactly your nudged=false + postTokens == prePruneTokens at 229k.

2. Even with the limit known, the 100% emergency threshold sits above the point where the request actually fails

The request is rejected when context + system prompt + tool schemas + max_tokens > 262144 — for you that's ~229k context (~87% of the window). But emergency tool-output truncation only arms at 100% of the limit (majorGcThresholdPercent: "100%", lib/messages/truncate-tools.ts:39). The in-flight safety net is armed at a point the request never reaches.

3. (Contributing) the growth nudge deadlocks at the wall

getCurrentTokenUsage (lib/token-utils.ts:9) reports the last successful assistant message's prompt size. Once every request is rejected, no new token data lands, the reported size stalls at ~229k, growth ≈ 0, and the growth-based nudge — the only mechanism that doesn't depend on the limit — can never fire again.

The compressions that did happen (totalPruneTokens 37k/230k) occurred while the session was still growing — consistent with the growth nudge working until the wall.

Fix plan (PR incoming):

1. Persist modelContextLimit (+ model identity) from the system.transform hook so the next spawned process loads it.
2. Lazy catalog hydration on miss in messages.transform (the server is guaranteed up by then — unlike at init).
3. Configurable fallback limit (compress.contextLimitFallback, default 128k) when the model declares none, plus a WARN log — the safety net stays armed.
4. Overhead-aware emergency threshold: min(configured, limit − system prompt − output reserve) so truncation fires before the rejection wall, not at 100%.
5. Loud ERROR log when an over-budget request can't be reduced in-flight. (The exit-0 empty response itself is opencode core behavior — the plugin can't set the exit code; worth reporting upstream to sst/opencode separately.)

Will post the PR link here once it's up. If you can share one of the state files (~/.local/share/opencode/storage/plugin/acp/ses_7fb*.json) I can confirm the block/limit fields match this analysis.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] Root cause confirmed and fixed — PR: #349

Why the safety net never engaged (all three from code, matching your logs):

1. The system hook is the only writer of state.modelContextLimit, and it runs after messages.transform within a request — so on the single request a spawned process handles, every percentage threshold (80% max/min, 98% emergency, 100% GC) resolved against undefined → all consumers no-op.
2. The system hook never persisted the limit it learned (saves only happen inside the messages-transform pipeline, before the same request's system hook) — so each fresh spawn started from undefined again. Learned and lost, every message.
3. The init-time catalog seed is fire-and-forget and races server readiness in spawned processes; nothing retried.

With the limit unknown, postTokens == prePruneTokens and nudged=false at every size are exactly what the code does — the prune itself is fine, it just never had a threshold to act on.

The fix (5 parts):

1. System hook persists modelContextLimit + model identity on change → spawned processes resume with the limit known.
2. One-shot lazy catalog hydration during the request on a catalog miss (server is guaranteed up then).
3. New compress.contextLimitFallback (default 128000, 0 disables) — bounds the conversation when the limit is genuinely unknown; drives nudges, emergency override, GC, and in-flight truncation. Real model limit always takes precedence.
4. In-flight truncation is now overhead-aware: fires at min(gc threshold, limit − systemPromptTokens − 16384) instead of 100% of the window. Your production numbers (262144 window, ~17K system, 229479 tokens) now trigger truncation at 228760 — there's a test using the exact 229479 figure.
5. If the post-transform context still exceeds the budget, ACP logs a loud ERROR ("ACP hard guard: …") with the budget breakdown.

On "surface an error instead of exit 0": the empty response + exit 0 is opencode-core behavior (the plugin can't set the exit code or reject the request) — the hard-guard ERROR gives the orchestrator a greppable signal in the ACP daily log; I'd file the exit-0 thing upstream to opencode separately.

Tests: 1049/1049 pass (+20 new); all 11 behavioral tests verified to fail with the fix reverted. Pre-existing #312 invalidation tests unchanged. Rollback: "contextLimitFallback": 0.

Two minor findings while in here (not blocking):

• CONFIGURATION.md documents maxContextLimit/minContextLimit defaults as 55%/45%, but the code defaults are 80%/80% — stale docs.
• The repo is not prettier-clean at HEAD under its own .prettierrc (printWidth: 100 vs the ~80-col style; prettier --write . would touch 412 files) — I left formatting untouched to keep the diff reviewable.

Still open: your state files would let me check one remaining question — with totalPruneTokens at 37k/230k, compress blocks did exist, so why didn't prune.messages entries actually remove covered messages in-flight? Happy to dig once I have a sample state file + the matching daily log.

[ranxianglei]

🤖 ework agent · vllm-qwen/qwen3.8-27b

[bot] PR #349 is ready — dual-agent review complete, all findings addressed, full CI green (pr-validation, build, test on Node 22 + 24, e2e, build-artifact).

• Branch: 2026-08-28_spawn-resume-context-limit (2 commits: 019616f fix + 24626ae review follow-ups)
• Suite: 1050/1050 locally and in CI; all 11 behavioral tests verified to fail with the fix reverted
• Review summary: fix: context-limit safety net for spawn+resume mode (#346) #349 (comment)

Per project rules the merge itself is human-only — please merge #349 when you're ready (CI auto-publishes on a release branch; this one lands on master as a normal fix PR).

Still open from my earlier comment: the state file + daily log would let me check why prune.messages entries (37k/230k totalPruneTokens) didn't actually remove covered messages in-flight in your production sessions — that's a separate question from the limit-unknown root cause, and I'd like to rule it out before calling this fully closed.