[bug] Post-compression sent-view undercounts prompt by ~70–80K → false idle → host-floor emergency re-nudge loop

[ranxianglei]

Summary

After a successful compression, ACP's own token accounting drops far below the provider-reported prompt size (a roughly constant ~70–80K offset), so the nudge state flips to idle. On the next turn the host-floor anchor (#257) snaps tokenCount back up to the real prompt size (~88–95% of the effective limit) and re-injects an EMERGENCY "compress now" nudge. The model compresses again, accounting drops again, and the loop repeats — the agent stops working every few turns just to compress.

Environment

• pi + billion-context-pi v0.1.56
• Model: local vLLM qwen3.8-27b, contextWindow=262144, maxTokens=131072
• Every turn logs [overflow-selfheal] event=output-headroom before=262144 after=131072 → effective limit = 131,072

Repro session

• Session id: 01a07b3c-ab19-7b19-8290-f7967a8221a6
• Log: ~/.pi/acp.log from line 10163 onward
• Compression points: 14:29:55 UTC (blocks b13+b14), 14:51:08 (b15+b16), 14:56:49 (b17)

Evidence

1. Pre-compression estimates match the provider exactly

On nearly every non-compression turn, [turn] tok == hostTokens, e.g. @14:27:14 tok=124726 hostTokens=124726. The estimator is correct while the conversation is uncompressed.

2. Post-compression: constant ~70–80K undercount

time (UTC)	[turn] tok	provider input+cacheRead	gap
14:31:30	31,745	79,788 + 22,528 = 102,316	70,571
14:56:15	34,220	110,570 (cacheR=0)	76,350
14:56:49	—	4,153 + 115,200 = 119,353	—

The #289 view-recount does not catch this: at 14:56:20 viewTokens=36,069 vs actual 116,261.

The gap stays in the same ~70–80K band across different compression sizes, which suggests a fixed component is missing from the post-compression sent-view rather than per-block summary miscounting (the 10 active tier-1 block summaries total only ~8–12K tokens).

3. Host-floor anchor then re-triggers EMERGENCY

Next turn after each compression, tokenCount = max(ownEstimate, lastProviderUsage) snaps back to 106,669 (81%) / 112,658 (86%) / 115,696 (88%) / 118,184 (90%) / 124,879 (95%) of the 131,072 limit → nudge escalates active→emergency and injects:

⚠️ Context limit reached — compress now. Prioritize consumed tool outputs.

(acp.log lines 13504-05 @14:27:14, 13848-49 @14:37:21, 13922-23 @14:48:06, 14122 @14:56:20; voice=emergency emergency=false reInject=false)

4. Loop cycle observed

emergency nudge → model compresses → accounting drops to 21–34K (16–26%, nudge=idle) → 1–2 work turns add tool outputs → host anchor snaps back to ~110–125K (88–95%) → emergency again.

User-visible: agent stopped every few turns to compress instead of working; user interrupted twice (stopReason=aborted @14:52:55, @14:57:08) and typed "继续" twice.

Secondary symptoms in the looped tail

• Duplicate compress of the identical range m00103..m00149 (second call failed: "Requested range(s) already compressed … Run acp_status")
• One malformed empty compress {} call (@14:52:55)
• Model thinking opens with "Need to compress the consumed ranges" on successive turns

These look like a weak local model (qwen3.8-27b) reacting to repeated emergency nudges; the root trigger is the accounting mismatch above.

Contributing factor

The effective window was halved by output-headroom (maxTokens=131072 reserved out of 262144), putting all nudge bands below the real post-compression prompt floor (~102–135K). Lowering maxTokens to 49152 removes the pressure, but the undercount itself should be fixed.

Suggested fix direction

Audit what the post-compression sent-view includes (estimateTokens(coreMessages, coveredIds, imageTokens) in src/index.ts): something fixed (~70–80K) is present in the actual request but absent from the view only after a compression has occurred. The #289 view-recount compares against the post-processTurn view, so it inherits the same blind spot.

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Root cause confirmed — this is a metering bug, not an estimator or model problem. I traced it in code and reproduced the mechanism.

src/index.ts sets tokenCount = estimateTokens(coreMessages, coveredIds, imageTokens) + systemPromptTokens, then floors it at the provider-reported prompt size — unless a successful compress landed after the last valid assistant usage (usageAnchorPredatesCompression()), in which case the floor was skipped entirely. That skip is the bug:

1. estimateTokens (src/tokens.ts) sums message text only. It structurally omits tool-definition schemas, ref-tag overhead, and the injected nudge. On your host that's a fixed ~70–80K — exactly the constant band you measured, and why it stayed flat across different compression sizes instead of scaling with block count (your "~8–12K of summaries" observation ruled that out correctly).
2. Pre-compression, the current-anchor floor lifted tokenCount up to the provider number — which is why non-compression turns "matched the provider exactly." The estimator was never wrong; the floor was masking its blind spot.
3. Post-compression, the stale-anchor skip dropped tokenCount back to the raw undercount → false idle. Next turn, fresh provider usage re-engaged the floor at full real size (~84–95% of 131072) → spurious EMERGENCY → you compress again → loop. The [Bug] Token 计数与压缩后状态不同步，导致低收益及零收益的连续压缩 #289 view-recount inherits the same blind spot because it compares against the post-processTurn view.

Layer judgment: the emergency re-nudges, duplicate compress of m00103..m00149, malformed compress {}, and "Need to compress…" thinking are all downstream symptoms. The single root cause is: post-compression, the meter reads the raw sent-view instead of the true prompt size. Fix the meter and the whole cascade stops.

Fix (deliberately diverges from the suggested direction): instead of adding the fixed component to the estimate — which would require measuring tool-schema tokens, unavailable and unstable inside the extension — I now floor at (lastProviderUsage − netReclaimedSinceAnchor) when the anchor is stale. netReclaimedSinceAnchor = Σ over active blocks whose creating compress toolResult lands after the last valid usage anchor of max(0, compressedTokens − summaryTokens). The provider number already counts all fixed overhead accurately, so subtracting only the tokens actually freed gives the exact post-compression size with no magic constant. New work added after the compress still grows the count via sentTokens; the non-stale path is unchanged; a stale-anchor compress that freed little still correctly trips.

Verified: typecheck ✓ · full suite 472 tests / 469 pass / 0 fail (3 pre-existing skips) ✓ · tsup build ✓. Added unit tests for the staleness delta (fresh/stale/pre-anchor-exclusion/inactive-skip/negative-clamp/multi-block) and rewrote the arbitration stale test to seed a real reclaiming block, plus a guard test so a tiny reclaim near the limit still fires.

PR: #328 (no version bump — release-branch territory per AGENTS.md).

Lowering maxTokens to 49152 remains a valid workaround for your local vLLM setup, but the undercount itself is fixed here.

One side note I did not fold into this fix: while tracing, I saw coreOutToAgentMessages drop rendered acp_summary_* messages from the rebuilt view returned to Pi. That does not affect this accounting loop (estimate and request agree on summaries either way), but if those summaries aren't reaching the model by some other path it would be a separate, larger issue. Flagging for your awareness — happy to investigate as a follow-up if you want.