Redesign #408 host-usage backfill: post-fold by default, baseline opt-in (opencode gap + whack-a-mole exemptions)

[ranxianglei]

来源: ranxianglei/billion-context#645 分析 codex >100% 上下文显示时发现。

Background

#408 introduced the host-usage backfill: the proxy adds the tokens folded out of the forwarded view back into input_tokens, so the host sees the uncompressed baseline. It was introduced to fix omp's accounting (omp's formula providerPromptTokens − historyRewriteTokensRemoved collapsed to ~0% because bili never called omp's recordAnchoredHistoryRewrite).

But the backfill is a one-size-fits-all solution that does not match every host's accounting semantics. It has required reactive exemptions for:

• pi (pi plugin 模式宿主 usage 回填未压缩基线:footer 显示 302.7%/1M 与实际请求(折叠后 ~57K)不符 #590, showed 302.7%)
• omp (omp 插件会话原生压缩被扩展全部取消后仍收 #408 usage 回填: omp footer 只增不减 (205%/1M) #623, showed 205%)
• codex ([Question] Previously, Codex displayed the context window size after compression, but it now appears to show an unexpected/incorrect value. #645, showed 138%) — just fixed in PR fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647

The opencode gap (concrete bug)

armHostUsageCredit (src/server.ts:1833) exempts only pi, omp, and codex (by UA). opencode is a bili agent plugin exactly like pi/omp — it self-announces via x-bili-plugin: opencode, cancels native compaction, and lets the proxy own the ACP tools (src/agent/opencode.ts: "mirrors the pi/omp plugin") — but it is not in the exempt list. So opencode receives the backfill and very likely shows the same >100% display that pi/omp/codex did; it simply hasn't been reported yet.

The design problem

The backfill adds "folded-out tokens" to input_tokens for every host, but each host has different accounting semantics. A number that is right for one host's display is wrong for another's. Exempting hosts one-by-one as they report a broken display is whack-a-mole.

Proposed direction

• Flip the default: hosts should report the post-fold (actually-forwarded) usage by default — that is the number the model really receives, and it is sane for any display.
• Make the baseline opt-in: the 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 uncompressed-baseline backfill should apply only to the specific host(s) whose accounting genuinely consumes it — via a per-host mechanism that matches their semantics, not a global input_tokens add.
• The ZCode 宿主 usage 回填未压缩基线:压缩后上下文显示成累加值,非实际转发量(v0.1.88 起;#645 的 ZCode 面) #648 hostUsageCredit: "off" switch (for zcode) is a step toward this, but it is a manual global opt-out. The cleaner end-state is: post-fold by default, baseline available where a host's accounting actually needs it.

Concrete steps

1. (Quick fix) Exempt opencode from the backfill (add it to the exempt list in armHostUsageCredit), like pi/omp.
2. (Redesign) Flip the default to post-fold usage and make the baseline opt-in (per host, matching each host's accounting semantics).

Impact

• The opencode gap affects opencode users now (a >100% context display).
• The whack-a-mole exemptions are a maintenance burden and a repeated source of bugs (pi plugin 模式宿主 usage 回填未压缩基线:footer 显示 302.7%/1M 与实际请求(折叠后 ~57K)不符 #590, omp 插件会话原生压缩被扩展全部取消后仍收 #408 usage 回填: omp footer 只增不减 (205%/1M) #623, [Question] Previously, Codex displayed the context window size after compression, but it now appears to show an unexpected/incorrect value. #645).

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

I dug into this before touching code. Two findings that change the proposed approach:

1. The opencode gap is real, but the mechanism differs from the issue — and step 1 (the quick fix) would not work as written

The issue says opencode "self-announces via x-bili-plugin: opencode" and is "a bili agent plugin exactly like pi/omp". The code says otherwise:

• devlog/2026-08-23_opencode-launcher/REQ.md lists exact conversation→session binding as an explicit non-goal and states: "opencode cannot stamp x-bili-plugin-* headers from a plugin".
• The thin plugin src/agent/opencode.ts only registers the /acp command (config + command.execute.before hooks). It has no session.created hook and never calls /__bili/plugin/register.
• Under bili opencode, opencode-acp self-disables (BILLION_CONTEXT_PROXY set, src/launcher.ts:1813) so the proxy owns the ACP tools → opencode runs in proxy mode (pluginMode=false, session.metadata.pluginAgent=undefined).
• src/server.ts:1346-1368: pluginAgent only comes from the x-bili-plugin header, a pending register, or a prior session value — none of which opencode produces. So it stays undefined.

Consequence: opencode is a plain proxy client, not a plugin. It receives the #408 backfill through the default path (the else branch of armHostUsageCredit), not a pluginAgent check. Adding "opencode" to the pluginAgent === "pi" || "omp" exempt list would match nothing — pluginAgent is undefined for opencode sessions. So step 1, as written, is a no-op.

2. Every host that has been reported shows the baseline as wrong — none has reported it as correct

• pi pi plugin 模式宿主 usage 回填未压缩基线:footer 显示 302.7%/1M 与实际请求(折叠后 ~57K)不符 #590: 302.7% → exempted
• omp omp 插件会话原生压缩被扩展全部取消后仍收 #408 usage 回填: omp footer 只增不减 (205%/1M) #623: 205% → exempted
• codex [Question] Previously, Codex displayed the context window size after compression, but it now appears to show an unexpected/incorrect value. #645: 138% → exempted (PR fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647, open)
• ZCode ZCode 宿主 usage 回填未压缩基线:压缩后上下文显示成累加值,非实际转发量(v0.1.88 起;#645 的 ZCode 面) #648: "cumulative, drifting baseline that overstates real context pressure" → global off switch (branch, unmerged)

#408 was introduced for omp, yet omp is now exempted. The uncompressed baseline is a virtual number (folded-out + forwarded) that drifts turn-to-turn and can exceed the window; no host's display consumes it correctly.

My recommendation (matches your proposed direction)

• Flip the default to post-fold — armHostUsageCredit stops arming the credit by default; the host sees the actually-forwarded (folded) usage, i.e. the real number the model receives. This fixes opencode and every other host with no per-host exemption list at all.
• Make the baseline opt-in via a config flag that builds on ZCode 宿主 usage 回填未压缩基线:压缩后上下文显示成累加值,非实际转发量(v0.1.88 起;#645 的 ZCode 面) #648's hostUsageCredit switch but inverts the default — e.g. hostUsageCredit: "postfold" | "baseline", default "postfold". This subsumes both fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647 (codex needs no UA exemption — it just gets the default) and ZCode 宿主 usage 回填未压缩基线:压缩后上下文显示成累加值,非实际转发量(v0.1.88 起;#645 的 ZCode 面) #648 (the off switch becomes the default; baseline is the opt-in).

Before I implement — two confirmations

1. Scope of the flip: OK to flip the default for all plain proxy clients (not just opencode)? This is a behavior change for anyone currently relying on the 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 baseline display.
2. Opt-in granularity: a global config flag (hostUsageCredit: "baseline") or a per-host map? Since no host is currently confirmed to need the baseline, I'd recommend the simpler global flag now and add per-host granularity only if a host is later found that needs it.

I'll hold the implementation until you confirm the direction. (Note: #647 and the #648 branch are both unmerged — I'll build on master and the PR will supersede them.)

[BsoBird]

Could you walk me through the root cause and background of #408?

1.Why does it depend on the baseline?
2. My understanding is that opencode's built-in auto-compaction must be disabled for ACP to work correctly. Given that, is the failure described in #408 specifically caused by the postfold path, or is something else going on?

[BsoBird]

I support this redesign. The problem with #408 was never "it fixed omp" — it was that omp's accounting semantics were made the global default for every host.

I would suggest not implementing this as a single global switch, but as a layered policy resolution ordered by identification confidence:

1. Wrap sessions identified via launcher / plugin registration (bili pi, bili codex, etc.): hard-code the per-host policy case by case.
2. Sessions carrying a trusted header such as x-bili-plugin: fall back to the matching host policy.
3. UA heuristics (e.g. codex): last resort only, and explicitly labeled as heuristic.
4. Fully unidentified proxy clients: use the global hostUsageCredit switch.

For the default, I would strongly recommend post-fold, with backfill kept only as a legacy / escape-hatch mode. If proxy-mode clients default to backfill, clients like ZCode (#648) and opencode (this issue) will re-run into >100% / accumulating displays. A global backfill default mostly just moves the whack-a-mole onto the user.

One omp-specific note: if omp still needs the uncompressed baseline, the right fix is to report the rewrite amount through a host-specific channel such as recordAnchoredHistoryRewrite, not by injecting the baseline into the global input_tokens. Based on the evidence in #590 / #623 / #647, pi / omp / codex all currently look better served by post-fold.

For PR #647: it is fine to merge it as a codex hotfix, but I would mark it as temporary. Long-term, the codex UA matcher should move into the policy table above as one UA-fallback row, instead of growing the armHostUsageCredit if-chain further.

Two acceptance criteria I would want:

• internal lastInputTokens / nudge / preflight always use the real post-fold value, independent of the host display policy;
• every host policy has a wire-level regression test, so we do not repeat the fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647 class of bug ("fixed codex, missed codex_sdk_ts").

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

@BsoBird good questions — let me walk through the #408 root cause, then respond to your design.

#408 root cause

The original failure (omp showing ~0%). omp's accounting formula is Math.max(0, providerPromptTokens − historyRewriteTokensRemoved). omp exposes a recordAnchoredHistoryRewrite API specifically for compression extensions, but bili never called it, so historyRewriteTokensRemoved stayed 0. When bili folded the request, the provider returned the small post-fold providerPromptTokens, and omp showed ~0% — the 50k–130k folded tokens "disappeared" from its ledger and its auto-compact safety net slept.

Why the fix became a baseline backfill (your Q1). The natural fix was to call recordAnchoredHistoryRewrite(removedTokens). But that API's semantics are providerPromptTokens − removed — it subtracts from the anchor (it exists for when the host's local history was deleted and the provider count is too large). In bili's case the anchor is already small (post-fold), so calling it would subtract more — the wrong sign. So the fix was flipped: instead of telling the host "I removed N tokens", bili backfills the usage it returns to the host — adding the folded-out credit back into input_tokens so the anchor carries the uncompressed baseline from the start, with zero omp-side changes. That's why #408 "depends on the baseline": the chosen fix was to make the host see the uncompressed baseline, because the host-specific channel had the wrong sign for this case.

Is the failure caused by the post-fold path (your Q2)? No — and the distinction matters:

• The post-fold path is correct: bili folds the request, the provider returns the post-fold usage, and that is the real number the model received. Returning it is honest.
• The original 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 failure (~0%) was a mismatch: omp's formula expected the host to report the rewrite amount, but bili didn't (and couldn't, via the wrong-sign API). So omp read the small post-fold value as ~0%.
• The new failure (>100% for pi/omp/codex/ZCode/opencode) is caused by the baseline backfill — the 代理压缩不上报改写量,宿主上下文会计塌缩至 ~0(omps recordAnchoredHistoryRewrite 零调用);且持久化 lastInputTokens 恢复无钳制,旧版负值升级后复活 #408 fix — which injects a virtual number (folded-out + forwarded) into input_tokens. That virtual number drifts turn-to-turn and can exceed the window.

So post-fold is the sane, correct default; the baseline backfill is the thing that's wrong for every host reported. On your opencode point: yes, opencode's auto-compact is disabled and the proxy owns compression — but that's orthogonal to the display bug. opencode is a plain proxy client that receives the backfill via the default path; the disabled auto-compact is what makes the proxy's folding authoritative, not the cause of the >100%.

Your layered policy proposal

I agree with the direction and the post-fold default. The layered resolution (launcher/plugin registration → trusted x-bili-plugin header → UA heuristic → global switch) is the right end-state — a policy table keyed by identification confidence, exactly what the issue's "per-host mechanism matching each host's semantics" asks for, and the natural home for the codex UA matcher from #647 as a labeled heuristic row.

Two refinements before implementing:

1. Right now the table is trivially "post-fold for every known host" (pi/omp/codex/opencode all want post-fold; no host is confirmed to need the baseline). So I'd implement the resolution machinery + post-fold default now, and keep the baseline as an escape-hatch mode (your "legacy/escape-hatch" framing) rather than wiring per-host baseline rows no host currently uses. The machinery is load-bearing; the per-host rows get added when a host is found that needs them.
2. The omp note is correct in principle but the API is wrong-sign (see Q1): recordAnchoredHistoryRewrite subtracts, so it can't carry a "the proxy folded N tokens, add them back" signal as-is. If omp (or any host) genuinely needs the baseline long-term, it needs a new host-specific channel (a "proxy added N tokens back" header/API), not the existing one. I'd file that as a separate upstream proposal if/when a host is found to need it.

I agree with both acceptance criteria:

• internal lastInputTokens / nudge / preflight always use the real post-fold value, independent of the host display policy — this already holds (hostCreditTokens is display-only; the ledger stays post-fold, verified in tests/host-usage-backfill.test.ts);
• every host policy has a wire-level regression test — I'll add one per policy row, including the codex UA row, so the fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647 class of bug ("fixed codex, missed codex_sdk_ts") is caught.

On #647: agree — fine to merge as a temporary codex hotfix, with the UA matcher moving into the policy table as a labeled heuristic row in this redesign.

What I need from @ranxianglei before implementing

Confirmation on (a) the post-fold default for all clients, and (b) scope of the machinery: build the full layered resolution now, or start with post-fold default + a global hostUsageCredit escape-hatch and grow the policy table as hosts are found? I lean toward the latter (simpler now, same end-state) — but the full table is defensible if you want the machinery in place up front. I'll implement as soon as you confirm.

[ranxianglei]

合并了#649 看看这个要更新吗

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Yes — #649 landing changes the design in a good way. Updated picture:

What #649 gives us (on master now, not yet released — it merged after v0.1.99):

• hostUsageCredit: "auto" | "off" config option. auto (default) = backfill ON for plain clients (pi/omp exempted); off = post-fold (no backfill).
• Because it's unreleased, no published version carries these values — I can rename them freely, zero migration cost.

How the redesign builds on it (updated):

1. Flip the default to post-fold and rename to match the meaning: hostUsageCredit: "postfold" | "baseline", default "postfold" (off→postfold, auto→baseline). The default is now the actually-forwarded usage for every host — opencode included — with no per-host exemption list at all.
2. Drop the pi/omp exemption (pluginAgent === "pi" || "omp", src/server.ts:1852) — redundant once the default is post-fold. Removes two of the whack-a-mole entries.
3. Supersede PR fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647 (codex) — still open. With the post-fold default, codex reports folded usage with no UA matcher, so fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647's isCodexClient check is unnecessary. I'd close it as superseded (its intent is absorbed by the new default). If codex ever needs special handling, per your/@BsoBird's note it belongs in a labeled heuristic row, not an if-chain.
4. Baseline stays the escape-hatch (hostUsageCredit: "baseline") for any host whose accounting genuinely consumes the uncompressed number — none confirmed today.

Net effect: one config knob, post-fold default, zero hardcoded host exemptions. The layered policy table @BsoBird sketched remains the documented end-state for when a host needs a specific baseline semantic; I wouldn't build the full table yet since every known host wants post-fold.

One scope call for you: go with this minimal flip-and-drop now, or build the full layered-resolution table up front? I recommend the minimal version — same end-state, less surface to test, and the table only earns its place once a host needs a non-default policy.

Say the word and I'll implement (branch off current master 175beb2; wire-level regression test per affected path; pre-flight typecheck/test/build).

[BsoBird]

@ranxianglei 默认策略还是 postfold 好一些,能从请求头中拿到agent type最好,直接根据type路由走case by case或者默认策略,如果都失败就下到默认策略,默认现在填写baseline,但是agent工具工作的时候,已经把压缩等于是交接给acp来做了,依然显示完整的baseline大小很奇怪.如果真的有这种agent,理论上应该提前再前面case by case的环节上解决掉,默认策略留个开关,可以再 baseline/postfold 切.是不是这样好一点?而不是现在写了好几个if条件做额外豁免

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷

Agreed on every point — and the reasoning nails why the current default is wrong: once an agent hands compression to ACP, the forwarded view is already folded, so the model physically receives far fewer tokens than the uncompressed baseline. Displaying the baseline then describes a context the model never saw — it overstates pressure and drifts turn-to-turn. Post-fold default is correct.

Your type-routed resolution is the right shape and it kills the whack-a-mole. Concrete plan (ready to implement once @ranxianglei gives the word):

Resolution order (no if-exemptions):

1. Detect agent type from the x-bili-plugin header (session.metadata.pluginAgent) → look up a per-host policy map.
2. No match → fall through to the global switch.
3. Global switch hostUsageCredit: "postfold" | "baseline", default "postfold".

Changes:

• src/config.ts: rename hostUsageCredit "auto"/"off" → "postfold"/"baseline", default "postfold" (safe — fix #648: add hostUsageCredit switch to opt out of #408 host-usage backfill #649 is unreleased, nothing to migrate).
• src/server.ts: replace the hostUsageCredit === "off" early-return plus the pluginAgent === "pi" || "omp" exemption with a single resolveHostUsagePolicy(pluginAgent, global) call. The per-host map starts empty (every known host is post-fold) — adding a host later is one map entry, not a new if.
• Supersede PR fix #645: suppress #408 host-usage backfill for codex (report folded usage) #647 (codex) — codex is a plain client, not in the map, so it gets post-fold by default; close it as superseded.
• Wire-level regression test per affected path (anthropic/openai/responses × default-postfold + baseline-override), honoring the acceptance criteria we agreed.

One scope note: the map is keyed on the announced agent type (the x-bili-plugin header), which covers the bili agent plugins (pi/omp/opencode). Plain clients (codex/zcode) carry no such header → they hit the global default (post-fold). If a plain client ever needs a non-default policy, that's where the UA-heuristic rung of the ladder comes in — I'd defer that until it's actually needed rather than build it speculatively.

@ranxianglei — green light and I'll branch off master 175beb2 and ship it.

[ranxianglei]

🤖 Powered by ework · qwen3.8-27b

[bot] 🏷 Done — implemented and opened PR #691.

First, what I verified (the opencode gap is subtler than the issue states). opencode's thin plugin (src/agent/opencode.ts) never registers with the proxy — no session.created, never calls /__bili/plugin/register. Under bili opencode, opencode-acp self-disables, so opencode actually runs as a plain proxy client (session.metadata.pluginAgent stays undefined). That means it receives the #408 baseline backfill through the default path, not through any plugin-agent branch. So the issue's proposed quick fix — adding "opencode" to the pluginAgent === "pi" || "omp" exempt list — would have been a no-op: pluginAgent is never "opencode". The real fix is structural, which is exactly the redesign you proposed.

What the PR does

• Flips the default to post-fold. Every host now sees the actually-forwarded (folded) usage — the tokens the model really receives — which is sane for any display. This fixes opencode (and keeps pi/omp/codex correct) with no exemption list at all.
• Makes the baseline opt-in via hostUsageCredit: "baseline", for the specific host(s) whose own accounting genuinely consumes the unfolded total.
• Replaces the if-chain with a resolver. New resolveHostUsagePolicy() consults a HOST_USAGE_POLICY map keyed on the announced agent type, falling back to the global switch. Adding a host = one map entry, not a new if — kills the whack-a-mole (pi plugin 模式宿主 usage 回填未压缩基线:footer 显示 302.7%/1M 与实际请求(折叠后 ~57K)不符 #590 / omp 插件会话原生压缩被扩展全部取消后仍收 #408 usage 回填: omp footer 只增不减 (205%/1M) #623 / [Question] Previously, Codex displayed the context window size after compression, but it now appears to show an unexpected/incorrect value. #645 / ZCode 宿主 usage 回填未压缩基线:压缩后上下文显示成累加值,非实际转发量(v0.1.88 起;#645 的 ZCode 面) #648).
• Backward compatible. Legacy values still parse: auto → baseline, off → postfold. Only the implicit default flips; anyone who explicitly set a value keeps their behavior.
• Dropped the now-redundant pi/omp/codex exemption branches and the codex UA parameter.

The internal ledger / nudge / preflight still use the real post-fold value regardless of display policy (the credit was always display-only). Added wire-level regression tests covering both the default-postfold plain-client path (the opencode fix) and the baseline opt-in control, plus a parser unit test.

Pre-flight: npm run typecheck clean · npm test 1319 pass / 0 fail / 2 skipped (e2e-gated) · npm run build success.

🔗 #691

中文摘要：把 #408 的宿主用量回填默认值从「未压缩基线」翻转为「实际转发的折叠用量」（对任意宿主都合理，顺带修掉 opencode 的 >100% 显示——它其实是普通代理客户端，加进 pluginAgent 豁免列表无效），基线改为按宿主可选开启（hostUsageCredit: "baseline"），并用「按宿主策略表 + 全局开关」取代逐宿主的 if 豁免链以终结打地鼠；旧配置值 auto/off 保持兼容。已开 PR #691，可合并（merge 需人工操作）。