diff --git a/CHANGELOG.md b/CHANGELOG.md index a13d1b19..68689cb8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,7 @@ Versions follow the merge of a `*_release-v*` branch; CI publishes to npm on tag ### Fixes - **Round-evidence closure for compress-reasoning drop (#651, #348 twin)**: the closure gate shipped with the reasoning drop — "compress call followed by a genuine user message" — is unreachable in long agentic sessions (no user messages after the opening prompt, observed: 0 drops while 30 compress rounds retained 20.6K/8.4K/10.6K-char thinking floors). A round now closes on tool-result evidence: the compress call's `tool-result` (matching `toolCallId`) exists at a later index and at least one message follows it. In-flight rounds (result missing or still the last message) stay untouched; per-provider `compress.providers..reasoning.drop=false` escape hatch preserved for reasoning-replay models (GLM). Mirrors billion-context-pi #348 (PR #349). +- **`hostUsageCredit` config switch: opt plain proxy clients out of the #408 uncompressed-baseline usage backfill (#648)**: plain anthropic proxy clients (ZCode — base-url → proxy, no `x-bili-plugin` header, no special UA) fell through every #408 backfill exemption (pi/omp by `pluginAgent`, codex by UA in #647) and got the full uncompressed-baseline backfill armed, so their UI showed a cumulative, drifting baseline (real folded value + per-compression backfill) instead of the actually-forwarded context — overstating real pressure and going non-monotonic on pure-append turns (est drift). New `hostUsageCredit` option (`"auto" | "off"`, default `auto` = current behavior; `BILI_HOST_USAGE_CREDIT` env / `hostUsageCredit` file key). `off` disables the #408 backfill entirely so the usage reported to the host is the actually-forwarded (folded) request, matching `[acp-usage] input=`. `auto` keeps today's behavior for everyone; ZCode users set `off` to get the actually-forwarded value. - **Lenient compress-arg parsing: salvage single-quoted JSON before hard rejection (#603)**: weak local models (reported via omp#121) emit `compress` args with single quotes (`{'content':[{'startId':...}]}`) — a malformation class the kernel's salvage ladder (fences, trailing commas, raw newlines, double-stringification, truncated/prose-wrapped arrays) does not cover, so the whole call was rejected `kind=malformed-json`, the round was wasted, and the model saw a FAILED result that can trigger tag-echoing. `parseCompressInput` now retries once through a quote-normalization pass when the kernel recovers zero ranges or reports invalid items: a state machine converts single-quoted strings to double-quoted ones (apostrophes inside double-quoted values are data and are copied verbatim; control characters inside single-quoted regions become JSON escapes), applied to raw-string args and to object inputs whose `content` value is a stringified array. The retry wins only when it recovers strictly more ranges — valid input is never rewritten — and salvaged ranges pass the same ref-validation gate as any other range, so the worst case is a wasted round, never a wrong compression. A `[acp-compress-input] quote-salvage: recovered N range(s)` warn logs each recovery for attribution. - **Forward-once-then-learn for image-dominated payloads — no false 502 on pixel-tile upstreams (#496)**: the default per-image estimate (`base64 length / 4`, uncapped) matches byte-billing relays but overestimates pixel-tile upstreams (official Anthropic/OpenAI) by up to ~200×, so a session whose *estimated* image floor alone exceeded the window was hard-failed with a 502 `preflight_compress_failed` ("Images alone account for ~N tokens") even though the real cost was a few K tokens — a regression vs master for official-API multimodal users (e.g. `bili claude` pasting screenshots). The fit gate now forwards ONCE instead of hard-failing when the over-window is attributable solely to the image estimate (`textEstimate < limit`), there is no upstream evidence of overflow yet (`lastInputTokens < limit` and no learned limit for the model), and images are present. The upstream then arbitrates billing: a pixel-tile upstream accepts (usage reports small → zero behavior change); a byte-billing relay rejects once, the existing self-heal learns the true window, and every subsequent request fails fast with an accurate message — exactly one rejected forward, strictly better than master's infinite 400 loop, no new knob. Also documents `BILI_IMAGE_TOKEN_CAP` (per-image estimate cap) in CONFIGURATION. diff --git a/CONFIGURATION.md b/CONFIGURATION.md index 6e3ad8dd..93274ab9 100644 --- a/CONFIGURATION.md +++ b/CONFIGURATION.md @@ -406,6 +406,7 @@ Environment variables take precedence over the config file. They are useful for | `ACP_LOG` | Set to `0` to disable request logging. | | `ACP_AUTO_UPDATE` | Set to `0` to disable auto-update checks. | | `ACP_UPDATE_TAG` | Dist-tag channel the auto-updater follows (default `latest`, e.g. `dev`). File-config key: `updateTag`. A `pr-N` preview tag is only followed when explicitly configured. | +| `BILI_HOST_USAGE_CREDIT` | `#408` host-usage backfill mode (file-config key: `hostUsageCredit`). `auto` (default) = the uncompressed-baseline backfill is armed for plain proxy clients (the bili-launched pi/omp extensions are exempted — their host-side compaction is cancelled, so the baseline drives nothing there). `off` = never backfill — the usage reported to the host is the actually-forwarded (folded) request, matching `[acp-usage] input=`. Use `off` for plain anthropic proxy clients (e.g. ZCode) whose UI would otherwise show the cumulative, drifting baseline as inflated context (#648). | | `ACP_PROVIDERS` | Path to an external `providers.json` (legacy / shared file). | | `BILI_REPLAY_RETRY_BASE_MS` | Base backoff delay (ms) for acp-loop replay retries after a transient upstream rejection (default `1500`; set `0` to disable the delay). See #189. | | `BILI_REPLAY_RETRY_MAX` | Total attempts for acp-loop replay retries (default `3`; set `1` to disable retries entirely — legacy fail-fast behavior). See #189. | diff --git a/src/config.ts b/src/config.ts index 2ba810e7..0f45759b 100644 --- a/src/config.ts +++ b/src/config.ts @@ -309,6 +309,15 @@ export type ProxyOptions = { autoUpdate: boolean; /** Dist-tag channel the auto-updater follows (default "latest"). */ updateTag: string; + /** #408 host-usage backfill mode. "auto" (default) = the uncompressed- + * baseline backfill is armed for plain proxy clients (the bili-launched + * pi/omp extensions are exempted — their host compaction is cancelled, so + * the baseline drives nothing on the host side). "off" = never backfill — + * the usage reported to the host is the actually-forwarded (folded) + * request, matching [acp-usage] input= (#648: plain anthropic proxy + * clients like ZCode otherwise show a cumulative, drifting baseline that + * overstates real context pressure). */ + hostUsageCredit: "auto" | "off"; logFile?: string; /** MITM transparent-proxy mode. When enabled, an HTTP CONNECT handler is * attached so clients that only know how to set HTTP_PROXY (ZCode with a @@ -446,6 +455,7 @@ export function loadOptions(env: NodeJS.ProcessEnv = process.env): ProxyOptions passthrough: passthrough.enabled, passthroughSource: passthrough.source, autoUpdate: (env.ACP_AUTO_UPDATE ?? (fileConfig.autoUpdate === false ? "0" : "1")) !== "0", + hostUsageCredit: parseHostUsageCredit(env.BILI_HOST_USAGE_CREDIT ?? fileConfig.hostUsageCredit), updateTag: (env.ACP_UPDATE_TAG ?? fileConfig.updateTag ?? "latest").trim() || "latest", logFile: env.ACP_LOG_FILE !== undefined ? (env.ACP_LOG_FILE || undefined) : fileConfig.logFile, mitm: { @@ -480,6 +490,7 @@ type FileConfig = { autoUpdate?: boolean; /** Dist-tag channel the auto-updater follows (default "latest"). */ updateTag?: string; + hostUsageCredit?: "auto" | "off"; upstreamProxy?: string; upstreamProxyMode?: string; logFile?: string; @@ -590,6 +601,10 @@ export function parseUpstreamProxyMode(value: string | undefined): UpstreamProxy return value === "manual" || value === "auto" ? value : "direct"; } +export function parseHostUsageCredit(value: string | undefined): "auto" | "off" { + return value === "off" ? "off" : "auto"; +} + export function parseCompressSettings(v: unknown): (CompressSettings & { injectTool?: boolean; injectNudge?: boolean }) | undefined { if (!v || typeof v !== "object" || Array.isArray(v)) return undefined; const obj = v as Record; diff --git a/src/server.ts b/src/server.ts index f00f367d..f6d507c7 100644 --- a/src/server.ts +++ b/src/server.ts @@ -1822,7 +1822,7 @@ function diagNudge(turn: { nudge?: { shouldInject: boolean; reason: string; cont return `[${sessionId}] nudge ${inject}: usage=${pct} (${tokenCount}/${limit}), growth=${growth}/${floor} (ref=${ref}, interval=${interval}), pendingT1=${pendingT1}/${interval}${modelTag}, reason="${n.reason.slice(0, 120)}"`; } -// #408/#590/#623: hosts that get the #408 uncompressed-baseline usage +// #408/#590/#623/#648: hosts that get the #408 uncompressed-baseline usage // backfill. pi's AND omp's bili extensions cancel the host's NATIVE compaction // so ACP owns compression — pi cancels auto-compaction, omp cancels ALL // compaction (its session_before_compact event carries no reason field, so @@ -1832,14 +1832,23 @@ function diagNudge(turn: { nudge?: { shouldInject: boolean; reason: string; cont // (#590 pi 302.7%, #623 omp 205%). Gate on pluginAgent so ONLY the // bili-launched extensions are exempted: plain proxy clients and codex // native-compact interception keep the #408 behavior (their native compaction -// stays live and consumes the baseline). +// stays live and consumes the baseline). The `hostUsageCredit` config option +// (#648) additionally lets a plain proxy client opt out entirely ("off") — +// ZCode and similar plain anthropic clients otherwise show the cumulative, +// drifting baseline as inflated context in their UI. function armHostUsageCredit( session: Session, originalMessages: CoreMessage[], processedMessages: CoreMessage[], + hostUsageCredit: "auto" | "off", log: (level: string, msg: string) => void, ): void { session.hostCreditTokens = 0; + // #648: "off" disables the #408 uncompressed-baseline backfill — the host + // sees the actually-forwarded (folded) request, matching [acp-usage] + // input=. Plain proxy clients (ZCode) otherwise show a cumulative, + // drifting baseline that overstates real context pressure. + if (hostUsageCredit === "off") return; if (session.metadata.pluginAgent === "pi" || session.metadata.pluginAgent === "omp") return; // #408: tokens folded out of the forwarded view vs the host's own (unfolded) // view — added back into the usage reported to the host so its anchor @@ -1990,7 +1999,7 @@ function prepareAnthropic( // identity chain (#268), not part of the Anthropic Messages API — strip it // so the real upstream never sees a field it doesn't know. delete (rebuilt as Record).prompt_cache_key; - armHostUsageCredit(session, originalMessages, processedMessages, log); + armHostUsageCredit(session, originalMessages, processedMessages, opts.hostUsageCredit, log); return { body: JSON.stringify(rebuilt), session, processedMessages, originalMessages, anthropicSystem: parsed.system, protocol: "anthropic", stream, compressInjected: injectTools, pluginMode, nudge, prompts, renderTags: "text-only" } as Prepared; } @@ -2245,7 +2254,7 @@ function prepareOpenai( if (stream && (rebuilt as Record).stream_options === undefined) { (rebuilt as Record).stream_options = { include_usage: true }; } - armHostUsageCredit(session, originalMessages, processedMessages, log); + armHostUsageCredit(session, originalMessages, processedMessages, opts.hostUsageCredit, log); // #532: title-gen side requests carry their own tiny system and would // clobber the conversation's measured overhead — skip them. if (!isTitleGen && openaiOutboundSystem !== undefined) { @@ -2501,7 +2510,7 @@ function prepareResponses( }); log("info", `[${sessionId}] responses forward tools=[${fwdTools.join(",")}] injectTool=${injectTools}${pluginMode ? " (plugin mode: wire injection suppressed)" : ""} NO_INJECT_TOOL=${!!process.env.ACP_NO_INJECT_TOOL} NO_COMPRESS_PROMPT=${!!process.env.ACP_NO_COMPRESS_PROMPT}`); } - armHostUsageCredit(session, originalMessages, processedMessages, log); + armHostUsageCredit(session, originalMessages, processedMessages, opts.hostUsageCredit, log); // #532: measure the outbound developer(system)+tools overhead for the panel. // On this wire the system rides the injected developer message outside the // fold space, so counting devContent + tools does not double-count the diff --git a/tests/host-usage-backfill.test.ts b/tests/host-usage-backfill.test.ts index fdd4cdf4..bf0a5981 100644 --- a/tests/host-usage-backfill.test.ts +++ b/tests/host-usage-backfill.test.ts @@ -7,7 +7,7 @@ import http from "node:http"; import { once } from "node:events"; import type { Config, CoreMessage } from "acp-kernel"; import { createCore, createInitialState, defaultConfig } from "acp-kernel"; -import type { Session } from "../src/session.ts"; +import { listSessions, _resetSessionsForTest, type Session } from "../src/session.ts"; import { runCompressLoop, createResponsesAdapter, createOpenaiAdapter, createAnthropicAdapter } from "../src/loop/index.ts"; import { backfillHostUsage, promptInputTotal, usageTotals } from "../src/util.ts"; import { pipePluginChatWithStrip, pipePluginResponsesWithStrip, pipePluginJson, _resetPluginStateForTest } from "../src/plugin.ts"; @@ -748,3 +748,165 @@ test("#623: omp plugin mode reports folded usage — host backfill suppressed", await new Promise((resolve, reject) => relay.close((e) => (e ? reject(e) : resolve()))); } }); + +// #648: ZCode — a plain proxy client on the anthropic wire (no x-bili-plugin +// header, no special UA) — must be able to opt out of the #408 +// uncompressed-baseline backfill via hostUsageCredit: "off", reporting the +// folded request's own usage (matching [acp-usage] input=). The control test +// pins the other side of the gate: an identical plain client on the default +// (hostUsageCredit: "auto") still gets the #408 backfill. The fold is real +// (the relay emits a compress tool_use), not a vacuous pass. + +const ZCODE_CONV_648 = "zcode-usage-648"; + +function zcodeSse(event: string, data: unknown): string { + return `event: ${event}\ndata: ${JSON.stringify(data)}\n\n`; +} + +function zcodeCompressToolUse(): string { + const args = JSON.stringify({ + content: [{ startId: "m00001", endId: "m00002", topic: "setup", summary: "MAIN-SUMMARY-SETUP-CONTEXT-FOLDED-BY-COMPRESSION-LONG-ENOUGH-FOR-KERNEL-MIN-LENGTH-CHECK" }], + }); + return [ + zcodeSse("content_block_start", { type: "content_block_start", index: 0, content_block: { type: "tool_use", id: "toolu_zcode_1", name: "compress", input: {} } }), + zcodeSse("content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "input_json_delta", partial_json: args } }), + zcodeSse("content_block_stop", { type: "content_block_stop", index: 0 }), + ].join(""); +} + +function zcodeNormalCompletion(inputTokens: number): string { + return [ + zcodeSse("message_start", { type: "message_start", message: { id: "msg_zcode", role: "assistant", usage: { input_tokens: inputTokens, output_tokens: 3 } } }), + zcodeSse("content_block_start", { type: "content_block_start", index: 0, content_block: { type: "text", text: "" } }), + zcodeSse("content_block_delta", { type: "content_block_delta", index: 0, delta: { type: "text_delta", text: "ok" } }), + zcodeSse("content_block_stop", { type: "content_block_stop", index: 0 }), + zcodeSse("message_delta", { type: "message_delta", delta: { stop_reason: "end_turn", stop_sequence: null }, usage: { output_tokens: 3 } }), + zcodeSse("message_stop", { type: "message_stop" }), + ].join(""); +} + +// 10 messages with filler; the sentinel sits in m00002 (the assistant message +// of the compressed head) so the fold is real and the post-fold upstream body +// provably drops the head content. +function zcodeConversation(): Array<{ role: string; content: string }> { + const headFiller = "lorem ipsum dolor sit amet consectetur adipiscing elit sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. ".repeat(28); + const tailFiller = "enim ad minim veniam quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat duis aute irure dolor in reprehenderit in voluptate. ".repeat(28); + const history: Array<{ role: string; content: string }> = []; + for (let i = 1; i <= 2; i++) { + history.push({ role: "user", content: `turn-${i}-marker question: ${headFiller}` }); + history.push({ role: "assistant", content: `turn-${i}-marker ${i === 1 ? "SENTINEL_FOLD_GONE " : ""}answer: ${headFiller}` }); + } + for (let i = 3; i <= 5; i++) { + history.push({ role: "user", content: `turn-${i} padding question: ${tailFiller}` }); + history.push({ role: "assistant", content: `turn-${i} padding answer: ${tailFiller}` }); + } + return history; +} + +async function withZCodeHarness(hostUsageCredit: "auto" | "off", fn: (h: { proxy: http.Server; upstream: http.Server; bodies: string[]; url: string }) => Promise): Promise { + const bodies: string[] = []; + const upstream = http.createServer((req, res) => { + const chunks: Buffer[] = []; + req.on("data", (c: Buffer) => chunks.push(c)); + req.on("end", () => { + bodies.push(Buffer.concat(chunks).toString("utf8")); + res.writeHead(200, { "content-type": "text/event-stream", "cache-control": "no-cache" }); + if (bodies.length === 1) { + res.write(zcodeSse("message_start", { type: "message_start", message: { id: "msg_zcode_1", role: "assistant", usage: { input_tokens: 1000, output_tokens: 3 } } })); + res.write(zcodeCompressToolUse()); + res.write(zcodeSse("message_delta", { type: "message_delta", delta: { stop_reason: "tool_use", stop_sequence: null }, usage: { output_tokens: 3 } })); + res.write(zcodeSse("message_stop", { type: "message_stop" })); + } else { + res.write(zcodeNormalCompletion(1000)); + } + res.end(); + }); + }); + upstream.listen(0, "127.0.0.1"); + await once(upstream, "listening"); + const upstreamPort = (upstream.address() as { port: number }).port; + _setStoreForTest(new SessionStore({ enabled: false })); + _resetSessionsForTest(); + setRegistryForTest({}); + const proxy = await startServer({ + port: 0, + host: "127.0.0.1", + upstream: "http://127.0.0.1", + routes: { [`http://127.0.0.1:${upstreamPort}`]: { models: { "claude-test": { context: 100_000 } } } }, + modelContextLimit: 100_000, + kernelConfig: defaultConfig(100_000), + compress: { injectTool: true, injectNudge: false }, + promptCache: { routing: "auto" }, + sessionHeader: "x-acp-session", + log: false, + debug: false, + passthrough: false, + autoUpdate: false, + hostUsageCredit, + mitm: { enabled: false, domains: [] }, + } as ProxyOptions); + await once(proxy, "listening"); + const proxyPort = (proxy.address() as { port: number }).port; + const h = { proxy, upstream, bodies, url: `http://127.0.0.1:${proxyPort}/bili/http://127.0.0.1:${upstreamPort}/v1/messages` }; + try { + await fn(h); + } finally { + proxy.close(); + await once(proxy, "close"); + upstream.close(); + await once(upstream, "close"); + } +} + +function zcodeInputTokensOf(raw: string): number { + const m = raw.match(/"input_tokens":(\d+)/); + assert.ok(m, `message_start usage missing: ${raw.slice(0, 400)}`); + return Number(m[1]); +} + +async function setupZCodeCompressedSession(h: { bodies: string[]; url: string }): Promise { + const r1 = await fetch(h.url, { + method: "POST", + headers: { "content-type": "application/json", "x-acp-session": ZCODE_CONV_648 }, + body: JSON.stringify({ model: "claude-test", max_tokens: 1024, stream: true, system: "You are a test assistant.", messages: zcodeConversation() }), + }); + assert.equal(r1.status, 200); + await r1.text(); + const s = listSessions().find((x) => x.meta.label === ZCODE_CONV_648); + assert.ok(s, "session exists"); + assert.ok((s!.state.blocks ?? []).some((b) => b.active), "setup created an active block (real fold)"); + assert.ok(h.bodies[0]!.includes("SENTINEL_FOLD_GONE"), "setup forwarded the unfolded head (sentinel present)"); + return h.bodies.length; +} + +test("#648: ZCode (anthropic wire, hostUsageCredit off) reports folded usage — host backfill suppressed", async () => { + await withZCodeHarness("off", async (h) => { + const afterSetup = await setupZCodeCompressedSession(h); + const r2 = await fetch(h.url, { + method: "POST", + headers: { "content-type": "application/json", "x-acp-session": ZCODE_CONV_648 }, + body: JSON.stringify({ model: "claude-test", max_tokens: 1024, stream: true, system: "You are a test assistant.", messages: zcodeConversation() }), + }); + assert.equal(r2.status, 200); + const raw = await r2.text(); + assert.equal(h.bodies.length, afterSetup + 1, "post-fold turn forwarded to upstream exactly once"); + assert.ok(!h.bodies[h.bodies.length - 1]!.includes("SENTINEL_FOLD_GONE"), "post-fold upstream body must not carry the folded head content"); + assert.equal(zcodeInputTokensOf(raw), 1000, "hostUsageCredit off must report the folded request's own usage — no uncompressed-baseline backfill (#648)"); + }); +}); + +test("#648 control: plain client (anthropic wire, hostUsageCredit auto) still gets the #408 backfill", async () => { + await withZCodeHarness("auto", async (h) => { + const afterSetup = await setupZCodeCompressedSession(h); + const r2 = await fetch(h.url, { + method: "POST", + headers: { "content-type": "application/json", "x-acp-session": ZCODE_CONV_648 }, + body: JSON.stringify({ model: "claude-test", max_tokens: 1024, stream: true, system: "You are a test assistant.", messages: zcodeConversation() }), + }); + assert.equal(r2.status, 200); + const raw = await r2.text(); + assert.equal(h.bodies.length, afterSetup + 1, "post-fold turn forwarded to upstream exactly once"); + assert.ok(!h.bodies[h.bodies.length - 1]!.includes("SENTINEL_FOLD_GONE"), "post-fold upstream body must not carry the folded head content"); + assert.ok(zcodeInputTokensOf(raw) > 1000, "plain proxy client with hostUsageCredit auto must still see the uncompressed baseline (#408)"); + }); +});