Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# Changelog

## Unreleased (master, since v0.1.38)
- **feat(rollover): 批量 rollover 压缩 — Prompt Cache 稳定性主杠杆 (closes #241)** — 就地压缩的真实大头是 Prompt Cache 失效(#80 提出、#240 确认):历史中间改写把压缩点之后的整个后缀踢出缓存前缀,后续每轮全价重算。rollover 模式(**默认开启**,`"rollover": false` 恢复旧行为)让 model-visible history 阶段内 append-only:`compress` 立即校验范围但只记录 pending(原文保持可见);新 `absorb` 工具把大型 tool output 蒸馏成模型自写的摘要、原文延迟到批量时移除;用量越过 `rollover.threshold`(默认 70%,低于 75% 强制 nudge 带)时一次性批量应用全部 pending(一次缓存失效摊薄整段),追加一次性 `▣ ACP rollover` 报告;`decompress`/`search_context` 结果只追加到尾部。pending 跨重启持久化(`.acp.json` 的 `rolloverPending` + `absorbed` 记录修复:`mergeInitialState` 原本丢弃 `absorbed` 字段)、`acp_status` 显示 pending 行、`/acp-rollover` 命令强制立即应用。kernel 侧零改动(全部 adapter 侧:pending 状态、延迟 apply、隐藏 pending compress 调用的 restore 修复——KEEP_LAST_ORPHANED=2 会把第 3 个 pending 调用从历史中间隐藏、破坏前缀稳定)。测试:新增 tests/rollover.test.ts 5 个(阶段内字节稳定含 ≥3 pending、阈值触发恰好一次改写后重新锚定、absorb 原文可见直到批量、检索结果纯尾部追加、手动 /acp-rollover);既有 60 处 createAcpExtension 调用显式 `rollover: false` 保留旧行为覆盖;prefix-stab 回归测试同处理
- **fix(nudge): 移除瞬态 compress 重试提示注入(closes #223,取代 #217)** — compress 失败后每次 LLM 调用重注入的 `compressRetryMessage` 瞬态 user 提示整体移除:对从不重试的模型,该提示无限追加(用户日志 ~400 次/小时、emergency pct 95→127%),即 #223 的"永远追加失败标记"。失败信息本身仍以 toolResult 形式持久留在 session 日志中(模型可见、可自我纠正,随正常压缩流程淘汰);issue #6 的 nudge 断路器保留:每用户轮 MAX_COMPRESS_ATTEMPTS=3 次失败/no-op 后 emergency nudge 停止重注入(kernel 紧急截断仍机械兜底),UI 提示改为 "nudge paused until the next user message (emergency truncation still active)"。`noteCompressOutcomes` 返回值去掉 `retryFor`
- **fix(tokens): 图片 token 计入发送视图估算 (closes #200)** — `extractText` 只投影 `type:"text"` 块,图片在 sent-view 估算中计 0 token:含图会话的 nudge/truncation/compress 仲裁系统性偏晚(只等真实 400 后 overflow-selfheal 被动触发),且 density 校准被 phantom gap 污染(provider 真实 usage 含图、估算不含 → 图片轮 dReal/dEst 爆表被 clamp 到 2.5×,纯文本轮又拉回 1.0,density 振荡且仍低估 5-10× → 过早/过晚压缩交替)。现在 `collectImageTokens` 按 `IMAGE_TOKEN_COST=1600`/张计入(仅视觉模型,`model.input` 含 `image`;非视觉模型 pi-ai 静默丢图、计 0),density 校准环自动收敛真实成本(A/B 实测收敛 ~0.98);出站 payload 字节不变(sha256 一致),前缀缓存不受影响 (#201)
- **fix(delegate): 并发多 agent 时失败必达,不再静默 (#16)** — async delegate 此前有三条失败路径完全不通知主模型(spawn error、结果持久化 error、`sendUserMessage` 注入丢失),模型未挂在 `acp_delegate_wait` 上时失败被吞,直到收尾汇总才发现少了结果。现在:所有终止路径 best-effort 注入 `FAILED ⚠️` 通知(带错误摘录,与 sync 路径对齐,明确提示"该任务结果缺失、收尾前决定是否重派");注入失败的 run 进入未送达集,随**下一个** delegate 通知或任何 delegate 工具结果(`acp_delegate`/`wait`/`cancel`)捎带 Recovery notice 补投;Recovery notice 的 delivered 标记改为 carrier 发送成功后才提交(发送抛错不再永久吞掉其他 run 的结果);system prompt 补充 FAILED/Recovery 通知说明
Expand Down
55 changes: 55 additions & 0 deletions CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,14 @@ All keys below are currently **ACTIVE**.
| `compress.emergencyThresholdPercent` | number \| string | `"95%"` | 🟢 ACTIVE | Context threshold that triggers emergency truncation. |
| `compress.nudgeGrowthTokens` | number | `50000` | 🟢 ACTIVE | Token growth step for soft compression nudges. |

**Rollover keys**

| Key | Type | Default | Status | Description |
|-----|------|---------|--------|-------------|
| `rollover` | boolean \| object | `true` | 🟢 ACTIVE | Batch rollover mode: `compress`/`absorb` are deferred and applied in one batch rewrite when context pressure crosses the threshold. |
| `rollover.enabled` | boolean | `true` | 🟢 ACTIVE | Enable batch rollover mode. `false` restores the legacy immediate in-place compression. |
| `rollover.threshold` | number \| string | `"70%"` | 🟢 ACTIVE | Context-usage threshold at which pending compressions/absorbs are applied in one batch. |

**Prompts keys**

| Key | Type | Default | Status | Description |
Expand Down Expand Up @@ -340,6 +348,53 @@ On `anthropic` / `claude-sonnet-4-5` the effective thresholds become `maxContext

---

## Rollover (Prompt Cache Stability)

Batch rollover mode — **on by default** — trades a little temporary context for dramatically fewer prompt-cache invalidations. In legacy mode every `compress` call rewrites the model-visible history in place, evicting the entire suffix after the compression point from the provider's cache prefix; every subsequent round re-pays full price for that suffix. Rollover mode makes history **append-only within a phase**:

1. **Deferred compress** — a `compress` call validates its ranges immediately (bad ranges still fail now, with errors) but only *records* them as pending. The range stays visible until the batch applies.
2. **`absorb` tool** — distills a large tool result into a compact summary you write; the original output is marked pending drop and stays visible until the batch applies (the summary is the durable record).
3. **Batch rollover** — when context usage crosses `rollover.threshold` (default 70%), all pending compressions and absorbs are applied in **one** rewrite: one cache invalidation, amortized over the whole phase. A one-shot `▣ ACP rollover | ...` report is appended to that round.
4. **Retrieval appends to the tail** — `decompress` / `search_context` results are tool results at the end of the history; the prefix is never touched.

Pending work is visible in `acp_status` (`Rollover: N pending ...` line) and survives restarts (persisted alongside the ACP state). To force the batch early, run `/acp-rollover`.

### Trade-off

Pending content occupies context until the rollover fires — that is the price of a stable cache prefix. The default threshold (70%) sits **below** the forced-nudge band (`compress.maxContextLimit`, 75%) so the rollover always applies before the nudge escalation, and the reclaimed tokens (typically tens of percent of the window) drop usage well back below the band in one step.

### `rollover`

- **Type:** `boolean | object`
- **Default:** `true`
- **Status:** 🟢 ACTIVE
- **Description:** Enable batch rollover mode. `false` (or `{"enabled": false}`) restores the legacy behavior where every `compress` call rewrites history in place immediately.

### `rollover.enabled`

- **Type:** `boolean`
- **Default:** `true`
- **Status:** 🟢 ACTIVE
- **Description:** Same as the `rollover` shorthand. `false` disables deferred compression: `compress` applies immediately, the `absorb` tool is not registered, and the rollover system-prompt section is omitted.

### `rollover.threshold`

- **Type:** `number | string`
- **Default:** `0.70` (or `"70%"`)
- **Status:** 🟢 ACTIVE
- **Description:** Context-usage threshold at which pending compressions/absorbs are applied in one batch. Accepts a ratio (`0.70`) or a percent string (`"70%"`). Keep it **below** `compress.maxContextLimit` so the rollover fires before forced nudges start. A higher value keeps the prefix stable longer at the cost of carrying more pending context; a lower value reclaims sooner.

```json
{
"rollover": {
"enabled": true,
"threshold": "70%"
}
}
```

---

## Prompts Customization

The `prompts` object overrides acp-kernel's **load-bearing** compression prompt rules — the verbatim instructions the model receives about *how* to write summaries (keep full file paths, function signatures, decisions and rationale; drop verbose logs, etc.). These four fields are embedded into the system prompt and the compression nudge text:
Expand Down
55 changes: 55 additions & 0 deletions CONFIGURATION.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,14 @@
| `compress.emergencyThresholdPercent` | number \| string | `"95%"` | 🟢 ACTIVE | 触发紧急截断的上下文阈值。 |
| `compress.nudgeGrowthTokens` | number | `50000` | 🟢 ACTIVE | 软压缩 nudge 的 token 增长步长。 |

**Rollover 键**

| 键 | 类型 | 默认值 | 状态 | 说明 |
|----|------|--------|------|------|
| `rollover` | boolean \| object | `true` | 🟢 ACTIVE | 批量 rollover 模式:`compress`/`absorb` 延迟生效,在上下文压力越过阈值时一次性批量应用。 |
| `rollover.enabled` | boolean | `true` | 🟢 ACTIVE | 启用批量 rollover 模式。`false` 恢复旧的就地立即压缩行为。 |
| `rollover.threshold` | number \| string | `"70%"` | 🟢 ACTIVE | pending 压缩/吸收一次性批量应用的上下文用量阈值。 |

**prompts 键**

| 键 | 类型 | 默认值 | 状态 | 说明 |
Expand Down Expand Up @@ -340,6 +348,53 @@ provider 的 key 是 **Pi provider 名**(如 `"anthropic"`、`"openai"`、`"zhip

---

## Rollover(Prompt Cache 稳定性)

批量 rollover 模式 —— **默认开启** —— 用一点临时 context 换取大幅减少的 prompt-cache 失效次数。旧模式下每次 `compress` 都在历史中间就地改写,把压缩点之后的整个后缀踢出 provider 缓存前缀;之后每一轮都按全价重算这段 input。rollover 模式让 model-visible history 在阶段内 **append-only**:

1. **延迟压缩** —— `compress` 调用立即校验范围(坏范围仍当场报错),但只把范围*记录*为 pending。原文保持可见,直到批量应用。
2. **`absorb` 工具** —— 把大型工具输出蒸馏成你写的紧凑摘要;原文标记为 pending drop,保持可见直到批量应用(摘要是持久记录)。
3. **批量 rollover** —— 当上下文用量越过 `rollover.threshold`(默认 70%)时,所有 pending 压缩与吸收**一次性**应用:一次缓存失效,摊薄到整个阶段。该轮追加一条一次性 `▣ ACP rollover | ...` 报告。
4. **检索结果追加到尾部** —— `decompress` / `search_context` 的结果是历史末尾的 tool result,前缀永不被触碰。

pending 工作显示在 `acp_status`(`Rollover: N pending ...` 行)中,并跨重启持久化(与 ACP state 一起保存)。想提前强制批量,运行 `/acp-rollover`。

### 权衡

pending 内容在 rollover 触发前一直占用 context —— 这是换取稳定缓存前缀的代价。默认阈值(70%)位于强制 nudge 带(`compress.maxContextLimit`,75%)**之下**,保证 rollover 总在 nudge 升级之前应用;回收的 token(通常是窗口的百分之几十)一步就把用量拉回带内。

### `rollover`

- **类型:** `boolean | object`
- **默认值:** `true`
- **状态:** 🟢 ACTIVE
- **说明:** 启用批量 rollover 模式。`false`(或 `{"enabled": false}`)恢复旧行为:每次 `compress` 立即就地改写历史。

### `rollover.enabled`

- **类型:** `boolean`
- **默认值:** `true`
- **状态:** 🟢 ACTIVE
- **说明:** 与 `rollover` 简写相同。`false` 禁用延迟压缩:`compress` 立即生效,不注册 `absorb` 工具,系统提示词中也不含 rollover 段落。

### `rollover.threshold`

- **类型:** `number | string`
- **默认值:** `0.70`(或 `"70%"`)
- **状态:** 🟢 ACTIVE
- **说明:** pending 压缩/吸收一次性批量应用的上下文用量阈值。接受比例(`0.70`)或百分号字符串(`"70%"`)。保持**低于** `compress.maxContextLimit`,让 rollover 在强制 nudge 开始之前触发。值越高,前缀稳定时间越长,但携带的 pending context 越多;值越低,回收越早。

```json
{
"rollover": {
"enabled": true,
"threshold": "70%"
}
}
```

---

## 提示词自定义

`prompts` 对象覆盖 acp-kernel 的**承重**压缩提示词规则——即模型收到的关于*如何*写摘要的逐字指令(保留完整文件路径、函数签名、决策与理由;丢弃冗长日志等)。这四个字段被嵌入系统提示词和压缩 nudge 文本:
Expand Down
18 changes: 17 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,19 @@ Each message gets an invisible `<acp>` ref tag (`m00001`, `m00002`, ...) visible

Pi's built-in auto-compaction is cancelled — billion-context is the sole context manager.

## Batch rollover — Prompt Cache stability

The dominant cost of in-place compression is not the summary's output tokens — it is the **prompt-cache invalidation**: rewriting history mid-stream evicts the entire suffix after the compression point from the provider's cache prefix, and every later round re-pays full price for it.

Batch rollover mode (on by default) makes the model-visible history **append-only within a phase**:

- `compress` validates its ranges immediately but only **records** them as pending — the range stays visible.
- `absorb` distills a large tool result into a summary you write; the original stays visible until the batch applies.
- When context usage crosses the rollover threshold (default **70%**, below the 75% forced-nudge band), all pending work is applied in **one** rewrite — one cache invalidation, amortized over the whole phase — and a one-shot `▣ ACP rollover | ...` report is appended.
- `decompress` / `search_context` results land at the tail of the history; the prefix is never touched.

Pending work shows up in `acp_status` and survives restarts; `/acp-rollover` forces the batch early. Set `"rollover": false` in `acp.json` to restore the legacy immediate-compression behavior. See [CONFIGURATION.md](./CONFIGURATION.md#rollover-prompt-cache-stability) for the trade-off and thresholds.

## Plugin compatibility & ordering

billion-context takes over context management by intercepting Pi's `context` event. **Pi has no plugin priority mechanism** — when multiple extensions register handlers for the same event, they run in a fixed sequence (load order), with no `priority`/`weight` field and no way for the user to control the order. The `context` event specifically is a *pipeline*: every handler receives the previous handler's output, there is no short-circuit, and the **last** handler has the final say over what reaches the model.
Expand All @@ -76,7 +89,8 @@ This has two practical implications:

| Tool | What it does |
|------|-------------|
| `compress` | Replace a contiguous message range with a detailed summary |
| `compress` | Replace a contiguous message range with a detailed summary (deferred to the next rollover in batch mode) |
| `absorb` | Distill a large tool result into a compact summary; the original is dropped at the next rollover |
| `decompress` | Restore a previously compressed block's content |
| `search_context` | Search compressed block summaries (and visible messages) by keyword |
| `acp_status` | Show context usage, compressed blocks, compressible ranges |
Expand Down Expand Up @@ -134,6 +148,8 @@ Blocks: 3 active (3.7K summary, 15.2K original compressed)
b3 (T2) 3.3K→1.0K age=1m "Architecture review"
```

In batch rollover mode the status also shows pending work (`Rollover: N pending compression(s) + M absorb(s) — ~X tokens pending (threshold 70%, current Y%)`), and `/acp-rollover` applies the pending batch immediately instead of waiting for the threshold.

## `/acp-subagents` command

**Optional, one-time setup — only if you also use [pi-subagents](https://github.com/nicobailon/pi-subagents).**
Expand Down
18 changes: 17 additions & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,19 @@ assign refs → sync blocks → prune → filter → hide calls → recommend

Pi 内置的自动压缩会被取消 —— billion-context 是唯一的上下文管理者。

## 批量 rollover —— Prompt Cache 稳定性

就地压缩的主要成本不是摘要的 output tokens,而是 **Prompt Cache 失效**:在历史中间改写会把压缩点之后的整个后缀踢出 provider 的缓存前缀,之后每一轮都要按全价重算这段 input。

批量 rollover 模式(默认开启)让 model-visible history 在阶段内 **append-only**:

- `compress` 立即校验范围(坏范围仍然当场报错),但只把范围**记录**为 pending —— 原文保持可见。
- `absorb` 把大型工具输出蒸馏成你写的摘要;原文保持可见,直到批量生效。
- 当上下文用量越过 rollover 阈值(默认 **70%**,低于 75% 强制 nudge 带)时,所有 pending 工作**一次性**应用 —— 一次缓存失效,摊薄到整个阶段 —— 并追加一条一次性的 `▣ ACP rollover | ...` 报告。
- `decompress` / `search_context` 的结果落在历史尾部,前缀永不被触碰。

pending 工作会显示在 `acp_status` 中,并跨重启持久化;`/acp-rollover` 可立即强制批量生效。在 `acp.json` 中设置 `"rollover": false` 可恢复旧的就地立即压缩行为。权衡与阈值详见 [CONFIGURATION.zh-CN.md](./CONFIGURATION.zh-CN.md#rollover-prompt-cache-稳定性)。

## 插件兼容性与排序

billion-context 通过拦截 Pi 的 `context` 事件接管上下文管理。**Pi 没有插件优先级机制** —— 当多个扩展为同一个事件注册 handler 时,它们按固定顺序(加载顺序)执行,没有 `priority`/`weight` 字段,用户也无法控制顺序。`context` 事件尤其是一个*管线*:每个 handler 都接收上一个 handler 的输出,没有短路,**最后一个** handler 对发给模型的内容拥有最终决定权。
Expand All @@ -75,7 +88,8 @@ billion-context 通过拦截 Pi 的 `context` 事件接管上下文管理。**Pi

| 工具 | 作用 |
|------|------|
| `compress` | 用详细摘要替换连续的消息范围 |
| `compress` | 用详细摘要替换连续的消息范围(批量模式下延迟到下一次 rollover 生效) |
| `absorb` | 把大型工具输出蒸馏成你写的紧凑摘要;原文在下一次 rollover 时移除 |
| `decompress` | 恢复之前压缩的块内容 |
| `search_context` | 按关键词搜索已压缩块摘要(及可见消息) |
| `acp_status` | 显示上下文用量、已压缩块、可压缩范围 |
Expand Down Expand Up @@ -132,6 +146,8 @@ Blocks: 3 active (3.7K summary, 15.2K original compressed)
b3 (T2) 3.3K→1.0K age=1m "Architecture review"
```

批量 rollover 模式下,状态面板还会显示 pending 工作(`Rollover: N pending compression(s) + M absorb(s) — ~X tokens pending (threshold 70%, current Y%)`),`/acp-rollover` 可立即应用 pending 批量,而不必等待阈值。

## `/acp-subagents` 命令

**可选、一次性设置——仅当你同时使用 [pi-subagents](https://github.com/nicobailon/pi-subagents) 时需要。**
Expand Down
Loading
Loading