fix(guardrail): generic tool-call repetition breaker (closes #308) - #311
Conversation
Greedy-decoding small models can loop on a byte-identical tool call
(e.g. acp_status {"scope":"uncompressed","view":"ranges"} x30), a
sequence-level attractor that token penalties cannot break and that the
compress-failure breaker never counts. Add a per-session consecutive-run
fingerprint guard: sha1(toolName + key-sorted canonical JSON(args)) —
arguments only, result content ignored. At warn (default 3) append a strong
warning to the matching toolResult; at abort (default 5) block the call,
abort the turn, and notify the terminal. Any arg/tool change or real user
message resets the run. Config: acp.json { "repetitionGuard": {enabled,warn,abort} },
default on.
ranxianglei
left a comment
There was a problem hiding this comment.
审查结论(来自 #308 提交者):LGTM,可合并
验证:全测试通过(484/0)、tsup 构建成功、与 #310 无文件冲突; 为 pi 官方支持的 tool_call 拦截方式(extensions.md "Can block")。
对照 issue 规格逐项确认:指纹 = sha1(toolName+键排序 canonical JSON)✓;只比参数不比结果✓;warn(3) 追加到 toolResult 尾部✓;abort(5) 拦截不执行 + ctx.abort() + 终端通知✓;参数变化/换工具/真实用户消息重置、extension 消息不重置✓;acp.json 可配 {enabled,warn,abort}✓;abort≤warn 时 clamp 到 warn+1✓。测试覆盖升级/重置/阈值/边界,事故形态(acp_status 同参 30 连发)会在第 3 次收到警告、第 5 次被拦。
后续可选项(不阻塞):
- 当前只统计严格连续相同指纹——A→B→A 交替环不会触发。已知三起事故均为纯连续,暂够;将来可改为滑动窗口内按指纹计数。
- issue #308 建议第 2 条(无进展工具调用计入 per-turn cap、nudge 增长放宽、循环段移出保护带)本 PR 未做,可另开小 PR。
ranxianglei
left a comment
There was a problem hiding this comment.
审查结论(来自 #308 提交者):LGTM,可合并
验证:全测试通过(484/0)、tsup 构建成功、与 #310 无文件冲突;{block:true,reason} 为 pi 官方支持的 tool_call 拦截方式(extensions.md "Can block")。
对照 issue 规格逐项确认:指纹 = sha1(toolName+键排序 canonical JSON)✓;只比参数不比结果✓;warn(3) 追加到 toolResult 尾部✓;abort(5) 拦截不执行 + ctx.abort() + 终端通知✓;参数变化/换工具/真实用户消息重置、extension 消息不重置✓;acp.json 可配 {enabled,warn,abort}✓;abort≤warn 时 clamp 到 warn+1✓。测试覆盖升级/重置/阈值/边界,事故形态(acp_status 同参 30 连发)会在第 3 次收到警告、第 5 次被拦。
后续可选项(不阻塞):
- 当前只统计严格连续相同指纹——A→B→A 交替环不会触发。已知三起事故均为纯连续,暂够;将来可改为滑动窗口内按指纹计数。
- issue #308 建议第 2 条(无进展工具调用计入 per-turn cap、nudge 增长放宽、循环段移出保护带)本 PR 未做,可另开小 PR。
ranxianglei
left a comment
There was a problem hiding this comment.
审查结论(来自 #308 提交者):LGTM,可合并
验证:全测试通过(484/0)、tsup 构建成功、与 #310 无文件冲突;{block:true,reason} 为 pi 官方支持的 tool_call 拦截方式(extensions.md "Can block")。
对照 issue 规格逐项确认:指纹 = sha1(toolName+键排序 canonical JSON)✓;只比参数不比结果✓;warn(3) 追加到 toolResult 尾部✓;abort(5) 拦截不执行 + ctx.abort() + 终端通知✓;参数变化/换工具/真实用户消息重置、extension 消息不重置✓;acp.json 可配 {enabled,warn,abort}✓;abort≤warn 时 clamp 到 warn+1✓。测试覆盖升级/重置/阈值/边界,事故形态(acp_status 同参 30 连发)会在第 3 次收到警告、第 5 次被拦。
后续可选项(不阻塞):
- 当前只统计严格连续相同指纹——A→B→A 交替环不会触发。已知三起事故均为纯连续,暂够;将来可改为滑动窗口内按指纹计数。
- issue #308 建议第 2 条(无进展工具调用计入 per-turn cap、nudge 增长放宽、循环段移出保护带)本 PR 未做,可另开小 PR。
# Conflicts: # CHANGELOG.md
📦 Built Extension ArtifactBranch: Option A — Install from npm PR tag (recommended)pi install npm:billion-context-pi@pr-311Each push to this PR publishes a new version under the Option B — Download artifact
tar xzf billion-context-pi-pr311.tgz
pi install ./packageThis comment is automatically updated on each push. |
|
Resolved the CHANGELOG.md conflict against master (which now carries #310: acp-kernel 0.0.55 + summary-fidelity): kept both sides' Unreleased entries. Post-merge verification on this branch: build OK, 589 pass / 0 fail / 3 skipped with kernel 0.0.55 — repetition-guard tests unaffected by the kernel bump. |
Problem
Fixes #308 — greedy-decoding small models get stuck re-emitting a byte-identical tool call every turn (
acp_status {"scope":"uncompressed","view":"ranges"}×30, context 57K→130K). This is a sequence-level attractor: the repetition crosses turn boundaries, so token-level penalties are powerless, and none of the existing defenses catch it:compress-retry-cappedbreaker only countscompressfailures (collectCompressOutcomesinsrc/index.tsskips non-compress results) —acp_statusalways succeeds → zero count;turnKey— during the loop there's no new user message, soturnKeystays constant and every non-emergency nudge is suppressed;src/tool-guardrails.tspreviously had only the bash output cap + timeout detection, no repetition detection.Fix (owner spec item #1 — core)
A per-session consecutive-run fingerprint guard in
src/tool-guardrails.ts:sha1(toolName + canonical-JSON(args)), where the JSON serialization sorts object keys → compares arguments only; key order and result content are ignored (the visible message count inside anacp_statusresult varies 24→26→28, but identical args still match);warn(default 3): append a strong warning to the matching toolResult ("stop issuing this identical call — change arguments/approach, or stop");abort(default 5): block the call (not executed), abort the current turn, and notify the terminal;Config:
acp.json { "repetitionGuard": { "enabled": true, "warn": 3, "abort": 5 } }— default on. Resolution lives insrc/config.ts(resolveRepetitionGuard,REPETITION_GUARD_DEFAULTS);abortis clamped to be> warn.Deferred (intentionally not in this PR)
The other three suggestions from the issue are deliberately split out:
acp_status/decompress/search_contextrepeats); the extra "tokenCount no-drop" signal is weaker and adds little value now.Happy to track these as follow-up issues if you want them.
Tests
+22 tests, all green (full suite 484 pass / 0 fail across repeated runs):
canonicalStringify(key-order independence, array ordering preserved, undefined-value drop, nesting),repetitionFingerprint(stable across key order, differs on arg change / tool change);RepetitionTracker: escalation none→warn→abort, reset on a different fingerprint (args or tool name),reset();pi+ctx): warn@3 without blocking, block+abort@5 (turn aborted exactly once, still blocked after), argument-change reset, user-input-resets-but-extension-input-does-not, fully inert whenrepetitionGuard: false, custom{warn:2, abort:3}thresholds.Docs updated:
CONFIGURATION.md/CONFIGURATION.zh-CN.md(new "Tool Call Repetition Guard" section + summary tables) andCHANGELOG.md.