Skip to content

fix(find): maximal stable find prompt prefix for prompt caching - #46

Open
Kizunad wants to merge 1 commit into
mainfrom
fix/cache-find-prefix-stability
Open

Kizunad wants to merge 1 commit into
mainfrom
fix/cache-find-prefix-stability

Conversation

@Kizunad

@Kizunad Kizunad commented Aug 10, 2026 •

Copy link
Copy Markdown
Owner

Summary

Engine half of the CACHE-ENGINE-SIDE ownership verdict. AxonHub is the larger and gating half (its probe sent an explicit cache_control breakpoint on 3,326 tokens, well over the 1,024 minimum, and got cache_creation=0 — the gateway creates no entries even when asked); the engine half done here is the concrete "variable sitting in the cacheable prefix" fix, as forward-structure readiness.

  • Move the find prompt's Assigned taxonomy dimension: value from block 10/12 to the last block. Diff, summaries, policy, and every instruction are byte-identical across a batch's 8 lenses; only the taxonomy value differs. With the variable value last, a breakpoint placed before it can share the maximum prefix across lenses instead of only the ~2,100-token instructions+skill block.
  • Correct the stale comment at schemaFeedbackPrompt that asserted the upstream prefix cache "still covers" the expensive part — the relay currently creates no prompt-cache entries even when explicitly asked, so the append-only shape is forward-structure, not a live guarantee.
  • New regression guard test/find-prefix-stability.test.mjs locks the order: taxonomy value must be the final block, diff must precede it, and everything before the taxonomy value must be byte-identical across lenses.

Quantified conditional saving

  • Typical find call ≈ 3,326 tokens (iteration-seat probe). Cross-lens shareable prefix grows from ~2,100 (instructions+skill block) to ~3,266 (through the diff).
  • Per batch of 8 lenses (uncached = 8 × 3,326 = 26,608): a working breakpoint + working gateway cuts find input from ~48% (old prefix) to ~62%.
  • Zero benefit today: the engine has no breakpoint mechanism (stdin→CLI, bwrap --clearenv + strict ALLOWED_ENV, no request body) and the relay does not create cache entries. This PR only changes what would be cacheable, never what is cached.

No stale-context risk

Nothing is cached by this change. When/if caching exists, a hit on the stable prefix requires a byte-identical prefix — same review, same diff/summaries/policy — which is exactly what the other lenses of the same batch send. A different PR's diff is a different prefix → cache miss. Stale reuse is structurally impossible.

Nonce anti-replay preserved

schemaFeedbackPrompt appends Repair attempt nonce: <uuid> after the base prompt, outside the cacheable prefix, so two identical consecutive failures still cannot replay a cached identical answer. The reorder does not move it; the property is unchanged.

Test plan

  • node --test test/: 228 tests / 220 pass / 7 skip (baseline 227 + new guard; the single fail is the known diff-input FIFO ETIMEDOUT flake, 8/0 in isolation).

🤖 Generated with Claude Code

Co-Authored-By: Claude noreply@anthropic.com
Claude-Session: w13-cache-followthrough

Summary by CodeRabbit

  • 改进

    • 优化查找阶段的信息组织顺序:调用方策略、摘要、路径和差异信息将优先于分类维度呈现。
    • 分类维度现位于提示内容的最后一个区块,相关提示修复信息追加至基础内容末尾。
  • 测试

    • 新增提示内容稳定性验证,确保不同分类配置下保持一致的前置内容,并维持差异信息与分类维度的顺序。

…aximal stable prefix

The find stage fans out one call per taxonomy dimension (8 lenses per diff
batch). The taxonomy value sat mid-block-order, so any prompt-cache breakpoint
before it could only share the ~2,100-token instructions+skill block across the
batch. Diff, summaries, policy, and every instruction are byte-identical across
lenses; only the taxonomy value differs.

Move the taxonomy value to the END of the block order. A breakpoint before it
now shares the maximum prefix (~3,266 tokens for a typical find call) across
the batch's 8 lenses - find input saving rises from ~48% to ~62% - but only IF
a breakpoint mechanism exists and the relay honors prompt caching, which it
does not today. This is forward-structure: it changes what is cacheable, not
what is cached. All prompt content is byte-identical, only the order of one
block changed.

The schema-repair nonce keeps its anti-replay property: it is appended AFTER
the base prompt, outside the cacheable prefix, so two identical consecutive
failures still cannot replay a cached answer. The reorder does not move it.

Verified: 228 tests, 220 pass, 7 skip (baseline 227 + the new prefix-stability
guard; the single fail under full-suite concurrency is the known diff-input
FIFO flake, 8/0 in isolation).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: w13-cache-followthrough
@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.

@coderabbitai

coderabbitai Bot commented Aug 10, 2026 •

Copy link
Copy Markdown

Review Change Stack

📝 Walkthrough

Walkthrough

本次变更调整 find 阶段提示词的区块顺序,并新增测试验证 taxonomy 位于末尾,以及不同 taxonomy 维度共享稳定前缀。

Changes

Find 提示词稳定性

Layer / File(s) Summary
提示词顺序与稳定性验证
src/claude-runner.mjs, test/find-prefix-stability.test.mjs
find 阶段先加入策略、摘要、路径和 diff,再加入 taxonomy。测试验证 taxonomy 位于最后一个区块,diff 位于 taxonomy 之前,并验证不同 taxonomy 维度的前缀保持字节级一致。注释说明当前中继不会创建提示词缓存。

Estimated code review effort: 1(简单)| ~5 分钟

Possibly related PRs

  • Kizunad/review#14:同样修改 src/claude-runner.mjs 的 find 阶段提示词结构。
  • Kizunad/review#37:同样修改 find 阶段提示词并增加提示词测试。

Poem

我是小兔,蹦过提示词边界,
让 diff 先行,taxonomy 收尾。
两种维度共享稳定前缀,
测试把顺序逐项核对。
咕咕一声,缓存注释也更新。

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed 标题准确概括了本次变更的主要目标:稳定 find 提示词前缀以支持提示词缓存。
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/cache-find-prefix-stability

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@test/find-prefix-stability.test.mjs`:
- Around line 26-36: Update the prompt loop in the find-prefix stability test to
retain each prompt’s expected taxonomy dimension and assert that the final block
contains that dimension’s serialized id, not just the “Assigned taxonomy
dimension:” label. Preserve the existing final-block and ordering assertions for
every prompt.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: a6b05fc5-3f9d-41f3-913c-0b765dc16f57

📥 Commits

Reviewing files that changed from the base of the PR and between ae90091 and b81a7cc.

📒 Files selected for processing (2)
  • src/claude-runner.mjs
  • test/find-prefix-stability.test.mjs

Comment on lines +26 to +36
for (const prompt of prompts) {
const blocks = prompt.split('\n\n');
const lastBlock = blocks[blocks.length - 1];
assert.ok(
lastBlock.startsWith('Assigned taxonomy dimension:'),
'the taxonomy value must be the final block of the find prompt so the diff/summaries/policy stay in the stable prefix',
);
assert.ok(
prompt.indexOf('Immutable pull-request diff batch:') < prompt.indexOf('Assigned taxonomy dimension:'),
'the diff must precede the taxonomy value so the diff is inside the cross-lens stable prefix',
);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🎯 Functional Correctness | 🟡 Minor | ⚡ Quick win

验证最终 block 中的 taxonomy id。

当前断言只检查最后一个 block 以 Assigned taxonomy dimension: 开头。如果实现保留标签但丢失或错误序列化 security/correctness,测试仍会通过。请保留每个 prompt 对应的 dimension,并断言最终 block 包含该 dimension 的 id。

建议补丁
-  const prompts = ['security', 'correctness'].map((dimension) =>
-    stagePrompt({ ...shared, taxonomy: { id: dimension } }, context),
-  );
+  const prompts = ['security', 'correctness'].map((dimension) => ({
+    dimension,
+    prompt: stagePrompt({ ...shared, taxonomy: { id: dimension } }, context),
+  }));

-  for (const prompt of prompts) {
+  for (const { dimension, prompt } of prompts) {
     const blocks = prompt.split('\n\n');
     const lastBlock = blocks[blocks.length - 1];
     assert.ok(
       lastBlock.startsWith('Assigned taxonomy dimension:'),
       'the taxonomy value must be the final block of the find prompt so the diff/summaries/policy stay in the stable prefix',
     );
+    assert.ok(
+      lastBlock.includes(dimension),
+      'the final block must contain the assigned taxonomy id',
+    );
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@test/find-prefix-stability.test.mjs` around lines 26 - 36, Update the prompt
loop in the find-prefix stability test to retain each prompt’s expected taxonomy
dimension and assert that the final block contains that dimension’s serialized
id, not just the “Assigned taxonomy dimension:” label. Preserve the existing
final-block and ordering assertions for every prompt.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant