feat(compact): reuse stable+context assembler prefix - #115
Conversation
Compaction shares Dream-owned common standing orders via CompactionPromptParts / OpenAIChatMessage envelopes (workspace context stays on the user message). Stacked on cleaned #113; no context pie reimplementation. Co-authored-by: Cursor <cursoragent@cursor.com>
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
codeant-ai
Bot
added
the
size:L
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
| # with live turns) and keeps workspace catalogues in the user message. | ||
| compact_prompt = CompactionPromptParts( | ||
| stable_prefix=assemble_stable_context_prefix( | ||
| stable=StablePromptBlock(role=None), |
There was a problem hiding this comment.
Suggestion: The compact request uses only the common standing orders, while live planner/generator/evaluator sessions use a role-specific <stable> block. Because the cache breakpoint is placed at the end of each complete stable block, these requests have different cache-prefix boundaries and will not reuse the live-turn cache as intended. Build the compact prefix from the same stable block or explicitly share a breakpoint at the common portion. [performance]
Severity Level: Major ⚠️
- ⚠️ Planner compaction misses live-turn cache reuse.
- ⚠️ Generator compaction incurs additional prompt processing.
- ⚠️ Evaluator summarisation may increase provider latency and cost.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/_factory.py
**Line:** 824:824
**Comment:**
*Performance: The compact request uses only the common standing orders, while live planner/generator/evaluator sessions use a role-specific `<stable>` block. Because the cache breakpoint is placed at the end of each complete stable block, these requests have different cache-prefix boundaries and will not reuse the live-turn cache as intended. Build the compact prefix from the same stable block or explicitly share a breakpoint at the common portion.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| memory_catalogue="", | ||
| ), | ||
| ), | ||
| workspace_context=context_block.render(), |
There was a problem hiding this comment.
Suggestion: Passing the entire rendered context places workspace-controlled AGENTS.md, governance, tool, skill, memory, and subagent text into the summariser's user prompt. The summariser's “reference only” wording does not prevent instruction-like content from influencing the generated rolling summary, so a workspace catalogue can inject or distort durable context that is later fed back into the session. Restrict this input to non-instructional catalogue data or treat it as untrusted data during summarisation. [security]
Severity Level: Major ⚠️
- ❌ Malicious AGENTS.md can distort rolling summaries.
- ⚠️ Compacted summaries persist injected workspace instructions.
- ⚠️ Later live turns consume the contaminated summary.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/_factory.py
**Line:** 831:831
**Comment:**
*Security: Passing the entire rendered context places workspace-controlled AGENTS.md, governance, tool, skill, memory, and subagent text into the summariser's user prompt. The summariser's “reference only” wording does not prevent instruction-like content from influencing the generated rolling summary, so a workspace catalogue can inject or distort durable context that is later fed back into the session. Restrict this input to non-instructional catalogue data or treat it as untrusted data during summarisation.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
| if isinstance(block, ToolResultBlock): | ||
| return block.content |
There was a problem hiding this comment.
Suggestion: The caller passes the unconstrained result of response.json() to _summary_text_from_payload, whose implementation assumes a mapping and calls .get(). A non-object JSON response from a gateway will raise AttributeError instead of the summariser's intended controlled RuntimeError, causing compaction to fail with an unrelated error. Validate that the decoded payload is a mapping before parsing it. [logic error]
Severity Level: Major ⚠️
- ⚠️ Full compaction omits tool failure metadata.
- ⚠️ Rolling summaries may misstate failed operations.
- ⚠️ Subsequent turns lose continuity about failed tools.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/services/compact/_summariser.py
**Line:** 111:112
**Comment:**
*Logic Error: The caller passes the unconstrained result of `response.json()` to `_summary_text_from_payload`, whose implementation assumes a mapping and calls `.get()`. A non-object JSON response from a gateway will raise `AttributeError` instead of the summariser's intended controlled `RuntimeError`, causing compaction to fail with an unrelated error. Validate that the decoded payload is a mapping before parsing it.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fixUse live stable_block.render() for compact cache prefix alignment, restrict workspace context to skill/memory catalogues only, add typed OpenAIAssistantToolMessage wire types, CompactionChatRequest, and validate summariser JSON payloads before parsing. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
1 participant
User description
Summary
assemble_stable_context_prefixand pass compact slices intomake_llm_summariser.Stack
Base:
feat/prompt-cache-control(#113) → this PR (Batch C).Test plan
uv run pytest tests/test_prompts/test_system_prompt.py tests/test_services/test_compact/test_summariser.py tests/test_factory.py -qMade with Cursor
CodeAnt-AI Description
Share stable session context with conversation compaction
What Changed
Impact
✅ Lower prompt processing cost during compaction✅ Consistent summarization instructions across live turns and compaction✅ Fewer missing tool-result details in conversation summaries💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.