feat(prompts): Hermes-style cache_control + stable tool prefix - #113
Conversation
🤖 CodeAnt AI — Review Status
|
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
codeant-ai
Bot
added
the
size:L
![codeant-ai[bot]](https://avatars.githubusercontent.com/in/646884?s=60&v=4){kind=small}
| # Claude-style deferred MCP stubs: keep name stable, shrink rare schemas. | ||
| if t.name.startswith("mcp_") or t.name.startswith("mcp__"): | ||
| description = f"[deferred schema] {description}" | ||
| params = {"type": "object", "properties": {}, "additionalProperties": True} |
There was a problem hiding this comment.
Suggestion: Replacing every mcp_ or mcp__ tool schema with an empty permissive object removes all required argument definitions from the model-visible contract. Dispatch still validates calls against the original Pydantic model, so required MCP arguments will be omitted and rejected; this also makes mcp_auth unusable because its required server_name, mode, and value fields are hidden. [incomplete implementation]
Severity Level: Major ⚠️
- ❌ MCP authentication calls fail before credentials are saved.
- ❌ Required-argument MCP tools cannot execute normally.
- ⚠️ Models receive unusable schemas for configured MCP tools.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/_factory.py
**Line:** 658:661
**Comment:**
*Incomplete Implementation: Replacing every `mcp_` or `mcp__` tool schema with an empty permissive object removes all required argument definitions from the model-visible contract. Dispatch still validates calls against the original Pydantic model, so required MCP arguments will be omitted and rejected; this also makes `mcp_auth` unusable because its required `server_name`, `mode`, and `value` fields are hidden.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| # Deterministic tool prefix for prompt-cache stability. | ||
| tools_wire.sort(key=lambda item: str(item["function"]["name"])) |
There was a problem hiding this comment.
Suggestion: This reorders the request-level tool schema away from ToolRegistry's documented canonical default order. The registry explicitly preserves that order as part of the stable API schema, so sorting all tools alphabetically changes the externally advertised tool contract and can invalidate consumers or prompt behavior that relies on the canonical ordering. Preserve the registry order or update the registry contract and all dependent callers consistently. [api mismatch]
Severity Level: Major ⚠️
- ⚠️ OpenAI tool schemas no longer match registry ordering.
- ⚠️ Prompt-cache prefixes change despite canonical registry ordering.
- ⚠️ Consumers comparing API schemas observe reordered tools.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/_factory.py
**Line:** 672:673
**Comment:**
*Api Mismatch: This reorders the request-level tool schema away from `ToolRegistry`'s documented canonical default order. The registry explicitly preserves that order as part of the stable API schema, so sorting all tools alphabetically changes the externally advertised tool contract and can invalidate consumers or prompt behavior that relies on the canonical ordering. Preserve the registry order or update the registry contract and all dependent callers consistently.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix| wire = conversation_to_openai_messages( | ||
| messages, | ||
| system_prompt=self._system_prompt, | ||
| prompt_cache=True, | ||
| ) |
There was a problem hiding this comment.
Suggestion: The live streamer unconditionally enables prompt_cache, even though ProviderCapabilities.prompt_cache defaults to false and this adapter is used for generic OpenAI-compatible endpoints. Vanilla OpenAI and other compatible providers may reject the nested cache_control content format, causing every streamed request to fail. Gate this option on the provider capability or configure it explicitly for providers that support the envelope. [api mismatch]
Severity Level: Critical 🚨
- ❌ Unsupported providers can fail every streamed turn.
- ❌ Provider failover may retry incompatible request envelopes.
- ⚠️ Default capability declarations do not control wire formatting.
Prompt for AI Agent 🤖
This is a comment left during a code review.
**Path:** src/dream/engine/_adapter_openai.py
**Line:** 205:209
**Comment:**
*Api Mismatch: The live streamer unconditionally enables `prompt_cache`, even though `ProviderCapabilities.prompt_cache` defaults to false and this adapter is used for generic OpenAI-compatible endpoints. Vanilla OpenAI and other compatible providers may reject the nested `cache_control` content format, causing every streamed request to fail. Gate this option on the provider capability or configure it explicitly for providers that support the envelope.
Validate the correctness of the flagged issue. If correct, How can I resolve this? If you propose a fix, implement it and please make it concise.
Once fix is implemented, also check other comments on the same PR, and ask user if the user wants to fix the rest of the comments as well. if said yes, then fetch all the comments validate the correctness and implement a minimal fix
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
f0946c6 to
6914cfa
Compare
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Rebase onto main (keep #112 context breakdown). Drop the PR's reimplemented context pie and getattr session wiring. Apply ≤4 ephemeral breakpoints via frozen OpenAIChatMessage envelopes when ProviderCapabilities.prompt_cache is set. Co-authored-by: Cursor <cursoragent@cursor.com>
6914cfa to
81cd408
Compare
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
codeant-ai
Bot
added
size:L
Replace to_wire dict helpers with OpenAIChatMessage.to_json_object / encode_openai_messages at the HTTP boundary only. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Compaction shares Dream-owned common standing orders via CompactionPromptParts / OpenAIChatMessage envelopes (workspace context stays on the user message). Stacked on cleaned #113; no context pie reimplementation. Co-authored-by: Cursor <cursoragent@cursor.com>
Drop unused JsonValue alias, move assistant tool_calls wire encoding into the OpenAI adapter, and inline message JSON at call sites. Co-authored-by: Cursor <cursoragent@cursor.com>
Thanks for using CodeAnt! 🎉We're free for open-source projects. if you're enjoying it, help us grow by sharing. Share on X · |
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
codeant-ai
Bot
added
size:L
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
c19cfa4 to
a2eed9d
Compare
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Narrow OpenAIChatMessage list before apply_cache_control and align httpx stream callable message param with StreamChatCompletion. Co-authored-by: Cursor <cursoragent@cursor.com>
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
Use live stable_block.render() for compact cache prefix alignment, restrict workspace context to skill/memory catalogues only, add typed OpenAIAssistantToolMessage wire types, CompactionChatRequest, and validate summariser JSON payloads before parsing. Co-authored-by: Cursor <cursoragent@cursor.com>
feat(compact): reuse stable+context assembler prefix
There was a problem hiding this comment.
divo12 has reached the 50-credit limit for trial accounts. To continue receiving code reviews, upgrade your plan.
1 participant
User description
Summary
cache_controlbreakpoints on live OpenAI stream turns (static<stable>prefix + trailing messages).Stack
Base:
feat/context-breakdown(#112) → this PR → Batch C (compact same assembler).Test plan
uv run pytest tests/test_prompts/test_cache_control.py tests/test_engine/test_adapter_openai.py tests/test_factory.py -qMade with Cursor
CodeAnt-AI Description
Add prompt caching to OpenAI-compatible conversation turns
What Changed
<stable>...</stable>section cache that shared prefix separately from the remaining system instructionsImpact
✅ More prompt-cache reuse across repeated conversations✅ Lower repeated prompt-processing cost✅ Stable system instructions cached independently💡 Usage Guide
Checking Your Pull Request
Every time you make a pull request, our system automatically looks through it. We check for security issues, mistakes in how you're setting up your infrastructure, and common code problems. We do this to make sure your changes are solid and won't cause any trouble later.
Talking to CodeAnt AI
Got a question or need a hand with something in your pull request? You can easily get in touch with CodeAnt AI right here. Just type the following in a comment on your pull request, and replace "Your question here" with whatever you want to ask:
This lets you have a chat with CodeAnt AI about your pull request, making it easier to understand and improve your code.
Example
Preserve Org Learnings with CodeAnt
You can record team preferences so CodeAnt AI applies them in future reviews. Reply directly to the specific CodeAnt AI suggestion (in the same thread) and replace "Your feedback here" with your input:
This helps CodeAnt AI learn and adapt to your team's coding style and standards.
Example
Retrigger review
Ask CodeAnt AI to review the PR again, by typing:
Check Your Repository Health
To analyze the health of your code repository, visit our dashboard at https://app.codeant.ai. This tool helps you identify potential issues and areas for improvement in your codebase, ensuring your repository maintains high standards of code health.