feat(serve): automatic shared-prefix write at the system/developer frontier - #152
Open
Astrangemaninhere wants to merge 1 commit into
Open
feat(serve): automatic shared-prefix write at the system/developer frontier#152Astrangemaninhere wants to merge 1 commit into
Astrangemaninhere wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 58ded12138
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Astrangemaninhere
force-pushed
the
pr-issue142
branch
from
September 1, 2026 13:19
0e4cb22 to
937abea
Compare
Author
|
这次的pr修改程度较小,希望能得到采纳;如果不能接受,希望能给出意见。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #142
What
Agent clients (Hermes, OpenCode, Pi, etc.) rarely send
prompt_cache_breakpoint; on master they always miss the shared system/tools head. This adds one automatic write candidate at the leading system/developer frontier so the existing shared-prefix machinery can publish a prefix those clients actually share.--no-auto-system-shared-prefix(extra write on by default) for the Responses path; the Chat path currently uses the default-on policy value (consistent with the issue discussion; wiring the flag into the Chat parse signature was deliberately left out to keep the diff minimal).shared_stable_prefixpath does the work.Per-file rationale (8 files, +65/-7, serve layer only)
src/serve/openai_common.h(+5) -OpenAIPromptCachePolicygainsauto_system_shared_prefix(defaulttrue). The policy is the single switch that reaches both Chat and Responses without threading a new parameter through every parser.src/serve/openai_common.cpp(+31/-1) - the actual feature:DefaultAutomaticcandidate on that last leading turn when unmarked (the existing explicit-breakpoint contract is untouched).frontend.cpprejects >4 markers); the explicit-write selection shrinks by one when the candidate is distinct, so a request with 3 explicit breakpoints + both automatic writes stays at 4.src/serve/serve_options.h(+3) -ServeOptionsgainsauto_system_shared_prefix = true.src/serve/serve_options.cpp(+4/-1) - parses--no-auto-system-shared-prefixand documents it in the usage text.src/serve/openai_responses.h(+8/-1) -OpenAIResponsesPromptRequestcarries the parsedcache_policyso the policy can be applied after prompt resolution (at parse time the leading turns are still ininput_turns/instructions, not ingeneration.messages).src/serve/openai_responses_request.cpp(+11/-3) - stores the policy on the prompt instead of applying it to the empty message list; the parser signature gains the opt-out bool from the HTTP layer.src/serve/openai_responses_state.cpp(+7) -resolve_openai_responses_promptapplies the policy oncegeneration.messagesis assembled (instructions + input turns), which is the earliest point the leading-instruction candidate can be placed correctly. (Fixes the review finding that Responses requests saw no system/developer turns.)src/serve/openai_responses_http.cpp(+3/-1) - passesoptions_.auto_system_shared_prefixinto the parser; without this one line the flag would exist but never reach Responses requests.Verification
da49c0d(WSL2, CUDA 13.3, sm_120a).--no-auto-system-shared-prefixrestores strict OpenAI-implicit behavior on the Responses path.Follow-ups agreed with the issue author (#142 discussion, not blockers)