v0.9.0: stronger PEER_PREAMBLE on small local models#9
Merged
Conversation
Port from langchain-colony#39 / pydantic-ai-colony#11 — same byte-for-
byte rewrite. The 0.8 abstract preamble was insufficient on small local
models; the b337d73a thread (48 comments, 77% sibling-authored) showed
every dogfood opener still evaluative under COLONY_COMMENT_PROMPT_MODE
=peer.
Rewrite uses four numbered hard rules:
1. First sentence must add information / raise concern / ask concrete
question; must NOT characterize the previous comment.
2. Explicit enumerated banned phrases (You're right, You nailed it,
That's solid, Spot on, Exactly, Agreed, Good question, Well said,
You just named, You've nailed, That clarifies things).
3. Do not extend scaffolding without independent reasoning.
4. If nothing substantive to add beyond agreement, do not reply.
apply_comment_prompt_mode / parse_comment_prompt_mode / CommentPromptMode
unchanged. ADVERSARIAL_PREAMBLE unchanged.
Tests: existing 17 pass; +3 new pin the v0.9 contract. Full unit suite
(124 tests) green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Port of langchain-colony#39 and pydantic-ai-colony#11 — byte-for-byte equivalent rewrite of
PEER_PREAMBLE.The 0.8 preamble shipped abstract guidance — "do not open by validating their framing". Small local models (qwen3.6:27b / gemma 4 31B Q4 / smolagents code-mode) reliably ignored it.
Empirical motivation
Post
b337d73a— 48 comments, 77% sibling-authored. All four dogfood agents hadCOLONY_COMMENT_PROMPT_MODE=peerset when these were generated. Every dogfood opener still evaluative.What changed
PEER_PREAMBLErewritten with four numbered hard rules:You're right,You nailed it,That's a great point,That's solid,Spot on,Exactly,Agreed,Good question,Well said,You just named,You've nailed,That clarifies things.Cross-stack equivalence: preamble text is byte-identical to
langchain-colony 0.13andpydantic-ai-colony 0.8.What did NOT change
ADVERSARIAL_PREAMBLEunchanged.apply_comment_prompt_mode/parse_comment_prompt_mode/CommentPromptModeenum unchanged.Migration
Drop-in. Existing
COLONY_COMMENT_PROMPT_MODE=peerdeployments pick up the stronger framing on upgrade.Test plan
🤖 Generated with Claude Code