Skip to content

Promote model roster: set cascade to mistral-small, gemma, qwen #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
morgaesis merged 1 commit into from
Jul 11, 2026
Merged

Promote model roster: set cascade to mistral-small, gemma, qwen #44

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,10 @@ gate:
# enabled: false
model:
name: deepseek/deepseek-v4-pro
cascade: [anthropic/claude-sonnet-4.6]
cascade:
- mistralai/mistral-small-3.2-24b-instruct
- google/gemma-3-27b-it
- qwen/qwen3-32b
apiBase: https://openrouter.ai/api/v1 # ignored from config by default; see note below
consensus: 1 # >1: only findings multiple models agree on survive
```
Expand Down
35 changes: 32 additions & 3 deletions src/config.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,11 @@
use crate::envelope::Severity;

pub const DEFAULT_MODEL: &str = "deepseek/deepseek-v4-pro";
pub const DEFAULT_CASCADE: [&str; 3] = [
"mistralai/mistral-small-3.2-24b-instruct",
"google/gemma-3-27b-it",
"qwen/qwen3-32b",
];
pub const DEFAULT_API_BASE: &str = "https://openrouter.ai/api/v1";

#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
Expand Down Expand Up @@ -118,7 +123,7 @@
gate_fail_on: GateLevel::Severity(Severity::Error),
gate_on_error: OnError::Block,
model: DEFAULT_MODEL.to_string(),
cascade: Vec::new(),
cascade: DEFAULT_CASCADE.iter().map(|m| (*m).to_string()).collect(),

Check warning on line 126 in src/config.rs

View check run for this annotation

postil-dev / postil/review

Default cascade sends code to three additional third parties without opt-in 

Changing the default `cascade` from empty to `[mistralai/mistral-small-3.2-24b-instruct,
google/gemma-3-27b-it, qwen/qwen3-32b]` means existing users who have not explicitly configured a
cascade will now send their source code to three extra external providers on every run. The updated
e2e assertions show the default request count jumping from 2 to 8 and token totals increasing
proportionally, confirming a significant rise in latency and cost. Confirm that expanding the
default data-sharing boundary to multiple new vendors without a major-version bump or explicit
opt-in is intentional and acceptable. If not, keep the default `cascade` empty and surface the new
roster only as an opt-in example in documentation.

@postil-dev postil-dev Bot Jul 11, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

warn Default cascade sends code to three additional third parties without opt-in
warn · confidence 0.90 · kind: humanEscalation

Changing the default cascade from empty to [mistralai/mistral-small-3.2-24b-instruct, google/gemma-3-27b-it, qwen/qwen3-32b] means existing users who have not explicitly configured a cascade will now send their source code to three extra external providers on every run. The updated e2e assertions show the default request count jumping from 2 to 8 and token totals increasing proportionally, confirming a significant rise in latency and cost. Confirm that expanding the default data-sharing boundary to multiple new vendors without a major-version bump or explicit opt-in is intentional and acceptable. If not, keep the default cascade empty and surface the new roster only as an opt-in example in documentation.

api_base: DEFAULT_API_BASE.to_string(),
consensus: 1,
guardrails: None,
Expand Down Expand Up @@ -464,8 +469,10 @@

model:
name: deepseek/deepseek-v4-pro
# cascade:
# - anthropic/claude-sonnet-4.6
cascade:
- mistralai/mistral-small-3.2-24b-instruct
- google/gemma-3-27b-it
- qwen/qwen3-32b
# apiBase: https://openrouter.ai/api/v1 # any OpenAI-compatible endpoint (Ollama, vLLM, Azure).
# # Ignored from config by default (a repo could redirect
# # the inference credential). Prefer POSTIL_API_BASE; to
Expand Down Expand Up @@ -546,6 +553,10 @@
f.write_all(content.as_bytes()).unwrap();
}

fn default_cascade() -> Vec<String> {
DEFAULT_CASCADE.iter().map(|m| (*m).to_string()).collect()
}

#[test]
fn defaults_are_low_noise() {
let c = Config::default();
Expand All @@ -557,6 +568,22 @@
));
}

#[test]
fn defaults_keep_primary_and_retry_roster_order() {
let c = Config::default();
assert_eq!(c.model, DEFAULT_MODEL);
assert_eq!(c.cascade, default_cascade());
assert_eq!(
c.model_chain(),
vec![
"deepseek/deepseek-v4-pro",
"mistralai/mistral-small-3.2-24b-instruct",
"google/gemma-3-27b-it",
"qwen/qwen3-32b",
]
);
}

#[test]
fn postil_yaml_wins_over_coderabbit() {
let dir = tempfile::tempdir().unwrap();
Expand Down Expand Up @@ -598,6 +625,8 @@
let mut c = Config::default();
c.apply_file(f).unwrap();
assert_eq!(c.max_findings, 20);
assert_eq!(c.model, DEFAULT_MODEL);
assert_eq!(c.cascade, default_cascade());
}

#[test]
Expand Down
31 changes: 27 additions & 4 deletions tests/e2e.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -662,10 +662,33 @@ async fn garbage_output_fails_closed_after_repair_attempt() {
let env: Value =
serde_json::from_str(&String::from_utf8(out.get_output().stdout.clone()).unwrap()).unwrap();
assert_eq!(env["findings"][0]["path"], ".postil/model-output");
assert_eq!(env["usage"]["promptTokens"], 160);
assert_eq!(env["usage"]["completionTokens"], 60);
// Initial call + repair call.
assert_eq!(server.received_requests().await.unwrap().len(), 2);
assert_eq!(env["usage"]["promptTokens"], 640);
assert_eq!(env["usage"]["completionTokens"], 240);
// Initial call + repair call for each default model in the retry roster.
let requests = server.received_requests().await.unwrap();
assert_eq!(requests.len(), 8);
let models: Vec<String> = requests
.iter()
.map(|request| {
request.body_json::<Value>().unwrap()["model"]
.as_str()
.unwrap()
.to_string()
})
.collect();
assert_eq!(
models,
vec![
"deepseek/deepseek-v4-pro",
"deepseek/deepseek-v4-pro",
"mistralai/mistral-small-3.2-24b-instruct",
"mistralai/mistral-small-3.2-24b-instruct",
"google/gemma-3-27b-it",
"google/gemma-3-27b-it",
"qwen/qwen3-32b",
"qwen/qwen3-32b",
]
);
}

#[tokio::test]
Expand Down
Loading