Repository navigation
qwen4_exp: warn when ROWINV runs without the in-pass checkpoint - #210
Conversation
ATLAS_QWEN4EXP_PREFILL_ROWINV=1 disables the tail-checkpoint split (a block-grid cut is off the 64-row GDN grid) and relies on ATLAS_QWEN4EXP_PREFILL_MIDCHUNK_CKPT=1 to capture the checkpoint in-pass. With ROWINV alone no tail SSM checkpoint is ever saved, so every warm turn recomputes its whole prompt, silently. Measured on Flash-Next FP4-Local agentic-webserver 50 (GB10): 0 checkpoints, TTFT p50 632 -> 3062 ms, 13.97 vs 12.43 s/turn. With MIDCHUNK_CKPT added: 307 intermediate hits, 11.76 s/turn. Warn once at the first last-chunk where it applies. No behaviour change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Nice catch on the silent no-checkpoint case. One suggestion: The patch below changes no behaviour. It moves the body into
The patch applies cleanly to diff --git a/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy.rs b/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy.rs
index 68c56380..df0e25ac 100644
--- a/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy.rs
+++ b/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy.rs
@@ -220,16 +220,27 @@ pub(in crate::model) fn tail_split_disabled() -> bool {
/// FP4-Local agentic-webserver, 0 checkpoints, TTFT p50 632 -> 3062 ms.
/// Warns once in that case.
pub(in crate::model) fn rowinv_tail_unsplit(in_pass: bool) -> bool {
+ static WARN: std::sync::Once = std::sync::Once::new();
let rowinv = crate::layers::ops::qwen4exp_rowinv::on();
+ rowinv_tail_unsplit_with(rowinv, in_pass, &WARN, || {
+ tracing::warn!(
+ "ATLAS_QWEN4EXP_PREFILL_ROWINV=1 without \
+ ATLAS_QWEN4EXP_PREFILL_MIDCHUNK_CKPT=1 saves no tail SSM checkpoint: \
+ warm turns will recompute their whole prompt"
+ )
+ })
+}
+
+/// [`rowinv_tail_unsplit`] with the flag, the once-guard and the warning
+/// passed in, so tests can drive it without the read-once env flag.
+fn rowinv_tail_unsplit_with(
+ rowinv: bool,
+ in_pass: bool,
+ warn: &std::sync::Once,
+ emit: impl FnOnce(),
+) -> bool {
if rowinv && !in_pass {
- static WARN: std::sync::Once = std::sync::Once::new();
- WARN.call_once(|| {
- tracing::warn!(
- "ATLAS_QWEN4EXP_PREFILL_ROWINV=1 without \
- ATLAS_QWEN4EXP_PREFILL_MIDCHUNK_CKPT=1 saves no tail SSM checkpoint: \
- warm turns will recompute their whole prompt"
- )
- });
+ warn.call_once(emit);
}
rowinv
}
diff --git a/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy_tests.rs b/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy_tests.rs
index eee53ac3..0a1845ae 100644
--- a/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy_tests.rs
+++ b/crates/spark-model/src/model/trait_impl/prefill_b/pc_policy_tests.rs
@@ -5,7 +5,7 @@
use super::{
Agreed, agree_restore, branch_checkpoint_at, branch_split_at, layer_write_floor, replay_floor,
- tail_cut,
+ rowinv_tail_unsplit_with, tail_cut,
};
const BS: usize = 16;
@@ -207,3 +207,44 @@ fn replay_floor_covers_the_replayed_matched_rows() {
assert_eq!(replay_floor(true, 256, 256, 256, 100), 0);
assert_eq!(replay_floor(false, 64, 0, 0, 70), 0);
}
+
+/// `ATLAS_QWEN4EXP_PREFILL_ROWINV` decides alone whether the last chunk
+/// runs unsplit; `in_pass` only gates the warning, which fires for ROWINV
+/// without the in-pass checkpoint and at most once per guard.
+#[test]
+fn rowinv_tail_unsplit_truth_table_and_warn_once() {
+ use std::cell::Cell;
+ use std::sync::Once;
+ for (rowinv, in_pass, unsplit, warns) in [
+ (false, false, false, 0),
+ (false, true, false, 0),
+ (true, true, true, 0),
+ (true, false, true, 1),
+ ] {
+ let guard = Once::new();
+ let fired = Cell::new(0);
+ for _ in 0..3 {
+ let got =
+ rowinv_tail_unsplit_with(rowinv, in_pass, &guard, || fired.set(fired.get() + 1));
+ assert_eq!(got, unsplit, "rowinv={rowinv} in_pass={in_pass}");
+ }
+ assert_eq!(fired.get(), warns, "rowinv={rowinv} in_pass={in_pass}");
+ }
+}
+
+/// The once-guard is shared across `in_pass` values: a warning already
+/// given is not repeated, and an `in_pass` call never consumes it.
+#[test]
+fn rowinv_tail_unsplit_warn_guard_is_shared() {
+ use std::cell::Cell;
+ use std::sync::Once;
+ let guard = Once::new();
+ let fired = Cell::new(0);
+ let bump = || fired.set(fired.get() + 1);
+ assert!(rowinv_tail_unsplit_with(true, true, &guard, bump));
+ assert!(!guard.is_completed());
+ assert!(rowinv_tail_unsplit_with(true, false, &guard, bump));
+ assert!(rowinv_tail_unsplit_with(true, true, &guard, bump));
+ assert!(rowinv_tail_unsplit_with(true, false, &guard, bump));
+ assert_eq!(fired.get(), 1);
+}This comment was prepared with AI assistance (OpenClaw agent) and reviewed by me. |
Co-Authored-By: maplepaladin73 <215220483+maplepaladin73@users.noreply.github.com> Co-Authored-By: Devin <158243242+devin-ai-integration[bot]@users.noreply.github.com>
|
Thanks @maplepaladin73. I applied your patch verbatim as d221f72 ( |
Problem
ATLAS_QWEN4EXP_PREFILL_ROWINV=1turns off the tail-checkpoint split, because a block-grid cut is off the 64-row GDN grid. It relies onATLAS_QWEN4EXP_PREFILL_MIDCHUNK_CKPT=1to capture the checkpoint in-pass instead. With ROWINV alone, no tail SSM checkpoint is ever saved, so every warm turn recomputes its whole prompt. Nothing reports this; the only trace is "Prefix cache hit … but no SSM snapshot" at INFO.Measured on Flash-Next FP4-Local, agentic-webserver 50, GB10, prefix cache on:
Change
pc_policy::rowinv_tail_unsplit(in_pass)replaces the inlineqwen4exp_rowinv::on()at the tail-split site and returns the same value. When ROWINV is on and the in-pass checkpoint is off, it logs one WARN. The guard is the existing split condition: last chunk, SSM model, snapshots on, prefix cache active. No behaviour change.Verified (sparky GB10)
cargo fmt --check,cargo clippy --release -p spark-model -- -D warnings,cargo test -p spark-model --lib pc_policy(11 passed).🤖 Generated with Claude Code