fix(qwen4_exp): accept compressed-tensors .weight_packed in the namespace probe

[AdamSWS]

Summary

audit_namespace() refuses to build any compressed-tensors NVFP4 checkpoint with qwen4_exp: 0 of N layers have routed experts, even though the format is detected correctly, every tensor loads, and the MoE loader behind the probe handles the packed layout. has_linear() recognises .weight and .trellis but not .weight_packed.

Root cause

crates/spark-model/src/weight_loader/qwen4_exp/probe.rs:

/// A projection is present in either the dense/NVFP4 layout (`.weight`) or the EXL3 one (`.trellis`).
fn has_linear(store: &WeightStore, stem: &str) -> bool {
    store.contains(&format!("{stem}.weight")) || store.contains(&format!("{stem}.trellis"))
}

In a compressed-tensors checkpoint the routed experts are stored as …mlp.experts.{e}.gate_proj.weight_packed (plus .weight_scale / .weight_global_scale), so has_linear is false for every layer and expert_tensors stays 0.

Only the experts are affected because, in a mixed-precision checkpoint, attention and the shared expert are FP8 with a plain .weight — which is why the same probe reports the GDN/full-attention split correctly while the MoE count is zero.

Reproduction

Any compressed-tensors NVFP4 Flash-Next checkpoint, e.g. RedHatAI/Qwen3.8-Flash-Next-NVFP4 (quant_method: compressed-tensors, format: nvfp4-pack-quantized):

INFO spark_model::quant_format: QuantFormat: compressed-tensors (format="mixed-precision"), 753 ignored module(s)
INFO spark::main_modules::serve_load: Quantization format: compressed-tensors (base variant CompressedTensors)
INFO spark_runtime::fast_weights: Fast-loaded 112326 weight tensors
INFO spark_model::weight_loader::qwen4_exp::probe: qwen4_exp namespace: 36/48 GDN + 12 full-attention layers, 0 MoE blocks

Error: Failed to build model

Caused by:
    qwen4_exp: 0 of 48 layers have routed experts — a layer whose experts silently fail to resolve
    would route into nothing

Verification

Built an engine from the equivalent downstream tree with only this change and served on 2× DGX Spark (GB10, sm_121) at TP2 over ConnectX-7:

• the probe reports every layer (was 0); the model builds and serves
• smoke check passes (17*23 = 391)
• single-stream decode 74.2 tok/s on code, 50.6 on prose, TTFT 0.12–0.18 s
• 33K-token cold prefill TTFT 8.47 s, then 44.9 tok/s decode
• 4 concurrent streams: 171.8 tok/s aggregate

Scope

has_linear also backs the lm_head, linear_attn.in_proj_qkv, self_attn.q_proj, indexer.index_qk_proj and vision probes, so fixing the helper makes those tolerant of packed layouts too rather than special-casing the MoE one. ModelOpt and EXL3 checkpoints are unaffected.

The same fix is open against a downstream fork as Enntity#1, where it was first found.

[github-actions]

All contributors have signed the CLA. Thank you!
_{Posted by the CLA Assistant Lite bot.}

[AdamSWS]

I have read the CLA Document and I hereby sign the CLA

[AdamSWS]

Added a regression test, compressed_tensors_layout_is_counted, mirroring the existing exl3_layout_is_counted: attention and lm_head FP8 with a plain .weight, routed experts NVFP4 with .weight_packed, BF16 n-gram shards deferred. Without the has_linear change it fails with 0 routed-expert layers — the same refusal seen in the field.

Verified by building the engine and running the crate's own test list, with spark-model:qwen4_exp::probe added to it:

test weight_loader::qwen4_exp::probe::tests::compressed_tensors_layout_is_counted ... ok
test weight_loader::qwen4_exp::probe::tests::exl3_layout_is_counted ... ok
test weight_loader::qwen4_exp::probe::tests::deferred_shards_are_counted ... ok
test weight_loader::qwen4_exp::probe::tests::missing_expert_still_refuses ... ok

Note the build I ran this in was pinned to a downstream fork's engine commit (that is the GB10 Flash-Next tree I have hardware for); the test itself touches only probe_tests.rs and is identical on both trees.