Qwen3-Next target crashes in speculative rollback hook: Qwen3NextGatedDeltaNet has no attribute in_proj_qkv

[JareyLi]

Body

## Summary

When using dflash-mlx with a Qwen3-Next target model, DFlash generation crashes in the speculative linear cache hook:

```text
'Qwen3NextGatedDeltaNet' object has no attribute 'in_proj_qkv'

Environment

• dflash-mlx: 0.1.5.1
• mlx-lm: >=0.31.x
• Platform: Apple Silicon / macOS
• Target model examples:
  • Qwen3-Coder-Next-8bit
  • Qwen3-Coder-Next-oQ6
• Draft model:
  • Qwen3-Coder-Next-DFlash

Error

DFlash streaming generation error: 'Qwen3NextGatedDeltaNet' object has no attribute 'in_proj_qkv'

Suspected cause

dflash_mlx/engine/target_qwen_gdn.py installs _install_speculative_linear_cache_hook() for linear attention layers.

Inside speculative_call, it assumes the GatedDeltaNet module has Qwen3.5-style projection attributes:

qkv = self.in_proj_qkv(inputs)
z_proj = self.in_proj_z(inputs)
b = self.in_proj_b(inputs)
a = self.in_proj_a(inputs)

But mlx_lm.models.qwen3_next.Qwen3NextGatedDeltaNet uses the Qwen3-Next projection layout:

self.in_proj_qkvz
self.in_proj_ba
self.fix_query_key_value_ordering(...)

So the hook works for Qwen3.5/Qwen3.6 GatedDeltaNet-style modules, but crashes for Qwen3-Next.

Expected behavior

dflash-mlx should either:

1. support Qwen3-Next GatedDeltaNet by using in_proj_qkvz + in_proj_ba, or
2. detect Qwen3-Next and disable the speculative recurrent rollback hook / mark it unsupported with a clear error.

Notes

This does not appear to be an mlx-lm missing attribute bug. The Qwen3-Next implementation intentionally uses fused in_proj_qkvz and in_proj_ba projections.

[bstnxbt]

Ok, i will support it

[bstnxbt]

Thanks for the report. I pushed a minimal fix in b734dbf.

What changed:

• the Qwen GDN target path now handles both the existing Qwen3.5/Qwen3.6 split projections (in_proj_qkv, in_proj_z, in_proj_b, in_proj_a) and the Qwen3-Next fused layout (in_proj_qkvz, in_proj_ba, fix_query_key_value_ordering(...));
• the fix is wired into both speculative rollback verify and DDTree tree verify;
• tiny Qwen3-Next tests now cover rollback verify, rollback replay/continuation, tree verify, tree acceptance commit, and continuation parity.

This is deliberately minimum support to stop the crash, not an optimized Qwen3-Next path. I only have a 64 GB local machine, so I cannot properly optimize or benchmark the large Qwen3-Coder-Next target path here. If you have a larger Apple Silicon setup and want to improve the optimized path, a focused PR would be welcome.

[JareyLi]

Thanks, your fix works for Qwen3-Coder-Next in oMLX.

One note: oMLX currently pins an older dflash-mlx revision that still expects make_draft_backend, so installing the latest dflash-mlx commit directly breaks oMLX with:

cannot import name 'make_draft_backend' from dflash_mlx.draft_backend

I backported just the Qwen3-Next fused GDN projection changes from b734dbf onto the older oMLX-compatible dflash-mlx revision, and DFlash now loads and runs successfully:

DFlashEngine loaded: target=Qwen3-Coder-Next-oQ6, draft=Qwen3-Coder-Next-DFlash
DFlash generation complete: 250 tokens, 19.7 tok/s, acceptance=45.2%, cycles=137

So the Qwen3-Next projection fix itself looks good. The remaining issue is only integration/API compatibility with the oMLX-pinned dflash-mlx version.