Request
Add a Glm5NextTargetOps target adapter so dflash-mlx can run
GLM-5.3-Flash (arch Glm5NextForConditionalGeneration, model_type: glm5_next)
with the incoai/GLM-5.3-Flash-DFlash2 drafter on Apple Silicon.
Context
- The DFlash2 draft checkpoint itself looks close to loadable: it declares
DFlash2DraftModel (block_size 8, selector_rank/top_k, two-tap conv), which
_get_dflash_model_classes already routes to DFlash2DraftModel. The blocker
is the target side — resolve_target_ops() only has QwenGdn / Gemma4 /
MuseGlimmer and raises NotImplementedError for glm5_next.
- The draft was trained for GLM-5.3-Flash and currently only runs on CUDA
(SGLang PR #36708; a community vLLM port on GB300 reports ~137 → 305–470
tok/s, 54–67 % acceptance), so the drafter transfers well — what's missing is
an MLX target backend.
What the backend needs (per your roadmap's "family-specific backend" contract)
- Hidden capture at target layers
[5, 14, 24, 33, 42]
(dflash_config.target_layer_ids, num_target_layers: 45).
- Hybrid stack: 34 KDA/linear-attention layers + 11 MLA/DSA layers with
slot-shared KV, mhc: true, DSA indexer on the sparse layers. The GB300 port
notes this "34 KDA/mamba + 11 MLA slot-sharing" KV grouping is the hard part.
- Rollback for the KDA recurrent state (analogous to the GatedDeltaNet
tape-replay path), plus family-specific attention masks / verify routing for
MLA+DSA layers.
Checkpoints
- Target (MLX):
pipenetwork/GLM-5.3-Flash-MLX-4bit (also 2/3/6-bit variants)
- Draft:
incoai/GLM-5.3-Flash-DFlash2 (bf16, 2.29 GB, CC BY-NC-ND 4.0)
Happy to test an experimental glm5_next backend on M-series hardware.
Request
Add a
Glm5NextTargetOpstarget adapter so dflash-mlx can runGLM-5.3-Flash(archGlm5NextForConditionalGeneration,model_type: glm5_next)with the
incoai/GLM-5.3-Flash-DFlash2drafter on Apple Silicon.Context
DFlash2DraftModel(block_size 8, selector_rank/top_k, two-tap conv), which_get_dflash_model_classesalready routes toDFlash2DraftModel. The blockeris the target side —
resolve_target_ops()only has QwenGdn / Gemma4 /MuseGlimmer and raises
NotImplementedErrorfor glm5_next.(SGLang PR #36708; a community vLLM port on GB300 reports ~137 → 305–470
tok/s, 54–67 % acceptance), so the drafter transfers well — what's missing is
an MLX target backend.
What the backend needs (per your roadmap's "family-specific backend" contract)
[5, 14, 24, 33, 42](
dflash_config.target_layer_ids,num_target_layers: 45).slot-shared KV,
mhc: true, DSA indexer on the sparse layers. The GB300 portnotes this "34 KDA/mamba + 11 MLA slot-sharing" KV grouping is the hard part.
tape-replay path), plus family-specific attention masks / verify routing for
MLA+DSA layers.
Checkpoints
pipenetwork/GLM-5.3-Flash-MLX-4bit(also 2/3/6-bit variants)incoai/GLM-5.3-Flash-DFlash2(bf16, 2.29 GB, CC BY-NC-ND 4.0)Happy to test an experimental glm5_next backend on M-series hardware.