Eval bug: DFlash2: --split-mode tensor aborts with SPLIT_AXIS_UNKNOWN on ROCm (Qwen3.8-27B, 2x RX 7900 XTX)

[a-n-t-0]

Name and Version

Current master:

b10660
6c84c7d model: add Qwen3.8-Flash-Next (qwen4exp) (#27742)

This includes the merged DFlash2 implementation from #27342.

Operating systems

Linux

GGML backends

HIP

Hardware

Ryzen 3600
Asus prime x570 pro (8x/8x)
2 x rx 7900 xtx

Models

Models

Target:

Qwen3.8-27B-Q8_0.gguf

DFlash2 drafter:

Qwen3.8-27B-DFlash2-Q8_0.gguf

Relevant configuration

--spec-type draft-dflash
--spec-draft-n-max 6
--spec-draft-p-min 0
--n-gpu-layers 999
--n-gpu-layers-draft 999

--flash-attn on
--ctx-size 196608
--cache-type-k f16
--cache-type-v f16

--split-mode tensor
--tensor-split 1,1
--fit off

Problem description & steps to reproduce

Description

DFlash2 works correctly with Qwen3.8-27B on ROCm when using --split-mode layer, but crashes during context initialization when using --split-mode tensor.

Importantly, tensor split itself works correctly on the same machine/model when using the native Qwen3.8 MTP drafter.

So the failure appears to be specifically triggered by the combination:

DFlash2 + SPLIT_MODE_TENSOR

rather than Qwen3.8 tensor parallelism alone.

Version

Current master:

b10660
6c84c7d5d model: add Qwen3.8-Flash-Next (qwen4exp) (#27742)

This includes the merged DFlash2 implementation from #27342.

Hardware / backend

• Linux
• ROCm / HIP + RCCL
• 2x AMD Radeon RX 7900 XTX
• gfx1100
• HIP_VISIBLE_DEVICES=1,0

Models

Target:

Qwen3.8-27B-Q8_0.gguf

DFlash2 drafter:

Qwen3.8-27B-DFlash2-Q8_0.gguf

Relevant configuration

--spec-type draft-dflash
--spec-draft-n-max 6
--spec-draft-p-min 0
--n-gpu-layers 999
--n-gpu-layers-draft 999

--flash-attn on
--ctx-size 196608
--cache-type-k f16
--cache-type-v f16

--split-mode tensor
--tensor-split 1,1
--fit off

DFlash2 loads and initializes correctly:

common_speculative_impl_draft_dflash: adding speculative implementation 'draft-dflash'
common_speculative_impl_draft_dflash: - n_max=6, n_min=0, p_min=0.00
common_speculative_impl_draft_dflash: - block_size=8, mask_token_id=248070, n_extract=5, sample_from_anchor=true

Immediately afterwards the server aborts with:

ggml/src/ggml-backend-meta.cpp:537:
GGML_ASSERT(ret.axis != GGML_BACKEND_SPLIT_AXIS_UNKNOWN) failed

The backtrace goes through:

ggml_backend_meta_get_split_state
ggml_backend_meta_buffer_init_tensor_impl
ggml_gallocr_alloc_graph
ggml_backend_sched_alloc_graph
llama_context::process_ubatch
llama_context::decode
common_context_can_seq_rm
server_context_impl::load_model

Control tests

Same target + DFlash2, changing only:

--split-mode layer

works correctly.

More importantly, on the same system:

Qwen3.8-27B + native MTP + --split-mode tensor

works correctly.

Therefore the reproducible matrix is:

Qwen3.8 + MTP     + tensor → OK
Qwen3.8 + DFlash2 + layer  → OK
Qwen3.8 + DFlash2 + tensor → ASSERT

This makes it look like DFlash2 introduces a graph operation / split-state propagation case that the tensor-parallel meta backend currently cannot resolve.

Possibly related to #27116 because the final assertion is the same, but unlike that report this reproduces with F16/F16 KV and tensor mode works when DFlash2 is not used.

Happy to test patches on 2x gfx1100 / ROCm.

First Bad Commit

No response

Relevant log output

Logs

[thatname]

I have the same error using CUDA.
PR #27858 fixed my error, you can try it.