Skip to content

fix(onnx-to-hip): resolve Neg in host slice extents - #1332

Merged
zz002 merged 1 commit into
mainfrom
fix/slice-dynamic-negative-start
Oct 27, 2026
Merged

zz002 merged 1 commit into
mainfrom
fix/slice-dynamic-negative-start

Conversation

@zz002

@zz002 zz002 commented Oct 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

SliceToHip now computes the true extent of a Slice whose start is -Shape(x)[k], instead of falling back to the data dim upper bound.

Related issue or design

None

Why

resolveHostIndex did not walk through Neg, so such a start failed to resolve and the extent fell back to the data dim. A graph that keeps the last S columns of the attention mask, Slice(attention_mask, -Shape(input_ids)[1], INT64_MAX), and gates a [B, S, ...] tensor with it via Where then gets a [B, total] mask at decode; Where broadcasts the [B, 1, ...] operand to [B, total, ...] and every downstream op sees the wrong sequence length. Prefill (S == total) hides it.

What

  • resolveHostIndex resolves onnx.Neg and hip.neg as 0 - x. Both spellings are accepted for the same reason as Sub/Add/Mul: the greedy driver does not order this pattern against the producer rewrites.

Test plan

  • LIT: test_slice.mlir tests 17 (ONNX spelling) and 18 (hip.neg spelling); full LIT suite passes
  • Numeric: test_shape_ops.py::TestSlice::test_slice_mask_tail_by_sequence_length for (S, total) = (1, 20), (3, 12), (7, 7) matches CPU EP exactly; the two decode cases fail without the fix

Notes for reviewers

None

SliceToHip sized a Slice whose start is -Shape(x)[k] by the full data dim,
because resolveHostIndex did not walk through Neg. Qwen3.8's EngramGate
padding slices attention_mask with start = -Shape(input_ids)[1]; at decode
the inflated extent broadcast the hidden state from [B,1,...] to
[B,total,...].
@github-actions

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

@github-actions

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 1.12256E-07 64 0
GroupQueryAttention_seq256 59.2561 2621440 0
MatMulNBits_o_seq128 259.905 368640 0
QMoE_seq128 24.2583 368640 0

Threshold: 0.01 | Run: 7915 - Commit: db0f9a6

@github-actions

github-actions Bot commented Oct 26, 2026 •

Copy link
Copy Markdown

Binary sizes

Baseline: run 40294400486 - 7579ebf (dll-sizes)

File This (MB) Main (MB) Delta (MB)
hipgpu.dll 59.31 59.31 0.00
custom_kernels_gfx1170.dll 5.55 5.55 0.00
custom_kernels_gfx11-generic.dll 5.79 5.79 0.00

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 8.76 7.42 272 3 1158
GroupQueryAttention_seq128 11265.60 2.54773 1 5 229
matmul_down_seq128 524.12 3.30 119 3 274

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 8.82 47.95 246 3 15505

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 8.83 10.23 245 3 15638

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 90.4 74.7 1.25 12.23
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 117.4 47.7 1.15 5.23

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 119 47.9

VLM Benchmark Results

Model TTFT (ms) TPS Output
gemma-4-E4B-it-fp16-int4-kquant-prune-logits 3966.8 7.3 This is a high-quality, outdoor photograph featuring a beautiful, fluffy white dog, likely a West Highland White Terrier or a si
Qwen3.8-27B-fp16-ve-fp16-int4-k_quant-gs128-text-dml-onnx 14451.0 14.0 The user wants a detailed description of the image. Let me carefully observe all elements before writing. Subject: A white do

Run: 7915 - Commit: db0f9a6

@zz002
zz002 merged commit 7f2f3ce into main Oct 27, 2026
12 checks passed
@zz002
zz002 deleted the fix/slice-dynamic-negative-start branch October 27, 2026 02:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants