Normalize W4A16 profiler scope grammar and report unique expert count

[roberteg16]

Summary

Two producer-side gaps that stop a W4A16 trace from becoming a bandwidth number.

1. Three spellings of one field. #1067 made the labels self-describing, but the group size is written three ways: bare g32 on the dense scopes, g=32 on the MoE ones, gs=/sk= on the AWQ GEMVs. Settle on key=value everywhere.

- wvsplitk_int4 1x43008x5376 g32 asym
+ wvsplitk_int4 1x43008x5376 g=32 asym

- awq_gemv_hip 5376x4096 gs=128 sk=2
+ awq_gemv_hip 5376x4096 g=128 split_k=2

hybrid_w4a16.py covers wvsplitk_int4, hybrid_dequant_w4a16 and hybrid_triton_w4a16 in one line — they share the _gz suffix.

2. The MoE scope carried no expert count. Weight slabs dominate that kernel's traffic and it loads one per block, but the count cannot be recovered downstream: a trace records expert_ids' length, not its contents, and several blocks can share an expert. So the block count is only an upper bound, and E/top_k a looser one. fused_moe_wvsplitk_int4 now emits the measured value:

fused_moe_wvsplitk_int4 4x1024x2048 E=256 top_k=8 g=128 sym experts=5

On the consumer side that is the difference between 5.23 MiB and 33.08 MiB of modelled traffic for a 32-block launch touching 5 distinct experts — 6.3x, in a byte model whose whole job is to be exact.

Cost when not profiling: none

The count needs a device->host sync, so _fused_moe_wvsplitk_scope() follows _moe_gemm_w4a16_scope() exactly: it returns contextlib.nullcontext() before touching the tensor unless VLLM_CUSTOM_SCOPES_FOR_PROFILING/VLLM_NVTX_SCOPES_FOR_PROFILING is set.

The guard has to live in a helper rather than inline in the f-string — Python evaluates call arguments eagerly, so an inline bincount would run on every decode step regardless. The sync perturbs the CPU timeline, not the GPU kernel duration the roofline measures.

Verified rather than argued: passing a sentinel whose every accessor raises, the scope returns a nullcontext untouched with profiling off. With it on, expert_ids = [3, 3, 7, 7, 7, 11, -1, -1] yields experts=3 — distinct ids, duplicates collapsed, -1 padding excluded. The block-count bound would have said 8.

Not a duplicate

• Keep W4A16 zero-points packed at 4 bits instead of expanding to 2 bytes #1073 (Keep W4A16 zero-points packed at 4 bits) touches hybrid_w4a16.py too, but the load path (process_weights_after_loading), not the profiler labels. Different concern, and complementary: its own description leans on the corrected byte accounting this labelling enables ("every wvsplitk_int4 shape sits at 187-204 GiB/s next to 216 GiB/s for the fp16 lm_head"). Expect a small contextual conflict in that file; happy to rebase behind it.
• [ROCm][Kernel] W4A16 MoE prefill: M-aware moe_align block size (1.6x TTFT) #1069, [ROCm][Kernel] W4A16 skinny GEMM: hand-asm kernel for gfx1151 #1045, [ROCm][MoE] W4A16 MoE routing-distribution benchmark suite for the gfx11 prefill GEMM #1020, MoE: Grouped Triton GEMM for TTFT improvements #970, Tune hybrid_triton_w4a16 prefill kernel for gfx1151 #879 are kernel/tuning work on W4A16; none touch record_function labels.
• No open PR matched a search for profiler-annotation changes.

Test plan

pytest tests/kernels/quantization/test_hybrid_w4a16_perf.py -> 137 passed, 3 failed.

The 3 failures are pre-existing flaky perf assertions, not regressions. Same three tests, 5 repetitions each:

branch	failures per repetition
gfx11 (base, without this change)	1, 1, 1, 0, 2
this branch	0, 1, 1, 1, 1

5 failures in 15 test-runs on base vs 4 in 15 here — indistinguishable. They also fail in the improvement direction (e.g. batch_size=1: 0.76 TFLOP/s (expected 0.68 +[-8,8]%) FAIL (improvement, +11.0%)), which a label change cannot cause. This is the golden-reproducibility issue already tracked on #1065 and noted in #1073.

That run is also the direct check that the in-repo label consumer still works: test_hybrid_w4a16_perf.py:627 reads the scope name back out of the profiler to report which path ran, and its output tags the rows [wvsplitk_int4] / [hybrid_triton_w4a16] — a broken parse would read [unknown].

End-to-end capture on gemma-4-31B-it-AWQ-4bit (decode, --profile --profile-record-shapes --profile-export-trace), with this branch installed:

• Labels come out in the new grammar: wvsplitk_int4 1x43008x5376 g=32 asym
• The roofline consumer reads them with zero warnings
• Per-shape bandwidth 188-204 GiB/s (81.8-88.8% of peak), matching the pre-change capture's 188.6-202.6 GiB/s within run-to-run noise

Also run:

• pre-commit — full hook set passed on commit (ruff, format, sign-off, config validation)
• Consumer side, 599 tests, against real traces in FaaSApps/rocm-scripts#878 — including both the old and new spellings, so old captures keep working

Still not run:

• A real MoE capture with VLLM_CUSTOM_SCOPES_FOR_PROFILING=1 end to end. No MoE AWQ checkpoint is cached on this machine (only GGUF ones and the dense gemma), so this would need a multi-GB download. The scope function itself is verified directly as described above, and the dense capture proves the label -> trace -> roofline path; what remains unproven is only that the MoE scope's window attaches to the right op in a real trace.

(An earlier revision of this PR said the test suite could not run at all. That was wrong: it needed amdsmi on PYTHONPATH — without it vLLM resolves to UnspecifiedPlatform and every gfx1151-gated test skips.)

AI assistance

This change was written with AI assistance (Claude). Per AGENTS.md: the analysis motivating it — the byte accounting, the annotation/consumer contract, the expert-count bound — was developed and verified interactively against real gfx1151 traces, and I have reviewed every changed line. Kept as a draft pending the MoE end-to-end capture above.