Keep W4A16 zero-points packed at 4 bits instead of expanding to 2 bytes

[mgehre-amd]

Keep W4A16 zero-points packed at 4 bits instead of expanding to 2 bytes

Draft. The change is a clear win for g=32 decode (+10% end to end) but a measured,
reproducible regression of up to −14% on g=128 asymmetric at batch size 4. I am opening
this to agree on how to handle that trade-off, not to merge as-is.

Problem

process_weights_after_loading unpacked the zero-points and stored them in the activation
dtype, so the skinny GEMM streamed 2 bytes per group for values that only ever span 0..15.
The checkpoint already ships them packed 8-per-int32 along N; expanding them cost 4x the DRAM
traffic.

On gemma-4-31B-it-AWQ-4bit gate_up (1x43008x5376, g=32, asymmetric) that is 14.5 MB per
call instead of 3.6 MB, against 115.6 MB of weights.

This matters because the kernel is at the memory roofline. Counting all bytes it streams
(weights + scale + zero-point), every wvsplitk_int4 shape sits at 187-204 GiB/s next to
216 GiB/s for the fp16 wvSplitK lm_head in the same trace. There is no scheduling headroom
left: two earlier experiments that rearranged loads without changing byte counts (a wider
A_CHUNK, staging scale/zp in LDS) both measured within noise. Removing bytes is the only
lever that moves this kernel.

Change

The kernel takes the packed tensor and extracts the nibble: row n -> word[n/8], bits
4*(n%8), matching the layout unpack_quantized_values_into_int32(packed_dim=0) inverts.

The unpacked values are still built at load time, but only transiently, for the Triton
scale/zp carrier and the optional dense dequant copy, both of which need them expanded.

Results (gfx1151)

Per-call kernel time, 5 reps, median:

shape	before	after	delta	byte reduction
gate_up 1x43008x5376	642.3 us	608.5 us	-5.3%	-7.5%
qkv 1x16384x5376	251.7 us	238.1 us	-5.4%	-7.5%
down 1x5376x21504	340.0 us	316.7 us	-6.8%	-7.5%

75-90% of the theoretical maximum realised, which is what you expect if time tracks bytes.

End to end on the same model (unprofiled, 10 prompts):

metric	before	after	delta
decode	9.0 tok/s	9.9 tok/s	+10.0%
TTFT	2297 ms	2093 ms	-8.9%
% of roofline	79.6%	87.6%	+8 pts

Kernel resource footprint is unchanged (VGPR 86, no spills, no scratch, LDS 65536) -- the
shift+mask absorbs into existing register pressure, which is free on a memory-bound kernel.

Known regression / why this is a draft

Re-running every golden entry that moved by more than the tolerance band, three reps each,
five are reproducible slowdowns with an exact shared signature:

shape	g	provider	bs	before	rerun	delta
2048x2048	128	-zp	4	2.23	1.92	-13.9%
2048x2560	128	-zp	4	2.41	2.17	-10.0%
6144x2048	128	-zp	4	2.80	2.53	-9.6%
2048x2048	128	-zp-bf16	4	2.08	1.88	-9.6%
4096x2048	128	-zp	4	2.64	2.41	-8.7%

All g=128, all bs=4, all asymmetric. That is a mechanism, not noise:

• at g=128 the zero-point tensor is a quarter the size, so the byte saving is only ~2.2%
• at bs=4 the kernel does more compute; currently the kernel doesn't run memory transfers and compute fully overlapped (vmcnt goes to 0 in the loop body); the nibble extract (shift + mask + int->float) costs extra compute

Testing

pytest tests/kernels/quantization/test_hip_w4a16.py -> 119 passed (106 pre-existing + 13
new).

This PR adds the value-level coverage that did not exist: nothing in the suite checked what
wvSplitK_int4_g actually computes, so a wrong nibble index would have been silent. New tests
compare against a reference dequant across fp16/bf16 x g32/g128 x three shapes, including
N=8 (exactly one packed word), plus a test that the op rejects act-dtype zero-points rather
than reinterpreting them.

End-to-end: gemma-4-31B-it-AWQ-4bit MM sanity check passes (-> 'Red'), 10/10 requests,
1280 output tokens.

Goldens deliberately not included

golden/hybrid_w4a16_gfx1151.json is not updated here. Regenerating it showed the expected
+6.3% median on g=32 asymmetric entries (vs +7.5% theoretical), and symmetric entries unmoved
as a control -- but re-running the outliers found 11 newly-recorded entries that do not
reproduce: the fresh median matches the previous value instead. That is the recording-run
reproducibility issue already open on #1065, and committing those goldens would bake in
failures. They should be regenerated once that is resolved.