llama: let the backend pick n_ubatch, and default to 2048 on RDNA3.5

[roberteg16]

Summary

The 512-token default physical batch predates these iGPUs. On gfx1151 a 512-token ubatch leaves the matrix cores short of work and re-reads the weights more times than necessary. Rather than hardcode a new number, this adds a way for a backend to state a preference.

• Adds LLAMA_N_UBATCH_AUTO, the new llama_context_default_params() value for n_ubatch. When set, llama_context queries each device for an optional ggml_backend_get_optimal_ubatch proc address. Devices with no preference return 0 and are ignored; if several disagree the smallest preference wins.
• Anything other than LLAMA_N_UBATCH_AUTO is passed through untouched, so an explicit -ub 512 still means 512.
• The CUDA/HIP backend returns 2048 for RDNA3.5 and 0 everywhere else, so no other architecture changes behaviour.
• llama-bench now reports the resolved value via llama_n_ubatch() rather than the requested one, since the request may be "auto".

Measured impact

gfx1151 (Radeon 8060S), Q4_K_M, -ngl 999 -d 0 -p 2048 -n 0. Both delta columns are against the same reference: today's default of -ub 512 with GGML_CUDA_GDN_CHUNKED unset.

• -ub 2048 — what this PR does on its own.
• -ub 2048 + CUDA: fused chunked gated_delta_net kernel (RDNA3.5) #54 — this PR together with the fused gated-delta-net kernel from CUDA: fused chunked gated_delta_net kernel (RDNA3.5) #54 enabled (GGML_CUDA_GDN_CHUNKED=1).

ᵈ marks rows where both columns are the same run: #54's kernel only applies to gated-delta-net models. * marks rows still >=3% relative stddev after re-measurement at -r 30.

model	repo	GDN	default (-ub 512)	-ub 2048	delta	-ub 2048 + #54	delta
Qwen2.5-0.5B-Instruct	bartowski	—	15606 ± 78	16021 ± 327	+2.7%	16021 ± 327 ᵈ	+2.7%
Qwen3.5-0.8B*	unsloth	yes	9206 ± 367	8097 ± 474	-12.1%	8828 ± 560	-4.1%
Qwen3-1.7B*	unsloth	—	5410 ± 176	5125 ± 144	-5.3%	5125 ± 144 ᵈ	-5.3%
gemma-2b-it	QuantFactory	—	4317 ± 124	4384 ± 126	+1.6%	4384 ± 126 ᵈ	+1.6%
SmolLM2-1.7B-Instruct	unsloth	—	4062 ± 101	3917 ± 87	-3.6%	3917 ± 87 ᵈ	-3.6%
gemma-4-E2B-it*	unsloth	—	3539 ± 109	3440 ± 157	-2.8%	3440 ± 157 ᵈ	-2.8%
Qwen2.5-3B-Instruct*	bartowski	—	3365 ± 133	3273 ± 80	-2.8%	3273 ± 80 ᵈ	-2.8%
Qwen2.5-3B-Instruct*	Qwen	—	3218 ± 113	3230 ± 45	+0.4%	3230 ± 45 ᵈ	+0.4%
gemma-3-4b-it	unsloth	—	2663 ± 77	2636 ± 36	-1.0%	2636 ± 36 ᵈ	-1.0%
Qwen3-4B	unsloth	—	2447 ± 67	2353 ± 54	-3.9%	2353 ± 54 ᵈ	-3.9%
Qwen3.5-35B-A3B	unsloth	yes	1697 ± 24	1606 ± 27	-5.3%	1789 ± 39	+5.4%
Qwen3.6-35B-A3B	unsloth	yes	1658 ± 45	1613 ± 48	-2.7%	1792 ± 28	+8.1%
Qwen2.5-7B-Instruct*	bartowski	—	1541 ± 88	1588 ± 18	+3.1%	1588 ± 18 ᵈ	+3.1%
gemma-4-26B-A4B-it*	unsloth	—	1527 ± 75	1668 ± 41	+9.3%	1668 ± 41 ᵈ	+9.3%
Llama-3.1-8B-Instruct	unsloth	—	1459 ± 27	1422 ± 16	-2.5%	1422 ± 16 ᵈ	-2.5%
Qwen3-8B	unsloth	—	1433 ± 31	1389 ± 10	-3.0%	1389 ± 10 ᵈ	-3.0%
GLM-4.7-Flash	unsloth	—	1013 ± 29	1132 ± 33	+11.7%	1132 ± 33 ᵈ	+11.7%
gemma-3-12b-it	ggml-org	—	868 ± 21	868 ± 9	-0.0%	868 ± 9 ᵈ	-0.0%
gemma-3-12b-it	unsloth	—	857 ± 14	866 ± 3	+1.0%	866 ± 3 ᵈ	+1.0%
Qwen3.6-27B	unsloth	yes	386 ± 2	379 ± 3	-1.8%	400 ± 4	+3.5%

	-ub 2048	-ub 2048 + #54
improve > +1%	5	8
flat (within ±1%)	4	4
regress > -1%	11	8
median	-2.1%	+0.2%
mean	-0.8%	+0.9%
worst	-12.1%	-5.3%
best	+11.7%	+11.7%

Only the four gated-delta-net rows differ between the two columns:

model	-ub 2048	-ub 2048 + #54
Qwen3.5-0.8B	-12.1%	-4.1%
Qwen3.5-35B-A3B	-5.3%	+5.4%
Qwen3.6-35B-A3B	-2.7%	+8.1%
Qwen3.6-27B	-1.8%	+3.5%

GDN's sequential scan gets slower as the ubatch grows, because the scan's serial length grows
with n_tokens — so a larger ubatch is actively wrong for those models today. #54's chunked
kernel removes that dependency, which is what turns three of those four rows positive.

Caveat on the second column

Merging #54 is not by itself sufficient. Its kernel is opt-in (GGML_CUDA_GDN_CHUNKED=1; unset means off, see ggml_cuda_gdn_chunked_supported), so the + #54 column describes a configuration, not the merge alone. The two changes are complementary and arguably should land together, or #54 should default its kernel on for RDNA3.5.

Qwen3.5-0.8B stays negative either way. At ~9k tok/s a 2048-token ubatch is most of the prompt, so there is little left to overlap.

Test plan

• Builds clean with HIP on gfx1151
• Default resolves to n_ubatch = 2048 on gfx1151; explicit -ub 512 / -ub 1024 / -ub 256 are preserved
• n_ctx clamping still applies (-p 512 yields n_ubatch = 512)