[zz002]

Summary

Build OGA v0.17.0 with the Qwen3.8-Flash-Next runtime: microsoft/onnxruntime-genai#2801 (the Generator-path runtime of #2714 plus the IndexerCache fix #2782, ported onto v0.17.0) and microsoft/onnxruntime-genai#2792 (onnxruntime-extensions pin with Qwen3_5Tokenizer).

Related issue or design

microsoft/onnxruntime-genai#2714, microsoft/onnxruntime-genai#2782, microsoft/onnxruntime-genai#2792, microsoft/onnxruntime-genai#2801

Why

The pinned OGA rejects the Qwen3.8 genai_config.json (Unknown value "rotary_position_ids"). #2714 adds that runtime but is based on OGA main after v0.17.0, so its diff does not apply to the tag alone. #2801 carries only the Generator-path part of #2714 (no Engine or CUDA changes) on top of v0.17.0, so no other main PRs are needed.

What

• OGA_PR_PATCHES (Windows and Linux): 2685 2691 2705 2712 2792 2801.
  • #2792 replaces 2763: it pins onnxruntime-extensions main plus Fix UTF-8 offset in the STL regex pre-tokenizer fallback microsoft/onnxruntime-extensions#1136 (the pre-tokenizer fix 2763 carried), which also brings Recognize Qwen3.5 tokenizer class names microsoft/onnxruntime-extensions#1142 (Qwen3_5Tokenizer). 2763 now merges OGA main into its branch, so its diff no longer applies on v0.17.0. Linux gets the pin too.
  • #2801 is applied after #2685, which rewrites the same extensions pin line, so the pin stays in its own PR.
• Exclude *.md from the applied diffs; the build does not read Markdown.
• Windows: retry the .diff fetches (-MaximumRetryCount 6), as Linux already does with curl.

Test plan

• The list applies cleanly on v0.17.0 with the CI git apply --3way command (locally)
• build-windows / build-linux green with this list (ci(oga): replace the Qwen3.8 patch chain with one v0.17.0 draft #1351)
• Qwen3.8-Flash-Next with the Windows gpu-test-package: image description correct; 1024-token prefill / decode on par with the earlier OGA build from #2714 plus its main dependencies
• GPU tests show no regression on the existing OGA models

Notes for reviewers

The OGA cache key only contains PR numbers, so pushing new commits to #2792 or #2801 does not invalidate it.

[github-actions]

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

1. The change is focused. Substantial work links the relevant issue
   or design discussion.
2. The PR documents relevant test results and updates affected
   documentation.
3. If AI tools provided substantial assistance, the description
   explains what was assisted and how it was validated, and commit
   trailers identify the tool. The contributor has reviewed and
   understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

[github-actions]

L2 Accuracy Results (EP vs CPU)

Model	Combined L2	Total Elems	Skipped NaN/Inf
conv_test_hybrid	1.12256E-07	64	0
GroupQueryAttention_seq256	59.2561	2621440	0
MatMulNBits_o_seq128	259.905	368640	0
QMoE_seq128	24.2583	368640	0

Threshold: 0.01 | Run: 8027 - Commit: 0973974

[github-actions]

Binary sizes

Baseline: run 40617608668 - cf8ff09 (dll-sizes)

File	This (MB)	Main (MB)	Delta (MB)
hipgpu.dll	59.34	59.34	0.00
custom_kernels_gfx1170.dll	5.55	5.55	0.00
custom_kernels_gfx11-generic.dll	5.79	5.79	0.00

MorphiZen EP Performance Results

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	8.77	7.43	245	3	1159
GroupQueryAttention_seq128	11451.60	2.53588	1	5	229
matmul_down_seq128	514.67	3.26	125	3	273

EPContext Export Performance

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	8.73	48.33	252	3	15505

EPContext Import Performance

Model	QPS	Session (s)	1st Infer (ms)	CPU%	Mem (MB)
full_model_seq128	8.80	10.55	253	3	15638

OGA Benchmark Results

Model	Warmup	Reps	Prompt Len	Gen Tokens	TTFT (ms)	TPS	Peak Mem (GB)	GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32	1	5	128	128	91.4	75.1	1.25	12.24
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml	1	5	128	128	118.5	47.7	1.14	5.23

OGA Wheel Smoke (Python benchmark_e2e.py)

Model	TTFT (ms)	TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml	118	47.9

VLM Benchmark Results

Model	TTFT (ms)	TPS	Output
gemma-4-E4B-it-fp16-int4-kquant-prune-logits	4001.2	7.8	This is a high-quality, outdoor photograph featuring a beautiful, fluffy white dog, likely a West Highland White Terrier or a si
Qwen3.8-27B-fp16-ve-fp16-int4-k_quant-gs128-text-dml-onnx	4910.2	14.0	The user wants a detailed description of the image. Let me carefully observe all elements before writing. Subject: A white do

TTFT / TPS vs main

Baseline: run 40617608668 - cf8ff09 | Gate: TTFT +20% / TPS -20%

Suite	Model	TTFT (ms)	Main	Delta	TPS	Main	Delta
oga	gpt-oss-20b-webgpu-int4-rtn-block-32	91.4	92.4	-1.1%	75.1	75.0	+0.2%
oga	Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml	118.5	119.4	-0.8%	47.7	47.7	0.0%
vlm	gemma-4-E4B-it-fp16-int4-kquant-prune-logits	4001.2	4096.2	-2.3%	7.8	7.3	+6.0%
vlm	Qwen3.8-27B-fp16-ve-fp16-int4-k_quant-gs128-text-dml-onnx	4910.2	4928.4	-0.4%	14.0	14.0	+0.1%

Run: 8027 - Commit: 0973974

[amd-mingw]

LGTM