[tawei-amd]

What

Vendors the qwen3_5vl engine libraries from FastFlowLM_IRON#134 (merged as a4b25ce6), CI run 41150478193: Linux and Windows, XRT and HRX (src/lib/{xrt,hrx}/libqwen3_5vl_npu.so, qwen3_5vl_npu.dll, .lib). sha256 of the vendored files equal the run's artifacts.

For Qwen3.5 0.8B, 2B and 4B (their model.q4nx has no model.embed_tokens, only the tied q8_0 lm_head), the engine used to unpack the whole lm_head into a vocab x dim bf16 table (485 MiB / 970 MiB / 1.2 GiB of host RAM) and read one row per token. It now maps the lm_head tensor read-only from model.q4nx and dequantizes the one row on demand; a prompt's rows are hinted to the OS before prefill. The 9B has a BF16 embed_tokens and is unchanged. Addresses the first two items of #807 for these models.

Docs: host-memory note on the 0.8B, 2B and 4B model cards in docs/docs/models/qwen.md.

Results (Linux XRT, ctx 4096, this branch built locally with the vendored CI libs vs the previous libs)

model	anon memory after chat	peak RSS
qwen3.5:0.8b	801 -> 301 MiB	1953 -> 1474 MiB
qwen3.5:2b	1230 -> 337 MiB (cache on), 1199 -> 207 MiB (--prefix-slots 0)	3888 -> 2908 MiB
qwen3.5:4b	1813 -> 567 MiB	5804 -> 4602 MiB

• Output identity: 11 scenarios per model (think on/off, 2.6k-token prompt, 4-turn prefix-cache growth, image request and repeat, cancel then a normal request), 2B with the prefix cache on and off: content, reasoning text, token counts and prefix-cache lookups identical to the previous libs (0 differing scenarios in 4 runs).
• Embedding rows bit-identical on all 248,320 vocab rows for 0.8B, 2B and 4B.
• Speed unchanged (2B decode 27.97 vs 27.98 tok/s, TTFT 0.448 vs 0.450 s).
• IRON#134 CI: DocVQA deltas all +0.0 (XRT and HRX). The 0.8B GSM8K "6.0 -> 0.0" there is the baseline's different FLM server version; same server, old vs new library gives identical answers on 50/50 questions (both 0/50).
• Under memory pressure (model file evicted from the page cache) a prompt of many new tokens starts ~0.7-0.9 s later on the 2B (1.16 s -> 1.8-2.0 s); after load the pages are warm.

Not tested locally: the Windows and HRX runtimes (built and benched by IRON CI).

Note: the loaded model's model.q4nx stays mapped. On Windows it cannot be replaced or deleted while the model is loaded (documented on the model cards); Linux is unaffected because downloads write .part and rename.

🤖 Generated with Claude Code