feat(gqh): Qwen3.8-27B GQH + DFlash2 SpecLA (I8 N=8) — staging for main - #34
Open
DeanoC wants to merge 27 commits into
Open
feat(gqh): Qwen3.8-27B GQH + DFlash2 SpecLA (I8 N=8) — staging for main#34DeanoC wants to merge 27 commits into
DeanoC wants to merge 27 commits into
Conversation
Wire the DSpark drafter heads (low-rank Markov bigram correction + confidence head) into the qwen35 spec-decode loop, so Qwen3.8-27B DSpark drafters (e.g. RadixArk/Qwen3.8-27B-DSpark) run with full head support: - spec loop: markov-corrected greedy chain (fused single-graph variant with non-fused fallback) replaces plain argmax projection when the drafter ships DSpark heads; DDTree candidate top-k gets the markov bias too. Env-gated: DFLASH_QWEN35_DSPARK, DFLASH_QWEN35_FUSED_DSPARK, DFLASH_QWEN35_DSPARK_TREE (all default on). - target capture layers now follow the drafter GGUF's dflash.target_layer_ids instead of the evenly-spaced derivation; the Qwen3.8 drafter is trained on layers 4/16/28/40/52, not 1/16/31/46/61. - draft loader: dflash.mask_token_id from the drafter GGUF wins over the family default (Qwen3.8 drafter uses 248077, default was 248070), and optional YaRN rope scaling keys are parsed into DraftWeights. - draft graph: rope calls honor the drafter's YaRN config (previously hardcoded plain NEOX rope). - Qwen35DFlashTarget exposes lm_head for the fused head path. - convert_dflash_to_gguf.py: handle single-file DSpark releases (markov/ confidence heads inline in model.safetensors), transformers>=5 nested rope_parameters and dflash_config.mask_token_id, and emit YaRN scaling metadata. The confidence-gate adaptive block length is not wired yet (q_len sizes the per-request step buffers); the chain runs with the gate off.
Verify/accept now run over v_len (the drafted chain's actual length) instead of the buffer-sizing q_len, so the DSpark confidence gate's adaptive block truncation is structurally supported. The gate itself stays off by default (DFLASH_QWEN35_DSPARK_CONFIDENCE_THRESHOLD=0): with the RadixArk Qwen3.8 drafter, any threshold in 0.1-0.5 truncates to the same short chain regardless of value, so the confidence scores coming out of the shared head path look mis-scaled and need a separate investigation before the gate can help. threshold=0 is bench-verified regression-free.
- ggml_ssm_conv_step: one kernel for the causal-conv decode/verify step (history window + silu(conv) + in-place history write-back + optional rollback window copy) replacing transpose/concat/ssm_conv/silu/cpy. - ggml_gated_delta_net_set_raw_gates: the GDN kernel applies sigmoid(beta) and softplus(alpha + dt_bias) * A itself. - ADD + RMS_NORM + MUL fusion (residual add materialized alongside the normalized output) in the CUDA/HIP graph evaluator. - legacy pool MAX_BUFFERS 256 -> 1024: LUCE_Q8_MEMO holds ~300 pooled buffers per evaluation; a full pool freed in-flight buffers with cudaFree and produced illegal memory accesses on long prefills.
Rename the RDNA small-tile macro to GGML_CUDA_MMQ_SMALL_TILE and apply it to IQ4_XS/Q4_K/Q5_K/Q6_K/Q8_0 in addition to the ROCmFPX formats. At spec-decode verify widths (N<=16) the 128-row tile leaves a 5120-row projection with only 40 blocks on a 64-CU gfx1201; 64x64/4-warp tiles measured +12-23% on those shapes (verify step 43.8 -> 39.7 ms on Qwen3.8-27B) at ~8% prefill cost.
- loader places attn_gate|attn_qkv and ssm_beta|ssm_alpha back to back and exposes zero-copy stacked aliases (L.wqkv_z, L.ssm_ba): one GEMV each instead of two (DFLASH_QWEN35_NO_STACK=1 disables). - FFN uses ggml_swiglu_split so the backend fuses gate/up/GLU into one vector kernel at decode. - DeltaNet block: single l2_norm over the q|k slab, ggml_ssm_conv_step, raw-gate gated_delta_net (in place, no state copy), no q/k head repeat (the kernel broadcasts). DFLASH_QWEN35_NO_FUSED_KERNELS=1 keeps the op-by-op graph for A/B. - DFLASH_KV_ROTATE=0 skips the FWHT K/Q rotation (precision-neutral with q8_0/f16 caches, two fewer launches per attention layer). Qwen3.8-27B IQ4_XS on R9700: plain decode 30.4 -> 33.8 tok/s with identical greedy output.
- Qwen35AdaptiveSpecPolicy: EMA of accepted draft tokens per step; below 0.8*(spec_step_ratio-1) the loop runs a burst of plain-decode steps (seed-only verify, no drafter/heads/snapshot/rollback, features still captured) and probes again afterwards. Env DFLASH_QWEN35_SPEC_STEP_RATIO (default 1.7, 0 disables) and DFLASH_QWEN35_AR_BURST (default 40). Low-acceptance prose 28.1 -> 32.4 tok/s, code/mixed unchanged. - Confidence gate now uses the fused Markov graph and truncates on the host; DFLASH_QWEN35_DSPARK_CONF_DEBUG=1 prints per-position scores. - spec-profile hooks for the chain path (project/snapshot/verify/ rollback/feature).
launch_fattn was told the vec kernel consumes D keys per step; it walks nthreads (128) per step, so a 256-key window at head_dim 256 ran as one block per head. Passing nthreads lets it use two blocks per head plus the combine pass: Qwen3.8-27B plain decode 34.3 -> 34.6 tok/s on R9700, identical output.
The first spec step after a plain-decode burst updates the acceptance EMA with alpha 0.5 so a stream that became predictable leaves plain decode immediately; step ratio and start value keep the measured best balance (45.7 / 31.8 / 40.4 tok/s code / prose / mixed).
The break-even acceptance now follows live EMAs of the spec-step and plain-step wall times (default 1.9 until both are measured), so it is right for any drafter block size (width-8 DSpark and width-16 DFlash measure ~1.8 on gfx1201).
DFlash 2 (z-lab/inco, e.g. z-lab/Qwen3.8-27B-DFlash2) is the DFlash backbone plus a grouped dynamic causal conv around attention and MLP in every layer and a candidate selector head (top-k lm_head candidates per block position, one path scored by a low-rank bigram form). - converter: maps attention_conv/mlp_conv (base kernels F32, kernel projections) and candidate_selector tensors, emits dflash2.* metadata, reads block_size from dflash_config, emits SWA pattern for drafters with causal sliding layers. - loader: DraftConvWeights per layer, DraftSelectorWeights, shape checks. - draft graph: conv prepare/finish (two taps over the block, per-element base + per-group dynamic coefficient) in both the stateless and the cached-KV builders. - selector chain: top-k via the target's GPU top-k (kMaxK 8 -> 16), one cached graph for hproj + codebook row gathers, host path search. - spec loop uses the selector before the DSpark/argmax paths. Qwen3.8-27B IQ4_XS on R9700, q8_0 drafter, greedy: 109.9 code / 50.7 prose / 111.8 mixed tok/s (DSpark drafter: 45.6 / 32.4 / 38.6); avg 5.9-6.0 accepted tokens per 8-token block on code, ~2.7 on prose.
With the 64-row/4-warp tile the mmq_x=32 instantiation runs at 180 GB/s on gfx1201 (17408x5120 IQ4_XS) against 443 GB/s at mmq_x=16 and 315 at 48, so N=17..32 batches (DDTree budgets, prefill remainders) took 2.4x longer than N=16 or N=40. Choose the next tile instead.
Resolves the conflicts with SpecLA (Luce-Org#606) and the packed concurrent prefill work (Luce-Org#595): - SSM_CONV op_params[0]: 1 stays the SpecLA heavy-light conv, the dflash fused conv step now uses 2; the CUDA dispatcher and supports_op handle both. - GDN raw-gate mode no longer uses src[8]/op_params[2] (compact-decode slot ids and the SpecLA marker): the loader builds one f32 [dt_bias | A] tensor per DeltaNet layer (TargetLayer::ssm_gate_ba, own small buffer) and ggml_gated_delta_net_set_raw_gates() attaches it as src[9] with op_params[10] = 1. - gated_delta_net.cu launchers carry both the active-slot arguments and the raw-gate parameters. - build_delta_net_block keeps main's token-axis segment structure; the stacked (z|qkv) and (beta|alpha) projections, the fused conv step, raw gates, single l2_norm over q|k and the head-broadcast shortcut apply on the plain single-sequence chain path only (ragged, compact-decode, SpecLA and chunked paths take main's materialized ops). - MMQ tiles: Q4_K keeps main's 128x64 (LUCEBOX_RDNA_MMQ_Y); IQ4_XS, Q5_K, Q6_K, Q8_0 and the ROCmFPX formats keep the 64x64 small tile. - Spec loop: DSpark Markov top-k stays available inside the non-conditional DDTree branch; SpecLA's conditional-draft path and draft-KV flag are preserved. Verified on lucebox8 (R9700): AR 34.6 tok/s, DFlash2 110/54/115 (code/prose/mixed), DSpark 48/32/40.
The DDTree verify path fell back to the generic per-token GDN kernel (61-196 us/layer on gfx1201) because the grouped-cols kernel had no parent_ids handling. Port the DFS branch-transition state reload into the grouped kernel: at parent_ids[t] != t-1 the register state shard reloads from the parent's stored intermediate state (same-thread read-after-write, no barrier), root-level siblings reset to the pre-block state, and intermediates are written in tree mode so later branches can read them. Verified numerically against the generic tree kernel on a 13-node branchy tree (max rel diff 8.4e-7, reduction-order noise only); end-to-end DDTree budget-12 on the R9700 matches text output on like-for-like runs at +2% tok/s.
DDTree branches were chosen by raw per-position top-k log-probs, ignoring the DFlash2 selector entirely (it only improved the chain path). Factor the chain selector into dflash2_score_candidates() + a host-side branch-conditioned topk, and feed DDTree through build_ddtree_conditional: each expansion scores candidates as logp + selector compatibility with the branch's actual parent, log-softmax-normalized per position so cumulative best-first comparisons across depths stay on a log-prob scale (without the normalization the raw dot term mis-allocates the budget: code 126 -> 106). DFLASH_QWEN35_DSPARK_TREE/raw top-k remain the fallback; DFLASH_QWEN35_DFLASH2_TREE=0 disables. R9700, budget 12 (code/prose/mixed tok/s): raw tree 126/56/112, selector tree 121/63/116, chain 112/62/124. The selector tree no longer collapses on low-acceptance content; chain remains the serving default.
…1201 matvec
Enables Qwen3.8-class GGUFs quantized to the GQH format produced by geo-quant.
Four rungs, appended after the last existing type so nothing renumbers:
108 GQH3 3.28125 bpw 256-weight superblock, 105 B (+5 B header in KV)
109 GQH2_H 2.28125 bpw 256-weight superblock, 73 B (+5 B header in KV)
110 GQH2_C 2.0625 bpw 256-weight superblock, 66 B (fp16 d in-block)
111 GQH4 4.28125 bpw 256-weight superblock, 137 B (+5 B header in KV)
A fixed-size ggml block cannot hold the 5-byte per-tensor prefix the wire puts
in front of the superblock stream, so the loader reads tensor_scale + grid code
from GGUF KV ('geoquant.gqh.headers') and registers them against the tensor's
device pointer (ggml_gqh_register). Decoding an unregistered GQH tensor aborts
rather than producing silent garbage. The registry is torn down before
ggml_free so a stale entry cannot shadow a later load that reuses the address.
What lands:
- ggml/src/gqh.{h,cpp} + gqh-tables.h: wire format, grids, CPU decode
- ggml/src/ggml-cuda/gqh.{cu,cuh}: the fused decode matvec, optimized for
gfx1201 (software-pipelined wire loads, exact-width ncols instantiations
1..5 with a shared activation read, occupancy-fit rows-per-warp arm, one
LDS level table per rung)
- ggml.c type traits, convert.cu dequant fallback, ggml-cpu decode hook
- ggml-cuda.cu dispatch hook (GGML_GQH_FUSED=0 forces the fallback)
- both mul_mat_vec_q gates consult ggml_cuda_qtype_has_no_mmvq(): GQH has no
vec_dot, so every path that can reach MMVQ must exclude it -- the shared
fusion gate and the unfused dispatch. Missing either aborts in mmvq's
default case.
- tests: test-gqh-cpu-decode + test-gqh-backend over 4 rungs x 3 shapes
against frozen f32 reference vectors, plus the header-KV parser test
Measured on an AMD AI PRO R9700 (gfx1201, 32 GB) with a 27B Qwen3.8 GQH
artifact (298 GQH4 / 10 GQH3 tensors, 13.2 GiB resident):
before: refuses to load ("invalid ggml type 111, should be in [0, 108)")
after: N=1 forward 32.8 ms, greedy decode 29.0 tok/s
ctest -R gqh: 25/25 pass, bit-identical to the f32 references.
Note the nextn/MTP blocks these artifacts carry are already handled by Luce-Org#610
(validate_embedded_nextn_blocks) -- no loader change is needed for them here.
The DFLASH27B_CHUNKED path was unusable on ROCm: a 512-token prefill forward took 39 s. Two causes, both fixed: - the [CS, CS] triangular solve at CS = 64 missed ggml-cuda's fast warp kernel (k <= 32) and fell into cublasStrsmBatched; CS = 32 keeps the solve on the fast path (graph cap raised to 32k nodes to match), - the per-chunk slices are strided views, which pushed every chunk matmul into cublasGemmBatchedEx; on ROCm that API stages its device pointer arrays through per-call pinned host allocations (~1 ms of hipHostMalloc/hipFree per node). ggml_cont on the sliced operands restores the strided-batched fast path. Result: 39 s -> 0.6 s per 512-token forward, output verified ~1e-6 against the sequential kernel at T = 64..2048 including padded chunks. Still OFF by default: on gfx1201 the sequential fused GDN kernel wins (514 ms vs 667 ms per forward; the ~20k-node chunk graph costs more in launches than it saves in serialization). DFLASH27B_CHUNKED=1 opts in, and the gate is per-call now, so enabling it no longer disables the raw-gate fusion on the decode path as a side effect. Also: env-gated DFLASH_PREFILL_TIMING=1 build/alloc/compute breakdown per prefill ubatch, and drop the ROCMFP requant experiment script that slipped into the merge commit (the format was refuted for this target). R9700 regression check (pure-IQ4_XS target): AR 36.4-36.6, DFlash2 spec 111/62/123 code/prose/mixed, prefill 1036/1102/1038 tok/s @512/2048/6000.
The fp64 RoPE path (required for Qwen3.5-family freq_base=1e7, see the fp32 precision wall note) computed pow(double, double) per element. On RDNA4 that libcall made rope_multi the second-largest prefill kernel: 692 us per launch at n_tokens=512 vs 76 us for the fp32 upstream kernel, ~33 ms of a 514 ms 512-token prefill forward. Replace pow() with binary exponentiation (<= 7 double multiplies for exponent < 128), keeping the large-freq_base precision to within 1 ulp. R9700: 512-token prefill forward 514 -> 414-423 ms (prefill ~996 -> ~1225 tok/s); DFlash2 spec decode 112.7/62.2/125.1 code/prose/mixed (from 111/61.5/123.4) with per-position acceptance identical.
The dense hybrid types compile their MMQ instances with the 64x64 small tile (GGML_CUDA_MMQ_SMALL_TILE), which wins 12-23% at spec-decode verify widths but re-streams the weights through narrow x-tiles at prefill widths (measured +16-18% kernel time at N=512 vs the 128x128 upstream shape). The tile shape is baked into every mmq.cuh constexpr via macros, so one TU can only hold one shape. Add big-tile twin instances for IQ4_XS/Q4_K/Q5_K/Q6_K/Q8_0 that re-include mmq.cuh inside namespace lucebox_mmq_big with no tile macro, giving the 128x128 shape distinct symbols, plus bridge functions and a runtime dispatch: RDNA4 + ncols_dst >= 256 takes the big tile (measured crossover: small wins to N=64, tie at 128, big wins 16-18% at 512); everything else keeps today's path. LUCE_MMQ_BIG_PREFILL=0 disables. gfx1151 behavior unchanged. R9700, Qwen3.8-27B pure-IQ4_XS: 512-token prefill forward 414 -> 365-374 ms (prefill ~1225 -> ~1385 tok/s, past upstream llama.cpp's 1366 pp512); generated output hash-identical; spec decode unchanged at 112.6/62.4/125.4 code/prose/mixed (verify widths never take the big tile).
Plain decode streams every weight byte exactly once per token, so caching the weight stream in L2 only evicts the activations and KV that other kernels reuse. Add a nontemporal load variant (HIP sc0/sc1 bypass hints) and use it for the IQ4_XS weight words in the MMVQ vec_dot; the q8_1 activation loads keep normal caching. Scope notes from measurement (R9700): the MMVQ weight reads are wave-contiguous full cache lines, so bypassing L2 is free there (GEMV 550 -> 554 GB/s, AR decode 36.6 -> 37.0 tok/s, +1%). The same hint in the MMQ tile loader was measured 32% SLOWER (457 -> 310 GB/s at N=8: tile blocks on different CUs share cache lines, and bypassing L2 amplifies DRAM traffic), so MMQ keeps cached loads. q8_0 qs is only 2-byte aligned and keeps get_int_b2. Spec decode unchanged at 112.9/62.4/125.4; outputs identical.
Merges Luce-Org#625 (DSpark/DFlash2 + Qwen3.8 spec-decode) onto the GQH-on-upstream staging branch. The published Geometric-AI Q3KXL GGUF stores attn_gate/attn_qkv as GQH4: each tensor has its own KV header, so the Luce-Org#625 zero-copy stacked GEMV would decode the second weight with the first header. Skip those pairs; q8_0 ssm_beta/ssm_alpha stacking stays. Adds the qwen38-27b model card (GGUF general.name "Qwen38 27b") and serving notes for the Q3KXL artifact. MTP nextn skip is already Luce-Org#610 on this base.
GQH has no MMVQ. The fused matvec hook was still gated on LUCE_MMVQ_MAX_NCOLS (default 3), so an 8-wide DFlash2 verify dequantized the whole weight matrix per step. On gfx1201 that was 6x slower than a fused N=5 and made spec slower than AR (HE 24 tok/s at AL 0.94). Stop using that cap for GQH. Exact-width arms now cover ncols 2..8 (DFlash2 default block size); 9..16 take the generic instantiation. ctest -R gqh 25/25, including fused nvec=8. Measured on R9700, GQH-Q3KXL + DFlash2, same flags as Luce-Org#625: HE-10 24.2 -> 96.1 tok/s aggregate (4.0x), AL unchanged code 23.9 -> 92.4, mixed 12.0 -> 46.9, prose 8.2 -> 29.6
DFlash2 fused-verify was f32 FMA on GQH3 5120x17408 ncols=8. The inner product now dequantizes the codebook to int8 once per block (grids are -1..+1 so the scale is 1/127), quantizes activations once per dispatch, and folds with v_sudot4. GGML_GQH_I8DOT=0 is the f32 bit-exact arm. Also: pair-fuse gate/up at N=8, fuse SwiGLU into the down-proj int8 pre-pass, and add test_gqh_pair_glu. ctest -R gqh forces I8DOT=0. gfx1201 R9700, Q3KXL + DFlash2 Q8_0, --fa-window 2048, q8_0 KV: HE-10 96.1 -> 140.5 tok/s, AL 7.44 unchanged pair-fused gqh3 5120x17408 ncols=8 and glu-fused 17408x8 both fire
Tree verify is 1+n_nodes columns, so --ddtree-budget 8 dispatched ncols=9 and missed the I8/ROWS=4/glu-fuse kernel (-25% vs chain). Cap GQH SpecLA budget to block_size-1 so 1+n_nodes stays 8. IQ4_XS is unchanged.
Evalplus grades sample['solution'] as a complete module. First-fence extraction plus missing prompt typing imports scored four correct GQH functions as NameError and HumanEval/83's formula sketch. Same generations, last-def fence + prompt imports: HE+ 149 → 154 vs IQ4 152. Same-arm floor is 0 (byte-identical jsonl across 3 GQH think-on repeats). The remaining 3.93-only miss is HumanEval/163. Adds harness/he_extract.py for the next HE+ pass.
Two conflicts, down from six against the pre-sync main — the other four were the muse-glimmer/upstream server divergence, now resolved on main itself. - model_capabilities: this branch added a dblock column, main added a muse-glimmer row. Kept both, with muse's dblock as kNever, consistent with main's own note that MuseBackendConfig carries no field for the options it marks Never. All seven rows verified column-aligned. - draft_graph: this branch folded four repeated ggml_rope_ext call sites into a draft_rope helper. Took the helper (it is the newer, exercised code) and confirmed every DraftWeights field it reads exists, and that the local rope_base it replaces is referenced nowhere else. One fix while resolving: draft_rope hardcoded GGML_ROPE_TYPE_NEOX. internal.h adds rope_type with an explicit comment that hardcoding it at the call sites "hid the assumption", and that choosing wrong is silent — the drafter emits plausible-but-unrelated tokens and only the acceptance rate drops. Folding the call sites into a helper must not re-hide that, so the helper now passes w.rope_type.
Collaborator
Author
|
Merged the upstream-synced Why this was The two real ones:
One bug fixed while resolving, and it is not cosmetic. That is a live path, not a hypothetical: Build details for anyone reproducing: the README's |
Collaborator
Author
|
Verified on a second machine with a fresh build tree, to check the merge is not toolchain- or arch-specific.
lucebox5 was a clean checkout of The rocWMMA path is worth calling out because it is only built when |
Collaborator
Author
Alignment against upstream Luce-Org#625 and Luce-Org#642Since both will land upstream before this work, I mapped where we sit and reconciled early. Result: Where we forked. Why reconciling was cheap — the two sides have been working in largely disjoint files:
(lines changed since the fork). Their heavy work is in What it actually cost:
Two things to watch. Both PRs are still open, and upstream's own stack is slightly out of sync — Luce-Org#642 is based on Luce-Org#625 at I pushed this as |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Staging branch that serves Geometric-AI/Qwen3.8-27B-GQH-Q3KXL on gfx1201 with DFlash2 SpecLA. Combines the GQH qtype family (108–111) with Luce-Org/lucebox#625 (DSpark/DFlash2). Not a drop-in merge of Luce-Org#625 — GQH projections stay unstacked because the published GGUF stores per-tensor KV headers.
Intended as the single place to fold this stack into
mainlater. Open against GeometricAGI/geo-luceboxmain, not Luce-Org yet.Serving
v_sudot4.GGML_GQH_I8DOT=0is the f32 bit-exact arm (ctest).--ddtree-budget 8is capped to 7 nodes so1+n_nodesstays ncols=8 (the I8/glu-fuse kernel). ncols=9 missed that arm (−25%).qwen38-dflash2-q8_0.ggufat 2,045,471,776 B (there is an 11 MB sibling that changes AL).gfx1201 R9700, 3.93 bpw GQH + DFlash2 Q8_0,
--fa-window 2048, q8_0 KV, think-off SpecLA: ~137–146 tok/s vs f32 GQH ~105. Luce-Org#625’s 159.8 IQ4_XS number did not reproduce on this box (same-box bartowski IQ4 is ~120–132).HumanEval+ (think-on, greedy, same draft)
Same-arm floor is 0 — greedy sequential SpecLA is bit-deterministic here (3× 3.93 GQH = 149, 2× IQ4 = 152, byte-identical jsonl). AIME ±3 does not apply.
The published −3 vs IQ4 was eval packaging, not bits:
from typingfrom the promptEvalplus grades
sample['solution']as a complete module. GQH often emitsdef f(x: List[...])without imports (NameErrorat exec, empty fail_tests: 8/12/20/22). HumanEval/83 wrote the right function as the last fence; first-fence extraction scored a unicode formula.harness/he_extract.pyis the durable fix.Remaining 3.93-only miss after packaging: HumanEval/163 (range scan vs digit scan). IQ4-only: 55, 134, 151.
Notes + samples:
notes/qwen38-gqh-heplus/.4.45 bpw band (old 2026-08-18 allocation)
Think-on HE+ 149 published / 150 with import header. Flat vs 3.93 on code; tok/s 112 think-on / 126 think-off (GQH4-heavy FFN, I8 still on). Quality is a lower bound on that band.
Test plan
ctest -R gqhwithGGML_GQH_I8DOT=0python3 -m unittest harness.tests.test_he_extract(orPYTHONPATH=harness python3 harness/tests/test_he_extract.py)main(this PR is the bookmark)