Skip to content

hk: faster G17P acceleration-structure refits - #79

Merged
DjDeveloperr merged 6 commits into
aurora-wipfrom
perf/rt-refit-20261026
Oct 27, 2026
Merged

DjDeveloperr merged 6 commits into
aurora-wipfrom
perf/rt-refit-20261026

Conversation

@DjDeveloperr

@DjDeveloperr DjDeveloperr commented Oct 26, 2026 •

Copy link
Copy Markdown

Summary

G17P acceleration-structure refits were about 3× behind macOS (refit of 1M triangles: 47.9 vs 152.7 Mtri/s). This follows #62 and makes them 2.70× faster at 1M, 2.04× at 64k and 1.32× at 4k. At 1M that is 135.9 Mtri/s, about 11% short of macOS.

  1. refit G17P acceleration structures from their retained encoding jobs.
    • A full build that allows updates now keeps its encoding jobs (24 B per job plus 4 B per leaf) instead of the retained IR and plan.
    • An update runs leaves → prepare → encode → envelope. There is no IR restore, no relink, no binary propagation and no plan walk.
    • TLAS refit computes the instance offsets itself.
  2. store G17P node records a line at a time. Triangle records and QTB blocks are written as four 16-byte stores, each lane stores its own root word, and QTB bytes are packed in registers without shared atomics. This is worth about 1%, since the encoder isn't store-bound, and it also touches BLAS builds.
  3. propagate G17P refits in their own dispatch. Inside the encoder, each device-scope acquire-release arrival at a parent waited for that thread's whole node's worth of global stores. That made propagation 9.3 of 18.1 ms at 1M. A separate one-thread-per-job dispatch now does the arrivals and encodes the inner jobs.
  4. encode G17P nodes without shared memory. The eight lanes of a node share its bounds and each QTB block's through clustered subgroup min/max. Every lane fits the root and its own block, child data stays in registers, and node encoding uses no shared memory and no barriers. This also speeds up BLAS builds.
  5. encode G17P refit inner jobs a level at a time. A full build that allows updates lists its inner jobs of the first 16 levels by level. An update encodes one level per dispatch, deepest first, eight lanes per job, with no arrivals; the per-job propagation remains only for deeper inner jobs. An update takes 4 to 16 levels by its leaf count. A failed job leaves NaN bounds so its ancestors fail with it.
  6. place G17P triangle updates where refit encoding reads them. A full build that allows updates keeps each triangle leaf's slot. An update reads each triangle from its geometry in primitive order and stores it at its slot, so the eight lanes of a leaf job read their children from consecutive slots and no IR leaves are written. Indirect builds and structures with a vertex table keep the previous path.

Measurements

Neo, same boot, 3 alternating rounds, refit Mtri/s (medians):

refit #62 commits 1–4 + 5 + 6
1M triangles 50.43 104.9 127.3 135.9 (2.70×)
64k 67.77 89.3 137.5 138.4 (2.04×)
4k 36.64 36.97 49.04 48.32 (1.32×)

Commit 6 costs about 0.7 µs per update at 4k for its extra dispatch. A scene whose BLASes refit every frame is unchanged (50.45 / 50.71 fps).

Other rows against #62, measured with commits 1–4 (commits 5 and 6 only change the update path): BLAS 1M / 64k +2.3% / +2.4%, fastbuild 1M +3.2%, TLAS 4096 +0.9%; ray throughput and the other scenes are unchanged within 0.6%. An updatable 1M acceleration structure shrinks from 334.9 to 322.3 MB, and its update scratch from 322.3 to 288.8 MB.

Testing

RT CTS, ray_query plus ray_tracing_pipeline (57,190 cases), with all six commits at 6 workers, update cases first: 47,101 Pass, 10,088 NotSupported. The 18,698 update cases had no failures. The #62 baseline is 47,100 Pass, 10,088 NotSupported, 2 crashes.

  • The one crash is ray_query.stress.geometry_shader.triangles, a kernel compute-watchdog timeout when it runs beside other traversal-heavy work. Run alone 10 times each, it passes on all six commits, on commits 1–4 and on hk: improve G17P shaders, submission and AS correctness #62 (about 6.3 s per run each). It's a known kernel issue under traversal contention, not a refit change.
  • Eleven update cases that hung an earlier version of commit 5 (a one-instance TLAS updated in place) pass when run one at a time.

What remains at 1M is mostly leaf generation and leaf encode.

An update restored the retained binary IR and the native plan into
scratch, relinked the IR's parents, propagated bounds up the binary tree
with one atomic per node, rebuilt every encoding job by walking the IR
again and only then encoded: most of the time went to work whose result
the full build had already known.

A full build that allows updates now keeps its encoding jobs instead:
for each job its slots, kind, child count, parent and position among its
parent's children, and the IR offsets of a leaf job's children. An
update regenerates the IR leaves and encodes every leaf job from them,
in one dispatch; a job hands its bounds to its parent, and the last
child of an inner job to arrive encodes that job's node and hands its
bounds on, up to the root. Neither the IR's internal nodes nor a plan
are read, and the retained data shrinks from the IR internal nodes plus
the plan to 24 bytes per job plus 4 per leaf.
Node encoding stored every word of a node with its own 32-bit store: a
triangle record took sixteen, a QTB block sixteen, and lane zero wrote
all sixteen words of the node's root after the others. Each triangle
also packed its twelve QTB vertex bytes with twelve atomic ORs on shared
memory.

Store triangle records and QTB blocks as four 16-byte stores, and have
every lane store its own 64-bit root word, so the eight lanes of a node
fill its root line together. A triangle's vertex bytes are three whole
words of its block, so it packs them in registers and stores them to
shared memory without atomics.

The stores go through helper functions: glslang's precise propagation
runs out of memory when such buffer-reference stores are written out in
main() of this shader.
Refit encoding propagated bounds from inside node encoding: once a leaf
job had stored its node, one lane arrived at the parent with a
device-scope acquire-release atomic, and the last child encoded the
parent. On G17P that atomic's fence waits for the thread's earlier
global stores, a whole node's worth, so propagation took 9.3 of the
18.1 ms that encoding took for a million triangles.

Node encoding now only leaves each leaf job's bounds at its position,
and a separate dispatch with one thread per job does the arrivals and
encodes the inner jobs. The encoding helpers that both dispatches use
move to agx3_node.h.
Node encoding passed each child's bounds, vertices, words and root word
through about 3.4 KB of shared memory per workgroup, with four barriers:
lane zero summed the node's bounds and fitted its root, the first lanes
fitted the QTB blocks, and every lane then read both back.

The eight lanes of a node now share its bounds and each QTB block's
through clustered subgroup minima and maxima (a workgroup is one 32-lane
subgroup), every lane fits the root and its own block itself, and a
node's checks are a clustered OR. Each lane keeps its child's vertices
and words in registers and writes its own QTB words; the first lane of a
block writes the block header. Encoding uses no shared memory and no
barriers.

Measured on Neo against the previous change, in the same boot,
alternating, three rounds (Mtri/s, medians):

  refit, 1048576 triangles       89.16 -> 98.87
  refit, 65536 triangles         84.66 -> 89.78
  BLAS build, 1048576            25.22 -> 25.63
  BLAS build, fast, 1048576      28.48 -> 29.17

TLAS builds, compaction, ray throughput and scenes are unchanged within
run-to-run spread.
Refit propagation encoded each inner job in the thread of the last of
its children to arrive: one lane per node, in divergent control flow,
with a device-scope arrival for every step up each path to the root. At
a million triangles it took 2.7 ms of a 10.6 ms update.

A full build that allows updates now also gives each inner job its
level, a third of the binary depth of its IR node, so that an inner
job's inner children are one level deeper, and keeps the inner jobs of
the first 16 levels listed by level after its other retained jobs. An
update encodes the inner jobs of one level per dispatch, deepest first,
eight lanes per job as node encoding does, with no arrivals; refit
propagation remains for any deeper inner jobs and stops at the levels.
An empty level still costs a dispatch, so an update takes about as many
levels as its tree is likely to have, 4 to 16 by its leaf count.

A job that fails now leaves NaN bounds, so that its ancestors fail with
it instead of encoding from bounds that were never written.

Measured on Neo against the previous change, in the same boot,
alternating, three rounds (Mtri/s, medians):

  refit, 1048576 triangles       104.9 -> 127.3
  refit, 65536 triangles          89.3 -> 137.5
  refit, 4096 triangles          36.97 -> 49.04

Builds, ray throughput and scenes, including a scene that refits its
structures every frame, are unchanged within run-to-run spread.
A triangle update generated IR leaves in primitive order, and refit
encoding then gathered them back through each leaf job's child offsets:
every lane's read waited on an offset read and landed on a line of its
own.

A full build that allows updates now also keeps each triangle leaf's
slot, the index of its child offset. An update reads each triangle from
its geometry in primitive order, as leaf generation does, and stores its
vertices and IDs at its slot in 48 bytes, so the eight lanes of a leaf
job read their children from consecutive slots and no IR leaves are
written. Updatable structures keep every primitive active, so the
update's active leaf count is the full build's. Indirect builds, whose
ranges are on the device, and structures with a vertex table, which is
built from the IR leaves, still generate IR leaves.

Measured on Neo against the previous change, in the same boot,
alternating, three rounds (Mtri/s, medians):

  refit, 1048576 triangles       127.3 -> 135.9
  refit, 65536 triangles         137.5 -> 138.4
  refit, 4096 triangles          49.04 -> 48.32

The small update pays for the extra dispatch. A scene that refits its
structures every frame is unchanged within run-to-run spread.
@DjDeveloperr
DjDeveloperr merged commit d00d104 into aurora-wip Oct 27, 2026
astaff pushed a commit to astaff/mesa that referenced this pull request Oct 28, 2026
aa733fc is the Mesa the J616s runs every day. Bring aurora-wip 2a5fd4a
into it: DJ's G17P refit and H-PLOC fixes (aurora-silicon#79, aurora-silicon#81), the deferred
occlusion resolver (aurora-silicon#82, aurora-silicon#83), the zink split-query readback fix (aurora-silicon#84)
and the virtio queue-limits fix (aurora-silicon#86).

The conflicts in the acceleration-structure sources and hk_query_pool.c
are resolved as in the earlier PR aurora-silicon#80 merge 8b7395e. That merge also
carried 2a541c6, which is held back here (see the PR).

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
astaff pushed a commit to astaff/mesa that referenced this pull request Oct 28, 2026
PR aurora-silicon#80 copied agx3_{leaf,envelope,plan}.comp to agx3_gen1_*.comp before
DJ's G17P refit series (aurora-silicon#79) landed, so after the merge the GEN1 build
programs that G17P and T6040 run were the pre-aurora-silicon#79 encoders, while their
refit programs came from the aurora-silicon#79 sources. The pre-aurora-silicon#79 leaf encoder does
not retain the encoding jobs refit reads, stores records through shared
memory, and its envelope clears instance pointers on refit too. The
aurora-silicon#79 agx3_{leaf,envelope,plan}.comp were still compiled, but nothing
used them.

The copies were byte-identical to the base before aurora-silicon#79, and the G17P
plan already matched the aurora-silicon#79 one apart from a comment. Build the
hk_agx3_gen1_* programs from the aurora-silicon#79 sources and drop the copies, so
GEN1 builds and refits come from one set of encoders again. M5 keeps
its agx3_m5_legacy_* programs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@RyanTheTide
RyanTheTide deleted the perf/rt-refit-20261026 branch October 30, 2026 03:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant