Skip to content

hk: run deep ray tracing pipelines as continuations on G17P (maxRayRecursionDepth 3 -> 31) - #278

Merged
DjDeveloperr merged 1 commit into
aurora-wipfrom
rtrec/hk-rt-continuations-pr
Nov 13, 2026
Merged

DjDeveloperr merged 1 commit into
aurora-wipfrom
rtrec/hk-rt-continuations-pr

Conversation

@DjDeveloperr

Copy link
Copy Markdown

G17P reported maxRayRecursionDepth = 3, so dEQP-VK.ray_tracing_pipeline.misc.recursiveTraces_{tri,AABB}_4..15 (24 cases) were NotSupported, and applications that need deeper recursion could not create their pipelines.

Why it was 3

The ray tracing megakernel inlines every stage once per trace depth (AGX has no call instruction). A closest-hit or miss shader with k trace sites therefore costs (k · stages)^depth copies, and hk_rt_inlining_fits() refuses anything over 1 M instructions.

What this changes

Pipelines with maxPipelineRayRecursionDepth > 3 now run their stages as continuations. Pipelines up to depth 3 compile exactly as before.

  • Split: every raygen, closest-hit, miss and callable stage is split at each traceRayEXT / executeCallableEXT with nir_lower_shader_calls, as hk's callable dispatcher already does.
    • The builtins a stage reads are hoisted to its entry, so a resumed part takes them from its spills.
    • Intersection and any-hit stages stay inlined in traversal.
  • Schedule: one loop per invocation over a pc.
    • The single traversal enters the closest-hit or miss shader the SBT names.
    • Every other pc runs one part, and each part is emitted once.
    • A part ending in a call pushes a 32-byte return record onto the invocation's scratch stack; a part that returns pops it.
    • The trace depth is a property of the stack, so code size no longer grows with depth.
    • The traversal state is emitted only when some stage traces.
  • Stack: reserved for the declared depth (plus the existing 31 callable frames).
    • A call that would leave the reservation is skipped, so no call sequence can overflow it.
    • Reservations above 48 KiB are refused, keeping the stack and register spills inside the 64 KiB that static thread-private offsets can address.
  • Fallback: the inlined megakernel is kept for a ray query or hit object whose uses span a call, and for hitObjectExecuteShaderEXT. Neither occurs in CTS above depth 3.

Tests on the Neo (J700, G17P, kernel 7.3.0-neo.final)

The tested builds carry this commit's RT code on top of the RT CTS fix stack (#270-#274) and add a default-off diagnostic environment switch, not part of this PR, that forces continuation mode at every depth.

Run Result
misc.recursiveTraces_{tri,AABB}_0..15 32/32 Pass, including the 24 previously NotSupported. Depth 15 takes ~5.3 s per case.
Depth-3 compile identity AGX_MESA_DEBUG=shaders NIR+AIR dumps of recursiveTraces_tri_3, a nested-callable case and a trace-in-loop case are identical to the base (0 of 839,013 / 98,428 / 89,700 lines differ).
ray_tracing_pipeline.* (8,442 cases, without the 1M-instance host builds) No change vs base except NotSupported → Pass for the 24.
ray_query.* (4,988) and every 10th ray_query.acceleration_structures.* (3,353) No change vs base.
Continuation forced at depth ≤ 3 on every 4th ray_tracing_pipeline.* case (2,111) All pass. The sweep found pipelines with no trace site failing creation; that bug is fixed here.
gpubench rt.rays (4 configs) and rt.scene medium/light, query and trace=pipeline, 2 rounds Within ±0.5% of base on every row; identical image CRCs.
GPU kernel log 0 fault lines over the RT runs.

Forcing continuation mode at depth ≤ 3 is 2.2-2.4× slower on rt.scene trace=pipeline (same images), so it stays limited to pipelines deeper than 3.

Unit test

test-hk-rt-cache-control builds a pipeline whose raygen, closest-hit and miss trace from every level, plus an any-hit and a recursive callable, at depths 3, 4, 8 and 31. It checks:

  • every call is inlined;
  • the code size does not grow with depth (1,286 / 1,287 / 1,289 instructions at 4 / 8 / 31, against 16,266 for the depth-3 megakernel);
  • the stack grows by exactly one frame and one record per level;
  • the stack formula.

meson test --suite asahi passes 131/131 in a debug build with asserts on (M5 local CI, since Actions is off for this repository).

Known limitations

  • Pipelines with thousands of shader groups compile much more slowly in continuation mode than inlined. With continuation forced, a raygen calling one of 4,096 callables took 86-307 s, against 2.5-5 s inlined. This only affects pipelines deeper than 3 with that many groups; no CTS case has one. A follow-up replaces the per-group select chains with balanced searches.
  • The fallbacks listed above can still hit the inlining limit at large depths.

Merge

This merges after the Neo batch merge gate passes and meson test --suite asahi passes on the exact merged tree.

🤖 Generated with Claude Code

The ray tracing megakernel inlines every stage once per trace depth, so a
closest-hit or miss with k trace sites costs (k * stages)^depth copies.
G17P therefore reported a maxRayRecursionDepth of 3, and
dEQP-VK.ray_tracing_pipeline.misc.recursiveTraces_*_4..15 were not
supported.

Pipelines deeper than that now split their raygen, closest-hit, miss and
callable stages at every call with nir_lower_shader_calls, as the callable
dispatcher already does, and run the parts from one scheduler loop. Every
part, and the traversal with the intersection and any-hit stages inlined
into it, is emitted once, so the code no longer grows with depth. Each
call pushes a return record onto the invocation's scratch stack. Builtins
a stage reads are hoisted to its entry, so resumed parts take them from
their spills. The record carries the vertex position address instead,
because an AABB hit has none to load.

The stack is reserved for the declared depth and 31 callable frames. A
call that would leave the reservation is skipped, so no call sequence can
overflow it. A ray query or hit object that lives across a call, or
hitObjectExecuteShaderEXT, keeps the inlined megakernel.

Pipelines up to depth 3 compile exactly as before. G17P now reports the
specification's maximum of 31.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr

DjDeveloperr commented Nov 13, 2026 •

Copy link
Copy Markdown
Author

Gate evidence for merging #278 (batch1, step 2 of a tree-checked sequence):

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant