Skip to content

Consolidate our Mesa work on aurora-wip - #87

Merged
RyanTheTide merged 296 commits into
aurora-wipfrom
consolidate/aurora-wip-20261028
Oct 28, 2026
Merged

RyanTheTide merged 296 commits into
aurora-wipfrom
consolidate/aurora-wip-20261028

Conversation

@RyanTheTide

Copy link
Copy Markdown

This brings our open Mesa work onto aurora-wip in one place, so the J616s daily driver, the M4 Pro branches and aurora-wip stop drifting apart.

The base is aa733fc, the Mesa the J616s runs every day (pushed as wip/j616s-daily-mesa-20261028). aurora-wip 2a5fd4a is merged into it, the AS conflicts are resolved the same way as the earlier #80 merge (8b7395e), and the GEN1 refit ABI follow-ups from that branch keep the tree building. Our remaining WIP sits on top.

What lands

PR or branch Result
wip/j616s-daily-mesa-20261028 (aa733fc) base, all of it
#80 wip/m4pro-mesa-tip-20261026 in, including 2a541c6
wip/m4pro-mesa-on-awip-20261028 (00cabf8) in, except f4b9ce3 and ef0db72 (the D3D11 regression, below)
wip/hk-occlusion-t6040-20261028 (7cc25f3) in, as 00cabf8
wip/m4pro-gpu-gap-20261027 480f485, 8ace18a, 1e878ff and ffc8242 are in. e8b9d04 is superseded by 2a541c6, which keeps the same payload on the stack
wip/m4pro-mesa-20261026 already present (85edec1 duplicates f1ef241, ce03198 is already applied)
wip/m4pro-mesa-build-base-20261026 already on aurora-wip through #77
#68 perf/agx2-parity-20261024, #69 perf/agx2-reset-only-20261025 in (51c0bfd, 3d00b6d, af16d98)
#65, #67 superseded. aa733fc's 33f2bf1 ports their layered/MSAA packets and per-level ZLS to the M4 Pro
#66 superseded. Colour MSAA attachments stay plain as #66 decided, and ee1ef4c covers the spill boundary for every format
#72 feat/g17p-emrt-transient-spill held. It adds boundary-copy work nobody has measured, 33f2bf1 deliberately left it off, and it no longer applies on the port
wip/m4pro-gpu-gap-20261027-fill-plain, -fill-tile held. Both are diagnostic probes that change production policy (plain colour attachments, 32x16 tiles) and say so in their messages

The D3D11 regression

The D3D11 dxbench on the J616s (2560x1600, 20 s, Aurora Proton) is bound by the RTE FMA tininess repair, because DXVK asks for RTE and the M4 Pro runs the repair on every FMA.

Build D3D11 fps
c29122c (before aa733fc) 6.4
aa733fc (branches around the repair for nonzero results) 98.2
aa733fc plus the f4b9ce3 + ef0db72 repair (3d147de9) 10.4
00cabf8 8.4

D3D12 stays at 214 fps on every build, and the native FMA checker gives identical output for all of them. f4b9ce3 and ef0db72 are the cause, so they stay off aurora-wip until the range guard keeps aa733fc's cheap path.

This branch measured on the J616s against aa733fc, alternating arms:

Build D3D11 fps
aa733fc 100.8, 99.7, 99.8, 99.7
this branch without 2a541c6 and 00cabf8 (23e8a4a) 100.6, 100.7, 100.7
this branch (b39f053) 100.5, 100.8

Already-merged history

aa733fc was built on a consolidation that merged DJ's agx3c (#47) and carries rebased copies of #73 and #78. Those commits come in with it. #47's head becomes an ancestor of this branch, so GitHub will show #47 as merged once this lands.

Testing

  • Fedora 45 aarch64 container, release build with -Dwerror=true and -Dbuild-tests=true: asahi and poly suites 111/111, hk query tests 5/5, dropped-signal 1/1, GEN1 ray query 12/12 repeats, RT core, the SSBO block limit under the drm shim for G13G, G14G, G16G, G16S and G17P, and the G16S and G17P extension tables. The same results on 23e8a4a and on b39f053 (an incremental rebuild with its two extra commits).
  • Release overlay ovl-rel-b39f0535.tar.zst, sha256 cb604df68e9358a1f7a9d79fcf34bdfabde3841e78732640042752e8ede8d3ed.
  • Still to run before this is anyone's daily driver: the M1 Pro / M2 perf gate (hk: reduce AGX2 reset work and G13X query overhead #68 and hk: avoid native graphics cache reset work on AGX2 #69 change the AGX2 reset and G13X submit paths) and a J616s Vulkan sanity pass.

🤖 Generated with Claude Code

A fragment shader that runs a ray query acquires the colour write class
(pixwait 4) ahead of its first traversal, so the linker turns its
epilog's read+write wait (pixwait 12) into a read wait (pixwait 8). It
built both the expected and the replacement instruction in the AGX3B
encoding only, so on M5 the AGX3C epilog never matched and every such
link failed (36 ray_query.helper_invocations.gpu cases ended in
vkEndCommandBuffer VK_ERROR_FEATURE_NOT_PRESENT).

Native M5 colour fragments with a ray query (the native G17G compiler,
intersection_query and intersector forms) show exactly this pair:
pixwait 4 immediately before intersect_ray, pixwait 8 before lstcol,
then pixrel 524. Encode the rewrite in the device's ISA.

Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_RQ_WATERTIGHT=1 enables it; default binaries are unchanged.

M5's triangle test is inclusive: when its edge function is exactly zero it
reports every triangle at that edge or vertex, each with a zero
barycentric there. Hardware probes (exact dyadic fixtures, 16384 generic
near-edge rays and the closedFan geometry) show near-edge rays otherwise
get exactly one triangle, but exact-edge rays get both. That double count
fails ray_query.watertightness.singlehit and
ray_tracing_pipeline.watertightness.closedFan/closedFan2. The hardware's
arithmetic is not exact (closedFan spoke rays get both neighbours although
exact arithmetic puts them strictly inside one), so a shader cannot
predict which candidates it reports. Apple's runtime query.next does not
filter them either.

When enabled, remember the distance of the last boundary triangle (some
barycentric within 2^-22 of zero, allowing for the hardware's ~1 ulp
barycentric rounding) a query let through, and drop a later boundary
triangle at the same distance: equal or adjacent 256-ulp distance keys.
Partners compute t from their own planes; on the CTS random-height mesh
they differ by more than 31 ulps at some diagonal pixels. Both are the
same point on the shared edge or vertex. A drop only follows its
delivered partner, so it cannot turn a hit into a miss. The key lives in
the candidate cursor's spare bits; no memory access.

Hardware (M5 Pro, R328): 34 singlehit/nomiss (compute, fragment, RT
stages) and closedFan/closedFan2 cases: 19 Pass / 15 Fail disabled,
34/34 Pass enabled. Vertex/tess/fragment singlehit and nomiss at one
worker, enabled: 11/11 Pass (geometry-stage cases fail pipeline creation
either way). vkQuake RT frames 80-81: images byte-identical to R337;
replay time within noise (15 interleaved pairs, median +0.04 ms/frame).

Cost when enabled, drm-shim shaderdb census of 1743 ray-query/RT shaders
against the default: +7.42% code bytes, median +35 instructions, median
GPR change 0; 30 shaders lose occupancy (28 at 1024 -> 896 threads) and 6
gain. Disabled: all 1743 shaders identical.

Signed-off-by: Diljit Singh <hey@djdev.me>
On M5 a vertex-stage draw picks its provoking vertex through the VDM
vertex program template, and hk left the PPP flat-shading vertex
at 0 for everything but mesh draws.
pipeline.*.extended_dynamic_state.*provoking_vertex_first_to_last* draw
oversized triangles with the last-vertex convention and shaded them from
the wrong vertex in every construction type (monolithic,
pipeline_library, fast_linked_library, shader_object_unlinked_spirv);
rasterization.provoking_vertex, whose triangles stay inside the viewport,
passed. Presumably clipping takes the flat inputs of the vertices it
generates from the PPP selector; that mechanism is inferred, not
measured.

Program the PPP selector from the same provoking state for all draws.

Hardware (M5 Pro, R328), diagnostic switch on the R337 base: the 20
two_draws_{dynamic,static} provoking cases across the four construction
types went from 12 Fail / 8 Pass to 20/20 Pass, and all 63
rasterization.provoking_vertex cases pass.

Signed-off-by: Diljit Singh <hey@djdev.me>
Fragment shaders that need the raster sample count (interpolateAtSample's
sample clamp, sample-position lowering) read samples_log2. Neither G17P
nor M5 has a grounded AGX3 encoding for that special register, so the
packer refuses it. hk folds it to the keyed count on G17P, but M5 keys the
count as m5_position_samples and was never specialized: every such
fragment shader failed to compile on M5. Pipelines creating it returned
VK_ERROR_UNKNOWN, and per-sample-count variants built at draw time ended
their command buffer with VK_ERROR_UNKNOWN
(pipeline.fast_linked_library.misc.frag_lib_varying_samples_{2,4,8},
interpolate_at_sample_no_sample_shading).

Specialize with the native sample count on both chips; M5 counts reach 8.

drm-shim: no remaining samples_log2 refusals; 15268 multisample/misc
shaders unchanged (5244 sampled cases).

Signed-off-by: Diljit Singh <hey@djdev.me>
…fault

Vulkan requires a ray that lands exactly on an edge shared by two
triangles to report a single hit; M5's inclusive triangle test reports
both. The filter fixes the 15 watertightness CTS failures (singlehit in
every stage, closedFan/closedFan2). Measured cost: +7.4% code in
ray-query/RT shaders, 30 of 1,743 shaders drop one occupancy step, no
measurable change in the vkQuake-RT replay (median +0.04 ms on ~3.1 ms
per frame, byte-identical images). Enabled by default at the user's
direction; ASAHI_M5_RQ_WATERTIGHT=0 turns it off for diagnostics.

Signed-off-by: Diljit Singh <hey@djdev.me>
Geometry ray queries were refused unless ASAHI_M5_GS_RAY_QUERY_DIAGNOSTIC=1,
after GS query draws requested GPU recovery twice. Those runs predate the
removal of dead native queries from the poly main/count kernels.

With the R338 stack and the queries enabled, all 578 geometry ray-query
cases of the mustpass (acceleration_structures, ray_flags,
traversal_control, builtin, watertightness, stress,
binding_model.*acceleration_structure*.geom) ran on M5 hardware with one
worker in 26 s: 299 Pass, 249 NotSupported, 30 Fail, no GPU recovery and
no long-running case. The 30 failures are
binding_model.mutable_descriptor.single.acceleration_structure_khr.*.geom
("expected 2, found 1"): the query's candidate-found result does not reach
the atomicAdd in the main GS kernel (open).

ASAHI_M5_GS_RAY_QUERY_DISABLE=1 refuses them for diagnosis. Concurrency
behaviour beyond one worker is not qualified here.

Signed-off-by: Diljit Singh <hey@djdev.me>
With independent kernel queues, advertise two more queue families next
to the universal one: compute+transfer and transfer-only, each with the
same queue count. Kernel queues carry no capability class (every queue
accepts render and compute commands) and hk records copies, clears and
blits the same way on any family, so these families only narrow what the
application may record. They reuse the universal queue implementation:
one kernel queue per Vulkan queue, created on first submission in the
device's single VM; no queue or VM is created for an unused queue.

This makes the CTS's exclusive compute/transfer queues, concurrent
sharing across families and cross-family queue pairs available. A
sampled hardware run (M5 Pro, R328 + R338 Mesa, 2 workers) of 1146 cases
that were NotSupported for queue reasons: 857 Pass, 282 NotSupported
(118 still lack a queue pair: they need two graphics families; the rest
for unrelated reasons), 7 Fail, all newly exposed compute variants (5 "M5
compute launch exceeds the current resource profile", 2 sampler
min_reduce compute image mismatches). A regression sample of 863
previously passing synchronization/api/compute cases: 863 Pass. No GPU
recovery.

Signed-off-by: Diljit Singh <hey@djdev.me>
Native G17G object/mesh pairs (lane-mesh nwit2 otpg/mtpg/mtpg0)
address one record through the payload address SR62/SR63: the object
stores its child grid (threadgroups_per_grid) at +0 and the payload from
+16, and the mesh reads its threadgroups_per_grid from +0 after the
payload-entry barrier. The record lives in the TA mesh rendering buffer
(register 0x1ca48), which the kernel only binds to task kicks.

With ASAHI_M5_TASK=1 and a kernel reporting
DRM_ASAHI_FEATURE_TASK_MESH_STORAGE (new uapi bit 13, with
DRM_ASAHI_RENDER_PROGRAM_TASK, bit 3):
- task shaders use the native threadgroup grid, rendezvous and
  payload-exit barrier, store the child grid at +0 of the record and
  accept payloads up to 16 KiB at +16;
- a mesh shader compiled for a task stage (no
  VK_SHADER_CREATE_NO_TASK_SHADER_BIT_EXT) reads gl_NumWorkGroups from
  the record instead of the API grid, and payloads from +16;
- task kicks carry DRM_ASAHI_RENDER_PROGRAM_TASK and set VDM mesh
  tail[0], as native object->mesh kicks do.

Without the feature only payload-free task shaders are accepted, as
before, and a task-stage mesh shader that reads gl_NumWorkGroups is
refused instead of silently reading the API grid. The
ASAHI_M5_TASK_PAYLOAD override is gone. drm-shim gains
ASAHI_SHIM_CHIP=g17s-task for host compile tests of the record path.

Not hardware-tested: needs a kernel with the task-only 0x1ca48 binding
and m5_mesh_storage_chunks != 0.

Signed-off-by: Diljit Singh <hey@djdev.me>
Mesh workgroups smaller than their export batch are padded and the API
body runs under the original invocation mask. The payload-entry barrier
inserted at the start of a payload- or record-reading mesh shader ended
up inside that mask. Native mesh main functions pass it with every lane
before the thread-index branch (lane-mesh nwit mpay, nwit2 mtpg); hoist
it out of the masked body the same way.

Only mesh shaders that read a task payload or the task record change.

Signed-off-by: Diljit Singh <hey@djdev.me>
hk_compile_shaders lowers every shader's I/O and system values (including
gl_NumWorkGroups to the draw root) before hk_compile_shader_impl, so the
task-record substitution placed in the impl never saw pipeline shaders: a
mesh shader under a task stage still read the API grid (hardware probe on
R328: per-pixel counts 0 0 0 10 instead of 1 2 3 4). Do the substitution,
or the refusal without the record, in both lowering paths.

Signed-off-by: Diljit Singh <hey@djdev.me>
Indirect, indirect-count and conditionally rendered task draws were
refused because the object dispatch was sized from a CPU grid. Record
them with one object workgroup and dispatch libagx_m5_task_dispatch in
the pre-graphics compute stream: it patches the object thread counts of
the VDM mesh dispatch record from the GPU grid and writes the draw's
execute word (grid nonzero, draw index below the count, predicate
passes). The task shader checks that word, through the new root field
mesh_task_flag, on entry and launches no mesh children for a disabled
draw; the word is zero for every other draw.

The kernel is admitted for the M5 (g17g) libagx target only, and the
g17s-task drm-shim chip now also reports COMPUTE_RESOURCES, as the
kernel does.

Hardware (R328, boot 410b5227, ASAHI_M5_TASK=1, no task record): all 360
payload-free conditional_rendering task cases (draw, draw_indirect,
draw_indirect_count) Pass, 0 GPU faults. Indirect task cases with
payloads need the task-record kernel.

Signed-off-by: Diljit Singh <hey@djdev.me>
An M5 task (object) shader with an image or texture read reached the
AGX3 packer with the 32-bit bindless form (uniform heap base plus a
register index), which has no grounded encoding, so every task shader
reading an image was refused. Treat the task stage like the mesh stage:
lower its reads to the heap-indexed descriptor-address form, stage
image-load coordinates as for mesh and kernel programs, and bind the
indexed heap pointers for a task program.

drm-shim (g17s-task, ASAHI_M5_TASK=1): the 10 synchronization.*_to_task
image cases now compile; their hardware run needs the task-record kernel.

Signed-off-by: Diljit Singh <hey@djdev.me>
A mesh workgroup padded to its export width, or a task workgroup of
fewer than three invocations, runs its API body under the exact
invocation mask. A user execution barrier in that body was refused,
since moving it under the mask changes the set of invocations that
reaches it. Split the body at top-level execution barriers instead:
each segment between barriers runs under the mask and the barriers stay
outside it, so every physical lane passes them, as Apple passes
barriers with all lanes enabled (cndst 0,6,0). Values that cross a
barrier are consumed only by API lanes; nir_repair_ssa gives the
padding lanes undefined copies. Barriers nested in control flow are
still refused.

Signed-off-by: Diljit Singh <hey@djdev.me>
Point meshes were limited to the size-only interface (position plus
point size) and private per-primitive vertices to triangles, so any
point or line mesh with a per-primitive output, and any point mesh with
a user varying, was refused.

Place the point-size word after the user words, in the UVS group order
agx_nir_lower_uvs uses for the raster state (unchanged for size-only
points, where it stays the plane after the position, as in the native
mpt witness), and give each point or line primitive one private vertex
per corner for its flat words, as triangles already have.

Host: agx_tests 440/440. drm-shim: 18 previously refused mesh-only cases
compile (max_points, maximize_primitives, max_mesh_output_primitives_256,
max_mesh_output_size_without_payload_per_vertex, provoking_vertex.lines
x4, the ten memory/group barrier mesh cases); no other mesh-list case
changes outcome. Not yet run on hardware.

Signed-off-by: Diljit Singh <hey@djdev.me>
The record task publishes its child grid from 16 bytes of threadgroup
memory past the body's own, so a task shader using all 32 KiB (Vulkan's
minimum maxTaskSharedMemorySize) was refused. In that case keep the grid
in registers until the body's closing rendezvous, then store it to word
0 of the body's memory, which no invocation uses any more, and pass one
more rendezvous before lanes 0..2 publish it. Task shaders with spare
threadgroup memory keep the native layout. The payload-free direct path
publishes from registers and never needed the extra words.

Host: agx_tests 441/441. drm-shim (g17s-task): properties.
task_shared_memory_size now compiles. Not yet run on hardware.

Signed-off-by: Diljit Singh <hey@djdev.me>
The frontend lowering ran agx_preprocess_nir_m5 on every stage except
mesh, and the M5 task branch preprocessed again after adding its
child-grid words, so every application shared-memory access in a task
shader had its byte offset divided into a word index twice: all four
invocations of a quad read and wrote the same word. Skip the frontend
pass for M5 task shaders as for mesh shaders; the task branch
preprocesses every shared access exactly once.

Hardware (R328, boot 0cbd0f25, ASAHI_M5_TASK=1, payload-free task):
a 128-invocation round trip through 1 KiB of task shared memory read
word (i & ~3) for word i before (0/128 correct) and is exact after;
the 16 KiB and 32 KiB write/barrier/read-modify-write probes
(task_payload_and_shared_memory_size's shape without a payload) go
from every word wrong to 0 mismatches, 0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
The native object program allocates threadgroup memory in 64-byte units
(at least 64 bytes), and refuses other sizes. The record path already
rounds its size; the payload-free direct path kept the application's
size, so a payload-free task shader with, e.g., 8 bytes of shared memory
was refused at draw time.

Hardware (R328, boot 0cbd0f25, ASAHI_M5_TASK=1): misc.barrier_in_task
goes from FEATURE_NOT_PRESENT to Pass (2/2), 0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
The M5 mesh dispatch record is an Index List VDM block (block type 3)
whose bits 8..15 are the Index List Primitive field. hk always wrote
the triangle value captured from native triangle meshes (0x70000600),
so the hardware handled point and line meshes as triangles: a point
mesh workgroup with 22 or more primitives drew nothing and a line mesh
with 22 or more lost its last lines (22 is where three index bytes per
primitive exceed 64 bytes), independent of the vertex count. Encode
points as 0 and lines as 1.

Hardware (R328, boot 3fc53692): point meshes of 21/22/32 primitives and
line meshes of 21/22/32 primitives, plus 22 points over 32 vertices and
a 22-line strip over 23 vertices, all draw exactly (were 0 drawn for
points >= 22, 2+ lines lost for lines >= 22); 0 GPU faults. The
point/line native mesh witnesses compile identical shader-side index
handling, so the shader was not the cause.

Signed-off-by: Diljit Singh <hey@djdev.me>
A mesh shader object compiled for triangles may still write
gl_PointSize: its polygon mode is dynamic and can rasterize the
triangles as points. The pipeline path drops such writes when it knows
the state, but a shader object kept them and was refused. Gather the
point-size word for every topology, after the user words as for points
(the agx_nir_lower_uvs group order that programs the raster state).

Hardware (R328, boot 3fc53692): mesh_shader.ext.smoke.shader_objects.
depth_only_triangles goes from refused to Pass, 0 GPU faults. drm-shim:
no other case in the 6056-case mesh list changes outcome.

Signed-off-by: Diljit Singh <hey@djdev.me>
A mesh shader writing one component of a per-vertex vector output
(gl_MeshVerticesEXT[i].gl_Position.y = ...) reaches nir_lower_io as an
array deref of a vector, which nir_lower_io stores to component 0, so
all four position components landed in x and the depth-only mesh tests
that write gl_Position component by component saw wrong depths. Run
nir_lower_array_deref_of_vec on mesh outputs first, as for other
per-vertex array I/O.

Hardware (R328, boot 3fc53692): the 8 smoke.*.depth_only_*_position_
components cases go from Fail to Pass (16/16 depth_only cases pass),
0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
Mesh shaders writing gl_CullPrimitiveEXT were refused. Keep one byte
per primitive in threadgroup memory (cleared before the body, so an
unwritten flag keeps its primitive), and before the export let one
lane compact the kept primitives' index bytes, and the words of
unwelded primitives, in API order and store the new primitive count;
the rendezvous that follows publishes them to the export.

Hardware (R328, boot 3fc53692): builtin.cull_primitives goes from
refused to Pass (2/2); misc.no_points/no_lines/no_triangles still pass;
0 GPU faults. agx_tests 442/442.

Signed-off-by: Diljit Singh <hey@djdev.me>
D16S8 is built from independent D16 and S8 planes, the same split
layout M5 already uses for D32_SFLOAT_S8_UINT, and every M5 path that
handles the split format (render pass depth/stencil planes, copies,
views, meta) already accepts the D16 plane; only the format-feature
query was restricted to G17P.

Hardware (M5 Pro, R328, boot 3fc53692, installed Mesa R338): a
stratified 477-case sample of the ledger's D16S8 NotSupported cases
(301 groups) now runs: 228 Pass, 0 Fail, 249 NotSupported for other
reasons (exclusive queues, depthBounds, sample counts, sparse, video);
the 324 D32S8 sibling cases are unchanged (178 Pass / 146 NS); 0 GPU
faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
M5 already stores optimal three-component images padded to four
components (layouts, views, render-target and epilog formats, clears
and buffer/image copies all take hk_padded_image_format on M5), but the
format-feature query only admitted the sRGB ones. Report the padded
formats' sampled, filtering, colour-attachment, blend, blit and transfer
features as on G17P; scaled formats stay unsupported on every chip and
linear three-component images still report nothing.

The 16 changed formats (R8G8B8, B8G8R8, R16G16B16, R32G32B32 UNORM/SNORM/
UINT/SINT/SFLOAT as applicable) now report the same optimal features as
G17P (drm-shim format dump).

Hardware (M5 Pro, R328, boot 3fc53692, installed Mesa R338): a
154-case stratified sample of the ledger's NotSupported cases naming
these formats: 70 Pass, 0 Fail, 84 NotSupported for other reasons
(linear/transfer clears, exclusive queues, ray-tracing vertex formats,
unformatted storage); 0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
VkDrmFormatModifierPropertiesList2EXT carries VkFormatFeatureFlags2, but
both modifier lists were filled from the 32-bit VkFormatProperties, so
every modifier lost the features that only exist as 64-bit flags, among
them HOST_IMAGE_TRANSFER, STORAGE_READ/WRITE_WITHOUT_FORMAT and
SAMPLED_IMAGE_DEPTH_COMPARISON. Fill both from the 64-bit sets (the
first list truncates as the API requires), and do not report host image
transfer for the compressed tiled modifier: images with host-transfer
usage are never compressed.

Hardware (M5 Pro, R328, boot 3fc53692): a 300-case sample of the
ledger's NotSupported image.host_image_copy.drm_format_modifier cases:
118 Pass, 0 Fail, 182 NotSupported for other reasons (3D or
depth/stencil-usage combinations, layouts without host copy); 0 GPU
faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
Count the primitives generated by the last pre-rasterization stage with
the helpers that already count them for the clipper statistics and the
transform feedback query:

- vertex-only draws: the assembled primitives, through the input
  assembly helper (a second dispatch when pipeline statistics are also
  being counted);
- tessellation without geometry: the tessellator's prefix sum, which
  now takes a third counter;
- geometry (including the transform-feedback passthrough): the
  geometry pipeline's per-stream generated count, into a new
  query_prims_generated_counter so a transform feedback query on the
  same stream keeps its own counter; an active query also enables the
  geometry count pass.

All of these run in the pre-graphics compute work, independent of
rasterizer discard, and availability is published after the graphics
work as for pipeline statistics. Meta draws are not counted. Advertise
the extension with rasterizer-discard and non-zero-stream support on M5
only.

Hardware (M5 Pro, R328, boot 3fc53692): a 724-case stratified sample of
the 107,866 transform_feedback.primitives_generated_query cases: 684
Pass, 40 Fail, 0 GPU faults. All 40 failures are
concurrent.pipeline_statistics_3, where a pipeline statistics query
active in the primary must also count draws recorded in a secondary
command buffer; secondary command buffers do not inherit active
pipeline statistics in hk, independent of this query.

Signed-off-by: Diljit Singh <hey@djdev.me>
hk_cmd_invalidate_graphics_state() runs after the secondaries are
replayed and clears all graphics state except the render pass. That also
cleared the bindings of the queries active in the primary: the pipeline
statistics report address and flags, the transform feedback and
primitives generated query reports, the mesh primitives query and the
occlusion mode/index. A query begun before vkCmdExecuteCommands then
stopped counting the primary's later commands.

Queries are not bound state; Vulkan keeps them active across executed
secondaries. Save and restore the active query bindings around the
reset.

Fixes transform_feedback.primitives_generated_query.concurrent.
pipeline_statistics_3 ("input assembly primitives == 32, expected 64"
in R340 qualification): 80/80 Pass on M5 (R342), 0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
…ews on M5

hk_meta_copy_get_image_properties() enabled depth_stencil_aspect_views
for split depth/stencil formats only for single-sampled images. A
multisampled D16S8 or D32S8 copy instead used the interleaved
R32G32_UINT alias with depth in component 0 and stencil in component 1.
That alias is a colour view, and a colour aspect of a depth/stencil
image selects image plane 0, the depth plane, for both aspects.

On M5, use the per-plane raw aliases (R16_UINT or R32_UINT for depth,
R8_UINT for stencil) at every sample count for image-to-image copies.
G17P keeps its previous multisample formats.

Fixes the 48 api.copy_and_blit.*.depth_stencil_msaa_copy D16S8 8-sample
failures of R340 qualification. On M5 (R342), all 576 non-D24
depth_stencil_msaa_copy cases: 288 Pass, 288 NotSupported (16/32/64
samples), 0 Fail, 0 GPU faults.

Signed-off-by: Diljit Singh <hey@djdev.me>
A transform feedback query's counters are written by the geometry
helpers in the pre-graphics stream of the render that draws, but its
availability was written to the general compute stream. That stream
stays open across render begins and is not ordered after later
pre-graphics streams, so with two render passes the second query could
be marked available before its counters were written, and
vkGetQueryPoolResults with WAIT returned zero counts. R340
qualification reported this as "[Query 1] xfbGenerated == 0" (251 cases)
and "[Query 1] xfbWritten == 0" (4), all primitives_generated_query.get
cases with transform feedback and two queries.

On M5, publish the availability in the current render's pre-graphics
stream after the helpers, or after the graphics work when there is no
such stream or no current render.

On M5 (R342): the 255 cases Pass in three repeats (765/765), 0 GPU
faults. In isolation the R340 baseline also passed them once (255/255),
so the hardware evidence of the race is the qualification run; the
ordering is from the stream code. A 2,300-case regression sample (500
primitives_generated_query, 400 transform_feedback, 1,000 query_pool
including 600 secondary/inherited, 400 multisample copy_and_blit) is
identical to the R340 baseline case by case (1,956 Pass, 344
NotSupported).

Signed-off-by: Diljit Singh <hey@djdev.me>
Sampler LOD clamps are 4.6 fixed point and the M5 (Gen5) packer truncates
them, so a positive clamp below 1/64 became 0. The hardware selects the
minification filter only for a clamped lambda above zero: with
minLod = 0.01, an explicit-LOD-0 lookup (what glslang emits for texture()
outside fragment shaders) was magnified, although Vulkan clamps lambda to
0.01 first and minifies. A hardware probe (8x1 texture, NEAREST mag,
LINEAR min, explicit LOD 0) returned the NEAREST texel for minLod 0.01
and the minified average/min/max results for minLod 1/64.

Pack such clamps as one 1/64 step. Every other value packs as before.
The older Gen4/HAL200 packer (__gen_pack_lod) truncates the same way and
is left as is here.

This fixes pipeline.*.sampler.*.min_reduce.*_compute and
min_filter.linear_compute, which run on the compute-only queue family
and failed with a weighted-average result: on M5 hardware all 1,826
pipeline.{monolithic,shader_object_unlinked_spirv}.sampler cases that
failed on R339/R340 pass, with no failure in a 1,200-case pipeline
sampler and 600-case texture filtering/mipmap regression sample.

Signed-off-by: Diljit Singh <hey@djdev.me>
agx3_ioto_uniform_program_desc_body_bound() charged one extended COPY
(10 bytes) per selected constant-program input half. The loader publishes
per loaded uniform dword (pack_cprog_copy): an aligned selected pair is a
single 32-bit COPY, so that term was about twice the real cost. The M5
compute generator checks the bound against its 6 KiB program buffer before
composing, so preamble shaders that import a few hundred uniform halves
were refused although they compose to 3.2-4.1 KiB, e.g. 396 inputs gave a
7,352-byte bound. hk then failed the command buffer with "M5 compute
launch exceeds the current resource profile" (VK_ERROR_FEATURE_NOT_PRESENT),
which is how all 297 ubo.*compute failures of the compute-only queue
family's qualification run ended.

Count the COPYs the loader emits instead: one for a selected pair at an
even compact TPS position, else one per selected half. Each COPY is still
charged 10 bytes (8 on M5), and the sink remains capacity-checked. G17P
uses the bound only as a validity check.

The new agx3_m5_programs case composes a 414-input layout and checks that
the bound fits the program buffer (it fails before this change) and still
covers the generated body for aligned pairs, lone halves and pairs at odd
compact positions. On M5 hardware all 297 ubo.*compute cases pass, and a
500-case ubo regression sample passes.

Signed-off-by: Diljit Singh <hey@djdev.me>
RyanTheTide and others added 22 commits October 26, 2026 23:46
The G16S compression port admitted layered and mipmapped D/S images and
standalone S8 to compression. Uploads then bypass the raw-copy path and
reach the graphics-copy gate, which supports D16/D32 only for a single
level/layer and refuses standalone S8. Legal commands fail with
VK_ERROR_FEATURE_NOT_PRESENT at EndCommandBuffer. A CPU-only recording
probe reproduces 13 failures on the port and none on the prior driver.

Keep the pre-port single-level/layer/sample D16/D32/D32S8 allocation
policy and restore its ZLS setup until all expanded D/S access paths are
implemented and hardware-qualified. Preserve the separate G16S colour
compression extension and AS-only construction work. Update the G16S
policy tests to pin the retained D/S boundary across transfer, feedback
and host-copy usages.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The native AS-only device deliberately leaves gen1_rq_instances and
gen1_rq_candidates clear. The shared build admission still required those
traversal flags for AABB BLAS and instance TLAS geometry, rejecting both
command and synchronous host builds with FEATURE_NOT_PRESENT.

Inside the native-AS path, admit AABB and instance geometry based on its
geometry/type contract. The native GEN1 encoders already handle these
record kinds. Keep intersector allocation, query/pipeline feature gates
and all traversal state unchanged.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The exact FP32 underflow repair can only change a native signed-zero result. Branch around its multiword integer arithmetic for nonzero results, retaining the existing repair unchanged for the boundary cases. This avoids paying the wide-integer cost on every FMA in explicit-RTE shaders. Keep constant-folded numerical coverage and add a cold-path structure check.
aa733fc is the Mesa the J616s runs every day. Bring aurora-wip 2a5fd4a
into it: DJ's G17P refit and H-PLOC fixes (#79, #81), the deferred
occlusion resolver (#82, #83), the zink split-query readback fix (#84)
and the virtio queue-limits fix (#86).

The conflicts in the acceleration-structure sources and hk_query_pool.c
are resolved as in the earlier PR #80 merge 8b7395e. That merge also
carried 2a541c6, which is held back here (see the PR).

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Carry the M5 shader programs from the t-3206 adaptation as dedicated variants. The M5 backend keeps its IR-topology refit arguments and scratch layout; GEN1 keeps the current retained-job refit path. Reconcile the per-format CPU guard with those two layouts.
PR #80 copied agx3_{leaf,envelope,plan}.comp to agx3_gen1_*.comp before
DJ's G17P refit series (#79) landed, so after the merge the GEN1 build
programs that G17P and T6040 run were the pre-#79 encoders, while their
refit programs came from the #79 sources. The pre-#79 leaf encoder does
not retain the encoding jobs refit reads, stores records through shared
memory, and its envelope clears instance pointers on refit too. The
#79 agx3_{leaf,envelope,plan}.comp were still compiled, but nothing
used them.

The copies were byte-identical to the base before #79, and the G17P
plan already matched the #79 one apart from a comment. Build the
hk_agx3_gen1_* programs from the #79 sources and drop the copies, so
GEN1 builds and refits come from one set of encoders again. M5 keeps
its agx3_m5_legacy_* programs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
G16S splits graphics uploads into one attachment view per destination
slice. For these views fragment Layer is always zero, even when the image
is an array or volume and the copied slice has a nonzero origin. Give
these uploads a separate meta key and constant-fold the layer coordinate,
source image-stride arithmetic and layer bounds check.

Keep helper termination and the source XY bounds clamp: unaligned edges
can otherwise speculate an out-of-range read. Keep format conversion,
compression classes, per-slice source rebasing and barriers unchanged.
The opt-in is restricted by agx_is_m4_pro(); other devices retain the old
key and shader. One-dimensional uploads retain their generic path.

Add a GPU readback oracle covering 110 format/shape/mip cases, including
padded source footprints, nonzero array/Z origins and untouched texels.
--record-only is CPU admission under drm-shim, never a GPU correctness pass.

EXPERIMENTAL: benchmark qualification pending. Supplied before: gpubench
070c2579 buf2img d1 2325.632 us, d4 1592.357 us on the measured tip.
After: pending J616s A/B/A/B and CTS. No measured speedup claimed and no
claim this explains the earlier 4911479..ce03198 regression: its exact
nine runtime shader records and upload helper metadata are unchanged.
Required CTS: dEQP-VK.api.copy_and_blit.*buffer_to_image* and
*image_to_buffer*, with array/3D/mip/offset/pitch and format coverage.

(cherry picked from commit 480f485)
An inline fragment program with its own constant program launches from the linked layout, while frame anatomy counted the API export main. Select the owned layout for T6040 render streams so profiling reflects the launched fragment program. Compute streams and other products retain their existing reporting.

CPU recording of the blended full-screen workload shows an 8-instruction API export main and a 13-instruction inline blend with four FMAs and no spills. This is an opt-in instrumentation correction, not a measured GPU performance fix. Target execution and hardware qualification remain pending; it changes no shader, barrier or launch state.

(cherry picked from commit 8ace18a)
A complete write conflicts with another pass footprint only when either
pass preserves partial tiles or inactive layers. Keep all complete-write
history for later partial passes, but skip the retirement ioctl and recent
index construction when neither this submission nor earlier history ever
preserved tiles. Track preserving evictions separately so ring wrap does
not invent a complete-vs-complete dependency. Preserve the existing query
and conservative eviction behavior for every other physical device.

The CPU shim reports 2055 SYNCOBJ_QUERY calls for 2056 repeated clear or
triangle submissions on the parent. This candidate targets those calls;
no J616s before/after GPU gain is measured yet. Native g3 clear d8 records
show submit CPU 8.8 us on 409edac versus 11.0-11.8 us on c29122c, with
nearly unchanged recording CPU, but this is not a claim that the ioctl
explains the source regression.

Tests cover wraparound, later partial writes, live and retired preserving
evictions, both mixed full/partial command-buffer orders, and the legacy
query policy. Target synchronization/synchronization2 and clear/blend CTS,
packet/shader equivalence and exact-parent A/B remain qualification gates.

(cherry picked from commit 1e878ff)
The submit-local footprint list orders different command buffers. A submit
with one command buffer can instead retain its recorded list and publish
it to queue history only after resolving all existing hazards. This removes
one transient allocation/copy/free on ordinary render submits. Preserve
record order, both engine identities, ring evictions, partial writes and
the original multi-buffer/other-device paths. Enable only on exact G16S.

Candidate only; no native performance gain or regression closure claimed.
Before: g3 standard tri d64 submit CPU about 13 us vs macOS about 1 us;
after: J616s A/B/A/B and frozen synchronization CTS subset pending in
gap/MEASURE-RUN7.md. This cost is not attributed wholly to footprint storage.
Production CPU differential checks cover 48 combinations of exact/legacy,
serialized/overlap, 0/1/2 buffers and 0/1/2/257 passes, ring wrap and retirement.
Hardware evidence must be attached before promotion. Other generations keep
the previous path and shader bytes.

(cherry picked from commit ffc8242)
AGX2 never populates the native graphics loader, launch, combined-sampler or spilled-render-target caches. Their storage is already zeroed when the command buffer is allocated. Leave that contiguous storage untouched on G13 and G14 resets, and reset all common state on either side. Native graphics resets keep the full clear.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
(cherry picked from commit 51c0bfd)
The per-recording background/end-of-tile cache is only populated by the native graphics path. G13 and G14 otherwise invalidate sixteen entries spread over 13.5 KiB on every command-buffer reset, despite all of them remaining unused. Keep the existing invalidation on other generations.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
(cherry picked from commit 3d00b6d)
G13X serializes render and compute work. Submit each control stream when it is ready so rendering can start while the following compute submission is prepared. This restores the upstream submission shape for post-graphics query availability. Keep batching on the other chips, including G14G and native AGX3.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
(cherry picked from commit af16d98)
Avoid the header allocation, command reallocation and free for small T6040 submits. util_dynarray retains its ordinary heap growth for larger payloads, and all other chip profiles keep their existing allocation path. This changes CPU payload storage only; commands, synchronization and shader code remain unchanged.

(cherry picked from commit 2a541c6)
A render has four timestamp slots. When a pass ends more occlusion
queries than its render has free slots, its deferred availability stream
is MOVABLE and stays as a compute command waiting on that render. The
reverse resolver then judged every earlier deferred stream against it:
the movable stream is compute waiting past the earlier render, so the
earlier stream could not be stamped and was dispatched in place. A
command buffer whose passes each end more than four queries therefore
emitted one compute command per pass, each with its cache flush and a
render to compute hand-off, where one command at the end would do.

Before judging an earlier stream, let it join the movable stream when
nothing else waits on it and the movable stream waits on a render at or
after its own. Renders complete in order, which the forward resolver
already relies on when it collects movable stores. Joining is preferred
to stamping the earlier render: the movable compute command is emitted
anyway, and a stamped render would make it wait for that render's
host-side retirement instead of in firmware. A stream that a later
reset, copy or other compute waits on still stays in place.

Under the asahi drm-shim with the G16S identity, a command buffer of
three passes with eight queries each now submits three renders and one
compute command instead of three renders and three compute commands.
One query per pass is unchanged: every render carries its own
availability and no compute command is emitted.

The reverse resolver is used on G17P only, so the other native AGX3
chips keep their current behaviour.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
(cherry picked from commit 00cabf8)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants