Skip to content

asahi: consolidate M4 Pro support on the native/M5 chip split - #80

Closed
RyanTheTide wants to merge 279 commits into
aurora-wipfrom
wip/m4pro-mesa-tip-20261026
Closed

RyanTheTide wants to merge 279 commits into
aurora-wipfrom
wip/m4pro-mesa-tip-20261026

Conversation

@RyanTheTide

@RyanTheTide RyanTheTide commented Oct 26, 2026 •

Copy link
Copy Markdown

Consolidate M4 Pro (T6040/G16S B1) bring-up on the Neo tip plus the existing native/M5 chip split. Retain physical chip identity, Borealis branding, zero conformance version pending qualification, GL HIGH-priority fallback, and packed-GS/immediate correctness fixes.

Keep bounded HAL200/Gen9 layered and 2x/4x colour compression selected only for T6040, including the colour-MSAA attachment spill restriction. Keep depth/stencil on the pre-port single-level/layer/sample D16/D32/D32S8 policy and its original ZLS setup. The expanded depth/stencil policy bypassed raw uploads and reached a graphics-copy gate that rejects compressed mips/arrays and standalone S8, causing VK_ERROR_FEATURE_NOT_PRESENT at EndCommandBuffer. A CPU-only probe reproduces 13 recording failures on the expanded policy and none on the prior driver; the depth-policy correction passes all 32 recording cases on G13X/G14G/G17P/G16S fixtures. Broader depth/stencil compression remains deferred until all access paths are implemented and qualified.

Separate native GEN1 AS construction from traversal on exact T6040. AS build/copy/refit uses ordinary compute with exact-prior and bounded-AS-build kernel capabilities (bits12+16); AS-only devices do not initialize intersectors or ray stacks. Query/pipeline traversal keeps its existing bits12+15 gate. Native AABB/instance construction also no longer depends on the traversal-state booleans deliberately absent on AS-only devices; both command and synchronous host builders share that admission. Intersectors and traversal gates are unchanged. No unsupported capability is forced.

Prior source validation: ARM64 and Fedora release WERROR builds, 110 CPU tests, GEN1 repeated 12/12, 611 RT-core checks, 450 unchanged serialized helper kernels, 36 CPU-only contract fixtures, and identical statistics/instructions for seven gpubench pipelines on M1 Pro, M2, Neo and M4 Pro compile targets. Both corrections build with WERROR and pass the 110 CPU tests. The depth correction also passes 12 shader comparisons, 36 contracts and AS API checks; the combined correction also passes 12 shader comparisons, 36 contracts and AS API checks. Triangle/AABB/instance command recording passes with and without host commands enabled; the prior candidate reproduces the AABB/instance rejections. Generated helper shader source is byte-identical. The AS-capable benchmark has a matching O3/NDEBUG WERROR Release build and retains query support skips. CPU probes execute no GPU work.

Queue submissions on exact T6040 start their ioctl payload on 1024 bytes of stack storage, with the existing dynarray heap fallback for larger payloads. Other chip identities retain their allocation path. This changes CPU allocation only; no command, synchronization or shader change is intended. A 32-process CPU-only real-ICD oracle matches payloads, sync descriptors and mapping records against the parent across small and heap-fallback submissions, timestamps, depths and synthetic cumulative-sync settings. Scoped untimed single-clear checks remove two reallocations and one free per submit. The matched Fedora release and ARM64 WERROR builds, 110 CPU tests, 611 RT-core checks, 36 contract fixtures and 12 shader/instruction comparisons pass for the new head. Hardware correctness and performance qualification remain required; no speedup is claimed.

Draft gates: follow the final landed t-3165 split; hardware-qualify the exact replacement overlay and review CTS/paired gpubench including queue rows. Establish the valid G16S firmware traversal/recovery contract before query/pipeline execution and RT performance comparisons. The shared GS correction intentionally fixes previously uncompileable mixed-type stores; byte identity applies to the recorded G16-only delta/sample, not the entire inherited consolidation. No merge readiness or measured performance gain is claimed.

The Gen10 compute textureQueryLod lowering reads min/max LOD, mip filter
and the signed LOD bias straight out of the descriptor's copy of the
Gen5 sampler word. Assert those bit positions against the packer for
the existing pinned words (bias 0, 1.5, -2.25, max LOD 14) and the base
state (min LOD 1, clamped max 15, linear mip filter, bias -1.015625).

Signed-off-by: Diljit Singh <hey@djdev.me>
A static-vertex-input VS containing an M5 native ray query reserves
uniform halves 0..167 for query state, and the prolog key and bound
uniform ranges place vertex buffer bases at 168. The no-prolog path only
applied that offset for Gen1 queries, so M5 query vertex shaders read
their attributes from query slots and traced rays from garbage origins.

Signed-off-by: Diljit Singh <hey@djdev.me>
M5 refused every mesh shader that writes a per-primitive user output,
gl_Layer or gl_ViewportIndex ("native mesh layout outside captured
triangle contract"), which covered ~600 EXT_mesh_shader, pipeline EDS,
library and shader-object CTS cases.

Give each primitive three private vertices instead: the API body writes
vertex words and per-primitive words to separate shared regions, then
SIMD group 0 copies, between two threadgroup barriers, the referenced
vertex words and the primitive's words to vertices 3p..3p+2 and rewrites
the index list to the identity. Per-primitive words are therefore flat
on every corner, so any provoking vertex selects them; positions,
winding, primitive order and interpolated values are unchanged. Layer
and viewport also fill the packed layer/viewport UVS word and, like the
M5 vertex stage (layer_as_varying), flat user words for fragment reads.
The fragment side reads per-primitive user/layer/viewport inputs as flat
32-bit inputs. Shaders are admitted while 3 * max_primitives <= 256 (the
native output-vertex limit); mesh-written primitive IDs and primitive
culling stay refused. The exported vertex count is reported as the mesh
max_vertices for the native output limits.

This is an emulation, not Apple's per-primitive record path (whose
storage/lifetime contract is still unproven on M5, R175); it costs at
most 3 * max_primitives exported vertices and only applies to shaders
that were refused before. Meshes without per-primitive outputs compile
byte-identically (host drm-shim: every R314 binary of all 6,056 mesh CTS
cases is reproduced).

Hardware (boot 2e8881b2, R239 kernel): a SIMD-group barrier between the
copy and the UVB stores exported stale words (api.draw_count_64 wrong
colours/missing triangles); a copy loop over all threads lost workgroup 2
of builtin.layer deterministically (cause not established); the SIMD 0
copy between threadgroup barriers passes both.

Signed-off-by: Diljit Singh <hey@djdev.me>
M5 vertex-stage draws select the provoking vertex through the VDM
vertex program template; native mesh draws have no such template
and always used the first vertex, so last-vertex mesh draws read flat
inputs from the wrong vertex (pipeline.*.extended_dynamic_state.
mesh_shader.*.provoking_vertex_first_to_last_*, mesh_shader.ext.misc.
clip_*_provoking_last).

For mesh draws, program the HAL300 PPP cull word's flat-shading vertex
((vertex + 1) << 7) instead, and recompute the selector when binding or
unbinding a mesh shader changes the topology. Vertex-stage draws keep 0
in the PPP word. On hardware the PPP selector makes the EDS last-vertex
mesh draws pass; putting the selector in the VDM mesh-state word 1
(bits 0..1, the vertex-outputs word-1 analogue) had no effect.

Signed-off-by: Diljit Singh <hey@djdev.me>
Lower M5 native ray queries in tessellation-evaluation shaders. The
hardware evaluation variant becomes a vertex program in poly_nir_lower_tes()
after the per-stage query scan, so flag it from the API stage, giving it the
vertex query state (uniform halves 0..167, SCS/IFT render program). An
evaluation variant run as a kernel before a geometry shader uses the compute
query ABI; admit its internal dispatch like the control and geometry
kernels.

Also stop the M5 indexed-read vertex shift from moving query helper loads:
with a native query the vertex ABI starts at half 168 and only loads above
it belong to the vertex ABI.

Hardware (R320a on the 329 previously refused evaluation cases): 274 Pass,
54 Fail with wrong results, 1 DeviceLost during a recovery shared with
concurrent geometry runs. Host: agx_tests 428/428, agx_scs_query adds the
evaluation stage.

Signed-off-by: Diljit Singh <hey@djdev.me>
…eries opt-in

poly_nir_lower_gs clones the (already lowered) geometry shader into the main
and count kernels and the raster vertex program. The main kernel has the API
stores stripped, so a native query whose results only fed them was dead work
that still ran on CDM ahead of the raster draw. agx_nir_m5_remove_dead_ray_queries
marks every value data- or control-dependent on a native query operation
(SCS loads/stores, enter/intersect/continue) and removes the query only when
no remaining side effect depends on it; otherwise only trivial phis are
folded. Host compile of 308 geometry ray-query CTS pipelines: no main or
count kernel keeps a query (e.g. builtin.flow.geom main 4994 -> 120 bytes);
the raster vertex program keeps it.

Geometry ray queries nevertheless stay refused by default: hardware runs of
GS query draws (before this removal) requested GPU recovery twice and the
cause is not isolated. ASAHI_M5_GS_RAY_QUERY_DIAGNOSTIC=1 enables them for
diagnosis; ASAHI_M5_RQ_DEAD_DIAG=1 reports a query kept live.

Host: agx_tests 428/428; agx_scs_query adds the geometry stage and a
dead/live removal regression.

Signed-off-by: Diljit Singh <hey@djdev.me>
gl_TessLevelOuter/Inner are compact float arrays. nir_lower_io turns an
indirect element index into an I/O offset counted in components, but poly's
TCS output and TES input addressing (tcs_out_addr, lower_tes_load) adds the
offset to the varying slot. A control shader writing
gl_TessLevelOuter[gl_InvocationID] therefore stored levels 1 and 2 into other
slots, left them zero and every patch was culled, so the evaluation shader
never ran. Convert the levels to vectors (nir_lower_tess_level_array_vars_to_vec)
so an indexed access becomes per-component write-masked stores or a vector
load and extract, which poly already addresses correctly.

This was the cause of the remaining evaluation-shader ray-query failures:
hardware, 55 previously non-passing evaluation cases -> 54 Pass, 1 Fail
(watertightness.singlehit.tese, which fails on every stage including
compute). Replacing the control shader of
binding_model...ray_query.regular.tese with one that writes every level from
each invocation made the unchanged evaluation shader pass, and a query-free
evaluation shader storing a constant wrote nothing before this change.

Limited to M5; the poly addressing is shared, so other chips are expected to
have the same defect but are untested here.

Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_RQ_KEEP_LOOPS=1 marks loops whose body runs a native ray query
nir_loop_control_dont_unroll, except loops that index a query array with a
non-constant index (unrolling those lets the arrays split into fixed native
contexts; rolling them grew rayqueryterminate from 27 KB to 337 KB).

Host compile of 36 ray_query.builtin cases (vert/tese/frag): total code
3,257,740 -> 568,988 bytes, largest graphics shader 370 KB -> 7 KB (the
evaluation-shader pixel loops), host compile wall time 9.3 s -> 2.0 s; only
the 8 evaluation-shader cases change. Hardware A/B over 1,088 common cases of
a 1,581-case loop-candidate list: identical statuses except two DeviceLost
cases in the default run, which pass with the policy. Both A/B runs stopped
at a GPU recovery before completion, so this stays default-off.

Signed-off-by: Diljit Singh <hey@djdev.me>
A vertex or compute loop that copies an image texel by texel waited on
one read at a time on M5: every image_load was ordered after the
previous image_write, and the pressure schedule put each write right
after its read. Apple's compiler issues four reads ahead of four writes
in the same loop (native725 vs_copy: smp x4, then imgwci x4).

- A storage-image read whose NIR access is ACCESS_CAN_REORDER and
  ACCESS_RESTRICT (and not coherent/volatile) drops its memory-order
  edges on M5, like texture_load. CAN_REORDER is nir_opt_access's
  statement that nothing in the shader writes the image (NonWritable or
  no writes); RESTRICT is vtn's default for an image not decorated
  Aliased, which the SPIR-V Simple/GLSL/Vulkan memory models allow the
  consumer to assume does not alias any other memory object declaration
  (SPIR-V 2.18.2 "Aliasing"; vtn_variables.c). So no store of the shader
  can write what the read observes.
- The occupancy-capped M5 sample-issue schedule now covers blocks that
  read at least four samples and write images or device memory: every
  such block of the non-compute geometry-side stages and compute loop
  bodies. Its window is anchored to the writes already issued, keeping
  four reads in flight ahead of the write that consumes the oldest.
- In those blocks, constants defined outside the block are
  rematerialized ahead of their first use. The 64-way unrolled 64x64
  copy otherwise held 64 coordinate constants live across the whole
  body (128 of its 150 halves), leaving no registers within its
  occupancy bucket for reads in flight; the body now peaks at 26 halves.

copy-z4 (6-vertex VS copying 4 z-slices of a 64x64x8 R32F image), cold
after 3 s idle on M5: 8.48 ms -> 2.42 ms; copy-z2 4.48 -> 1.24 ms;
copy-z8 4.79 ms (native Metal under capture: 5.575 ms); alu-r8
unchanged at 3.43 ms.

Signed-off-by: Diljit Singh <hey@djdev.me>
9275604f4c3 dropped a reorderable image load's memory-order edges in
every dependency graph, including the bottom-up pressure schedule that
runs first on all stages. Without those edges the pressure order sank
such loads next to their uses: a host replay of the vkQuake-RT frames
80-81 capture on the noop drm-shim compiled 3 of its 133 binaries
differently from the base, all compute post-process passes with two or
three image loads (5060- and 532/528-byte programs), and in the
532-byte one both SMPs moved below independent arithmetic they used to
precede.

Only the M5 copy schedule (the write-anchored sample-issue schedule of
copy blocks) now drops those edges. The replay compiles all 133
binaries byte-identically to the base again; the 64x64 vertex copy
loop still compiles to the hardware-measured binary (9434 bytes, same
hash as before this change).

Signed-off-by: Diljit Singh <hey@djdev.me>
nir_lower_int64 expands every 64-bit udiv/umod into a fully unrolled
64-step division (about 780 AGX instructions per scalar division whatever
the operands). The native G17G compiler instead aligns the divisor with the
dividend using leading-zero counts and loops only over the quotient bits
that can be set, with a native 64-bit subtract (for
`long a / long b`: 124 static instructions including the
loop).

agx_nir_lower_divmod64 emits that algorithm for 64-bit udiv, umod, idiv,
irem and imod on M5, before nir_lower_int64 lowers the remaining 64-bit
compares and shifts. The loop runs msb(n) - msb(d) + 1 times when n >= d;
division by zero (undefined) returns quotient 0 and remainder n.

The raster vertex shader of spirv_assembly.type.vec4.i64.div_geom shrinks
from 9405 to 1217 instructions. GPU time on M5 of a vertex loop of 3x50
i64vec4 divisions: signed 2.89 -> 0.25 ms, unsigned 2.77 -> 0.34 ms; at
the CTS size (3x390) signed is 2.0-3.3 ms, where the old lowering
extrapolates to about 23 ms, past the TA progress check. Results match a
CPU reference (0 mismatches). New host test checks all five operations
against C on 16x15 operand pairs by constant-folding the unrolled loop.

The pass and its host test already exist for G17P, which builds the loop's
64-bit add and subtract from 32-bit halves. This change runs the pass on M5
as well and lets it keep those operations native there; the host test
checks both forms.

Signed-off-by: Diljit Singh <hey@djdev.me>
On M5 (G17G/G17S) allocate_uvb completes on scoreboard slot 1. Every
native write_uvb helper in the captured G17G witnesses waits slot 1 on
the instruction right after allocate_uvb (cndst with f4 set, 1e00002e or
9e60002e). Mesa issued the UVSTs without that wait.

On hardware this lost the exports of alternate mesh workgroups: of 8
one-triangle workgroups, 2, 4 and 6 drew nothing (1, 3 and 32 threads
per workgroup alike; also when a task stage launched 8 children), and
dEQP-VK.mesh_shader.ext.builtin.work_group_id_in_mesh failed every run.
With the wait, all eight workgroups draw in every run.

Retire any older slot-1 generation before the allocation claims the
slot, then wait for the allocation. Only AGX3C changes.

Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_TASK=1 (with ASAHI_MESH=native-probe, G17S only) advertises
taskShader and runs the API task shader as the native object program of
the mesh draw, in place of the mesh-only passthrough object:

- agx_nir_lower_m5_task publishes the EmitMeshTasks grid with UVSW from
  lanes 0..2 of SIMD group 0. A payload-free task takes the grid of
  invocation 0 from registers, as the native passthrough object does.
  Payload-carrying tasks use the native threadgroup grid, rendezvous and
  payload-exit barrier; workgroups under three invocations are padded to
  three lanes with the API body masked, as for mesh workgroups.
- Task payloads keep the native SR62/SR63 payload address. A read-only
  probe (ASAHI_MESH_PP_PROBE=payload, added here) reads SR62=SR63=0 on
  the current boot, whose provider loads asahi with
  m5_mesh_storage_chunks=0 (no 0x1ca48 mesh rendering buffer). Payload
  accesses are therefore refused unless ASAHI_M5_TASK_PAYLOAD gives a
  byte limit.
- hk_draw_mesh_native sizes the object dispatch from the API task grid
  (direct draws only; indirect, draw-count and conditional task draws
  are refused), publishes the task's draw root (DrawID, NumWorkGroups)
  and gives the kernel the task's thread count and threadgroup memory.
- Task shaders read gl_NumWorkGroups and gl_DrawID from the draw root,
  like mesh shaders. Known gap: a mesh shader under a task stage still
  reads the API task grid as gl_NumWorkGroups.

Hardware (boot c73879b2, R239 kernel): payload-free tasks of 1 and 32
invocations launch data-dependent child grids correctly (per-pixel draw
counts 4 3 2 1 for EmitMeshTasks(id+1) over 4 tasks). The first attempt,
which used the threadgroup/rendezvous shape and tail[0]=1 for a
payload-free task, caused one GPU recovery; tail[0] stays at the
mesh-only value.

Default behaviour (no ASAHI_M5_TASK) is unchanged.

Signed-off-by: Diljit Singh <hey@djdev.me>
Native G17G point and line mesh witnesses (lane-mesh/cwit-meshtopo mpt,
mptc, mln) use the triangle layout with one and two packed index bytes
per primitive in the 16-byte-aligned index region, and points carry
their size in the plane right after the four position planes (plane 4,
before any user plane).

agx_nir_lower_m5_mesh now admits points and lines: index bytes, the
exported index words and the index-region size scale with the
primitive's vertex count, and a point mesh exports PSIZ at plane 4. Only
the size-only point interface (positions + size) is admitted, since the
generic UVS order places PSIZ after user varyings; per-primitive outputs
(private vertices) stay triangle-only. As on G17P, a point mesh that does
not write gl_PointSize gets the default size 1.0.

These were refused at pipeline creation before. Hardware (boot cc58080f,
R328 kernel, 7283-case mesh/non-mesh set, jobs 4): 271 more Pass than the
previous commit (256 shader_module_identifier pipeline_from_id point
meshes, single_point/single_line, point_size, many_mesh_work_groups_*,
misc.state.*), 0 regressions, 0 GPU faults. depth_only_points* (8) and
two line query cases still fail at runtime.

Signed-off-by: Diljit Singh <hey@djdev.me>
SPIR-V EmitMeshTasksEXT terminates the invocation, so vtn follows the
launch with a halt. With the launch inside an if/else both branches
halted before the appended child-count epilogue, which then published
undefined counts. A task shader has no other halt source: turn the halts
into returns and lower them, so the launch ends main.

Hardware (opt-in ASAHI_M5_TASK=1, boot cc58080f): emit_in_control_flow
and emit_in_control_flow_bad_emit_last now pass; the 1296-case task set
gives 10 Pass, 0 GPU faults (the rest refuse payloads, indirect or
conditional task draws, or are NotSupported).

Signed-off-by: Diljit Singh <hey@djdev.me>
vkCreateRayTracingPipelinesKHR read each group's
pShaderGroupCaptureReplayHandle unconditionally. Replaying the vkQuake-RT
frames 80-81 capture with gfxreconstruct segfaulted there with every
library since the feature was exposed: the capture holds a 0-byte
SetRayTracingShaderGroupHandles block for pipeline 458 (its capture-time
device did not support handle capture/replay), and gfxreconstruct, which
enables the feature whenever the device has it, then passes
NULL + 4 * group as each group's handle pointer (0x4 for group 1; the
fault was ldr w24, [x0] with x0 = 4).

A handle pointer inside the zero page cannot address application memory,
and handle 0 is never issued (handles start at 1), so fail such groups
with VK_ERROR_INVALID_OPAQUE_CAPTURE_ADDRESS, the code for a replay
handle the implementation cannot reuse, rather than dereferencing them.

Signed-off-by: Diljit Singh <hey@djdev.me>
vkAllocateMemory rounded allocationSize up to the 16 KiB page and passed
the result to agx_bo_create() unchecked. A 0-byte request (and any size
whose rounding wraps to 0) became a 0-byte BO request that only failed
in the kernel (GEM_CREATE: ENOSPC, "BO creation failed: size=0").

Such requests come from cooperative_matrix.op_constant_null in the
deqp build we run: its iterator does not stop when the remaining
device configuration (FP32 x FP32 -> FP32, 8x8x8) is absent from its
own static list, which still excludes FP32 A/B operands, so the
instance runs a zero-initialized configuration and sizes its buffers 0.

An allocation that imports nothing must have allocationSize > 0, and
no size beyond the memory heap can ever be satisfied, so return
VK_ERROR_OUT_OF_DEVICE_MEMORY for both before touching the BO layer.
Imports keep their existing path.

Signed-off-by: Diljit Singh <hey@djdev.me>
A fragment shader that runs a ray query acquires the colour write class
(pixwait 4) ahead of its first traversal, so the linker turns its
epilog's read+write wait (pixwait 12) into a read wait (pixwait 8). It
built both the expected and the replacement instruction in the AGX3B
encoding only, so on M5 the AGX3C epilog never matched and every such
link failed (36 ray_query.helper_invocations.gpu cases ended in
vkEndCommandBuffer VK_ERROR_FEATURE_NOT_PRESENT).

Native M5 colour fragments with a ray query (the native G17G compiler,
intersection_query and intersector forms) show exactly this pair:
pixwait 4 immediately before intersect_ray, pixwait 8 before lstcol,
then pixrel 524. Encode the rewrite in the device's ISA.

Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_RQ_WATERTIGHT=1 enables it; default binaries are unchanged.

M5's triangle test is inclusive: when its edge function is exactly zero it
reports every triangle at that edge or vertex, each with a zero
barycentric there. Hardware probes (exact dyadic fixtures, 16384 generic
near-edge rays and the closedFan geometry) show near-edge rays otherwise
get exactly one triangle, but exact-edge rays get both. That double count
fails ray_query.watertightness.singlehit and
ray_tracing_pipeline.watertightness.closedFan/closedFan2. The hardware's
arithmetic is not exact (closedFan spoke rays get both neighbours although
exact arithmetic puts them strictly inside one), so a shader cannot
predict which candidates it reports. Apple's runtime query.next does not
filter them either.

When enabled, remember the distance of the last boundary triangle (some
barycentric within 2^-22 of zero, allowing for the hardware's ~1 ulp
barycentric rounding) a query let through, and drop a later boundary
triangle at the same distance: equal or adjacent 256-ulp distance keys.
Partners compute t from their own planes; on the CTS random-height mesh
they differ by more than 31 ulps at some diagonal pixels. Both are the
same point on the shared edge or vertex. A drop only follows its
delivered partner, so it cannot turn a hit into a miss. The key lives in
the candidate cursor's spare bits; no memory access.

Hardware (M5 Pro, R328): 34 singlehit/nomiss (compute, fragment, RT
stages) and closedFan/closedFan2 cases: 19 Pass / 15 Fail disabled,
34/34 Pass enabled. Vertex/tess/fragment singlehit and nomiss at one
worker, enabled: 11/11 Pass (geometry-stage cases fail pipeline creation
either way). vkQuake RT frames 80-81: images byte-identical to R337;
replay time within noise (15 interleaved pairs, median +0.04 ms/frame).

Cost when enabled, drm-shim shaderdb census of 1743 ray-query/RT shaders
against the default: +7.42% code bytes, median +35 instructions, median
GPR change 0; 30 shaders lose occupancy (28 at 1024 -> 896 threads) and 6
gain. Disabled: all 1743 shaders identical.

Signed-off-by: Diljit Singh <hey@djdev.me>
On M5 a vertex-stage draw picks its provoking vertex through the VDM
vertex program template, and hk left the PPP flat-shading vertex
at 0 for everything but mesh draws.
pipeline.*.extended_dynamic_state.*provoking_vertex_first_to_last* draw
oversized triangles with the last-vertex convention and shaded them from
the wrong vertex in every construction type (monolithic,
pipeline_library, fast_linked_library, shader_object_unlinked_spirv);
rasterization.provoking_vertex, whose triangles stay inside the viewport,
passed. Presumably clipping takes the flat inputs of the vertices it
generates from the PPP selector; that mechanism is inferred, not
measured.

Program the PPP selector from the same provoking state for all draws.

Hardware (M5 Pro, R328), diagnostic switch on the R337 base: the 20
two_draws_{dynamic,static} provoking cases across the four construction
types went from 12 Fail / 8 Pass to 20/20 Pass, and all 63
rasterization.provoking_vertex cases pass.

Signed-off-by: Diljit Singh <hey@djdev.me>
Fragment shaders that need the raster sample count (interpolateAtSample's
sample clamp, sample-position lowering) read samples_log2. Neither G17P
nor M5 has a grounded AGX3 encoding for that special register, so the
packer refuses it. hk folds it to the keyed count on G17P, but M5 keys the
count as m5_position_samples and was never specialized: every such
fragment shader failed to compile on M5. Pipelines creating it returned
VK_ERROR_UNKNOWN, and per-sample-count variants built at draw time ended
their command buffer with VK_ERROR_UNKNOWN
(pipeline.fast_linked_library.misc.frag_lib_varying_samples_{2,4,8},
interpolate_at_sample_no_sample_shading).

Specialize with the native sample count on both chips; M5 counts reach 8.

drm-shim: no remaining samples_log2 refusals; 15268 multisample/misc
shaders unchanged (5244 sampled cases).

Signed-off-by: Diljit Singh <hey@djdev.me>
…fault

Vulkan requires a ray that lands exactly on an edge shared by two
triangles to report a single hit; M5's inclusive triangle test reports
both. The filter fixes the 15 watertightness CTS failures (singlehit in
every stage, closedFan/closedFan2). Measured cost: +7.4% code in
ray-query/RT shaders, 30 of 1,743 shaders drop one occupancy step, no
measurable change in the vkQuake-RT replay (median +0.04 ms on ~3.1 ms
per frame, byte-identical images). Enabled by default at the user's
direction; ASAHI_M5_RQ_WATERTIGHT=0 turns it off for diagnostics.

Signed-off-by: Diljit Singh <hey@djdev.me>
Geometry ray queries were refused unless ASAHI_M5_GS_RAY_QUERY_DIAGNOSTIC=1,
after GS query draws requested GPU recovery twice. Those runs predate the
removal of dead native queries from the poly main/count kernels.

With the R338 stack and the queries enabled, all 578 geometry ray-query
cases of the mustpass (acceleration_structures, ray_flags,
traversal_control, builtin, watertightness, stress,
binding_model.*acceleration_structure*.geom) ran on M5 hardware with one
worker in 26 s: 299 Pass, 249 NotSupported, 30 Fail, no GPU recovery and
no long-running case. The 30 failures are
binding_model.mutable_descriptor.single.acceleration_structure_khr.*.geom
("expected 2, found 1"): the query's candidate-found result does not reach
the atomicAdd in the main GS kernel (open).

ASAHI_M5_GS_RAY_QUERY_DISABLE=1 refuses them for diagnosis. Concurrency
behaviour beyond one worker is not qualified here.

Signed-off-by: Diljit Singh <hey@djdev.me>
With independent kernel queues, advertise two more queue families next
to the universal one: compute+transfer and transfer-only, each with the
same queue count. Kernel queues carry no capability class (every queue
accepts render and compute commands) and hk records copies, clears and
blits the same way on any family, so these families only narrow what the
application may record. They reuse the universal queue implementation:
one kernel queue per Vulkan queue, created on first submission in the
device's single VM; no queue or VM is created for an unused queue.

This makes the CTS's exclusive compute/transfer queues, concurrent
sharing across families and cross-family queue pairs available. A
sampled hardware run (M5 Pro, R328 + R338 Mesa, 2 workers) of 1146 cases
that were NotSupported for queue reasons: 857 Pass, 282 NotSupported
(118 still lack a queue pair: they need two graphics families; the rest
for unrelated reasons), 7 Fail, all newly exposed compute variants (5 "M5
compute launch exceeds the current resource profile", 2 sampler
min_reduce compute image mismatches). A regression sample of 863
previously passing synchronization/api/compute cases: 863 Pass. No GPU
recovery.

Signed-off-by: Diljit Singh <hey@djdev.me>
Native G17G object/mesh pairs (lane-mesh nwit2 otpg/mtpg/mtpg0)
address one record through the payload address SR62/SR63: the object
stores its child grid (threadgroups_per_grid) at +0 and the payload from
+16, and the mesh reads its threadgroups_per_grid from +0 after the
payload-entry barrier. The record lives in the TA mesh rendering buffer
(register 0x1ca48), which the kernel only binds to task kicks.

With ASAHI_M5_TASK=1 and a kernel reporting
DRM_ASAHI_FEATURE_TASK_MESH_STORAGE (new uapi bit 13, with
DRM_ASAHI_RENDER_PROGRAM_TASK, bit 3):
- task shaders use the native threadgroup grid, rendezvous and
  payload-exit barrier, store the child grid at +0 of the record and
  accept payloads up to 16 KiB at +16;
- a mesh shader compiled for a task stage (no
  VK_SHADER_CREATE_NO_TASK_SHADER_BIT_EXT) reads gl_NumWorkGroups from
  the record instead of the API grid, and payloads from +16;
- task kicks carry DRM_ASAHI_RENDER_PROGRAM_TASK and set VDM mesh
  tail[0], as native object->mesh kicks do.

Without the feature only payload-free task shaders are accepted, as
before, and a task-stage mesh shader that reads gl_NumWorkGroups is
refused instead of silently reading the API grid. The
ASAHI_M5_TASK_PAYLOAD override is gone. drm-shim gains
ASAHI_SHIM_CHIP=g17s-task for host compile tests of the record path.

Not hardware-tested: needs a kernel with the task-only 0x1ca48 binding
and m5_mesh_storage_chunks != 0.

Signed-off-by: Diljit Singh <hey@djdev.me>
Mesh workgroups smaller than their export batch are padded and the API
body runs under the original invocation mask. The payload-entry barrier
inserted at the start of a payload- or record-reading mesh shader ended
up inside that mask. Native mesh main functions pass it with every lane
before the thread-index branch (lane-mesh nwit mpay, nwit2 mtpg); hoist
it out of the masked body the same way.

Only mesh shaders that read a task payload or the task record change.

Signed-off-by: Diljit Singh <hey@djdev.me>
hk_compile_shaders lowers every shader's I/O and system values (including
gl_NumWorkGroups to the draw root) before hk_compile_shader_impl, so the
task-record substitution placed in the impl never saw pipeline shaders: a
mesh shader under a task stage still read the API grid (hardware probe on
R328: per-pixel counts 0 0 0 10 instead of 1 2 3 4). Do the substitution,
or the refusal without the record, in both lowering paths.

Signed-off-by: Diljit Singh <hey@djdev.me>
Indirect, indirect-count and conditionally rendered task draws were
refused because the object dispatch was sized from a CPU grid. Record
them with one object workgroup and dispatch libagx_m5_task_dispatch in
the pre-graphics compute stream: it patches the object thread counts of
the VDM mesh dispatch record from the GPU grid and writes the draw's
execute word (grid nonzero, draw index below the count, predicate
passes). The task shader checks that word, through the new root field
mesh_task_flag, on entry and launches no mesh children for a disabled
draw; the word is zero for every other draw.

The kernel is admitted for the M5 (g17g) libagx target only, and the
g17s-task drm-shim chip now also reports COMPUTE_RESOURCES, as the
kernel does.

Hardware (R328, boot 410b5227, ASAHI_M5_TASK=1, no task record): all 360
payload-free conditional_rendering task cases (draw, draw_indirect,
draw_indirect_count) Pass, 0 GPU faults. Indirect task cases with
payloads need the task-record kernel.

Signed-off-by: Diljit Singh <hey@djdev.me>
An M5 task (object) shader with an image or texture read reached the
AGX3 packer with the 32-bit bindless form (uniform heap base plus a
register index), which has no grounded encoding, so every task shader
reading an image was refused. Treat the task stage like the mesh stage:
lower its reads to the heap-indexed descriptor-address form, stage
image-load coordinates as for mesh and kernel programs, and bind the
indexed heap pointers for a task program.

drm-shim (g17s-task, ASAHI_M5_TASK=1): the 10 synchronization.*_to_task
image cases now compile; their hardware run needs the task-record kernel.

Signed-off-by: Diljit Singh <hey@djdev.me>
A mesh workgroup padded to its export width, or a task workgroup of
fewer than three invocations, runs its API body under the exact
invocation mask. A user execution barrier in that body was refused,
since moving it under the mask changes the set of invocations that
reaches it. Split the body at top-level execution barriers instead:
each segment between barriers runs under the mask and the barriers stay
outside it, so every physical lane passes them, as Apple passes
barriers with all lanes enabled (cndst 0,6,0). Values that cross a
barrier are consumed only by API lanes; nir_repair_ssa gives the
padding lanes undefined copies. Barriers nested in control flow are
still refused.

Signed-off-by: Diljit Singh <hey@djdev.me>
RyanTheTide and others added 24 commits October 26, 2026 06:52
Classify register-indexed SCS loads alongside the existing per-lane SCS loads in divergence analysis. Materialize the high-byte coverage/stencil mask as an ordinary 16-bit register value before bitwise operations instead of constructing an invalid oversized IR immediate. Keep validation bounds, encoding domains and all depth-export assertions unchanged.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>

Assisted-by: LLM
Keep every original valid/unaligned SSBO check at a usable std140 boundary within the advertised limit. Respect stage storage-block counts, asserting rejection rather than skipping unsupported stages, and add aligned over-limit checks. Mark application ray-query fixtures scratch-capable as the real Vulkan driver and Gen1 fixture do; internal SCS forms retain unspillable keys. Driver limits, spill guards and existing shader assertions are unchanged.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>

Assisted-by: LLM
…solidation

Merge aurora-wip at f2c9dc3 (acb2189 plus #62, 309c3fe, 18
commits) into the consolidated tree da770a4. #62 branched at 28f65a6,
before the consolidation's M5 (G17G/G17S) snapshot made several G17P-only
paths shared with M5, so both sides rewrote the same native AGX3 code.
Each conflict is resolved by chip, keeping both designs: G17P and the other
native AGX3 chips take #62's, M5 keeps the consolidation's. Nothing new is
configurable.

Native AS encoder (agx3_leaf.comp, hk_build_interface.h, bvh/meson.build):
- The two encoders are separate sources selected by the device's chip
  (hk_agx3_as_format_for_chip(), new hk_agx3_as_format.h). bvh/agx3_*.comp
  are the consolidation's M5 encoders, unchanged but for one field name
  (m5_aabb_stride, below) (8-lane node workgroups
  through the batch job map, 48-byte bounding-box records, 1 KiB images).
  bvh/agx3_gen1_*.comp are #62's G17P encoders, unchanged (four node jobs
  per 32-lane workgroup with a GPU-written indirect dispatch, triangle-pair
  records, 64-byte records, 16 KiB images), built as the hk_agx3_gen1_*
  programs hk_acceleration_structure.c already used.
- The consolidation's -DHK_AGX3_GEN1 builds of agx3_*.comp were G17P's
  previous encoder; preprocessed with HK_AGX3_GEN1=1 they match the merge
  base 28f65a6 (only redundant zero stores, reordered exclusive stores and
  a refactored helper differ), which #62 builds on. #62 supersedes them for
  G17P, so they are no longer built. M5 never used them.
- Without the split the auto-merge combined the consolidation's G17P
  shaders (one node per workgroup) with #62's G17P dispatch (four per
  workgroup, indirect counts), and put #62's plan-state tail into M5's
  scratch layout too.
- hk_agx3_as_layout.h derives image alignment and the plan tail from the
  format: GEN1 status follows the 48-byte plan state plus encode dispatch
  arguments (#62), M5 keeps its 32-byte plan state.
- hk_build_interface.h stays shared, holding both sides' additions (#62's
  leaf counts and plan pairs fields, the M5 batch job structures). M5's
  push and batch structures therefore grow by #62's fields, which M5
  leaves zero and its encoders never read; its ABI asserts are updated
  (leaf args 32, plan args 120, batch job 152, first_job at 140). Separate
  per-chip structs of the same name would break hk_cmd_encode_agx3_nodes(),
  which forwards one struct to the M5 backend.
- #62 renamed the hk_agx3_plan_args word that M5 uses for its diagnostic
  bounding-box record stride (reserved) to plan_restored, its G17P update
  flag. Each format keeps its meaning of that word: M5 code names it
  m5_aabb_stride, an alias defined and documented next to the field, so
  neither side's push layout changes.
- Build-time asserts pin each layout (plan tail, record and slot size,
  alignment, encoder workgroup) next to the backend that uses it.
- AS entry points dispatch on hk_agx3_as_format_for_chip(), which maps
  exactly what agx_chip_g17_m5() mapped. Each format's refit, traversal
  (compiler/agx3_rt_gen1_* for GEN1), serialization, relocation, copies and
  property queries (libagx_gen1_as_* vs libagx_agx3_as_*) were already per
  format and are unchanged.
- tests/test-hk-agx3-as-format.c checks for every chip the selected format,
  its scratch and image layout, and the encoder and refit SPIR-V its
  pipelines are created from (workgroup shape per format and variant).

Queue signals (hk_queue.c): the in-ioctl cumulative output
(DRM_ASAHI_SYNC_TIMELINE_SYNCOBJ_CUMULATIVE) is used exactly where #62
enabled it: G17P with DRM_ASAHI_FEATURE_CUMULATIVE_SIGNALS. M5 keeps the
consolidation's signalling unchanged (cumulative transfer after the ioctl
with INDEPENDENT_QUEUES, chunked timelines); the two never combine because
chunked timelines are M5-only.

Occlusion availability (hk_query_pool.c, hk_cmd_buffer.h): native AGX3
pools take #62's 64-bit availability slots in a bound timestamp buffer and
render-carried availability, chosen at pool creation by
agx_native_graphics(); M5 pools keep 32-bit slots, the queued 32-bit store
and lazy counter assignment. At query end M5 transform feedback queries
still publish after their pre-graphics helpers; only native occlusion pools
can take the stamped path. Both sides' state fields are kept.

Compute uniform DMA (hk_cmd_buffer.c, hk_cmd_dispatch.c, hk_shader.c):
#62's shared hk_g17p_dma_ranges() serves graphics on every chip (the
consolidation's loop was the same code) and compute on native AGX3; M5
compute keeps hk_m5_compute_uniform_dma_ranges() appended by its caller.
hk_g17p_compute_uniform_ranges() takes both the chip (M5 root and texture
uniforms) and the command buffer (#62, NULL when prewarming) and counts
spans only off M5, so no chip adds them twice. The compute DMA enable is
#62's term on native AGX3 or the consolidation's on M5. The Gen1 ray-query
range limit becomes HK_G17P_COMPUTE_MAX_RANGES (#62): the consolidation's
8-range limit assumed G17P compute had no spans.

Mechanical: asahi_drm.h keeps the M5 feature bits 17-28 and #62's
CUMULATIVE_SIGNALS (bit 29) and cumulative sync type (2); no value
collides. hk_cmd_draw.c uses #62's two-attempt monolithic compile with
the consolidation's inline-samples layout check inside it (inline samples
are M5-only, uniform exports G17P-only, so they never combine), and the
cached-program check passes own_layout=false.
test-hk-sync-prior.c's link stub from #62 (0b03b01) defines
hk_flush_deferred_clears_why(), the function behind the consolidation's
hk_flush_deferred_clears() macro.

Not resolved by chip because they are chip-agnostic: #62's common Vulkan
runtime update-path changes (leaf.h, vk_acceleration_structure.c,
vk_bvh_*) also apply to M5 builds.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register an exit handler before redirecting stderr so CHECK exits replay the captured loader error and VkResult. Restore and close the saved descriptor on the normal path as well. This makes the missing-driver control directly report VK_ERROR_INCOMPATIBLE_DRIVER instead of producing an empty log.
T6040 is admitted through the G17P profile, so the G16G-only check let it inherit a CTS version before the M4 Pro had completed qualification. Check its physical chip identity as well. API and feature exposure stay unchanged.
The gallium driver advertises HIGH and REALTIME context priorities but
created every context that asked for them as MEDIUM, so a compositor
requesting a high-priority EGL context (Hyprland/aquamarine log "Got a
high priority context") got no priority over its clients.

Ask the kernel for DRM_ASAHI_PRIORITY_HIGH for HIGH and REALTIME
contexts. Kernels grant it only on G17, and only to a caller with
CAP_SYS_NICE or to the DRM master (a render-node client is never the
master, so a compositor needs CAP_SYS_NICE when it creates the
context). On any refusal fall back to MEDIUM, as before.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
Output packing can place float and integer components in one slot. Preserve their bits and component types through GS lowering.

Generated-by: LLM
Signed-off-by: Eryk Wieliczko <eryk@wieliczko.ninja>
Select-guard rewriting could generate #256 for an eight-bit immediate. Restrict candidate constants to valid encodings; equivalent comparisons remain available.

Generated-by: LLM
Signed-off-by: Eryk Wieliczko <eryk@wieliczko.ninja>
Select G13X, G14G and the admitted T6040 G16S identity explicitly instead of falling back to G13G. Model the M4 Pro two-cluster ten-core helper tables from public Mesa #55, with no render ray-query feature. These are compile-only fixtures: the inherited timing limits are not hardware measurements, and shim submissions still execute no GPU work.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep both packed-output checks inside the type-pair loop. Brace the gtest assertion so its macro expansion cannot bind an else to the surrounding immediate check under Werror.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Add a separate g16s-rt fixture with the exact-prior and render-ray-query bits and eight advertised queues. Keep plain g16s as the missing-capability control. This exercises production capability gates without forcing features on a real device.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Reuse the public Gen9 layered/MSAA packet and per-level ZLS work from Mesa PRs #65 and #67 on the admitted T6040 profile. Its AGX3B, HAL200, Gen9 descriptor and sampler axes match the native profile established by #55.

Select expanded array/cube/3D colour, transfer-only 2x/4x colour, layered/MSAA depth-stencil and single-sample depth-stencil mip policies by the physical T6040 identity. Preserve the prior Neo policy and Gen10 packet cases. Colour MSAA render attachments stay plain under #66 because a legal MRT mix can spill them; #72 adds unmeasured boundary-copy work and is not silently enabled.

Keep all source-level RT gates unchanged. Retain existing policy regressions and add the public packet, metadata and per-level ZLS checks alongside a separate G16S policy suite. Hardware and performance qualification remain required before merge.

Source: #65
Source: #67
Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The imported ZLS fixture includes agx_helpers and its generated GLSL builtin type header. Declare the NIR dependency so a clean build supplies that generated include path.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Give the inherited compressed Gen9 layer-mode fixture its required extension and metadata address. The imported G16S policy test must also retain the already-implemented M5 shared-modifier eligibility. Both changes adapt public draft fixtures to the consolidated tree; allocation policies are unchanged.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Reuse #66 allocation-policy coverage across all46 admitted colour formats, sample/layer counts and input/transient attachment combinations. Transfer-only compressed MSAA must not make any later legal colour-attachment spill depend on an unavailable decompression path.

Source: #66
Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The G16S compression port admitted layered and mipmapped D/S images and
standalone S8 to compression. Uploads then bypass the raw-copy path and
reach the graphics-copy gate, which supports D16/D32 only for a single
level/layer and refuses standalone S8. Legal commands fail with
VK_ERROR_FEATURE_NOT_PRESENT at EndCommandBuffer. A CPU-only recording
probe reproduces 13 failures on the port and none on the prior driver.

Keep the pre-port single-level/layer/sample D16/D32/D32S8 allocation
policy and restore its ZLS setup until all expanded D/S access paths are
implemented and hardware-qualified. Preserve the separate G16S colour
compression extension and AS-only construction work. Update the G16S
policy tests to pin the retained D/S boundary across transfer, feedback
and host-copy usages.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The native AS-only device deliberately leaves gen1_rq_instances and
gen1_rq_candidates clear. The shared build admission still required those
traversal flags for AABB BLAS and instance TLAS geometry, rejecting both
command and synchronous host builds with FEATURE_NOT_PRESENT.

Inside the native-AS path, admit AABB and instance geometry based on its
geometry/type contract. The native GEN1 encoders already handle these
record kinds. Keep intersector allocation, query/pipeline feature gates
and all traversal state unchanged.

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Avoid the header allocation, command reallocation and free for small T6040 submits. util_dynarray retains its ordinary heap growth for larger payloads, and all other chip profiles keep their existing allocation path. This changes CPU payload storage only; commands, synchronization and shader code remain unchanged.
@RyanTheTide

Copy link
Copy Markdown
Author

Superseded by #87, merged into aurora-wip in the 2026-10-28 consolidation.

@RyanTheTide
RyanTheTide deleted the wip/m4pro-mesa-tip-20261026 branch October 28, 2026 04:26
astaff pushed a commit to astaff/mesa that referenced this pull request Oct 28, 2026
aa733fc is the Mesa the J616s runs every day. Bring aurora-wip 2a5fd4a
into it: DJ's G17P refit and H-PLOC fixes (aurora-silicon#79, aurora-silicon#81), the deferred
occlusion resolver (aurora-silicon#82, aurora-silicon#83), the zink split-query readback fix (aurora-silicon#84)
and the virtio queue-limits fix (aurora-silicon#86).

The conflicts in the acceleration-structure sources and hk_query_pool.c
are resolved as in the earlier PR aurora-silicon#80 merge 8b7395e. That merge also
carried 2a541c6, which is held back here (see the PR).

Assisted-by: LLM
Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
astaff pushed a commit to astaff/mesa that referenced this pull request Oct 28, 2026
PR aurora-silicon#80 copied agx3_{leaf,envelope,plan}.comp to agx3_gen1_*.comp before
DJ's G17P refit series (aurora-silicon#79) landed, so after the merge the GEN1 build
programs that G17P and T6040 run were the pre-aurora-silicon#79 encoders, while their
refit programs came from the aurora-silicon#79 sources. The pre-aurora-silicon#79 leaf encoder does
not retain the encoding jobs refit reads, stores records through shared
memory, and its envelope clears instance pointers on refit too. The
aurora-silicon#79 agx3_{leaf,envelope,plan}.comp were still compiled, but nothing
used them.

The copies were byte-identical to the base before aurora-silicon#79, and the G17P
plan already matched the aurora-silicon#79 one apart from a comment. Build the
hk_agx3_gen1_* programs from the aurora-silicon#79 sources and drop the copies, so
GEN1 builds and refits come from one set of encoders again. M5 keeps
its agx3_m5_legacy_* programs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants