Repository navigation
asahi: consolidate M4 Pro support on the native/M5 chip split - #80
Closed
RyanTheTide wants to merge 279 commits into
Closed
RyanTheTide wants to merge 279 commits into
RyanTheTide wants to merge 279 commits into
Conversation
The Gen10 compute textureQueryLod lowering reads min/max LOD, mip filter and the signed LOD bias straight out of the descriptor's copy of the Gen5 sampler word. Assert those bit positions against the packer for the existing pinned words (bias 0, 1.5, -2.25, max LOD 14) and the base state (min LOD 1, clamped max 15, linear mip filter, bias -1.015625). Signed-off-by: Diljit Singh <hey@djdev.me>
A static-vertex-input VS containing an M5 native ray query reserves uniform halves 0..167 for query state, and the prolog key and bound uniform ranges place vertex buffer bases at 168. The no-prolog path only applied that offset for Gen1 queries, so M5 query vertex shaders read their attributes from query slots and traced rays from garbage origins. Signed-off-by: Diljit Singh <hey@djdev.me>
M5 refused every mesh shader that writes a per-primitive user output,
gl_Layer or gl_ViewportIndex ("native mesh layout outside captured
triangle contract"), which covered ~600 EXT_mesh_shader, pipeline EDS,
library and shader-object CTS cases.
Give each primitive three private vertices instead: the API body writes
vertex words and per-primitive words to separate shared regions, then
SIMD group 0 copies, between two threadgroup barriers, the referenced
vertex words and the primitive's words to vertices 3p..3p+2 and rewrites
the index list to the identity. Per-primitive words are therefore flat
on every corner, so any provoking vertex selects them; positions,
winding, primitive order and interpolated values are unchanged. Layer
and viewport also fill the packed layer/viewport UVS word and, like the
M5 vertex stage (layer_as_varying), flat user words for fragment reads.
The fragment side reads per-primitive user/layer/viewport inputs as flat
32-bit inputs. Shaders are admitted while 3 * max_primitives <= 256 (the
native output-vertex limit); mesh-written primitive IDs and primitive
culling stay refused. The exported vertex count is reported as the mesh
max_vertices for the native output limits.
This is an emulation, not Apple's per-primitive record path (whose
storage/lifetime contract is still unproven on M5, R175); it costs at
most 3 * max_primitives exported vertices and only applies to shaders
that were refused before. Meshes without per-primitive outputs compile
byte-identically (host drm-shim: every R314 binary of all 6,056 mesh CTS
cases is reproduced).
Hardware (boot 2e8881b2, R239 kernel): a SIMD-group barrier between the
copy and the UVB stores exported stale words (api.draw_count_64 wrong
colours/missing triangles); a copy loop over all threads lost workgroup 2
of builtin.layer deterministically (cause not established); the SIMD 0
copy between threadgroup barriers passes both.
Signed-off-by: Diljit Singh <hey@djdev.me>
M5 vertex-stage draws select the provoking vertex through the VDM vertex program template; native mesh draws have no such template and always used the first vertex, so last-vertex mesh draws read flat inputs from the wrong vertex (pipeline.*.extended_dynamic_state. mesh_shader.*.provoking_vertex_first_to_last_*, mesh_shader.ext.misc. clip_*_provoking_last). For mesh draws, program the HAL300 PPP cull word's flat-shading vertex ((vertex + 1) << 7) instead, and recompute the selector when binding or unbinding a mesh shader changes the topology. Vertex-stage draws keep 0 in the PPP word. On hardware the PPP selector makes the EDS last-vertex mesh draws pass; putting the selector in the VDM mesh-state word 1 (bits 0..1, the vertex-outputs word-1 analogue) had no effect. Signed-off-by: Diljit Singh <hey@djdev.me>
Lower M5 native ray queries in tessellation-evaluation shaders. The hardware evaluation variant becomes a vertex program in poly_nir_lower_tes() after the per-stage query scan, so flag it from the API stage, giving it the vertex query state (uniform halves 0..167, SCS/IFT render program). An evaluation variant run as a kernel before a geometry shader uses the compute query ABI; admit its internal dispatch like the control and geometry kernels. Also stop the M5 indexed-read vertex shift from moving query helper loads: with a native query the vertex ABI starts at half 168 and only loads above it belong to the vertex ABI. Hardware (R320a on the 329 previously refused evaluation cases): 274 Pass, 54 Fail with wrong results, 1 DeviceLost during a recovery shared with concurrent geometry runs. Host: agx_tests 428/428, agx_scs_query adds the evaluation stage. Signed-off-by: Diljit Singh <hey@djdev.me>
…eries opt-in poly_nir_lower_gs clones the (already lowered) geometry shader into the main and count kernels and the raster vertex program. The main kernel has the API stores stripped, so a native query whose results only fed them was dead work that still ran on CDM ahead of the raster draw. agx_nir_m5_remove_dead_ray_queries marks every value data- or control-dependent on a native query operation (SCS loads/stores, enter/intersect/continue) and removes the query only when no remaining side effect depends on it; otherwise only trivial phis are folded. Host compile of 308 geometry ray-query CTS pipelines: no main or count kernel keeps a query (e.g. builtin.flow.geom main 4994 -> 120 bytes); the raster vertex program keeps it. Geometry ray queries nevertheless stay refused by default: hardware runs of GS query draws (before this removal) requested GPU recovery twice and the cause is not isolated. ASAHI_M5_GS_RAY_QUERY_DIAGNOSTIC=1 enables them for diagnosis; ASAHI_M5_RQ_DEAD_DIAG=1 reports a query kept live. Host: agx_tests 428/428; agx_scs_query adds the geometry stage and a dead/live removal regression. Signed-off-by: Diljit Singh <hey@djdev.me>
gl_TessLevelOuter/Inner are compact float arrays. nir_lower_io turns an indirect element index into an I/O offset counted in components, but poly's TCS output and TES input addressing (tcs_out_addr, lower_tes_load) adds the offset to the varying slot. A control shader writing gl_TessLevelOuter[gl_InvocationID] therefore stored levels 1 and 2 into other slots, left them zero and every patch was culled, so the evaluation shader never ran. Convert the levels to vectors (nir_lower_tess_level_array_vars_to_vec) so an indexed access becomes per-component write-masked stores or a vector load and extract, which poly already addresses correctly. This was the cause of the remaining evaluation-shader ray-query failures: hardware, 55 previously non-passing evaluation cases -> 54 Pass, 1 Fail (watertightness.singlehit.tese, which fails on every stage including compute). Replacing the control shader of binding_model...ray_query.regular.tese with one that writes every level from each invocation made the unchanged evaluation shader pass, and a query-free evaluation shader storing a constant wrote nothing before this change. Limited to M5; the poly addressing is shared, so other chips are expected to have the same defect but are untested here. Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_RQ_KEEP_LOOPS=1 marks loops whose body runs a native ray query nir_loop_control_dont_unroll, except loops that index a query array with a non-constant index (unrolling those lets the arrays split into fixed native contexts; rolling them grew rayqueryterminate from 27 KB to 337 KB). Host compile of 36 ray_query.builtin cases (vert/tese/frag): total code 3,257,740 -> 568,988 bytes, largest graphics shader 370 KB -> 7 KB (the evaluation-shader pixel loops), host compile wall time 9.3 s -> 2.0 s; only the 8 evaluation-shader cases change. Hardware A/B over 1,088 common cases of a 1,581-case loop-candidate list: identical statuses except two DeviceLost cases in the default run, which pass with the policy. Both A/B runs stopped at a GPU recovery before completion, so this stays default-off. Signed-off-by: Diljit Singh <hey@djdev.me>
A vertex or compute loop that copies an image texel by texel waited on one read at a time on M5: every image_load was ordered after the previous image_write, and the pressure schedule put each write right after its read. Apple's compiler issues four reads ahead of four writes in the same loop (native725 vs_copy: smp x4, then imgwci x4). - A storage-image read whose NIR access is ACCESS_CAN_REORDER and ACCESS_RESTRICT (and not coherent/volatile) drops its memory-order edges on M5, like texture_load. CAN_REORDER is nir_opt_access's statement that nothing in the shader writes the image (NonWritable or no writes); RESTRICT is vtn's default for an image not decorated Aliased, which the SPIR-V Simple/GLSL/Vulkan memory models allow the consumer to assume does not alias any other memory object declaration (SPIR-V 2.18.2 "Aliasing"; vtn_variables.c). So no store of the shader can write what the read observes. - The occupancy-capped M5 sample-issue schedule now covers blocks that read at least four samples and write images or device memory: every such block of the non-compute geometry-side stages and compute loop bodies. Its window is anchored to the writes already issued, keeping four reads in flight ahead of the write that consumes the oldest. - In those blocks, constants defined outside the block are rematerialized ahead of their first use. The 64-way unrolled 64x64 copy otherwise held 64 coordinate constants live across the whole body (128 of its 150 halves), leaving no registers within its occupancy bucket for reads in flight; the body now peaks at 26 halves. copy-z4 (6-vertex VS copying 4 z-slices of a 64x64x8 R32F image), cold after 3 s idle on M5: 8.48 ms -> 2.42 ms; copy-z2 4.48 -> 1.24 ms; copy-z8 4.79 ms (native Metal under capture: 5.575 ms); alu-r8 unchanged at 3.43 ms. Signed-off-by: Diljit Singh <hey@djdev.me>
9275604f4c3 dropped a reorderable image load's memory-order edges in every dependency graph, including the bottom-up pressure schedule that runs first on all stages. Without those edges the pressure order sank such loads next to their uses: a host replay of the vkQuake-RT frames 80-81 capture on the noop drm-shim compiled 3 of its 133 binaries differently from the base, all compute post-process passes with two or three image loads (5060- and 532/528-byte programs), and in the 532-byte one both SMPs moved below independent arithmetic they used to precede. Only the M5 copy schedule (the write-anchored sample-issue schedule of copy blocks) now drops those edges. The replay compiles all 133 binaries byte-identically to the base again; the 64x64 vertex copy loop still compiles to the hardware-measured binary (9434 bytes, same hash as before this change). Signed-off-by: Diljit Singh <hey@djdev.me>
nir_lower_int64 expands every 64-bit udiv/umod into a fully unrolled 64-step division (about 780 AGX instructions per scalar division whatever the operands). The native G17G compiler instead aligns the divisor with the dividend using leading-zero counts and loops only over the quotient bits that can be set, with a native 64-bit subtract (for `long a / long b`: 124 static instructions including the loop). agx_nir_lower_divmod64 emits that algorithm for 64-bit udiv, umod, idiv, irem and imod on M5, before nir_lower_int64 lowers the remaining 64-bit compares and shifts. The loop runs msb(n) - msb(d) + 1 times when n >= d; division by zero (undefined) returns quotient 0 and remainder n. The raster vertex shader of spirv_assembly.type.vec4.i64.div_geom shrinks from 9405 to 1217 instructions. GPU time on M5 of a vertex loop of 3x50 i64vec4 divisions: signed 2.89 -> 0.25 ms, unsigned 2.77 -> 0.34 ms; at the CTS size (3x390) signed is 2.0-3.3 ms, where the old lowering extrapolates to about 23 ms, past the TA progress check. Results match a CPU reference (0 mismatches). New host test checks all five operations against C on 16x15 operand pairs by constant-folding the unrolled loop. The pass and its host test already exist for G17P, which builds the loop's 64-bit add and subtract from 32-bit halves. This change runs the pass on M5 as well and lets it keep those operations native there; the host test checks both forms. Signed-off-by: Diljit Singh <hey@djdev.me>
On M5 (G17G/G17S) allocate_uvb completes on scoreboard slot 1. Every native write_uvb helper in the captured G17G witnesses waits slot 1 on the instruction right after allocate_uvb (cndst with f4 set, 1e00002e or 9e60002e). Mesa issued the UVSTs without that wait. On hardware this lost the exports of alternate mesh workgroups: of 8 one-triangle workgroups, 2, 4 and 6 drew nothing (1, 3 and 32 threads per workgroup alike; also when a task stage launched 8 children), and dEQP-VK.mesh_shader.ext.builtin.work_group_id_in_mesh failed every run. With the wait, all eight workgroups draw in every run. Retire any older slot-1 generation before the allocation claims the slot, then wait for the allocation. Only AGX3C changes. Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_TASK=1 (with ASAHI_MESH=native-probe, G17S only) advertises taskShader and runs the API task shader as the native object program of the mesh draw, in place of the mesh-only passthrough object: - agx_nir_lower_m5_task publishes the EmitMeshTasks grid with UVSW from lanes 0..2 of SIMD group 0. A payload-free task takes the grid of invocation 0 from registers, as the native passthrough object does. Payload-carrying tasks use the native threadgroup grid, rendezvous and payload-exit barrier; workgroups under three invocations are padded to three lanes with the API body masked, as for mesh workgroups. - Task payloads keep the native SR62/SR63 payload address. A read-only probe (ASAHI_MESH_PP_PROBE=payload, added here) reads SR62=SR63=0 on the current boot, whose provider loads asahi with m5_mesh_storage_chunks=0 (no 0x1ca48 mesh rendering buffer). Payload accesses are therefore refused unless ASAHI_M5_TASK_PAYLOAD gives a byte limit. - hk_draw_mesh_native sizes the object dispatch from the API task grid (direct draws only; indirect, draw-count and conditional task draws are refused), publishes the task's draw root (DrawID, NumWorkGroups) and gives the kernel the task's thread count and threadgroup memory. - Task shaders read gl_NumWorkGroups and gl_DrawID from the draw root, like mesh shaders. Known gap: a mesh shader under a task stage still reads the API task grid as gl_NumWorkGroups. Hardware (boot c73879b2, R239 kernel): payload-free tasks of 1 and 32 invocations launch data-dependent child grids correctly (per-pixel draw counts 4 3 2 1 for EmitMeshTasks(id+1) over 4 tasks). The first attempt, which used the threadgroup/rendezvous shape and tail[0]=1 for a payload-free task, caused one GPU recovery; tail[0] stays at the mesh-only value. Default behaviour (no ASAHI_M5_TASK) is unchanged. Signed-off-by: Diljit Singh <hey@djdev.me>
Native G17G point and line mesh witnesses (lane-mesh/cwit-meshtopo mpt, mptc, mln) use the triangle layout with one and two packed index bytes per primitive in the 16-byte-aligned index region, and points carry their size in the plane right after the four position planes (plane 4, before any user plane). agx_nir_lower_m5_mesh now admits points and lines: index bytes, the exported index words and the index-region size scale with the primitive's vertex count, and a point mesh exports PSIZ at plane 4. Only the size-only point interface (positions + size) is admitted, since the generic UVS order places PSIZ after user varyings; per-primitive outputs (private vertices) stay triangle-only. As on G17P, a point mesh that does not write gl_PointSize gets the default size 1.0. These were refused at pipeline creation before. Hardware (boot cc58080f, R328 kernel, 7283-case mesh/non-mesh set, jobs 4): 271 more Pass than the previous commit (256 shader_module_identifier pipeline_from_id point meshes, single_point/single_line, point_size, many_mesh_work_groups_*, misc.state.*), 0 regressions, 0 GPU faults. depth_only_points* (8) and two line query cases still fail at runtime. Signed-off-by: Diljit Singh <hey@djdev.me>
SPIR-V EmitMeshTasksEXT terminates the invocation, so vtn follows the launch with a halt. With the launch inside an if/else both branches halted before the appended child-count epilogue, which then published undefined counts. A task shader has no other halt source: turn the halts into returns and lower them, so the launch ends main. Hardware (opt-in ASAHI_M5_TASK=1, boot cc58080f): emit_in_control_flow and emit_in_control_flow_bad_emit_last now pass; the 1296-case task set gives 10 Pass, 0 GPU faults (the rest refuse payloads, indirect or conditional task draws, or are NotSupported). Signed-off-by: Diljit Singh <hey@djdev.me>
vkCreateRayTracingPipelinesKHR read each group's pShaderGroupCaptureReplayHandle unconditionally. Replaying the vkQuake-RT frames 80-81 capture with gfxreconstruct segfaulted there with every library since the feature was exposed: the capture holds a 0-byte SetRayTracingShaderGroupHandles block for pipeline 458 (its capture-time device did not support handle capture/replay), and gfxreconstruct, which enables the feature whenever the device has it, then passes NULL + 4 * group as each group's handle pointer (0x4 for group 1; the fault was ldr w24, [x0] with x0 = 4). A handle pointer inside the zero page cannot address application memory, and handle 0 is never issued (handles start at 1), so fail such groups with VK_ERROR_INVALID_OPAQUE_CAPTURE_ADDRESS, the code for a replay handle the implementation cannot reuse, rather than dereferencing them. Signed-off-by: Diljit Singh <hey@djdev.me>
vkAllocateMemory rounded allocationSize up to the 16 KiB page and passed the result to agx_bo_create() unchecked. A 0-byte request (and any size whose rounding wraps to 0) became a 0-byte BO request that only failed in the kernel (GEM_CREATE: ENOSPC, "BO creation failed: size=0"). Such requests come from cooperative_matrix.op_constant_null in the deqp build we run: its iterator does not stop when the remaining device configuration (FP32 x FP32 -> FP32, 8x8x8) is absent from its own static list, which still excludes FP32 A/B operands, so the instance runs a zero-initialized configuration and sizes its buffers 0. An allocation that imports nothing must have allocationSize > 0, and no size beyond the memory heap can ever be satisfied, so return VK_ERROR_OUT_OF_DEVICE_MEMORY for both before touching the BO layer. Imports keep their existing path. Signed-off-by: Diljit Singh <hey@djdev.me>
A fragment shader that runs a ray query acquires the colour write class (pixwait 4) ahead of its first traversal, so the linker turns its epilog's read+write wait (pixwait 12) into a read wait (pixwait 8). It built both the expected and the replacement instruction in the AGX3B encoding only, so on M5 the AGX3C epilog never matched and every such link failed (36 ray_query.helper_invocations.gpu cases ended in vkEndCommandBuffer VK_ERROR_FEATURE_NOT_PRESENT). Native M5 colour fragments with a ray query (the native G17G compiler, intersection_query and intersector forms) show exactly this pair: pixwait 4 immediately before intersect_ray, pixwait 8 before lstcol, then pixrel 524. Encode the rewrite in the device's ISA. Signed-off-by: Diljit Singh <hey@djdev.me>
ASAHI_M5_RQ_WATERTIGHT=1 enables it; default binaries are unchanged. M5's triangle test is inclusive: when its edge function is exactly zero it reports every triangle at that edge or vertex, each with a zero barycentric there. Hardware probes (exact dyadic fixtures, 16384 generic near-edge rays and the closedFan geometry) show near-edge rays otherwise get exactly one triangle, but exact-edge rays get both. That double count fails ray_query.watertightness.singlehit and ray_tracing_pipeline.watertightness.closedFan/closedFan2. The hardware's arithmetic is not exact (closedFan spoke rays get both neighbours although exact arithmetic puts them strictly inside one), so a shader cannot predict which candidates it reports. Apple's runtime query.next does not filter them either. When enabled, remember the distance of the last boundary triangle (some barycentric within 2^-22 of zero, allowing for the hardware's ~1 ulp barycentric rounding) a query let through, and drop a later boundary triangle at the same distance: equal or adjacent 256-ulp distance keys. Partners compute t from their own planes; on the CTS random-height mesh they differ by more than 31 ulps at some diagonal pixels. Both are the same point on the shared edge or vertex. A drop only follows its delivered partner, so it cannot turn a hit into a miss. The key lives in the candidate cursor's spare bits; no memory access. Hardware (M5 Pro, R328): 34 singlehit/nomiss (compute, fragment, RT stages) and closedFan/closedFan2 cases: 19 Pass / 15 Fail disabled, 34/34 Pass enabled. Vertex/tess/fragment singlehit and nomiss at one worker, enabled: 11/11 Pass (geometry-stage cases fail pipeline creation either way). vkQuake RT frames 80-81: images byte-identical to R337; replay time within noise (15 interleaved pairs, median +0.04 ms/frame). Cost when enabled, drm-shim shaderdb census of 1743 ray-query/RT shaders against the default: +7.42% code bytes, median +35 instructions, median GPR change 0; 30 shaders lose occupancy (28 at 1024 -> 896 threads) and 6 gain. Disabled: all 1743 shaders identical. Signed-off-by: Diljit Singh <hey@djdev.me>
On M5 a vertex-stage draw picks its provoking vertex through the VDM
vertex program template, and hk left the PPP flat-shading vertex
at 0 for everything but mesh draws.
pipeline.*.extended_dynamic_state.*provoking_vertex_first_to_last* draw
oversized triangles with the last-vertex convention and shaded them from
the wrong vertex in every construction type (monolithic,
pipeline_library, fast_linked_library, shader_object_unlinked_spirv);
rasterization.provoking_vertex, whose triangles stay inside the viewport,
passed. Presumably clipping takes the flat inputs of the vertices it
generates from the PPP selector; that mechanism is inferred, not
measured.
Program the PPP selector from the same provoking state for all draws.
Hardware (M5 Pro, R328), diagnostic switch on the R337 base: the 20
two_draws_{dynamic,static} provoking cases across the four construction
types went from 12 Fail / 8 Pass to 20/20 Pass, and all 63
rasterization.provoking_vertex cases pass.
Signed-off-by: Diljit Singh <hey@djdev.me>
Fragment shaders that need the raster sample count (interpolateAtSample's
sample clamp, sample-position lowering) read samples_log2. Neither G17P
nor M5 has a grounded AGX3 encoding for that special register, so the
packer refuses it. hk folds it to the keyed count on G17P, but M5 keys the
count as m5_position_samples and was never specialized: every such
fragment shader failed to compile on M5. Pipelines creating it returned
VK_ERROR_UNKNOWN, and per-sample-count variants built at draw time ended
their command buffer with VK_ERROR_UNKNOWN
(pipeline.fast_linked_library.misc.frag_lib_varying_samples_{2,4,8},
interpolate_at_sample_no_sample_shading).
Specialize with the native sample count on both chips; M5 counts reach 8.
drm-shim: no remaining samples_log2 refusals; 15268 multisample/misc
shaders unchanged (5244 sampled cases).
Signed-off-by: Diljit Singh <hey@djdev.me>
…fault Vulkan requires a ray that lands exactly on an edge shared by two triangles to report a single hit; M5's inclusive triangle test reports both. The filter fixes the 15 watertightness CTS failures (singlehit in every stage, closedFan/closedFan2). Measured cost: +7.4% code in ray-query/RT shaders, 30 of 1,743 shaders drop one occupancy step, no measurable change in the vkQuake-RT replay (median +0.04 ms on ~3.1 ms per frame, byte-identical images). Enabled by default at the user's direction; ASAHI_M5_RQ_WATERTIGHT=0 turns it off for diagnostics. Signed-off-by: Diljit Singh <hey@djdev.me>
Geometry ray queries were refused unless ASAHI_M5_GS_RAY_QUERY_DIAGNOSTIC=1,
after GS query draws requested GPU recovery twice. Those runs predate the
removal of dead native queries from the poly main/count kernels.
With the R338 stack and the queries enabled, all 578 geometry ray-query
cases of the mustpass (acceleration_structures, ray_flags,
traversal_control, builtin, watertightness, stress,
binding_model.*acceleration_structure*.geom) ran on M5 hardware with one
worker in 26 s: 299 Pass, 249 NotSupported, 30 Fail, no GPU recovery and
no long-running case. The 30 failures are
binding_model.mutable_descriptor.single.acceleration_structure_khr.*.geom
("expected 2, found 1"): the query's candidate-found result does not reach
the atomicAdd in the main GS kernel (open).
ASAHI_M5_GS_RAY_QUERY_DISABLE=1 refuses them for diagnosis. Concurrency
behaviour beyond one worker is not qualified here.
Signed-off-by: Diljit Singh <hey@djdev.me>
With independent kernel queues, advertise two more queue families next to the universal one: compute+transfer and transfer-only, each with the same queue count. Kernel queues carry no capability class (every queue accepts render and compute commands) and hk records copies, clears and blits the same way on any family, so these families only narrow what the application may record. They reuse the universal queue implementation: one kernel queue per Vulkan queue, created on first submission in the device's single VM; no queue or VM is created for an unused queue. This makes the CTS's exclusive compute/transfer queues, concurrent sharing across families and cross-family queue pairs available. A sampled hardware run (M5 Pro, R328 + R338 Mesa, 2 workers) of 1146 cases that were NotSupported for queue reasons: 857 Pass, 282 NotSupported (118 still lack a queue pair: they need two graphics families; the rest for unrelated reasons), 7 Fail, all newly exposed compute variants (5 "M5 compute launch exceeds the current resource profile", 2 sampler min_reduce compute image mismatches). A regression sample of 863 previously passing synchronization/api/compute cases: 863 Pass. No GPU recovery. Signed-off-by: Diljit Singh <hey@djdev.me>
Native G17G object/mesh pairs (lane-mesh nwit2 otpg/mtpg/mtpg0) address one record through the payload address SR62/SR63: the object stores its child grid (threadgroups_per_grid) at +0 and the payload from +16, and the mesh reads its threadgroups_per_grid from +0 after the payload-entry barrier. The record lives in the TA mesh rendering buffer (register 0x1ca48), which the kernel only binds to task kicks. With ASAHI_M5_TASK=1 and a kernel reporting DRM_ASAHI_FEATURE_TASK_MESH_STORAGE (new uapi bit 13, with DRM_ASAHI_RENDER_PROGRAM_TASK, bit 3): - task shaders use the native threadgroup grid, rendezvous and payload-exit barrier, store the child grid at +0 of the record and accept payloads up to 16 KiB at +16; - a mesh shader compiled for a task stage (no VK_SHADER_CREATE_NO_TASK_SHADER_BIT_EXT) reads gl_NumWorkGroups from the record instead of the API grid, and payloads from +16; - task kicks carry DRM_ASAHI_RENDER_PROGRAM_TASK and set VDM mesh tail[0], as native object->mesh kicks do. Without the feature only payload-free task shaders are accepted, as before, and a task-stage mesh shader that reads gl_NumWorkGroups is refused instead of silently reading the API grid. The ASAHI_M5_TASK_PAYLOAD override is gone. drm-shim gains ASAHI_SHIM_CHIP=g17s-task for host compile tests of the record path. Not hardware-tested: needs a kernel with the task-only 0x1ca48 binding and m5_mesh_storage_chunks != 0. Signed-off-by: Diljit Singh <hey@djdev.me>
Mesh workgroups smaller than their export batch are padded and the API body runs under the original invocation mask. The payload-entry barrier inserted at the start of a payload- or record-reading mesh shader ended up inside that mask. Native mesh main functions pass it with every lane before the thread-index branch (lane-mesh nwit mpay, nwit2 mtpg); hoist it out of the masked body the same way. Only mesh shaders that read a task payload or the task record change. Signed-off-by: Diljit Singh <hey@djdev.me>
hk_compile_shaders lowers every shader's I/O and system values (including gl_NumWorkGroups to the draw root) before hk_compile_shader_impl, so the task-record substitution placed in the impl never saw pipeline shaders: a mesh shader under a task stage still read the API grid (hardware probe on R328: per-pixel counts 0 0 0 10 instead of 1 2 3 4). Do the substitution, or the refusal without the record, in both lowering paths. Signed-off-by: Diljit Singh <hey@djdev.me>
Indirect, indirect-count and conditionally rendered task draws were refused because the object dispatch was sized from a CPU grid. Record them with one object workgroup and dispatch libagx_m5_task_dispatch in the pre-graphics compute stream: it patches the object thread counts of the VDM mesh dispatch record from the GPU grid and writes the draw's execute word (grid nonzero, draw index below the count, predicate passes). The task shader checks that word, through the new root field mesh_task_flag, on entry and launches no mesh children for a disabled draw; the word is zero for every other draw. The kernel is admitted for the M5 (g17g) libagx target only, and the g17s-task drm-shim chip now also reports COMPUTE_RESOURCES, as the kernel does. Hardware (R328, boot 410b5227, ASAHI_M5_TASK=1, no task record): all 360 payload-free conditional_rendering task cases (draw, draw_indirect, draw_indirect_count) Pass, 0 GPU faults. Indirect task cases with payloads need the task-record kernel. Signed-off-by: Diljit Singh <hey@djdev.me>
An M5 task (object) shader with an image or texture read reached the AGX3 packer with the 32-bit bindless form (uniform heap base plus a register index), which has no grounded encoding, so every task shader reading an image was refused. Treat the task stage like the mesh stage: lower its reads to the heap-indexed descriptor-address form, stage image-load coordinates as for mesh and kernel programs, and bind the indexed heap pointers for a task program. drm-shim (g17s-task, ASAHI_M5_TASK=1): the 10 synchronization.*_to_task image cases now compile; their hardware run needs the task-record kernel. Signed-off-by: Diljit Singh <hey@djdev.me>
A mesh workgroup padded to its export width, or a task workgroup of fewer than three invocations, runs its API body under the exact invocation mask. A user execution barrier in that body was refused, since moving it under the mask changes the set of invocations that reaches it. Split the body at top-level execution barriers instead: each segment between barriers runs under the mask and the barriers stay outside it, so every physical lane passes them, as Apple passes barriers with all lanes enabled (cndst 0,6,0). Values that cross a barrier are consumed only by API lanes; nir_repair_ssa gives the padding lanes undefined copies. Barriers nested in control flow are still refused. Signed-off-by: Diljit Singh <hey@djdev.me>
Classify register-indexed SCS loads alongside the existing per-lane SCS loads in divergence analysis. Materialize the high-byte coverage/stencil mask as an ordinary 16-bit register value before bitwise operations instead of constructing an invalid oversized IR immediate. Keep validation bounds, encoding domains and all depth-export assertions unchanged. Signed-off-by: Ryan Murray <ryan@aurorasilicon.org> Assisted-by: LLM
Keep every original valid/unaligned SSBO check at a usable std140 boundary within the advertised limit. Respect stage storage-block counts, asserting rejection rather than skipping unsupported stages, and add aligned over-limit checks. Mark application ray-query fixtures scratch-capable as the real Vulkan driver and Gen1 fixture do; internal SCS forms retain unspillable keys. Driver limits, spill guards and existing shader assertions are unchanged. Signed-off-by: Ryan Murray <ryan@aurorasilicon.org> Assisted-by: LLM
…solidation Merge aurora-wip at f2c9dc3 (acb2189 plus #62, 309c3fe, 18 commits) into the consolidated tree da770a4. #62 branched at 28f65a6, before the consolidation's M5 (G17G/G17S) snapshot made several G17P-only paths shared with M5, so both sides rewrote the same native AGX3 code. Each conflict is resolved by chip, keeping both designs: G17P and the other native AGX3 chips take #62's, M5 keeps the consolidation's. Nothing new is configurable. Native AS encoder (agx3_leaf.comp, hk_build_interface.h, bvh/meson.build): - The two encoders are separate sources selected by the device's chip (hk_agx3_as_format_for_chip(), new hk_agx3_as_format.h). bvh/agx3_*.comp are the consolidation's M5 encoders, unchanged but for one field name (m5_aabb_stride, below) (8-lane node workgroups through the batch job map, 48-byte bounding-box records, 1 KiB images). bvh/agx3_gen1_*.comp are #62's G17P encoders, unchanged (four node jobs per 32-lane workgroup with a GPU-written indirect dispatch, triangle-pair records, 64-byte records, 16 KiB images), built as the hk_agx3_gen1_* programs hk_acceleration_structure.c already used. - The consolidation's -DHK_AGX3_GEN1 builds of agx3_*.comp were G17P's previous encoder; preprocessed with HK_AGX3_GEN1=1 they match the merge base 28f65a6 (only redundant zero stores, reordered exclusive stores and a refactored helper differ), which #62 builds on. #62 supersedes them for G17P, so they are no longer built. M5 never used them. - Without the split the auto-merge combined the consolidation's G17P shaders (one node per workgroup) with #62's G17P dispatch (four per workgroup, indirect counts), and put #62's plan-state tail into M5's scratch layout too. - hk_agx3_as_layout.h derives image alignment and the plan tail from the format: GEN1 status follows the 48-byte plan state plus encode dispatch arguments (#62), M5 keeps its 32-byte plan state. - hk_build_interface.h stays shared, holding both sides' additions (#62's leaf counts and plan pairs fields, the M5 batch job structures). M5's push and batch structures therefore grow by #62's fields, which M5 leaves zero and its encoders never read; its ABI asserts are updated (leaf args 32, plan args 120, batch job 152, first_job at 140). Separate per-chip structs of the same name would break hk_cmd_encode_agx3_nodes(), which forwards one struct to the M5 backend. - #62 renamed the hk_agx3_plan_args word that M5 uses for its diagnostic bounding-box record stride (reserved) to plan_restored, its G17P update flag. Each format keeps its meaning of that word: M5 code names it m5_aabb_stride, an alias defined and documented next to the field, so neither side's push layout changes. - Build-time asserts pin each layout (plan tail, record and slot size, alignment, encoder workgroup) next to the backend that uses it. - AS entry points dispatch on hk_agx3_as_format_for_chip(), which maps exactly what agx_chip_g17_m5() mapped. Each format's refit, traversal (compiler/agx3_rt_gen1_* for GEN1), serialization, relocation, copies and property queries (libagx_gen1_as_* vs libagx_agx3_as_*) were already per format and are unchanged. - tests/test-hk-agx3-as-format.c checks for every chip the selected format, its scratch and image layout, and the encoder and refit SPIR-V its pipelines are created from (workgroup shape per format and variant). Queue signals (hk_queue.c): the in-ioctl cumulative output (DRM_ASAHI_SYNC_TIMELINE_SYNCOBJ_CUMULATIVE) is used exactly where #62 enabled it: G17P with DRM_ASAHI_FEATURE_CUMULATIVE_SIGNALS. M5 keeps the consolidation's signalling unchanged (cumulative transfer after the ioctl with INDEPENDENT_QUEUES, chunked timelines); the two never combine because chunked timelines are M5-only. Occlusion availability (hk_query_pool.c, hk_cmd_buffer.h): native AGX3 pools take #62's 64-bit availability slots in a bound timestamp buffer and render-carried availability, chosen at pool creation by agx_native_graphics(); M5 pools keep 32-bit slots, the queued 32-bit store and lazy counter assignment. At query end M5 transform feedback queries still publish after their pre-graphics helpers; only native occlusion pools can take the stamped path. Both sides' state fields are kept. Compute uniform DMA (hk_cmd_buffer.c, hk_cmd_dispatch.c, hk_shader.c): #62's shared hk_g17p_dma_ranges() serves graphics on every chip (the consolidation's loop was the same code) and compute on native AGX3; M5 compute keeps hk_m5_compute_uniform_dma_ranges() appended by its caller. hk_g17p_compute_uniform_ranges() takes both the chip (M5 root and texture uniforms) and the command buffer (#62, NULL when prewarming) and counts spans only off M5, so no chip adds them twice. The compute DMA enable is #62's term on native AGX3 or the consolidation's on M5. The Gen1 ray-query range limit becomes HK_G17P_COMPUTE_MAX_RANGES (#62): the consolidation's 8-range limit assumed G17P compute had no spans. Mechanical: asahi_drm.h keeps the M5 feature bits 17-28 and #62's CUMULATIVE_SIGNALS (bit 29) and cumulative sync type (2); no value collides. hk_cmd_draw.c uses #62's two-attempt monolithic compile with the consolidation's inline-samples layout check inside it (inline samples are M5-only, uniform exports G17P-only, so they never combine), and the cached-program check passes own_layout=false. test-hk-sync-prior.c's link stub from #62 (0b03b01) defines hk_flush_deferred_clears_why(), the function behind the consolidation's hk_flush_deferred_clears() macro. Not resolved by chip because they are chip-agnostic: #62's common Vulkan runtime update-path changes (leaf.h, vk_acceleration_structure.c, vk_bvh_*) also apply to M5 builds. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Register an exit handler before redirecting stderr so CHECK exits replay the captured loader error and VkResult. Restore and close the saved descriptor on the normal path as well. This makes the missing-driver control directly report VK_ERROR_INCOMPATIBLE_DRIVER instead of producing an empty log.
T6040 is admitted through the G17P profile, so the G16G-only check let it inherit a CTS version before the M4 Pro had completed qualification. Check its physical chip identity as well. API and feature exposure stay unchanged.
The gallium driver advertises HIGH and REALTIME context priorities but created every context that asked for them as MEDIUM, so a compositor requesting a high-priority EGL context (Hyprland/aquamarine log "Got a high priority context") got no priority over its clients. Ask the kernel for DRM_ASAHI_PRIORITY_HIGH for HIGH and REALTIME contexts. Kernels grant it only on G17, and only to a caller with CAP_SYS_NICE or to the DRM master (a render-node client is never the master, so a compositor needs CAP_SYS_NICE when it creates the context). On any refusal fall back to MEDIUM, as before. Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com> Assisted-by: LLM
Output packing can place float and integer components in one slot. Preserve their bits and component types through GS lowering. Generated-by: LLM Signed-off-by: Eryk Wieliczko <eryk@wieliczko.ninja>
Select-guard rewriting could generate #256 for an eight-bit immediate. Restrict candidate constants to valid encodings; equivalent comparisons remain available. Generated-by: LLM Signed-off-by: Eryk Wieliczko <eryk@wieliczko.ninja>
Select G13X, G14G and the admitted T6040 G16S identity explicitly instead of falling back to G13G. Model the M4 Pro two-cluster ten-core helper tables from public Mesa #55, with no render ray-query feature. These are compile-only fixtures: the inherited timing limits are not hardware measurements, and shim submissions still execute no GPU work. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Keep both packed-output checks inside the type-pair loop. Brace the gtest assertion so its macro expansion cannot bind an else to the surrounding immediate check under Werror. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Add a separate g16s-rt fixture with the exact-prior and render-ray-query bits and eight advertised queues. Keep plain g16s as the missing-capability control. This exercises production capability gates without forcing features on a real device. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Reuse the public Gen9 layered/MSAA packet and per-level ZLS work from Mesa PRs #65 and #67 on the admitted T6040 profile. Its AGX3B, HAL200, Gen9 descriptor and sampler axes match the native profile established by #55. Select expanded array/cube/3D colour, transfer-only 2x/4x colour, layered/MSAA depth-stencil and single-sample depth-stencil mip policies by the physical T6040 identity. Preserve the prior Neo policy and Gen10 packet cases. Colour MSAA render attachments stay plain under #66 because a legal MRT mix can spill them; #72 adds unmeasured boundary-copy work and is not silently enabled. Keep all source-level RT gates unchanged. Retain existing policy regressions and add the public packet, metadata and per-level ZLS checks alongside a separate G16S policy suite. Hardware and performance qualification remain required before merge. Source: #65 Source: #67 Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The imported ZLS fixture includes agx_helpers and its generated GLSL builtin type header. Declare the NIR dependency so a clean build supplies that generated include path. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Give the inherited compressed Gen9 layer-mode fixture its required extension and metadata address. The imported G16S policy test must also retain the already-implemented M5 shared-modifier eligibility. Both changes adapt public draft fixtures to the consolidated tree; allocation policies are unchanged. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Reuse #66 allocation-policy coverage across all46 admitted colour formats, sample/layer counts and input/transient attachment combinations. Transfer-only compressed MSAA must not make any later legal colour-attachment spill depend on an unavailable decompression path. Source: #66 Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The G16S compression port admitted layered and mipmapped D/S images and standalone S8 to compression. Uploads then bypass the raw-copy path and reach the graphics-copy gate, which supports D16/D32 only for a single level/layer and refuses standalone S8. Legal commands fail with VK_ERROR_FEATURE_NOT_PRESENT at EndCommandBuffer. A CPU-only recording probe reproduces 13 failures on the port and none on the prior driver. Keep the pre-port single-level/layer/sample D16/D32/D32S8 allocation policy and restore its ZLS setup until all expanded D/S access paths are implemented and hardware-qualified. Preserve the separate G16S colour compression extension and AS-only construction work. Update the G16S policy tests to pin the retained D/S boundary across transfer, feedback and host-copy usages. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
The native AS-only device deliberately leaves gen1_rq_instances and gen1_rq_candidates clear. The shared build admission still required those traversal flags for AABB BLAS and instance TLAS geometry, rejecting both command and synchronous host builds with FEATURE_NOT_PRESENT. Inside the native-AS path, admit AABB and instance geometry based on its geometry/type contract. The native GEN1 encoders already handle these record kinds. Keep intersector allocation, query/pipeline feature gates and all traversal state unchanged. Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Avoid the header allocation, command reallocation and free for small T6040 submits. util_dynarray retains its ordinary heap growth for larger payloads, and all other chip profiles keep their existing allocation path. This changes CPU payload storage only; commands, synchronization and shader code remain unchanged.
Author
|
Superseded by #87, merged into aurora-wip in the 2026-10-28 consolidation. |
astaff
pushed a commit
to astaff/mesa
that referenced
this pull request
Oct 28, 2026
aa733fc is the Mesa the J616s runs every day. Bring aurora-wip 2a5fd4a into it: DJ's G17P refit and H-PLOC fixes (aurora-silicon#79, aurora-silicon#81), the deferred occlusion resolver (aurora-silicon#82, aurora-silicon#83), the zink split-query readback fix (aurora-silicon#84) and the virtio queue-limits fix (aurora-silicon#86). The conflicts in the acceleration-structure sources and hk_query_pool.c are resolved as in the earlier PR aurora-silicon#80 merge 8b7395e. That merge also carried 2a541c6, which is held back here (see the PR). Assisted-by: LLM Signed-off-by: Ryan Murray <ryan@aurorasilicon.org> Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
astaff
pushed a commit
to astaff/mesa
that referenced
this pull request
Oct 28, 2026
PR aurora-silicon#80 copied agx3_{leaf,envelope,plan}.comp to agx3_gen1_*.comp before DJ's G17P refit series (aurora-silicon#79) landed, so after the merge the GEN1 build programs that G17P and T6040 run were the pre-aurora-silicon#79 encoders, while their refit programs came from the aurora-silicon#79 sources. The pre-aurora-silicon#79 leaf encoder does not retain the encoding jobs refit reads, stores records through shared memory, and its envelope clears instance pointers on refit too. The aurora-silicon#79 agx3_{leaf,envelope,plan}.comp were still compiled, but nothing used them. The copies were byte-identical to the base before aurora-silicon#79, and the G17P plan already matched the aurora-silicon#79 one apart from a comment. Build the hk_agx3_gen1_* programs from the aurora-silicon#79 sources and drop the copies, so GEN1 builds and refits come from one set of encoders again. M5 keeps its agx3_m5_legacy_* programs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidate M4 Pro (T6040/G16S B1) bring-up on the Neo tip plus the existing native/M5 chip split. Retain physical chip identity, Borealis branding, zero conformance version pending qualification, GL HIGH-priority fallback, and packed-GS/immediate correctness fixes.
Keep bounded HAL200/Gen9 layered and 2x/4x colour compression selected only for T6040, including the colour-MSAA attachment spill restriction. Keep depth/stencil on the pre-port single-level/layer/sample D16/D32/D32S8 policy and its original ZLS setup. The expanded depth/stencil policy bypassed raw uploads and reached a graphics-copy gate that rejects compressed mips/arrays and standalone S8, causing VK_ERROR_FEATURE_NOT_PRESENT at EndCommandBuffer. A CPU-only probe reproduces 13 recording failures on the expanded policy and none on the prior driver; the depth-policy correction passes all 32 recording cases on G13X/G14G/G17P/G16S fixtures. Broader depth/stencil compression remains deferred until all access paths are implemented and qualified.
Separate native GEN1 AS construction from traversal on exact T6040. AS build/copy/refit uses ordinary compute with exact-prior and bounded-AS-build kernel capabilities (bits12+16); AS-only devices do not initialize intersectors or ray stacks. Query/pipeline traversal keeps its existing bits12+15 gate. Native AABB/instance construction also no longer depends on the traversal-state booleans deliberately absent on AS-only devices; both command and synchronous host builders share that admission. Intersectors and traversal gates are unchanged. No unsupported capability is forced.
Prior source validation: ARM64 and Fedora release WERROR builds, 110 CPU tests, GEN1 repeated 12/12, 611 RT-core checks, 450 unchanged serialized helper kernels, 36 CPU-only contract fixtures, and identical statistics/instructions for seven gpubench pipelines on M1 Pro, M2, Neo and M4 Pro compile targets. Both corrections build with WERROR and pass the 110 CPU tests. The depth correction also passes 12 shader comparisons, 36 contracts and AS API checks; the combined correction also passes 12 shader comparisons, 36 contracts and AS API checks. Triangle/AABB/instance command recording passes with and without host commands enabled; the prior candidate reproduces the AABB/instance rejections. Generated helper shader source is byte-identical. The AS-capable benchmark has a matching O3/NDEBUG WERROR Release build and retains query support skips. CPU probes execute no GPU work.
Queue submissions on exact T6040 start their ioctl payload on 1024 bytes of stack storage, with the existing dynarray heap fallback for larger payloads. Other chip identities retain their allocation path. This changes CPU allocation only; no command, synchronization or shader change is intended. A 32-process CPU-only real-ICD oracle matches payloads, sync descriptors and mapping records against the parent across small and heap-fallback submissions, timestamps, depths and synthetic cumulative-sync settings. Scoped untimed single-clear checks remove two reallocations and one free per submit. The matched Fedora release and ARM64 WERROR builds, 110 CPU tests, 611 RT-core checks, 36 contract fixtures and 12 shader/instruction comparisons pass for the new head. Hardware correctness and performance qualification remain required; no speedup is claimed.
Draft gates: follow the final landed t-3165 split; hardware-qualify the exact replacement overlay and review CTS/paired gpubench including queue rows. Establish the valid G16S firmware traversal/recovery contract before query/pipeline execution and RT performance comparisons. The shared GS correction intentionally fixes previously uncompileable mixed-type stores; byte identity applies to the recorded G16-only delta/sample, not the entire inherited consolidation. No merge readiness or measured performance gain is claimed.