Skip to content

drm/asahi: g16: Keep compute commands in flight; one 64 MiB PB heap per VM (J773g) - #2419

Open
DjDeveloperr wants to merge 7 commits into
integration/aurorafrom
djdev/g16-in-flight
Open

DjDeveloperr wants to merge 7 commits into
integration/aurorafrom
djdev/g16-in-flight

Conversation

@DjDeveloperr

@DjDeveloperr DjDeveloperr commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

On the J773g Mac mini (M4, G16G B0, 25G83 firmware), G16 had two structural gaps against macOS 27.2 on the same machine. It published one GPU command at a time and built the next only after the previous one retired on the host, so a 1.5 µs dispatch cost ~270 µs per submission at any depth. And every rendering process pinned ~0.5 GiB: two 256 MiB parameter-buffer heaps plus its arenas.

Stacking: the first commit is #1543 (@aeries0x, PB epoch reuse), included because commits 2-6 build on its epoch handling. Merge #1543 first; the six commits after it are this PR. Hardware results are with #2239 (J773g S10) as well, as on the mini.

Commits

  1. Keep compute commands in flight. Compute commands queue behind the same VM's in-flight compute on the one firmware compute queue; renders stay one at a time.
    • Each queued command has its own retained unit and its own UMA slot (1 + unit). With every unit in slot 1 (the old two-owner design), queued long compute triggered firmware recovery within seconds: the firmware sets up the next command while the current one runs.
    • The owner ledger generalizes to N owners that retire in publication order.
    • Each command posts its own submit record (the firmware ignores a WRITE advance without one).
    • Flights with later flights behind them retire on "passed" receipts; the newest keeps the exact one.
    • Same-queue compute dependencies follow the published frontier; renders and cross-engine dependencies still need completion.
    • Only the same VM queues (with six dEQP clients, compute queued behind another VM's produced wrong images and a firmware stall), and nothing queues behind a command with user timestamps (the host writes those at retirement; a queued query copy ran before they existed).
    • A VM that submits compute back to back gets 6 units (lazily; others keep 2).
    • Firmware moved its new state to the heap: it is built on the probe's stack, and the larger frame overflowed the 32 KiB stack.
    • debugfs aurora-g16/tune exposes depth, units and the repoll interval.
  2. Share one PB heap between a VM's render allocators. Renders are serialized and retire with their blocks returned.
  3. Let a closed VM's arenas go; slot leases keep their own. Hardware-slot leases held the whole VM state (every arena, PB heap included) until another VM displaced them; leases now hold exactly their arena, and the VM's cache drops its arenas at client close.
  4. Keep never-exposed PB epochs for the next renders. Render submit CPU went from ~190-435 µs to 35-46 µs per vkQueueSubmit (the epoch allocation, mappings and L2 sync were wasted with epoch reuse). The cache drops spares at close; a spare holds its VM.
  5. Size the PB heap at 64 MiB. The 25G83 firmware copes with a short PB without the host. With 8 MiB posted, heavy and tiler-overflow renders still complete, no growth event arrives, and PB-manager counters move only for the scenes that overran it. Heap 16-256 MiB made no difference on gpubench's heavy/tvb rows. Tunable via pb_heap_mb.
  6. Skip the submit-path publication while a render runs. Nothing is published behind a render, so a job submitted during one waits for the poll after its retirement anyway; the submit ioctl's full publication pass (ring drain, health and power checks) under the device lock only delayed the retire-to-publish path. render.light d8 389-396 → 344 µs, d64 368 → 345 µs.

Results (J773g Mac mini, kernel 7.3.0-ARCH-m4drv8; macOS 27.2 on the same machine)

gpubench, µs per submit; stock = the kernel's own aurora.ko, run back to back with this PR's module.

row stock this PR macOS
compute.light d8 / d64 268 / 269 100 / 101 26.4 / 19.7
compute.heavy alu d4 / d64 887 / 880 556 / 554 (= GPU time 554) 539 / 538
render.light tri d1 988 560 212
render.light tri d8 350 344 46.7
render.light tri d64 294 345 42.2
render.heavy fill 1080p d1 / d16 1316 / 801 1000 / 755 502 / 246
concurrent compute+render d4 464 323 54.9
render submit CPU 190-435 35-46 1-2
Geekbench 6.7.1 GPU (Vulkan vs Metal), ms/iteration macOS before (#2239+#1543) this PR this PR + Mesa packing (aurora-silicon/mesa#488)
Background Blur 9.14 82.4 34.7 23.2
Face Detection 6.17 122.8 50.9 38.8
Feature Matching 14.96 89.7 40.4 37.5
Particle Physics 31.17 145.4 67.0 64.2
Stereo / Gaussian / Edge / Horizon 7.4 / 9.4 / 10.3 / 13.8 21.8 / 22.9 / 16.1 / 22.4 18.6 / 22.5 / 13.4 / 18.4 18.8 / 22.6 / 13.3 / 18.2

Memory (unevictable delta per process, settled): 8 × render.light 158 MB (stock 551, macOS 60); 4 × render.heavy 100 / 124 MB (stock 324 / 348, macOS 111 / 95). Idle desktop after boot: 0.47 GB unevictable (before 2.75 GB). Compute-only process: 52 MB with 2 units, 149 MB with the 6 units a pipelining client gets.

Regressions against stock (same kernel)

  • render.light d64: 294 → 345 µs (d8 and d1 are faster). The per-render firmware turnaround (gap) went from ~302 to ~322 µs; I have not found the cause.
  • Compute-only processes pin more: 8 × compute.light, 60 → 174 MB unevictable per process (macOS 78). A process that submits compute back to back gets 6 units, each with its own arena; aurora-g16/tune compute_units=2 keeps 52 MB but gives back the pipelining.
  • The full gpubench plan was run with commits 1-5 only (before commit 6). Slower than stock there: compute processes in separate VMs (compute.heavy procs 893 → 1016-1025 µs, compute.light procs 261-267 → 283-310 µs; cross-VM commands still go one at a time and pay one more check per turnaround), render.heavy fill d2-d64 by 5-15%, render.light d32/d64, and mem.buffer 64 MiB / mem.image 4096 cycles (1373 → 1725 / 1317 → 1826 µs). The timestamp query rows look worse only because the baseline without queries got faster: commands with timestamps still run one at a time.

Tests

  • Vulkan CTS 1.4.7.1, 1/16 sample (198,732 cases), 6 jobs, commits 1-5: 84,219 Pass, 0 Fail, 0 Crash, 1 QualityWarning, no GPU recovery (before: 84,183 Pass / 0 Fail). Final module (all six): the groups that failed during development (api, binding_model, draw, host_image_copy, fast_linked_library, image, sampler, stencil, timestamp) rerun at 0 Fail, timestamp 18/18.
  • GLES/EGL (public GL plan, 6 jobs, commits 1-5): 0 Fail, 0 Crash. dEQP-EGL 2,360 Pass; dEQP-GLES2 17,051; GLES3 44,567; GLES31 37,543 (incl. multisample); KHR-GLES2/3/31/32 468 / 6,479 / 4,060 / 1,403; KHR-GL46 13,846 Pass of 15,000 run (the 25-minute budget ended with 4,211 left).
  • packtest (Mesa submit packing): 21/21 PASS.
  • tools/testing/selftests/drivers/gpu/asahi-m4/run-host-tests.py: 23/23. Harness mocks were updated for the new calls, and host cases were added for the N-owner ledger, the published frontier, in-order "passed" retirement and cache close.

Not done

  • Render pipelining (TA of the next render during the current FS). It needs per-slot render UMA slots and one PB manager shared across consecutive renders (the macOS model). render.light d64 is still ~345 µs vs macOS 42.
  • Cross-VM queuing is wrong on this firmware path as built (see commit 1). macOS gives each context its own queues.
  • PB growth. The heap is a fixed 64 MiB; the firmware handles overflow itself. Host-side growth between epochs from the overflow counters is possible.

🤖 Generated with Claude Code

aeries0x and others added 7 commits November 19, 2026 08:57
Every G16 render gets a fresh parameter buffer epoch at SUBMIT and an
InitBM ahead of its TA work, so the firmware rebuilds the allocator
state for each render. Reusing the epoch instead cuts a small triangle
render at queue depth 1 on a base M4 MacBook Pro (J604) from 1121 to
492 us.

With this change, the next render of the same VM on a pass slot gets the
slot's owner epoch, without a reseed or InitBM, if that epoch's render
retired normally and was exposed. InitBM binds the device-wide slot to
one epoch and VM, so this happens only while no other epoch was handed
to the slot since; after another VM's render on the slot, after a failed
or aborted render, or after any firmware recovery, the next render
starts fresh. SUBMIT still allocates a fresh epoch for every render, so
one is always at hand. Renders are bound, published and retired one at a
time, so the slot's owner is the epoch it is bound to whenever reuse is
allowed.

The reused epoch is the slot's existing owner lease, so this holds no
epoch, VM or admission permit beyond what the lease already holds: a
slot binding is tracked by a counter, not by a reference. A host test
covers when an epoch may be reused and that nothing else holds it.

aurora.g16_pb_reuse=0 restores a fresh epoch and InitBM per render.
The stats line counts reused and fresh bindings.

Assisted-by: LLM
Signed-off-by: Friedrich <261542475+aeries0x@users.noreply.github.com>
G16 published one command at a time and built the next only after the previous one fully
retired on the host, so every compute command paid the whole firmware and host round trip:
on the J773g Mac mini a 1.5 us dispatch cost ~270 us per submission at any queue depth, and
dependent work (one command buffer per dispatch, as Geekbench records it) was bound by it.

Queue compute commands behind in-flight compute commands on the one firmware compute queue:

- A VM gets more retained compute units, each in its own hardware UMA slot (unit k in slot
  1 + k). All units in slot 1, as before, is not safe once commands are queued: the firmware
  starts the next command while the previous one still runs, and with long commands two
  owners of one slot then corrupt each other (firmware recovery within seconds on queued ALU
  work). With one slot per unit, ~115k queued commands ran without a recovery.
- The displacement ledger generalizes to N owners: owners retire in publication order, and an
  owner is reusable once any later publication retired (the firmware finished it before
  starting that one). Publication stays serialized for the render pass slots.
- Every queued command posts its own submit record and doorbell (the firmware does not pick up
  a WRITE advance without one) behind unconsumed ones, with room checks before anything is
  visible.
- A flight with later compute flights behind it retires on "passed" receipts: its stamp,
  queue READ cursor and the submit pipe's cursors have moved past it. The newest flight keeps
  the exact receipt.
- A compute command may follow its queue's published (not yet retired) compute predecessors:
  the firmware compute queue runs commands in publication order, each to its Finalize.
  Renders and every cross-engine dependency still wait for completion, and nothing is
  published behind a render or a failed, doomed or recovering flight.
- Only the same VM's commands queue behind each other: with six dEQP clients, compute queued
  behind another VM's lost writes (image and sampler cases failed) and stalled the firmware
  queue. Nothing queues behind a command with user timestamps either: the host writes them
  when the command retires, and a query copy queued behind it ran before they existed and
  never finished.
- Recovery matches a snapshot entry against every flight and marks every unfinished one; a
  failed flight dooms the later flights of its queue (their commands would follow a hole).
- Two units per VM as before; a VM that submits compute back to back (three submissions within
  2 ms each, or three compute commands in one submission) gets four more, mapped at
  0x7090000000, so ordinary clients keep the old memory cost.
- Up to 16 jobs per logical queue reach the engine (was 2).
- The engine statistics and the compute owner ledger move to the heap: `Firmware` is built on
  the probe's stack, and its larger frame overflowed the 32 KiB kernel stack.
- `aurora-g16/tune` (debugfs, root) sets the depth (compute_flights; 1 = the serialized engine),
  the units of a growing VM and the completion repoll interval for measurements.

J773g Mac mini (G16G B0, 25G83), gpubench, before -> after (macOS 27.2 on the same machine):
compute.light d8 268 -> 136 us/submit, d64 269 -> 125 (gap between commands 266 -> 16.5 us;
macOS 19.7), compute.heavy alu d4 874 -> 557, d64 871 -> 554 (GPU time 554; macOS 540),
compute.heavy bw d16 1667 -> 1275 (macOS 1437).

Signed-off-by: Diljit Singh <hey@djdev.me>
Every rendering VM pinned two 256 MiB parameter-buffer heaps, one per render allocator (pass
slot), so each rendering process cost ~0.5 GiB of unevictable memory and a few dozen clients
(a dEQP-EGL run at six jobs) exhausted a 24 GiB Mac mini. macOS 27.2 on the same machine adds
~24 MiB of wired memory per rendering process.

A render is published only while nothing else is in flight and retires with its PB blocks
returned to its manager, so the two pass slots' PB managers never hand out heap blocks at the
same time. Back both with one heap at the first slot's address.

Signed-off-by: Diljit Singh <hey@djdev.me>
A hardware slot's last owner stays bound until a later owner displaces it, because the
firmware may still write that owner's descriptor then. The leases held the owner's whole VM
state, and with it every arena of the VM: a closed client whose compute unit was the last in
some UMA slot kept its 256 MiB PB heap and its compute pools pinned until another VM happened
to use that slot. With more compute units, slots used only by clients that queue compute are
displaced rarely, and closed clients stayed pinned.

Each lease now holds exactly the arena of its unit (compute) or pass slot (render) beside the
VM state, and the VM's cache lets go of its arenas when the client closes the VM. What jobs
and leases still need, they hold themselves; the rest is freed at close.

Signed-off-by: Diljit Singh <hey@djdev.me>
Each render submission allocated a fresh PB epoch: three firmware objects, their GPU and
firmware mappings and a shared-L2 synchronisation. With PB epoch reuse most renders never
expose that epoch, and its deferred teardown unmaps it again. Render submission cost ~390 us
of CPU per vkQueueSubmit on the J773g Mac mini (compute ~25 us).

Return a retired job's never-exposed epochs to its VM (up to four, under a spinlock: a job may
drop in any context) and hand them to the next render submissions before allocating. A spare
keeps the budget permit it was created with. A spare holds its VM, which holds the cache, so
the cache drops its spares, and takes no more, when the client closes the VM.

Signed-off-by: Diljit Singh <hey@djdev.me>
The parameter-buffer heap was the whole 256 MiB window, fully backed and pinned for every
rendering VM. The 25G83 firmware copes with a PB that runs out without the host: with only
64 blocks (8 MiB) posted, heavy and tiler-overflow renders still complete, no growth event
arrives, and counters in the PB manager move only for the scenes that overran it.

Back 64 MiB of the window by default and name only the backed part in the page and block
lists. `aurora-g16/tune` sets the heap of VMs created later (pb_heap_mb, 8..256) and caps the
blocks a fresh epoch posts (pb_blocks), for measurements.

Signed-off-by: Diljit Singh <hey@djdev.me>
Nothing is published behind a render, so a job submitted while one is in flight waits for the
poll pass after its retirement anyway. With more jobs admitted per queue, each such submit
still ran a full publication pass (ring drain, health and power checks) under the device lock
and delayed the retire-to-publish path: gpubench render.light at depth 8 took 389-396 us per
submission. Enqueue only; the next poll publishes it (344 us; stock 350, macOS 47).

Signed-off-by: Diljit Singh <hey@djdev.me>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants