Skip to content

hk: resolve each occurrence of a command buffer in a submission - #444

Merged
RyanTheTide merged 1 commit into
aurora-silicon:aurora-wipfrom
jgoyvaerts:pr/hk-simultaneous-use-occurrences
Nov 19, 2026
Merged

RyanTheTide merged 1 commit into
aurora-silicon:aurora-wipfrom
jgoyvaerts:pr/hk-simultaneous-use-occurrences

Conversation

@jgoyvaerts

Copy link
Copy Markdown

What

A simultaneous-use command buffer may occur more than once in one vkQueueSubmit, for example {c0, c1, c0}. hk_sync_resolve_submit() kept the resolved barrier and prior fields in each hk_cs, so the second occurrence of c0 overwrote those of the first before the payload was built. The first occurrence then went out with a wait on a command after it; the G17 kernel refuses that (refused a submission (EINVAL)) and the device is lost.

Every occurrence is now resolved completely, the first-command chain and the vertex-half split included, before the next one starts, and its fields are saved to a per-queue array in submission order. queue_submit() and the sibling-queue check read that array, and the payload loop loads each entry into its stream before emitting it.

A sparse bind that only flushes the sparse table has no stream to resolve, so the array is not resized for it: util_dynarray_resize() returns NULL for zero entries of a never-allocated array, which lost the device on a queue's first such bind (vkd3d-proton's tiled-resource tests).

Tests: test-hk-sync-prior gains test_repeated_recording ({a, b, a}, exact and legacy priors, with and without the first-command chain); the new hk-g17p-sparse-bind-first-submit shim test binds one tile as a queue's first submission.

Testing

M4 Pro (J616s, G17P), kernel 7.3.0-m4.167, release builds, guarded runs, 0 GPU recoveries.

🤖 Generated with Claude Code

A simultaneous-use command buffer may occur more than once in one
vkQueueSubmit, for example {c0, c1, c0}. hk_sync_resolve_submit()
kept the resolved barrier and prior fields in each hk_cs, so the
second occurrence of c0 overwrote those of the first before the
payload was built. The first occurrence then went out with the
second's fields: a wait on a command after it, which the G17 kernel
refuses (barrier past the commands before it, EINVAL) and the device
is lost.

Resolve every occurrence completely, the first-command chain and the
vertex-half split included, before the next one starts, and save its
fields to a per-queue array in submission order. queue_submit() and
the sibling queue check read that array, and the payload loop loads
each entry into its stream before emitting it.

A sparse bind that only flushes the sparse table has no stream to
resolve: the array is not resized for it, since util_dynarray_resize()
returns NULL for zero entries of a never-allocated array, which lost
the device on a queue's first such bind (vkd3d-proton's tiled-resource
tests). The hk-g17p-sparse-bind-first-submit shim test binds one tile as
a queue's first submission.

Signed-off-by: Jonathan Goyvaerts <jonathan.goyvaerts@gmail.com>
Assisted-by: LLM
@DjDeveloperr

Copy link
Copy Markdown

Hardware evidence for this bug from the M5 Pro driver-overhead lane, which also covers G16G (M4 mini):

  • M5 Pro (G17S): a SIMULTANEOUS_USE command buffer listed three times in one VkSubmitInfo ({cb0, cb1, cb1, cb1, cb4}) makes the kernel refuse the submit (refused a submission (EINVAL)) and the device is lost. This happens with packing off on every build tested, including one based on aurora-wip without this lane's changes.
  • M4 mini (G16G): the M4 mini lane reproduced the same failure with packing off (vkQueueSubmit = -4).
  • Packing hides it: with submit packing (hk: pack runs of plain compute command buffers into one command #488) on, the occurrences are copied into one packet stream and the case passes. A pack-on PASS is therefore not evidence of a fix.
  • Same cause as described here: the later occurrence overwrites the resolved barrier and prior fields in hk_cs. The kernel's log line names the group's first command (SET_COMPUTE_RESOURCES), not the refused dispatch.

The lane fixed it differently: it split such a batch into consecutive ioctls in which each recording occurs once. That fix passed on the M5:

  • the repro, with packing off and on;
  • the CTS lane's 64-case regression list (synchronization{,2}.basic.*, api.command_buffers.record_simul_use_* / submit_twice_* / secondary_execute_twice, transform_feedback.simple.draw_indirect_counter_resubmit*): 63 Pass, 1 NotSupported, with packing on and off.

This PR keeps one ioctl per batch, which is the better shape, so we won't open a competing PR. We're happy to run the same repro and regression list on the M5 with this PR applied if that helps it land. The mustpass cases with this pattern are image.general_layout.astc_sample.* (5), which are NotSupported on M5, so CTS alone doesn't catch it there.

🤖 Generated with Claude Code

@RyanTheTide
RyanTheTide merged commit ded68fe into aurora-silicon:aurora-wip Nov 19, 2026
RyanTheTide added a commit to jgoyvaerts/mesa that referenced this pull request Nov 19, 2026
Keep both sets of shim tests in src/asahi/meson.build and
src/asahi/vulkan/meson.build next to the ones aurora-silicon#444 added.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
RyanTheTide added a commit to jgoyvaerts/mesa that referenced this pull request Nov 19, 2026
Keep both sets of shim tests in src/asahi/meson.build and
src/asahi/vulkan/meson.build next to the ones aurora-silicon#444 added.

Signed-off-by: Ryan Murray <ryan@aurorasilicon.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants