Skip to content

hk: submit packing follow-ups from the #488 reviews - #558

Open
DjDeveloperr wants to merge 2 commits into
aurora-wipfrom
djdev/g16-pack-followup
Open

DjDeveloperr wants to merge 2 commits into
aurora-wipfrom
djdev/g16-pack-followup

Conversation

@DjDeveloperr

Copy link
Copy Markdown

Follow-ups to #488 (merged as 11ee623) from its third and fourth independent reviews (comments 7025084567 and 7025630906) and the M5 lane's re-review. One commit, no change in what packs on G16G or G17S today.

Changes

  • G16G atomics and ordering rationale (fourth review, finding 2). The comments said the G16 kernel runs one command of the device at a time, but linux#2419 keeps several commands of a VM in flight. They now say what still holds: G16 runs every compute command of a VM in order on its one firmware compute queue, never beside another VM's. The behaviour is unchanged.
  • HAL200 boundary limited to the G16 parts (finding 4). G17A was let in only because hk_cdm_memory_scope_form() doesn't list it. It sits between G17P and M5, which both have tagged caches and a memory-scoped flush form, so it gets no boundary until its caches are known. No current kernel admits G17A.
  • G15 wording (nit). G15 stays out because packing there isn't qualified (kernel deadline, attachment limits), not because its barrier differs. The test name says so too.
  • HK_PACK_ATOMICS on G16G (nit). An unparsable value keeps the default (on), as before hk: pack runs of plain compute command buffers into one command #488.
  • Strong-flush tails, reported and tested.
    • hk_pack_admits_strong_tail() says whether a member may end with a strong flush without memory scope, as an application barrier at the end of a command buffer leaves it.
    • It is true at the HAL200 boundary, which upgrades that record in place. It is false at the visible-completion boundary (M5), whose members must end with the memory-scoped flush.
    • The packing trace prints it as strong_tails=, and a test pins it to what hk_pack_run_member() accepts on G17S and on G16G.
    • mesa#506, which admits those tails on M5, should flip this one function.
  • aurora-packtest must-pack follows the boundary (M5 lane).
    • Cases whose command buffers end with a dispatch (multi-tail, disjoint) must pack wherever packing is on.
    • Cases ending with a barrier (chain242, chain300, loops, chain-all, multi) must pack only where the trace reports strong_tails=1, so M5 rows without hk: pack M5 command buffers that end in a barrier without memory scope #506 no longer FAIL them by design.
    • With no trace available, only must-pack cases fail (nit).
  • hk_pack_test: exact-prior queue history (finding 5). This is the reviewer's test: a render forgotten in an earlier submission on an exact-prior queue splits, as the resolver resolves it (last_absolute, not ever). A G17A row is also added.

Checks

  • Local, host aarch64, clang 23.1.1, release, -j2, 2 GiB cap:
    • libvulkan_asahi.so and aurora-packtest build;
    • meson test --suite asahi: 139 Ok, 0 Fail;
    • hk_pack_test: 142 checks, 0 failures; random harness 3180 packets, history harness 4872 packets, 0 failures;
    • no warnings in the changed files.
  • Hardware (J773g, M4, G16G). The driver part of this commit ran on the M4 mini on 2026-11-19, as part of the alias-fix candidate driver built on top of it: aurora-packtest 22/22 PASS with packing on, every trace line strong_tails=1. This head's aurora-packtest (the boundary-aware must-pack) hasn't run on hardware yet. It runs in the next M4 slot, and an M5 row with and without hk: pack M5 command buffers that end in a barrier without memory scope #506 would check the strong_tails=0 path.

Not in this PR

  • Two GPU addresses on G16G. The stale reads found on the M4 row (hk_cdm_alias_scope() false on G16G) get their own PR once the fix candidate's full CTS has run on the M4. That PR also adds the alias-* aurora-packtest cases.
  • Looping streams. The one-looping-stream cap costs Feature Matching and Particle Physics their packing; that's confirmed as the only split reason. Recovering it needs a run-time bound for looping members, as a separate change.

🤖 Generated with Claude Code

- The G16G rationale for packing global atomics, and the ordering note
  of the planner, now say what linux#2419 keeps true: the G16 kernel
  runs every compute command of a VM in order on its one firmware
  compute queue, never beside another VM's. It no longer runs one
  command of the device at a time.
- The HAL200 boundary is limited to the G16 parts. G17A, the other
  HAL200 part, sits between G17P and M5, which both have tagged caches
  and a memory-scoped flush form; it has no boundary until its caches
  are known. G15's exclusion now gives its real reason: packing there
  is not qualified, not that its barrier differs.
- An unparsable HK_PACK_ATOMICS keeps G16G's default (on), as before
  #488.
- hk_pack_admits_strong_tail() says whether a member may end with a
  strong flush without memory scope, as an application barrier at the
  end of a command buffer leaves it: at the HAL200 boundary, which
  upgrades it in place, and not at the visible-completion boundary
  (M5), whose members must end with the memory-scoped flush. The
  packing trace reports it as strong_tails=, and a test pins it to
  what hk_pack_run_member() takes on G17S and G16G.
- aurora-packtest's must-pack check follows it: cases whose command
  buffers end with a dispatch (multi-tail, disjoint) must pack wherever
  packing is on; those ending with a barrier (chain242, chain300,
  loops, chain-all, multi) only where the trace reports strong_tails=1.
  With no trace available, only must-pack cases fail.
- hk_pack_test: an exact-prior queue whose earlier submission forgot a
  render splits as the resolver resolves it (last_absolute, not ever;
  from the fourth review), and G17A has no boundary.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr

Copy link
Copy Markdown
Author

Independent review of 88881d60 (on aurora-wip 11ee623)

Verdict: ready to merge. Nothing blocks. Four low-severity notes and two nits are below. Fold them in if convenient; none needs a hardware rerun.

1. "No change in what packs on G16G or G17S": verified

  • Diff of the driver code with comments stripped (hk_pack.c, hk_pack.h, hk_queue.c, 11ee623 → head). There are exactly three changes:

    1. the G16 chip list in hk_pack_boundary();
    2. debug_parse_bool_option(opt, true) for HK_PACK_ATOMICS on G16G;
    3. the strong_tails= field in the two trace lines.

    Nothing else reads the boundary: the only callers are hk_pack.c and hk_queue.c.

  • Boundary table. I built hk_pack_test against the base and the head hk_pack.c and printed hk_pack_boundary() for all 19 chips, with the visible-completion feature off and on. The tables differ in exactly two rows, both G17A: the boundary goes from STRONG_FLUSH to NONE with the feature off and on. G16P/G/S/C, G17P, the three M5 parts, G15 and AGX2 are identical.

  • HK_PACK_ATOMICS on G16G. Only an unparsable value changes, and it now gives on instead of off. Unset, 0/n/no/f/false and 1/y/yes/t/true behave as before.

2. The items, checked by mutation

Host build: x86_64 Arch, clang, release, -j2, MemoryMax=2G. libvulkan_asahi.so and aurora-packtest build with no warnings in the changed lines. meson test --suite asahi: 139 Ok, 0 Fail. hk_pack_test: 142 checks, 0 failures, history harness 4872 packets.

I applied six mutations, one at a time, and rebuilt hk_pack_test for each. Each one fails exactly one check, and it is the new check for that item:

Mutation Failing check
Restore "HAL200 && no memory-scope form" in hk_pack_boundary() G17A: HAL200 but not G16, no boundary
hk_sync_queue_kinds() returns ever in exact mode as well exact: a render forgotten in an earlier submission splits
hk_pack_admits_strong_tail() returns boundary != NONE M5: no strong-flush tail, and the trace says so
hk_pack_admits_strong_tail() returns false G16G: a strong-flush tail, and the trace says so
G16 member check accepts only weak_barrier_end G16G: a strong-flush tail, …
M5 member check relaxed as in #506 (strong_flush_end == end only) M5: no strong-flush tail, …

So the exact-vs-legacy test (finding 5) does tell the two lookups apart, and no pre-existing check does: in the second mutation it is the only failure.

The aurora-packtest must-pack logic was checked with a harness that calls trace_line() and must_pack() directly:

  • strong_tails=0 makes chain242 not must-pack and leaves disjoint must-pack;
  • =1 makes both must-pack;
  • with no line seen yet, the value is -1, which counts as must-pack (the old behaviour);
  • after a render, chain-all is not must-pack.

The case lists are classified correctly: multi-tail and disjoint end with an OP_MIX, and the other five end with OP_BARC/OP_BAR. With no trace available, only must-pack cases FAIL.

The G16G rationale now matches linux#2419 as written there: "compute commands queue behind the same VM's in-flight compute on the one firmware compute queue", and "only the same VM queues".

Findings (most severe first, all low)

L1. hk_pack_admits_strong_tail() mirrors hk_pack_run_member() instead of being used by it. hk_pack.c:56, hk_pack.c:101

L2. With a driver older than this commit, aurora-packtest still fails barrier-ended cases on M5. tests/aurora_packtest.c:812, :841, :969

  • Scenario. This head's packtest runs against an M5 driver without the strong_tails= field. The value stays -1, which counts as must-pack, so chain242 etc. FAIL as before. The value is also sticky: a line without the field keeps the previous value.
  • Fix. None needed: falling back to must-pack is the right default. Say in the comment (or the PR text) that the relaxation needs a driver that has this commit, so an M5 row on an older driver isn't misread.

L3. The HK_PACK_ATOMICS nit has no test. hk_queue.c:2460

  • Scenario. hk_pack_atomics_allowed() is static in hk_queue.c, which no unit test links, so the false → true default isn't pinned. I verified it by reading debug_parse_bool_option().
  • Fix. Optional: move the env-to-bool decision into a small pure helper in hk_pack.c and add a two-line check.

L4. Wording in the G16G atomics rationale. hk_queue.c:2445-2446

  • "none runs beside a packet out of order" mixes overlap with order. The property atomics need is that the firmware runs the queued commands one after another and never beside another queue's.
  • Fix. "…several commands of one VM may be in flight there, but the firmware runs them one after another, never beside a packet."

Nits

  • hk_pack.c:204 (108 columns) and hk_queue.c:2383 (83 columns): the comment rewrites weren't reflowed.
  • hk_device.h:452 still says "The other HAL200 parts (G16) have only the plain strong flush". With the new G17A note in hk_pack.c, it should leave G17A out or say its caches are unknown.

Independent review. Scope: the diff, plus local build and tests on the host. Not run on hardware.

🤖 Generated with Claude Code

From the review of #558:

- hk_pack_run_member() now asks hk_pack_admits_strong_tail() whether a
  member may end with a strong flush without memory scope, instead of
  restating the rule. The packing trace reports the same answer, so a
  change to one (mesa#506 admitting such tails on M5) changes both, and
  aurora-packtest's must-pack check follows it.
- hk_pack_atomics_choice() is the HK_PACK_ATOMICS decision, now a pure
  helper with a test: on by default on G16G and opt-in elsewhere with one
  kernel queue; an unparsable value keeps the default.
- aurora-packtest says that only a driver with the strong_tails= field
  relaxes the barrier-ended must-pack cases; with an older one on M5
  without #506 they FAIL as before.
- Comments: the G16G atomics rationale says the firmware runs a VM's
  compute commands one after another, never beside a packet; G17A's
  caches are not known (hk_device.h); two paragraphs reflowed.

Signed-off-by: Diljit Singh <hey@djdev.me>
@DjDeveloperr

Copy link
Copy Markdown
Author

Thanks. All fixed in one commit on top, new head bc0f0dc:

  • L1: hk_pack_run_member() now takes its tail rule from hk_pack_admits_strong_tail(). A member qualifies when it ends with a strong flush that has memory scope and visible completion, or that the boundary admits plain. At the HAL200 boundary it may also end with an ordering barrier. The trace reports the same function, so hk: pack M5 command buffers that end in a barrier without memory scope #506 really flips one place, and the existing G17S/G16G check pins both together.
  • L2: the comments at the trace parser and must_pack() say that only a driver with the strong_tails= field relaxes the barrier-ended cases. With an older driver on M5 without hk: pack M5 command buffers that end in a barrier without memory scope #506, they FAIL as before.
  • L3: the decision is now hk_pack_atomics_choice(), a pure helper in hk_pack.c. hk_pack_test checks it: G16G on by default, 0 off, unparsable keeps on; M5 off by default, 1 with one kernel queue on, with two off, unparsable keeps off.
  • L4: reworded to "…several commands of one VM may be in flight there, but the firmware runs them one after another, never beside a packet."
  • Nits: both paragraphs reflowed (no line over 80 in hk_pack.c/hk_pack.h/hk_device.h or the packing block of hk_queue.c). hk_device.h now says G17A's caches are not known, so hk_pack_boundary() gives it no packing.

Local checks at this head (host aarch64, clang 23.1.1, release, -j2, 2 GiB cap):

  • meson test --suite asahi: 139 Ok, 0 Fail.
  • hk_pack_test: 150 checks, 0 failures; random harness 3180 packets, history harness 4872 packets, 0 failures.
  • No warnings in the changed files.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant