Repository navigation
hk: VK_EXT_memory_priority and VK_EXT_pageable_device_local_memory - #243
Conversation
|
@erikwb the driver side reads correctly. This now conflicts in include/drm-uapi/asahi_drm.h with #242, which merged. Please rebase, keeping both sets of additions, and sync the header from aurora-silicon/linux#1610 once that merges so the ioctl id and feature bit 35 match what the kernel ships. Minor: |
Under memory pressure the kernel may release the memory of an idle VM's
private BOs, which its next submission then waits to get back. Memory
priority 1.0 now asks the kernel to release a memory object's BO after
the VM's other BOs when it reclaims the VM's idle memory, and only if
theirs was not enough (DRM_IOCTL_ASAHI_GEM_RESIDENCY, a hint), whether
given at allocation
(VkMemoryPriorityAllocateInfoEXT) or later (vkSetDeviceMemoryPriorityEXT);
any other priority returns it to the kernel's default policy. That is
the only distinction the kernel makes, so it is the whole mapping.
Freeing the memory object clears the request, since the BO may be
reused from the BO cache. A refused request is logged and leaves the
priority as it was (the call cannot fail).
Both extensions and their features are advertised only when GET_PARAMS
reports DRM_ASAHI_FEATURE_GEM_RESIDENCY (and not over virtio). The
include/drm-uapi copy gains the GEM_RESIDENCY ioctl, its struct and
flag, and feature bit 35.
Tested on the M5 Max with a kernel that has GEM_RESIDENCY:
dEQP-VK.memory.pageable_allocation.* 202 pass and
dEQP-VK.memory.binding.priority{,_dynamic}.* 112 pass (all skipped
without this change). A 64 MiB device-local allocation at priority 1.0
raises the kernel's user_kept count by 64 MiB; setting it to 0.5, back to
1.0 and freeing it give 0, 64 MiB and 0. With the feature bit masked out
of GET_PARAMS, as on a kernel without it, neither extension nor feature is
listed, no GEM_RESIDENCY call is made, and those cases are skipped as
before. dEQP-VK.memory.allocation.basic.* is unchanged.
Developed with Claude Code.
Signed-off-by: Erik Bourget <erik@ebourget.net>
565e4f7 to
fc1d7d9
Compare
|
Thanks. Rebased onto aurora-wip with #242 (fc1d7d9):
Retested on the rebased base (the testing section has the new numbers): the priority CTS cases pass, the results with the feature bit masked equal the base case for case, and #242's external_memory_host list is unchanged. Heads-up: #259 adds feature bit 36 at the same place in the header, so whichever of the two lands second will need a trivial rebase. |
mesa#264 added DRM_ASAHI_FEATURE_COMPUTE_VISIBLE_COMPLETION (bit 34) at the place in include/drm-uapi/asahi_drm.h where this branch adds DRM_ASAHI_FEATURE_GEM_RESIDENCY (bit 35). Both are kept, in bit order. No other change. Signed-off-by: Diljit Singh <hey@djdev.me>
|
I pushed aurora-silicon/linux#1610 is merged ( Next steps, then the merge:
|
DjDeveloperr
left a comment
There was a problem hiding this comment.
Approving and merging at 6abda25d7909.
- Independent review (with linux#1610): no blocker.
- The header hunks are byte-identical to the kernel's, which is merged as
954a6a74. - Priority 1.0 keeps an object resident; any other value returns it to the default.
- Freeing a memory object clears its request before the BO goes back to the cache.
- A refused request is logged, and the priority stays as it was.
- Both extensions are advertised only with bit 35, and not over virtio.
- The header hunks are byte-identical to the kernel's, which is merged as
- Local CI on the M5 Pro (VK CTS lane slot m5-vkcts-mg243c): the arm is this head merged into aurora-wip
2e14956a88a(0d86debb055, tree1397fb2b2196).- The asahi workflow builds clean.
meson test --suite asahi: 131 Ok, 0 Fail.
- Hardware:
- An earlier J714s run of this series, on a kernel with GEM_RESIDENCY, gave priority/pageable 426/426.
- Your J716c rerun after the rebase passes the priority CTS cases.
- The only code change since then is the conflict-only merge I pushed.
#259 adds bit 36 at the same place, so it needs one more conflict-only resolution after this lands.
Problem
Under memory pressure the kernel may release the memory of an idle VM's
private BOs, and the VM's next submission then waits to get it back. An
application has no way to say which allocations should stay resident. The
kernel change in aurora-silicon/linux#1610 adds
DRM_IOCTL_ASAHI_GEM_RESIDENCYto ask that an object's memory be released last.
Change
include/drm-uapi/asahi_drm.hgains theGEM_RESIDENCYioctl (the idafter
GEM_USERPTR), its struct and flag, andDRM_ASAHI_FEATURE_GEM_RESIDENCY(bit 35), as in drm/asahi: GEM_RESIDENCY, keep an object's memory resident linux#1610.the VM's other BOs when it reclaims the VM's idle memory, and only if
theirs was not enough (a hint: see drm/asahi: GEM_RESIDENCY, keep an object's memory resident linux#1610 for what
reclaim reaches),
whether given at allocation (
VkMemoryPriorityAllocateInfoEXT) or later(
vkSetDeviceMemoryPriorityEXT); any other priority returns it to thekernel's default policy, the only distinction the kernel makes. Freeing
the memory object clears the request, since the BO may be reused from the
BO cache. A refused request is logged and leaves the priority as it was.
GET_PARAMSreports
DRM_ASAHI_FEATURE_GEM_RESIDENCY(and not over virtio).Testing
Hardware: M5 Max MacBook Pro (J716c), kernel 7.3.0-ewb121+ (has
GEM_RESIDENCY and GEM_USERPTR), Mesa 26.3.0-devel (
aurora-wip1f629c9plus this commit; Honeykrisp built alone). Builds with no new warnings.
Vulkan CTS with deqp-runner (2643 cases:
dEQP-VK.memory.pageable_allocation.*,dEQP-VK.memory.binding.priority{,_dynamic}.*,dEQP-VK.memory.allocation.basic.*,dEQP-VK.api.info.get_physical_device_properties2.*andvulkan1p2_limits_validation.*):The 314 new passes are all 202
pageable_allocationand all 112binding.priorityandbinding.priority_dynamiccases. With the feature bitmasked out, the results equal the base case for case.
The 2443-case list of #242 (
dEQP-VK.memory.external_memory_host.*,memory.allocation.basic.*,memory.mapping.suballocation.full.*, theexternal memory info cases, the properties and limits cases above and
dEQP-VK.api.device_init.create_device*) gives 1510 passes and 933 skips onboth the base and this branch, case for case.
The CTS priorities stay below 1.0, so a small Vulkan program checked the
residency request itself: a 64 MiB device-local allocation at priority 1.0
raises the kernel's
user_kept(GPUgpu_memoryin sysfs) from 0 to67108864; setting the priority to 0.5, back to 1.0 and freeing the memory
give 0, 67108864 and 0, with four
GEM_RESIDENCYcalls that all succeed.With the feature bit masked out the program sees
memoryPriorityandpageableDeviceLocalMemoryfalse and noGEM_RESIDENCYcall is made.The masked runs used an
LD_PRELOADshim that clears the bit fromDRM_IOCTL_ASAHI_GET_PARAMS, which is how the driver sees a kernel withoutGEM_RESIDENCY.
Depends on
The
GEM_RESIDENCYandGEM_USERPTRparts of the header match itscurrent head.
aurora-wipalone. hk: hk_compute_timeout_ms sets the watchdog allowance of compute work #259 adds featurebit 36 at the same place in
include/drm-uapi/asahi_drm.h, so whicheverof the two merges second conflicts textually there; keeping both
additions resolves it.
Not covered
was not exercised from Mesa.
🤖 Generated with Claude Code