Skip to content

【Vulkan后端增强】Enhance Vulkan Renderer backend - #1171

Open
LFRon wants to merge 15 commits into
linuxdeepin:masterfrom
LFRon:feat-and-refactor/enable-vulkan-renderer
Open

LFRon wants to merge 15 commits into
linuxdeepin:masterfrom
LFRon:feat-and-refactor/enable-vulkan-renderer

Conversation

@LFRon

@LFRon LFRon commented Jul 18, 2026 •

Copy link
Copy Markdown
Contributor

该PR是 #1032 的更新版, 改动的wlroots由外侧转为treeland内侧的wlroots

Summary by Sourcery

Enhance the compositor’s Vulkan backend with synchronized Qt/wlroots rendering, robust resource lifetimes, Vulkan effects and cursor handling, capture support, and related Wayland protocol integration.

New Features:

  • Add a Vulkan rendering path that shares wlroots and Qt Vulkan resources with explicit image ownership, synchronization, render-target preservation, and texture sampling support.
  • Support Vulkan backdrop rendering for effects, including blur, rotation, content compositing, and direct hardware-cursor rendering.
  • Expose renderer-backed linux-dmabuf and presentation-time Wayland globals for Vulkan sessions.
  • Add Vulkan-specific frame tracing, presentation feedback, failure handling, and cleanup diagnostics.
  • Provide Vulkan capture support by dumping rendered buffers to normalized images before copying them to clients.

Bug Fixes:

  • Fix Vulkan texture format/view handling, queue-family transitions, DMA-BUF synchronization, staging uploads, and render-buffer lifetime management.
  • Prevent stale or mismatched surface textures, cursor resources, swapchains, and deferred QRhi resources from being reused or destroyed prematurely.
  • Improve capture validation and copying for invalid images, crop regions, formats, strides, and buffer-access failures.
  • Preserve existing non-Vulkan rendering behavior while making texture-provider and scene-graph paths resilient to missing textures.

Enhancements:

  • Integrate the vendored wlroots Vulkan extensions directly into Waylib and coordinate Qt Quick frame submission with wlroots queue operations.
  • Add Vulkan-specific render-pass dependencies, shared depth-stencil attachments, asynchronous staging support, and batched texture barriers.
  • Refine Vulkan cursor placement, scaling, fallback behavior, and output scheduling.
  • Add presentation feedback tracking for textured surfaces and output commits.

Build:

  • Detect wlroots Vulkan support and conditionally locate and link Vulkan.
  • Build the vendored wlroots before Waylib and include new protocol and tracing sources.

Documentation:

  • Update Waylib build instructions to use the default CMake configuration.

Chores:

  • Update Vulkan and wlroots integration APIs and compatibility wrappers across the embedded renderer.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @LFRon, your pull request is larger than the review limit of 150000 diff characters

@deepin-ci-robot

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: LFRon

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@deepin-ci-robot

Copy link
Copy Markdown

Hi @LFRon. Thanks for your PR.

I'm waiting for a linuxdeepin member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 3 times, most recently from 3d74e19 to 49ea9e2 Compare July 21, 2026 03:34
@LFRon

LFRon commented Jul 21, 2026

Copy link
Copy Markdown
Contributor Author

@zccrs 对了大佬, 这个实现里我顺便把CPU-GPU的等待模型改成了GPU内异步等待模型降低了画面延迟且减少了开销, 目前测试下来没啥问题, 大佬是否能参考一下(

@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 3 times, most recently from c0942c0 to ec9999c Compare July 22, 2026 13:41
@LFRon LFRon changed the title 【Beta】Enable Vulkan Renderer 【重大更改】Add Vulkan Renderer backend Jul 22, 2026
@LFRon

LFRon commented Jul 22, 2026

Copy link
Copy Markdown
Contributor Author

@zccrs 大佬, 我做了Blur效果支持, 目前我这边测试没问题:

treeland侧的提交: c3e99a7

wlroots侧的提交:
ec9999c

希望对大佬有参考(

@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch from ec9999c to 32acd40 Compare July 22, 2026 13:43
@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 3 times, most recently from 9e19f70 to ac5f77d Compare July 24, 2026 03:26
@deepin-bot

deepin-bot Bot commented Jul 24, 2026

Copy link
Copy Markdown

TAG Bot

New tag: 0.8.16
DISTRIBUTION: unstable
Suggest: synchronizing this PR through rebase #1203

@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 9 times, most recently from a973cd3 to 9990d35 Compare July 30, 2026 07:50
@deepin-bot

deepin-bot Bot commented Jul 31, 2026

Copy link
Copy Markdown

TAG Bot

New tag: 0.8.17
DISTRIBUTION: unstable
Suggest: synchronizing this PR through rebase #1229

@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch from 9990d35 to f47d018 Compare July 31, 2026 06:00
@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 10 times, most recently from 8769afc to 0bfcf85 Compare September 20, 2026 03:35
@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch 4 times, most recently from 4ee4af9 to f30b0ee Compare September 23, 2026 09:57
Expose the Vulkan texture and render-buffer primitives Waylib needs to
share wlroots-owned images with the Qt Quick QRhi bridge.

- Add an image_usage out-parameter to vulkan_import_dmabuf() so callers can
  learn the actual usage flags a render image was created with.
- Store it on struct wlr_vk_render_buffer and return it through
  waylib_vk_renderer_get_render_buffer_attribs() via a new usage field on
  struct wlr_vk_image_attribs.
- Add waylib_vk_renderer_get_queue() to expose the VkQueue so Qt and
  wlroots can be verified to share the same queue.

No GLES2 behavior is affected; these are additive waylib bridge helpers.
Enable the device capabilities Qt Quick needs to implement the Vulkan
blur/backdrop effect (RenderBufferBlitter) with split render passes, and
round out the device extensions Qt documents for an adopted VkDevice.

- Probe and enable VK_KHR_separate_depth_stencil_layouts on the device and
  expose it via waylib_vk_renderer_has_separate_depth_stencil_layouts() so
  Qt can safely split and resume render passes around a depth-stencil pass.
- Advertise transfer-source usage for render modifiers and record the max
  transfer-source extent, so render buffers can be created with
  VK_IMAGE_USAGE_TRANSFER_SRC_BIT when supported (used to sample the output
  buffer as the blur source).
- Include color-attachment reads in both compatible render-pass external
  dependencies so preserved color attachments (effect split passes) observe
  the load operation as well as attachment writes. This fixes color issues
  in apps such as Youdao Cloud Notes that rely on preserved contents.
- Qt Quick RHI adopts wlroots' VkDevice when the Qt Quick bridge runs on
  this renderer, and
  QRhiVulkanInitParams::preferredExtensionsForImportedDevice() lists
  VK_KHR_create_renderpass2, VK_KHR_depth_stencil_resolve and
  VK_EXT_vertex_attribute_divisor as expected on the imported device;
  without them Qt silently loses functionality on a per-driver basis.
  Enable all three whenever the driver offers them and decouple
  create_renderpass2 from the separate-depth-stencil-layouts probe.
  VK_KHR_swapchain is deliberately not added: the compositor drives Qt
  through an offscreen QQuickRenderControl and never presents a swapchain.

No GLES2 behavior is affected; these are additive waylib bridge helpers.
Bridge the Qt/QRhi render path to wlroots' Vulkan textures when both
share the same VkQueue, so client DMA-BUF/shm textures can be sampled by
Qt Quick without per-frame CPU stalls.

- waylib_vk_renderer_prepare_texture_for_sampling() /
  finish_texture_sampling(): record the queue-family-ownership and layout
  transition for one imported texture on Qt's command buffer.
- Texture sync batch (begin/flush/abort_texture_sync_batch): collect the
  foreign-texture DMA-BUF sync_files for a frame, import them into reusable
  binary semaphores, and submit a single wait-only bridge command buffer
  that also records an all-commands memory barrier, so the later Qt
  submission observes the producer's writes.
- Texture barrier batch (begin/flush/abort_texture_barrier_batch): defer the
  per-texture acquire/release barriers and record a single vkCmdPipelineBarrier
  per phase instead of one per texture.
- Same-frame aliasing dependency barrier: a render buffer written by the
  current frame's command buffer (output/layer/cursor buffers) can be
  re-imported as a sampled texture within that same command buffer
  (layer-buffer compositing, cached compositor render targets). The two
  are separate logical VkImages aliasing the same DMA-BUF memory, and per
  §Memory Aliasing the ordering between uses of two aliases must be a
  memory dependency whose access scope covers both: neither the
  ownership-transfer acquire (its source access mask is ignored) nor an
  image memory barrier naming the *sampled* image covers writes performed
  through the render buffer image. So remember such producer buffers
  (frame_render_buffers) and, when the acquire batch flushes, record one
  global (resource-free) VkMemoryBarrier, MEMORY_WRITE -> MEMORY_READ,
  ALL_COMMANDS -> FRAGMENT_SHADER - the same shape the bridge above uses.
- WAR direction of the same aliasing: prepare_texture_for_sampling()
  registers every sampled buffer (frame_sampled_buffers, deduplicated and
  append-only like frame_render_buffers) and
  waylib_vk_renderer_record_render_buffer_acquire() records an additional
  global MEMORY_READ -> MEMORY_WRITE dependency for such a buffer, so a
  render pass through the render buffer image can never begin before
  aliasing sampling finished.
- waylib_vk_renderer_record_render_buffer_acquire() declares oldLayout
  VK_IMAGE_LAYOUT_GENERAL on the first acquire as well: PREINITIALIZED
  asserts host-written memory, which is never true for a freshly imported
  DMA-BUF scanout buffer; GENERAL is the single bookkeeping layout of this
  renderer's acquire/release ping-pong (matching
  waylib_vk_renderer_get_render_buffer_attribs() and the sampled-texture
  path), which also removes an oldLayout-01197 mismatch for validation.
- waylib_vk_renderer_set_stage_async_enabled() + vulkan_submit_stage_async():
  submit SHM staging uploads without blocking the CPU; the texture-sync
  bridge orders the upload before the consumer samples it. WLR_VK_FORCE_STAGE_BLOCK
  and WLR_VK_FORCE_SYNC_POLL force the legacy blocking paths.
- wlr_vk_texture_get_image_attribs(): report the layout of DMA-BUF textures
  from the owned state instead of the stale "transitioned" flag, so the
  reported layout matches the acquire/release
  GENERAL <-> SHADER_READ_ONLY_OPTIMAL ping-pong; pixel textures keep the
  permanent post-upload layout.
- destroy_render_buffer(): wait for the renderer timeline to catch up with
  the last submit point instead of vkQueueWaitIdle(). The async staging
  submissions added here make a non-empty queue a legitimate state on this
  path, while every wlroots submission (sync/async stage, bridge, render
  pass) completes by signalling the timeline semaphore, so matching
  timeline_point is the exact and cheaper equivalent.
- get_command_buffer(): fail with a log when every slot of the fixed pool
  is recording; the bridge and staging submissions added here made that
  state reachable.
- Add new renderer state (timeline semaphore, semaphore pool, barrier arrays,
  stage-async flags, alias-tracking lists) with matching create/destroy
  handling.

No GLES2 behavior is affected.

Follow-up hardening pass (folded):
- Separate the Qt Quick RHI / readback acquire bookkeeping from wlroots'
  render-pass 'owned' flag with a dedicated qt_sampling_acquired flag, and
  self-link the foreign-texture list node in vulkan_texture_destroy(). An
  earlier wlroots render pass that failed between add and drain could leave
  owned=true and permanently suppress acquire for Qt and for readback; the
  foreign list entry of a destroyed texture is now unlinked instead of being
  left dangling.
- Reset the frame alias lists at frame completion
  (waylib_vk_renderer_reset_frame_alias_lists(), called by waylib after the
  Qt offscreen frame has CPU-waited): same-queue submission order already
  orders the previous frame's GPU work ahead of the next frame's, so entries
  only need to live for one frame. This bounds the CPU scan cost and the
  conservative global alias barriers to the frame that created them.
- Reclaim binary-semaphore slots used by a failed bridge submit
  (discard_texture_sync_sems + slot recreation in acquire_texture_sync_sem):
  a semaphore whose wait submission failed has consumed the temporary
  sync_fd import and cannot be reused for a new import, and its slot would
  otherwise be stuck at release_point=UINT64_MAX.
- Add the out-of-frame readback gate
  waylib_vk_texture_begin_readback()/end_readback(): reads of DMA-BUF
  imported wlr_textures via wlr_texture_read_pixels() wait the buffer's
  DMA-BUF sync_files and record the FOREIGN<->own ownership transition
  around the staging command buffer (same-queue ordering covers the
  execution dependency; the staging submit still CPU-waits). The gate
  refuses multi-plane/YCbCr textures and textures whose foreign ownership
  is taken by a live acquire.

Failure-path hardening (folded):
- The barrier batch now keeps the per-texture bookkeeping next to each
  deferred barrier (texture_acquire_textures/texture_release_textures) and
  rolls it back whenever the batch is aborted or cannot be recorded
  (rollback_deferred_texture_barriers): a dropped acquire leaves the image
  FOREIGN-owned (it must be re-acquired on the next use), a dropped release
  leaves the graphics queue owning it (it must not be acquired again). A
  flush without a command buffer fails closed instead of silently dropping
  the batch, and aborting an active batch rolls it back before rearming.
  WAYLIB_VK_TEST_FLUSH_FAIL_AT=<n> (one-shot, test-only) makes the n-th
  flush behave as if no command buffer were available, so the rollback path
  can be exercised on a live compositor.
- Widen the batched acquire/release barrier stage masks from
  VK_PIPELINE_STAGE_FRAGMENT_SHADER_BIT to ALL_COMMANDS: the release's first
  scope must cover every stage that may still read the texture (not only
  fragment sampling), and the acquire barriers are recorded with the same
  shape.
- vulkan_submit_stage_async() cleans up a failed submit (drop the stage
  buffer, recycle its command buffer slot) and
  reclaim_pending_stage_buffers() reclaims still-staged buffers from the
  renderer teardown instead of leaking them with the device.
- waylib_vk_texture_end_readback() keeps the acquire when the foreign release
  cannot be recorded (fail closed), so the texture is never sampled or read
  again without its matching release.
Make Qt Quick (QRhi) render the compositor canvas, layer buffers, and
cursor buffers directly into renderer-allocated DMA-BUFs, so every frame
is produced in one offscreen QRhi pass and handed to wlroots as a plain
wlr_buffer - no intermediate copies, and GLES2/Pixman paths keep their
existing behavior.

- WRenderHelper: wrap renderer render buffers and wlroots textures into
  QRhiTexture/QQuickRenderTarget objects with exact-format Vulkan image
  views (including the opaque alpha swizzle), record acquire/release
  ownership-transfer barriers around Qt's use of a render buffer, and
  defer retired render-buffer/swapchain destruction until the frame's GPU
  work has completed.
- WOutputRenderWindow/WBufferRenderer: run outputs and layers inside one
  synchronous offscreen frame (beginFrameChecked/endFrameChecked); release
  every render buffer before the frame is submitted and retire replaced
  swapchains afterwards; expose the last buffer per output viewport for
  cache sampling.
- WOutput::configurePrimarySwapchain()/configureCursorSwapchain(): hand
  the replaced swapchain out to the caller instead of destroying it
  immediately, so its buffers can outlive GPU work still referencing them.
- WSGTextureProvider/WSurfaceItem: keep a client buffer and its wlroots
  texture as one ownership tuple, rebuild Qt wrappers when the buffer
  changes, and defer texture cleanup to render jobs so GPU sampling has
  finished before destruction.
- Harden WOutputHelperPrivate::acquireBuffer(): this legacy path has no
  Vulkan callers and destroys a replaced swapchain immediately, which can
  race with GPU work still referencing its buffers; refuse it on the
  Vulkan renderer instead of introducing an untested retire mechanism.
- Add the renderer-backed linux-dmabuf v4 protocol wrapper and the VKTRACE
  logging helper (WAYLIB_VULKAN_TRACE) covering frames, passes, texture
  wrapping, sampling dispositions, cleanups, and output present/commit
  results.

Follow-up hardening pass (folded):
- Refuse to record wlroots' queue-family ownership barriers inside an active
  QRhi render pass: VUID-vkCmdPipelineBarrier-oldLayout-01181 /
  -srcQueueFamilyIndex-01182 forbid in-pass layout transitions and ownership
  transfers, so a barrier injected between QRhi passes by mistake would be
  undefined as well as mis-accounted. The render-buffer acquire/release
  wrappers and texture-sampling prepare now check
  QVkCommandBuffer::recordingPass (fail-closed, qCCritical + frame-fail
  fallback like the existing barriers).
- Call waylib_vk_renderer_reset_frame_alias_lists() from doRender once the
  frame's endFrameChecked has waited for the GPU, keeping the wlroots alias
  tracking per-frame.
- Escalate consecutive beginFrameChecked() errors (non-device-lost) to a
  fatal render error: those frames never reach endFrameChecked(), so retired
  resource cleanup would freeze while the compositor keeps skipping frames.

Follow-up hardening (folded): null m_handle in WLinuxDmabufV1::destroy() so
handle()/global() return null instead of a dangling pointer once wlroots has
destroyed the global from its display destroy listener.
Follow-up hardening (folded): close the offscreen QRhi frame even when the
flushTextureSyncBatch() synchronization before submission fails. Leaving the
frame open makes every later beginFrame() a silent no-op inside
QQuickRenderControl (Qt requires endFrame() first), freezing the output on the
last good image; and at shutdown the QRhi cleanup callbacks run while QRhi is
still inFrame, so a backend finish() from such a callback ends the abandoned
primary command buffer and aborts Mesa's radv in the amdgpu command-buffer
finalization (observed as the Waydroid shutdown crash with large
foreign-texture sync batches). endFrameChecked() now always runs after
beginFrameChecked() succeeded; its submission result only gates the normal
frame-completion path, matching the previous semantics on the flush-ok side.
Report wp_presentation feedback for the frames the Vulkan path commits:
WPresentation wraps wlr_presentation, and WOutputRenderWindow collects the
surfaces textured on the current output
(markSurfaceTexturedForPresentation) and submits
surfaceTexturedOnOutput() for them per output commit
(submitPresentationFeedbackForOutput), so clients asking for presentation
feedback get their texture/present events from the output that actually
committed the frame.

Follow-up hardening (folded): null m_handle in WPresentation::destroy() so
handle() returns null instead of a dangling pointer once wlroots has destroyed
the global from its display destroy listener.
Render the cursor QML item into a cursor-plane scanout buffer and hand it
to the KMS cursor plane, instead of always compositing it through the
canvas. This keeps the cursor cheap on the Vulkan path, where a canvas
repaint per pointer move would cost a full synchronous frame.

- SourceOutput.qml detaches the cursor item geometry from the live pointer
  position (pinned at (0,0)) when an output layer carries the cursor, so
  moving the pointer no longer dirties the main scene. The on-screen
  placement is computed from wlroots cursor/output coordinates during
  layer rendering, and moves go through move_cursor; woutputitem
  schedules an explicit output frame for the placement update, since the
  detached geometry no longer dirties the scene itself.
- OutputHelper::tryToHardwareCursor(): on Vulkan, re-render the cursor QML
  item directly into a cursor-format scanout buffer (sampling the already
  FOREIGN-released layer buffer would yield undefined contents), falling
  back to software compositing when no suitable cursor size exists or the
  plane rejects the buffer.
- Defer set_cursor() until the frame's command buffer has been submitted
  and completed (endFrameChecked waits on a fence): the buffer handed to
  the cursor plane is produced by that command buffer, so committing it
  earlier could let the plane scan out content the GPU has not rendered
  yet. move_cursor stays in-frame for pointer latency; the deferred commit
  is cancelled whenever the frame does not complete or the fallback to the
  software cursor is taken, and a failed deferred set_cursor keeps the
  cursor on the software canvas path until its content is re-rendered.
Advertise the legacy wl_drm global for Vulkan only when the renderer
exposes an implicit-modifier DMA-BUF texture format. This prevents
legacy-buffer clients such as Youdao Note from selecting DRM formats
the Vulkan renderer cannot import, which showed up as wrong colors and
flicker, while leaving the non-Vulkan wl_drm path unchanged.
Folded follow-ups: track live WRenderBufferBlitter item count
(WRenderBufferBlitter::liveCount()) and skip the per-beginRender scene walk
entirely when no blitter exists, and gate the Vulkan sampling prepare/finish/
barrier-batch-flush wrappers against QRhi passes (VUID-vkCmdPipelineBarrier-
oldLayout-01181 / -srcQueueFamilyIndex-01182) like the render-buffer paths.

Follow-up hardening (folded): VulkanDepthStencilAllocation::release() stays
away from QRhi and native vk when the backend still is recording a frame (e.g.
a shutdown forced by an earlier fatal render error), because finish() on such
a leaked frame would end the abandoned command buffer and can abort the
driver. The native image and memory are then left for the wlroots device
teardown, which reclaims them together with the device.

Follow-up hardening (folded): every BufferData that owns QRhi render-target
objects now registers a keyed QRhi::addCleanupCallback
(registerVulkanRhiTeardownCleanup). At shutdown the window tears the
QQuickRenderControl down while WRenderHelper still caches its render targets;
QRhi objects destroyed after the backend's VmaAllocator are unreclaimable and
vmaDestroyAllocator() aborts the process with "Some allocations were not
freed before destruction of this memory block!" (observed in Waydroid /
dde-lock shutdown cores). The callback runs from ~QRhi's runCleanup() - the
one point while the VkDevice is still alive - and calls resetWindowRenderTarget()
first.

Follow-up hardening (folded): WRenderHelper::addVulkanTeardownSweep() arms one
sweep callback per Vulkan QRhi (registered right after createRhi in
initRCWithRhi, and kept idempotently per BufferData): at ~QRhi it
force-destroys every still-registered Buffer/Texture/RenderBuffer QRhiResource
before the allocator dies. Such resources are produced by client scene-graph
items / QQuickWindow caches that outlive the render control and are outside
of waylib's ownership; Qt itself only reports them in ~QRhiImplementation,
which runs after the VMA assert would already have fired. The per-resource
qCWarning (type + Qt object name) makes any leaked owner identifiable in
normal logs. Qt's destroy() paths are null-checked/idempotent, so owners
deleting the C++ objects later stay safe.

Depth-less backdrop targets (folded): the backdrop clear/preserve/resume
targets are created without a depth attachment. WRenderBufferNode never
advertises DepthAwareRendering, so Qt keeps the 2D depth buffer disabled for
the whole batch renderer that contains a blitter; the attachment was only
adding a full-attachment depth load/store (and a depth-stencil pass barrier)
to every pass split. createRhiRenderTarget() gained a withDepthStencil opt-out
that only the backdrop set uses (all other callers, including GLES2, keep
their depth attachment exactly as before), the resume target no longer needs
PreserveDepthStencilContents, the native depth-stencil texture/allocation
machinery is deleted, and WRenderBufferNode::prepare() fails closed if a
depth-using renderer ever reaches the blitter.

Fast paths (folded): renderVulkan() returns early when the node has no content
(nothing consumes the backdrop snapshot, so the canvas copy and the pass split
are skipped), the blitter detection respects item visibility (an invisible
blitter cannot request the backdrop), and both scene-graph walks run
allocation-free (QVarLengthArray + childItems instead of per-beginRender
QList/QSet copies).

Teardown fix (folded): ~Node() hands the content subtree back from the clip
node before deleting it, so the clip node is not leaked when the node is
destroyed while clipping is active.
Folded follow-up: the NVIDIA forceSoftwareCursor workaround (pre-Vulkan,
wlr_egl context handoff) is now gated to the OpenGL renderer; the Vulkan
backend shares the wlroots VkDevice and defers set_cursor() until the cursor
buffer has been submitted and fenced, so forcing a canvas-composited cursor
there costs a full synchronous frame per pointer move. The workaround still
applies unchanged on GLES2 while a renderer is being selected at startup.
Folded follow-up for Vulkan-correct capture and preview paths:

- WBufferDumper::dumpBufferToImage() on the Vulkan renderer now takes the
  wlroots readback gate (waylib_vk_texture_begin_readback/end_readback): a
  DMA-BUF-imported wlr_texture is FOREIGN-owned and behind a producer fence
  between frames, so reading its pixels without the gate (wlr_texture_
  read_pixels -> upstream vulkan_read_pixels, which assumes a GENERAL
  layout and a queued producer) is undefined contents and a validation
  error. The gate acquires/releases the foreign ownership around the
  staging command buffer.
- SHM-backed buffers (XWayland, wlr_allocator shm) are read out directly
  via CPU pixel access; importing them as staging textures and letting the
  upstream read path walk into them is precisely the layout/ownership
  mistake avoided above.
- WTextureCapturer::doGrabToImage() on Vulkan previously read back the
  provider's QRhiTexture (a foreign-owned imported image) through QRhi; it
  now reads the backing wlr_buffer through WBufferDumper instead, so remote
  sources and portal previews never touch the image without a matching
  acquire. GLES2 readback is unchanged.

Follow-up hardening (folded):
- CaptureContextV1::failFrame() funnels every failure path through one helper
  that sends failed, drops the frame and disconnects imageReady, so a late
  imageReady can never emit a buffer after failed.
- WTextureCapturer::doGrabToImage() falls back to
  WOutputViewport::lastBuffer() when the provider has no buffer of its own
  (the primary output viewport on Vulkan), so previews keep working without
  touching the foreign-owned image; the WPointer-backed last buffer stays
  valid until the next output commit.
Rebuild the effect on RenderBufferBlitter: the blitter snapshots the
surrounding canvas into the effect's source texture, and the effect renders it
back through Qt Quick Effects (MultiEffect / glass), keeping the GLES2 blur
look while the compositor canvas stays a single offscreen QRhi pass.

Follow-up hardening (folded): the blitter item is now visible only while the
effect is enabled and its content is rendered onscreen, mirroring the content
loader's own condition (visible: effectEnabled && !contentOffscreen). Hiding
the blitter keeps WBufferRenderer::hasActiveRenderBufferBlitter() from
requesting the Vulkan backdrop, so a disabled or offscreen-rendered effect no
longer pays for a canvas copy and a render-pass split.
Clients select buffer formats that the compositor must import, wrap into
Qt, and sample. Two gaps let them end up with formats that render as a
blank white screen:

- Restrict the advertised dmabuf/shm texture format sets to the subset
  the Qt wrapper displays exactly:
  waylib_vk_renderer_restrict_texture_formats() rebuilds
  dev->dmabuf_texture_formats and dev->shm_texture_formats from a DRM
  format whitelist, and Helper::init applies it before wl_shm and
  linux-dmabuf hand the sets to clients. Verified against RADV: 17/40
  dmabuf and 17/31 shm formats kept - the dropped ones are the 16-bit
  packed, 3-channel, and YCbCr families QRhi cannot map.
- Wrap packed 10-bit A2R10G10B10 buffers (XWayland depth-30 windows,
  e.g. Command & Conquer: Mental Omega under Wine) exactly instead of
  rejecting them: Qt maps both 10-bit channel orders to RGB10A2 and its
  Vulkan backend derives an A2B10G10R10 initial view, which cannot be
  created on a non-MUTABLE A2R10 image. Construct the QVkTexture
  manually with the exact wlroots format and the alpha swizzle instead of
  createFrom().

XWayland selects buffer formats from the X visual depth and does not
consult the advertised sets, so the exact wrapping is required even with
the restriction in place; both mechanisms share the same goal - no
client-visible format may fall outside what the compositor can display.

Verified with Mental Omega: 817 XR30/AR30 wraps with zero discards, the
format sets restricted, and no other rendering behavior changed. GLES2
is not touched.

Folded follow-up placement note: WRenderHelper::resetVulkanFrameAliasLists()
definition sits directly after this commit's restrictVulkanTextureFormats()
(the call site and header declaration were folded earlier in the render
buffer integration commit).

Follow-up hardening (folded): both restricted sets are rebuilt into
temporaries and committed together, so a failure halfway through cannot leave
the dmabuf set restricted while the shm set stays untouched (or vice versa);
and the updateVKTexture() comment no longer points at the removed
attachment-only depth helper.
… die

Two Wayland-session shutdown crashes, both in the compositors' seat
teardown cascade (observed as Waydroid / dde-lock coredumps):

1. treeland_keyboard_state_notify_v1 attached a WScopedListener to the
   seat keyboard-groups wlr_keyboard events.modifiers signal, but never
   detached it: WServerInterface::destroy() only removed the global, and
   SeatsManager emitted seatRemoved after server->detach() had already run
   WSeat::destroy. wlroots wlr_keyboard_finish() hard-asserts an empty
   listener list, so every session exit with the module attached aborted
   with SIGABRT (wlr_keyboard.c:181). destroy()/~private now tear down the
   listeners and the SeatManager connections (restart-safe re-setup stays
   idempotent via the seat-keyed dedup in connectKeyboardGroup), and all
   three SeatsManager removal paths (removeSeat, loadConfig, destructor)
   emit seatRemoved before detach, so native listeners are unlinked while
   the keyboard is alive.

2. WSeat wrappers are registered as server interfaces and ~WServer
   qDeleteAll()s that list, while SeatsManager kept raw pointers and
   deleted the same objects in its own destructor: whichever cascade ran
   second jumped through a freed WSeat vtable (SIGSEGV inside
   ~SeatsManager). createSeat now hooks QObject::destroyed to prune
   m_seats/m_defaultSeat (the hook only ever compares pointer values),
   and ~SeatsManager disconnects the hooks and emits seatRemoved up front,
   so both destruction orders are safe.
@LFRon
LFRon force-pushed the feat-and-refactor/enable-vulkan-renderer branch from f30b0ee to afb0b6c Compare September 30, 2026 03:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants