Repository navigation
Custom shaders (shader engine 7B, first 7C slices) plus capture, ignore_alpha, focus and spring fixes - #4
Merged
Merged
Conversation
Capturing a single toplevel rendered the raw window through AsRenderElements with only its resolved opacity, so a frosted or water-glass window lost its glass the moment OBS's window source or a per-window screenshot captured it alone. Output-level captures were fine because they already go through desktop_render_elements. The toplevel branch now asks glass_layer_elements for that one window and relocates the result from its output-local rect onto the capture canvas, behind the window's own surfaces, in the same order the live desktop uses. block_capture still skips both the surfaces and the glass.
Parses the opt-in custom effect surface from SHADER_DESIGN.md without
any render path yet: a `shaders { enabled }` master that defaults off,
and named `shader "name" { stage { file, params { } } }` definitions.
A later definition with the same name replaces the earlier one whole,
so a stage list is always the written order of one block. Anything the
first render slice can't honor as written (other scopes, sources or
invalidation modes, chains, save/textures, bad params, the 32
definition / 4 stage / 8 parameter caps) rejects the definition with a
panel warning instead of rendering something other than what was asked.
shader_effect.rs holds the fixed contract. A user file defines
`vec4 tide_effect(vec2 uv)`; a token-aware check rejects host-owned
directives, declarations and identifiers, and the host wrapper adds the
version, Smithay's defines marker, every uniform, rounded clipping,
opacity and main, with #line mapping back to the user's file. Files load
at config time relative to the main config, through a nonblocking open
that refuses FIFOs and devices, 64 KiB per file and 1 MiB per config.
Each path joins source_paths, so the existing watcher reloads on a
shader edit or late creation.
`rule { shader = name }` names a custom effect definition and
`shader = none` clears an earlier match; the last matching rule wins,
and an invalid name warns and leaves the earlier value alone. Each
mapped window's assignment lands in window_shader_assignments, which
follows window_glass_modes exactly: the same refresh sites, cleared on
reload and evicted in detach_mapped_toplevel. The name is kept as
written and resolved per frame, so toggling the master switch or
editing a definition needs no per-window re-resolve.
This is the first slice that puts a custom shader's pixels on screen. A
window whose rule assigns a loaded single-stage definition, with
`shaders { enabled = true }`, now gets a backdrop capture regardless of
`water_effects`, and glass_layer_elements draws the wrapped `.frag`
through compile_custom_texture_shader in the glass pass's z-slot instead
of water or frost. Without a program the window falls back to its
normal glass path, and `water_effects` keeps its old meaning there.
Programs are cached per definition name together with the parameter
schema they were built for. A changed source compiles once, at most one
new program per glass_layer_elements call and frame. A failed source is
never retried until it changes, and the last good program keeps drawing
until then. A failure goes to the persistent warning panel from an idle
callback, so the panel's reserved strip never changes mid-frame, and a
reload re-reports a source that still fails. Removing a definition
drops its program.
u_time and u_delta come from a small per-window clock that only samples
when the element's content commit changes, so time alone never
schedules a redraw. The clock is evicted on unmap, pruned on reload and
cleared on lock. The per-window capture path from 7B-i picks the effect
up through the same function.
The shader_effect tests compile and draw through Mesa's software EGL
device where one exists: passthrough is pixel exact and upright, params
bind, the host clamps premultiplied RGB, applies opacity and rounding,
reports a real driver compile failure once and keeps the last good
program. They skip when no software device is present.
The engine stays undocumented in DOCUMENTATION.md until Phase 7B is whole, so this entry says plainly what is owed and that the GLSL tests run on a software renderer, not hardware.
Showing a window's glass in its own capture meant showing its backdrop, and that texture is shared with the live frame: a block_capture window or layer behind it would reach the capture through the glass even though capturing the blocked window directly renders nothing. When any blocked window or layer overlaps the captured window, its glass is now left out of the capture instead.
Each capture pass now admits shaded windows against the design's caps before anything allocates: at most 128 windows drawing a custom effect, and 64 MiB of their captures counted at each definition's render_scale. A refused window takes its normal path, shows one warning on the panel and is admitted again once there is room. A shaded window's backdrop is now captured at its definition's render_scale (default 2) instead of the glass-wide backdrop_capture_scale, so that key finally means what it says. `tidectl perf` gains a custom_shaders object (definitions, programs, compiles, compile failures, instances, capture bytes, bypassed windows, total bypasses and content updates), and the text summary prints one line for it while the master switch is on.
Smithay reports a failed compile as a bare ShaderCompileError and sends the driver's text only to the journal, so the panel could say that a shader broke but not why. After a failure the fragment is compiled once more on its own to read the info log, and its first error line goes to the panel, bounded to 240 bytes. Mesa's `1:LINE(COL)` prefix is rewritten to the user's file line, since source string 1 is the user file in the wrapper. A reload re-reports the same reason while the source still fails. Cached programs are now scoped to the renderer's context ID the same way caustics already are, so a replacement renderer never draws with a program from the old context.
`workspace_rule { shader = name }` gives every window on that workspace
a custom effect unless its own rule says otherwise, and a window rule's
`shader = none` now opts out of the workspace default instead of being
dropped. This matches the design's order: workspace default, then the
last matching window rule, with the master switch over both. The
default is read live from the window's current workspace, so moving a
window re-resolves it on the next frame. A borrowing lookup keeps the
per-frame resolution from cloning rules.
A definition may now have up to four stages. Every stage but the last
renders into an offscreen target sized to the window's capture, in
written order, right after the backdrop capture and before the visible
bind, where offscreen work is safe on both backends. The last stage
draws in the glass slot as before.
A stage reads the previous stage's output as `tex` by default, or
`source = backdrop` / `source = "get:name"` for the backdrop or the
output of an earlier stage that set `save = "name"`. `textures { }`
binds up to four more sampler2D uniforms from the same sources, on
texture units 1 to 4. Reading a name no earlier stage saved, saving one
name twice, or saving from the last stage (which draws on screen) is
rejected at load time, so the written order is the whole dependency
graph and no cycle check is needed.
Intermediate stages rerender only when the capture, a stage program or
a parameter changed. Their u_time/u_delta come from the same clock the
last stage reuses, and they are never clipped or faded; the last stage
applies rounding and opacity once. Programs are cached per stage.
Admission counts one extra capture-sized target per offscreen stage
against the 64 MiB budget, and `tidectl perf` reports stage bytes.
Smithay sets sampling state only on the texture it binds to unit 0, so
an extra unit now gets linear, clamped sampling explicitly. Without it
a stage output read only through `textures { }` sampled as an
incomplete, black texture, which the new GL tests caught.
The layer alpha mask is an offscreen target that is only ever sampled
through texture unit 1. Smithay sets sampling state only on the texture
it binds to unit 0 at draw time, so the mask kept GL's mipmapped default
filter, sampled as an incomplete texture that reads alpha 1 everywhere,
and `layer_rule { ignore_alpha }` frosted transparent regions exactly as
if it were unset. Binding the mask now sets nearest, clamped sampling.
A new test renders frost through Mesa's software GLES with a fully
transparent and a fully opaque mask: before this change the transparent
one still frosted.
share/shaders/ gains tint.frag, a separable Gaussian blur split into
blur-h.frag and blur-v.frag for a two-stage chain, and edge-mix.frag,
which shows an extra `textures { }` binding. A test checks each against
the fragment contract and compiles it through Mesa's software GLES
where available, and checks that the documented two-pass blur keeps a
flat input flat. Another test evaluates the documentation's two config
examples verbatim, so the documented syntax can't drift from what
parses.
DOCUMENTATION.md gets a Custom shaders reference: the master switch, definitions and stages, the fragment contract and uniform table, a porting note for Hyprland-style screen shaders, errors and reloads, budgets, and how effects appear in captures and on lock. The rule and workspace_rule tables gain `shader`, the `water_effects` row notes that custom shaders keep their own capture, and the tidectl perf paragraph mentions the new summary. README mentions the feature once, and the CHANGELOG entry now describes the finished Phase 7B instead of the groundwork, alongside the ignore_alpha fix.
A window over the instance or payload cap still drew its custom effect whenever it was also eligible for glass, because the glass pass never checked the bypass set. It does now, and a bypassed window also drops its effect state and stage textures. Admission only counted windows that already had effect state, so turning the feature on over many large windows admitted all of them in one pass before any cap applied. Windows admitted earlier in the same pass now count at their wanted size, which makes "checked before anything allocates" true. Effect state and stage textures are released together with the window's capture when it is hidden or leaves every output, so they no longer pin VRAM or count against the instance cap. A chain's version hashed the capture's version, which restarts in every new capture, so a window whose capture was recreated could skip rerendering and show stale stage output. The capture's identity is part of the key now, and so are each stage's `source` and `textures` bindings, for the chain and for the last stage's commit. Rewiring a stage without editing its file therefore redraws, and the cached program adopts the new binding sources along with new param values. Repeated panel warnings are no longer appended again, so a window flipping in and out of the budget can't grow the panel without bound. Six or eight hex digits with a letter now read as a color param even when they would also parse as a float (`1E0000`). The three copies of the shader resolution collapse into Config::shader_for, which also puts the master-switch check in one place, and workspace_shader no longer sits between resolve_workspace_rule and its doc comment.
cycle_focus raised whichever window it focused. Tiled windows never overlap each other, so raising one only ever lifts it above the floating windows that overlap it, breaking the floating-above-tiled invariant until the next retile re-raises them. Every other raise site already skips tiled windows; this one now does too, the same way directional focus does. It is the likely cause of a floated window ending up behind its tiled sibling in the 2026-09-12 live session.
The 2026-09-12 live pass found that an out-of-range spring and one that never settles logged the same generic easing warning, so there was no telling which to fix. SpringCurve::try_new now names the failed check and the three curve keys log that reason when the value is a spring.
`layer_rule { shader = name }` gives a bar, launcher or other layer
surface a custom effect over its backdrop, the first 7C slice from
SHADER_DESIGN.md. Layers only ever get an effect from an explicit rule;
they never inherit a window or workspace shader. The last matching rule
with a `shader` wins and `shader = none` clears it. With `blur = true`
too, the effect replaces the layer's frost, which comes back while the
shader has no compiled program.
The layer capture pass resolves each mapped layer's assignment into
layer_shader_assignments, so the shared resolver, budget admission, the
render_scale capture divisor and the pre-bind chain runner all work for
layers unchanged. The last stage draws without rounding and honors
`ignore_alpha`: the custom element now takes the layer's composed alpha
mask on its own texture unit, and a pixel whose layer alpha is under the
threshold draws nothing, so a transparent bar region is never shaded.
A software-GLES test covers a transparent and an opaque mask. Admission
now takes a candidate list so windows and layers share it.
Both visible backends extend the frame's glass layers with
layer_glass_elements, but the output-capture path in capture.rs built
only window glass, so a `layer_rule { blur = true }` bar appeared
unblurred in every screenshot and screencast. The capture path now adds
the same layer glass, which also carries layer shaders.
Output captures blacked out a block_capture layer's rect only after rendering, while frost or a displacing shader over it had already spread its pixels past that rect, and adding layer glass to captures made this reachable for bars. Glass overlapping a blocked layer is now left out of output captures. A layer or window that lost its shader but kept frost or glass also kept its effect state and stage textures, which still counted against the budget. Effect state is now dropped whenever the surface no longer resolves an effect, not only when its capture goes. The compile budget was per call site, so the chain passes, the glass slots and a screenshot could each compile a program in one frame. It is now one budget per output frame, reset at the start of the capture pass and shared by every consumer until the next. The window and layer last stages shared about eighty duplicated lines; one custom_shader_last_stage helper serves both, and one surface_shader resolver backs every lookup, so layer assignments go through Config::shader_for like windows. Layer assignments are cleared on reload so a screenshot served before the next frame can't use a removed effect. With shaders and frost both off the layer pass returns at once again, and the window pass builds no candidates while shaders are off.
Fi3w0
changed the title
`source = "shader:<name>"` starts a definition from another definition's output, so a blur can be tinted or edged without copying its stages. The source's stages, recursively expanded, are spliced in front at load time, after files have loaded so each file is read once. Its last stage saves under a host-reserved `tide_source_N` name, the definition's first stage reads it positionally, and `backdrop` in the definition's own stage sources and texture bindings becomes that saved output. The source's own stages still read the real backdrop, so the chain runner, program cache and budgets need no changes. An unknown source, a loop, a save name the source already uses, or a spliced chain over four stages leaves the definition out with a warning. A software-GLES test checks that `backdrop` in a sourced definition samples the source's output rather than the capture.
Fi3w0
changed the title
depth { enabled } now defaults to false in DepthConfig and in the
generated first-run config, which gains an explicit
depth { enabled = false } block. Unfocused windows no longer fade into
the blue cool_color wash after 30 s or turn into title cards after 60 s
unless a config opts in with enabled = true.
depth_defaults_match_generated_config asserts the disabled default for
both sources; the tier-boundary test now enables depth explicitly
instead of relying on the default. DOCUMENTATION.md and CHANGELOG.md
updated.
WaylandDndGrabHandler was an empty impl, so Smithay's default dnd_requested cancelled every drag source immediately. Dragging files, text or links between apps did nothing, and Firefox treated every tab drag as a drop outside the window and detached the tab. Start a DnDGrab for pointer and touch drags, keep the client's icon surface, draw it at the pointer with the cursor elements on the udev backend (skipped while locked), send it frame callbacks, and clear it on drop or cancel.
Offscreen captures render at scale 1.0 in output-local physical pixels but used the on-screen wallpaper element, which is sized in logical pixels and relies on the frame's scale. On a 1.25x output the wallpaper inside glass, layer glass, shader and workspace-transition captures was drawn at 80% size from the top-left, so frost showed a shrunken wallpaper shifted away from the real one. Captures now build the wallpaper at the output's physical size, rounded like the visible frame.
DOCUMENTATION.md says 0 disables cursor hiding, but the key used the duration parser that rejects zero, so cursor_hide_after = 0 (or 0ms) was ignored with a warning. Use the zero-allowing parser that exists for exactly this case.
spawn split commands on whitespace only, so no keybind, startup spawn or tidectl spawn could pass an argument containing a space, and the doc comment's advice to use sh -c "..." could never work. The splitter now understands single quotes, double quotes and backslash escapes, still without any shell expansion; an unterminated quote falls back to the old whitespace split so existing configs keep working. tidectl quotes spawn arguments that need it when joining them, parses --json/--socket/--help only before the command so a spawned program's own flags reach it, and treats `tidectl help` as help instead of sending it to the compositor as an unknown action.
The gpu check probed three hard-coded paths and listed /dev/dri/card0 under "render nodes". Enumerate /dev/dri instead, and report renderD* nodes and card* nodes separately, so a card1-only machine is seen and the labels are right.
Requests were only handled once a newline arrived, so a client that closed its write side after an unterminated line got no reply at all: both a truncated request and a complete one sent without a trailing newline (printf ... | socat) were silently dropped. Treat EOF as the end of the last line, so the request is answered or rejected with the usual invalid-request error. Only the read step changed; the request handling moved out of the read match unchanged.
pointer_modifier had no row in the top-level settings table, so nothing said it is separate from @mod and defaults to SUPER (the cause of Alt+drag not moving windows with an older config). The spawn key, spawn: action and --spawn flag now describe the quoting rules.
output { } blocks only took effect when an output was set up, so a
reload that moved, scaled or rotated a monitor was accepted and then
ignored. The wlr-output-management apply path (layout validation,
transform/scale/position change, floating-window translation, layer
re-arrange, retile) is now a shared apply_output_changes() that config
reload uses too. Outputs without a block keep their state. A different
mode or enabled = false still needs a modeset at output setup, so those
now produce a notice on the warning panel instead of being ignored
silently. The transform conversion moved to
OutputTransformConfig::to_transform() for both callers.
TECHNICAL_REPORT.md: the standalone DRM backend is verified on a real RTX 3060 with two outputs at mixed scale; drop the open "Nvidia native run" item. report.md: the validation pass with IPC robustness, idle and nine-glass-window cost, the Nvidia PSS breakdown (driver mappings, no leak), the fixes found during the pass, and what is still open.
Portal screen sharing from native Vesktop through the TideWM ScreenCast backend worked on the RTX 3060 setup. Note that src/screencast only logs failures, so a successful session leaves no trace in the log.
Fi3w0
marked this pull request as ready for review
September 25, 2026 01:20
Fi3w0
marked this pull request as draft
September 25, 2026 01:21
Fix standalone Nvidia issues found during hardware validation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this does
This PR finishes shader engine Phase 7B from
SHADER_DESIGN.md(v3), turns on the public, documentedshaders { enabled }switch, and starts 7C with explicit layer assignments and definitions that build on other definitions. It also fixes the bugs listed under Fixes, all found along the way. It rebuilds 7A, 7B-i and 7B-ii from scratch, because the local3b86703/de39f05/8296f62commits weren't pushed. Expect conflicts with those when rebasing; names follow the roadmap ledger (shaders_enabled,shader_definitions,window_shader_assignments,resolved_shader_assignment,validate_fragment_contract) to make the mapping easier.Custom shaders (7A, 7B-ii to 7B-iv)
shaders { enabled }(default off, independent ofwater_effects) gates namedshader "name" { stage { file = "x.frag"; params { } } }definitions. A window selects one withrule { shader = name }, or a workspace withworkspace_rule { shader = name }. A window rule, includingshader = none, wins over the workspace default.vec4 tide_effect(vec2 uv). A token-aware check rejects host-owned directives, declarations and identifiers. The host wrapper owns#version, Smithay's defines marker, every uniform,main, premultiplied clamping, rounding and opacity, and maps driver line numbers back to the user's file with#line..fragonly, nonblocking open that refuses FIFOs and devices, 64 KiB per file, 1 MiB per config. They joinsource_paths, so saving a shader hot-reloads it.ContextLostrule). Stages chain positionally, cansave = "name", and read saved outputs throughsource = "get:name"ortextures { name = "get:name" | backdrop }(up to four extra samplers on units 1 to 4). Forward references, double saves and saving from the last stage are rejected at load, so the written order is the whole dependency graph.render_scale(default 2) sets each effect's capture divisor. Over-budget windows render normally with one warning. Lock clears all effect state.tidectl perfgains acustom_shaderssummary.layer_rule { shader = name }, never a window or workspace one. The effect replaces the layer's frost if it also blurs, and the rule'signore_alphamasks it out of transparent pixels through a composed-alpha mask on its own texture unit.source = "shader:<name>"starts a definition from another one's output, sosoft-blurcan be tinted without copying its stages. The source's stages (recursively) are spliced in front at load time, its output is saved under a host-reservedtide_source_Nname, andbackdropin the definition's own stages means that output. The chain runner, cache and budgets are unchanged. Unknown sources, loops, save clashes and spliced chains over four stages are rejected with a warning (5c63b90).scopeother thanwindow,invalidate = always|manual) are rejected with a visible warning instead of being half applied.share/shaders/shipstint, a two-pass Gaussianblur-h/blur-v, andedge-mix. DOCUMENTATION.md has a full "Custom shaders" reference, and README mentions it once.Fixes
f1c58c4): capturing one window (OBS window source, per-window screenshot) dropped its frost or water glass. It now relocates the live glass layer onto the capture canvas.2baa205): the shared backdrop could show ablock_capturewindow or layer through the glass, so the glass is left out of a per-window capture whenever blocked content overlaps.layer_rule { ignore_alpha }never masked anything (dc6a2ee): the mask is only sampled on texture unit 1, and Smithay sets filtering only on the unit-0 texture, so the mask was an incomplete texture that reads alpha 1. A GL test reproduced this before the fix.567107b):cycle_focuswas the one raise site without a tiled check. This is the likely cause of the "floated window behind its tiled sibling" note from the 2026-09-12 live session, but it's unconfirmed live.7c410cb): both live backends add layer glass, butcapture.rsonly built window glass, so a blurred bar showed up unblurred in every capture. Glass overlapping ablock_capturelayer is now left out of output captures (0b65d1e), since blur or a displacing shader would spread its pixels past the blacked-out rect.75ff5c0): a refusedspring(...)curve now says whether a coefficient was out of range or it never settles.The branch also went through two code review passes. The first round of fixes is in
a3a2b2f: bypass enforcement, same-pass admission, stale chain versions, rewiring detection, VRAM release for hidden windows, deduplicated warnings, and1E0000-style color params. The second is in0b65d1e: one compile budget per output frame, effect state dropped when a surface loses its shader, one shared last-stage helper and resolver for windows and layers, and layer assignments cleared on reload.How it was verified
No nested or native compositor was launched; the live-session notes say not to. Everything below is automated.
cargo fmt --all -- --checkis clean.cargo test --locked --all-features --all-targets: 539 compositor + 12 tidectl + 9 wavefmt tests pass (master had 480 compositor tests).cargo clippy --locked --all-targets --all-features -- -D warningsis clean, apart from onenonminimal_boolin existing code (smart_attach_ocean_floating, from 650284c). The toolchain in the cloud session (clippy 1.94) flags it; CI's stable clippy passed on master, so I left it alone.cargo build --release --locked --all-featurespasses.The shader and frost GL tests create a headless
GlesRendereron Mesa's software EGL device (llvmpipe) and really compile and draw. They check that passthrough is pixel exact and upright, thatuv(0,0) is top-left, and that params, opacity, premultiplied clamping and rounding apply. Chains carry outputs forward in order and upright,save/getand extra units bind correctly, driver errors map to the user's line, the last good program is kept, rewiring or a new capture counts as a change, the shipped examples compile, a sourced definition'sbackdropsamples its source's output, andignore_alphamasks a transparent layer for both frost and a layer shader. The tests skip where no software device exists. This is a real GLSL ES compile and draw, not a hardware pass.The documentation's config examples are parsed verbatim in a test.
cargo fmtandcargo clippypasscargo testpassesBehavior verified live (described above), or this is a pure-logic change
TEST NEXT LIVE SESSION (native udev/DRM):
.fragand the shipped blur chain on a kitty rulewater_effects = falsewith a shader still drawingblock_capturewindowignore_alphaon a transparent full-output layer, with frost and with alayer_rule { shader }source = "shader:soft-blur"definition over the shipped blurAnything touching the RAM budget?
Yes, but only when opted in. With
shaders.enabledoff, or no window assigned, nothing new is allocated. The per-frame cost is a few map lookups per placed window. A shaded window allocates its backdrop capture atrender_scaleplus one capture-sized target per offscreen stage, all within the 64 MiB and 128-window caps. Stage targets are released with the capture when the window is hidden, unmapped, bypassed or locked. Drawing a custom element builds one small uniformVec, and only on redraw. Program sources are sharedArcs, so the steady-state frame compares pointers and doesn't hash. I have no release-build PSS numbers because there was no hardware in this session; please measure withtidectl perf(see itscustom_shadersobject) during the live pass.