Skip to content

Luau art: a home visual below the composer and a generative new-thread background - #924

Draft
jsgrrchg wants to merge 3 commits into
zeronsh:mainfrom
jsgrrchg:zeron/luau-custom-art-support
Draft

jsgrrchg wants to merge 3 commits into
zeronsh:mainfrom
jsgrrchg:zeron/luau-custom-art-support

Conversation

@jsgrrchg

@jsgrrchg jsgrrchg commented Oct 16, 2026 •

Copy link
Copy Markdown
Contributor

Builds on the native Luau visuals from #900 to let users put their own art on the new-thread view. There are two new slots, both set in Settings → Appearance:

  • Home visual: any visual script, drawn below the blank new-thread composer. It can be interactive (canvases, scenes, shaders, controls).
  • Generative background: a ui.shader script as an alternative to the background image, drawn behind the composer. It moves but takes no input.

Summary

Home visual

  • new_thread_visual (device-local) holds the script. Apply runs zeron_visual::check off the main thread and saves only if it passes; errors come back with their line. Clear removes it.
  • VisualFrame::from_script runs a script already in hand with the same runtime as agent visuals.
  • The shell places it from last frame's composer and column bounds. It fades with the Home selectors, leaves room for the bottom notices, clips at the terminal, and fades out at its bottom when it is taller than the room.
  • ComposerInput gains a CodeInput key context, where Enter breaks the line, and a monospace option for code fields.

Generative background

  • A background source setting, Image or Generative, sits next to the image. The image and the script are both kept, so switching loses neither. Choosing or shuffling an image switches back to Image.
  • Apply checks that the script runs and returns a ui.shader without assets, and samples one frame's color for "Match wallpaper colors".
  • Frames render at the background's size and paint through the same composer mask, fades, frost opacity, and dock dissolve as the image. Effects and framing don't apply.
  • Cost controls:
    • frames render at half the display density, at most 1600 px on a side;
    • at most 30 fps, with one frame in flight;
    • frames render only while the background paints;
    • reduced motion and "Pause animations in background" hold a still frame.
  • Without a GPU, the image shows instead.
  • Readiness crossfades only between sources. A live source's frames replace each other in place, including mid-crossfade.
  • The voice stage shows the same background.

Shared pieces

  • ShaderPlayer is extracted from VisualFrame. It renders one shader's frames for agent visuals and for the background.
  • ShaderSpec::fade_edges lets a host that masks the frame itself turn the edge fade off. Mobile and agent visuals keep it on.
  • ScriptCard with a ScriptSpec is the settings card both slots share: code field, Apply, Clear, check errors.
  • new-project-fixture takes ZERON_FIXTURE_HOME_VISUAL=<file.luau> and ZERON_FIXTURE_GENERATIVE=<file.luau>.

Test plan

  • cargo test -p zeron-ui --lib -- settings:: composer:: visuals::: 357 passed.
  • Focused run, -- new_thread_generative new_thread_background_effects settings::tests visuals: 89 passed. It includes the new tests:
    • the first shader is the background, and scripts without one are rejected with the error;
    • the background takes no pointer and spins on its own;
    • live frames replace in place, and only switching sources crossfades;
    • settings round-trip and stay omitted when unset.
  • cargo test -p zeron-visual-gpu -- light_fades_out: a host-masked frame keeps its edges.
  • cargo check for zeron-ui, zeron-visual-gpu (tests and examples), and zeron-mobile.
  • Manual, new-project-fixture on Xvfb with lavapipe:
    • an animated shader and an animated canvas draw below the composer;
    • Apply shows a broken script's error and keeps the saved one;
    • a Pac-Man canvas game with an autopilot demo takes clicks to play;
    • a halftone "tokyo rain" shader renders as the background, masked and faded like the image, and animates.
  • Not done: a real (non-fixture) app run, light themes, macOS/Windows, and an agent MCP tool for setting either script.

View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

…poser

Settings → Appearance → Home visual takes a Luau script in the format
agents use with show_visual and draws it in the room between the blank
new-thread composer and the window bottom.

- The script is checked with zeron_visual::check off the main thread
  before it is saved, so a broken script never reaches the home view;
  its error comes back with the line. Clear removes it.
- The setting (`new_thread_visual`) is device-local and omitted from the
  file when unset.
- VisualFrame::from_script runs a script already in hand, with the same
  runtime as agent visuals (canvases, scenes, shaders).
- The shell places it from last frame's composer and column bounds,
  fades it with the Home selectors, leaves room for the bottom notices,
  clips it at the terminal, and fades a visual taller than the room out
  at its bottom instead of cutting it off.
- Code fields: a CodeInput key context where Enter breaks the line, and
  a monospace ComposerInput option.
- new-project-fixture takes ZERON_FIXTURE_HOME_VISUAL=<file.luau>.
…omposer

Appearance gains a background source, Image or Generative. Generative
shows the user's Luau script, whose ui.shader renders at the background's
size and paints through the same composer mask, fades, and crossfade as
an image. It moves (time, spin, feedback buffers) but takes no input.
Both the image and the script are kept, so switching loses neither.

- Applying the script checks that it runs and returns a shader without
  assets, and samples one frame's color for "Match wallpaper colors".
  Choosing or shuffling an image switches back to it.
- Frames render at half the display's density (at most 1600 px on a
  side), at most 30 per second, one in flight, and only while the
  background paints; reduced motion and background pause hold a still.
  Without a GPU the image shows instead. Effects and framing don't apply.
- Readiness crossfades only between sources: a live source's frames
  replace each other in place, including mid-crossfade.
- ShaderPlayer, extracted from VisualFrame, renders one shader's frames
  for agent visuals and the background alike.
- ShaderSpec::fade_edges lets a host that masks the frame itself turn the
  edge fade off.
- The script settings card is shared: ScriptCard with a ScriptSpec for
  the home visual and for the generative background.
- The voice stage shows the same background.
- new-project-fixture takes ZERON_FIXTURE_GENERATIVE=<file.luau>.
Frames rendered at half the display's density and were capped at 1600 px on
the longest side, so on a 1x display a large window stretched the art two to
three times and fine detail (halftone dots, neon lettering) went soft. Render
at the window's logical resolution instead, capped by area (about 4 MP) so a
wide window scales down evenly. 2x displays render as before.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant