Skip to content

Tao: NativeView/TextureView headful monkeys and the embed bugs they caught - #659

Merged
kdroidFilter merged 5 commits into
nucleus-2.6from
feat/tao-native-view-monkeys
Sep 7, 2026
Merged

Tao: NativeView/TextureView headful monkeys and the embed bugs they caught#659
kdroidFilter merged 5 commits into
nucleus-2.6from
feat/tao-native-view-monkeys

Conversation

@kdroidFilter

@kdroidFilter kdroidFilter commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

New stage-2 headful cases (decorated-window-tao), run on X11 (Xvfb + openbox, AWT Robot) and on a native Wayland session:

  • NativeView monkey: a BasicTextField, a Compose button, a real native text widget embedded through NativeView (GtkEntry / NSTextField / EDIT, handed out by new nativeDiag* bridge entry points) and a Compose button drawn over the embed. Alternating click storms, a right click on the embed, a resize storm (discrete, burst, animated, interactive corner drag) and a 150-action random walk, each with an in-process driver and a real-pointer Robot driver. Asserts Compose keeps counting clicks, a single keyboard owner, letters and caret keys landing where the focus says, a stable I-beam over the field, the embed on its Compose slot, no leaked probe, a live main dispatcher.
  • TextureView monkey: four TextureViews fed by the platform test producers from their own threads plus an in-process renderer on the scene's Skia context (rememberTaoGpuRenderContext); mount/unmount/swap/close-under-view/hide-show/minimize/DPI/frame bursts.
  • Shared MonkeySupport (dispatcher watchdog, journal echoed to stderr, seed), PointerDrivers, NativeProbe; -Dnucleus.tao.headful.monkeyScript replays a journal; the suite filter accepts a|b.

Bugs the monkeys found, all fixed in this PR:

  • NativeView disposed the platform view before detaching it (SIGSEGV in nativeDetach); nativeViewHost() returned a new object per call (detach/attach churn on every recomposition of the window root); a late setFrame after detach touched the widget.

  • Linux: a right click forwarded to an embed lost its release to the widget's context-menu grab and every later Compose click was dead. The host records buttons forwarded to an embed, heals from GDK's live button mask on the next motion and releases phantoms before a new press.

  • Linux: printable keys never reached a focused GTK embed (toplevel IME + Stop), and arrows moved GTK focus into the embed (move-focus binding). Tao propagates keys to a foreign focus widget and stops the event once Compose consumed it.

  • Linux: GTK's map-time default focus landed on the embed (two carets). Focus sink, keyboard reclaim on a Compose-kept press, clearFocus on a forwarded press.

  • Linux: the embed trailed its slot through a resize; nativeSetFrame now relayouts the overlay synchronously (0 px behind measured, was 12–40 px).

  • Linux: SIGSEGV in Skia after a Wayland hide/show for rememberTaoGpuRenderContext consumers — the old GL host bound the new EGL context for a closed DirectContext.

  • All platforms: native popup layer windows leaked when an owner window closed during a Dialog's disappearance animation, leaving an invisible click-eating window. Hosts now close surviving layers on detach.

  • Linux/Wayland: the embed peeled off the Compose hole by a frame through a resize (a sub-surface position is parent state, applied on GTK's commit, one paint after Compose swapped its desync buffer). During a resize burst with an embed the content sub-surface runs in set_sync mode and a toplevel draw is queued after each swap, so hole and embed land in the same commit.

  • Linux/Wayland: nativeSetContentOffset committed GTK's toplevel surface out of band; that races GDK's end_paint/after_paint buffer bookkeeping and aborts the process in cairo (reproduced by the texture monkey after minimize/restore cycles). It now queues a GTK draw instead.

Windows, once the monkeys were run there (the native probe hands out a plain EDIT control):

  • A press forwarded to an embedded child HWND is replayed to the owner HWND a moment later — forwarding moves Win32 focus and the queue hands the message to the window that owns the pixels. Dispatched twice, Compose held a press with no release and every later click was dead. The host recognises the replay of the overlay event it just handled and drops it, matched on button, position and recency.
  • The child captures the mouse on that press and keeps every later message, so neither the scene nor its blending overlay sees the pointer again. The capture is handed straight back, and a release the child swallowed is healed from Win32's own button state.
  • Keys kept going to an embed clicked into earlier while Compose showed a focused text field. A press Compose keeps takes Win32 focus back (the macOS makeFirstResponder equivalent), and a press handed to the embed clears the Compose focus.
  • A child's handler may run a modal loop — an EDIT opens its context menu from WM_RBUTTONUP and does not return until dismissed. Everything but the press is now posted rather than sent, and the Tao loop drains the main dispatcher on a wake: Windows derives MainEventsCleared from a WM_PAINT that a modal loop never generates, so the app's coroutines used to stop for as long as the menu was up.
  • The overlay input replay repeated whatever message it saw last regardless of what the caller was forwarding; it only replays the matching one now.
  • The scene placed a click where the last move it saw left the pointer. Tao reports a button without a position and drops a WM_MOUSEMOVE whose coordinate equals the last one it saw, while every move over an embed goes to the blending overlay, which Tao knows nothing about — a pointer that left the embed and came back to a point Tao had seen before got no move at all, so the click landed on the embed the user had just left, and every click after it was dead. When the position came from the overlay the host asks Win32 where the pointer is and walks the scene there first.
  • A native modal loop on the loop thread stops Tao producing MainEventsCleared, which is where the Windows redraws asked for during a batch were served: the window froze for as long as an embedded EDIT's context menu was up, with no frames, no recomposition, and anything queued behind a frame never running. The wake that drains the dispatcher inside such a loop now serves those redraws too.
  • A press handed to an embed cleared the Compose focus through the frame queue, so when that press was the one opening the embed's menu the clear waited for a frame that could not come and the field kept its caret beside the embed's. It goes through the main dispatcher instead.
  • Handing an event to a child HWND runs its handler on this thread, and anything it pumps swallows the redraw the window had pending; the coalescing latch then suppressed every later request. Reset after a forwarded dispatch, like the other nested-pump callers.
  • Closing a native popup layer is idempotent now: the detach sweep above closes surviving layers itself and Compose then disposes the same layer as the composition unwinds, which released the native panel twice and took the process down.

macOS (real desktop, both drivers):

  • NSTextField.mouseDown: / NSTextView.mouseDown: run trackMouse:untilMouseUp: (and a right click may pop an NSMenu) on the Tao thread. A synthetic press never returned, and the suite watchdog halted after 900s on the first NativeView case. Tracking and menu selectors are skipped; a press only makeFirstResponders. Compose focus is cleared on a forwarded press, a Compose-kept press takes first-responder back, synthetic keys go to the embed when it holds the keyboard, and a late setFrame after detach is refused.
  • nucleusIOSurfaceTextureSource now retains the IOSurface. Closing the producer under a live TextureView used to free the surface; remounting then SIGSEGV'd in IOSurfaceGetPixelFormat.

Test plan

  • native view and texture view monkey filters green on X11 (Robot included) and on Wayland
  • full X11 headful suite: 235 run / 53 skipped; remaining failures are the #582 clipboard cases (wl-copy on the box) and two known WM-placement flakes that pass 3/3 in isolation
  • ktlintCheck, detekt, apiCheck, unit tests green
  • Windows (real desktop, both drivers): the native view and texture view monkey filters green, 13 run / 0 failed, and both monkeys green over four seeds (7, 999, 12345, 20260903)
  • full Windows headful suite: 234 run / 54 skipped / 3 failed — one tab-drag WM flake that passes in isolation, and the two dialog appearance disappearance cases, which fail the same way on this branch before these fixes (hide gone: in-scene=null native=null, pre-existing on Windows)
  • ktlintCheck, detekt, apiCheck, unit tests green on Windows
  • macOS (real desktop, both drivers): 12 native view cases green (synthetic + robot storms, right click, resize, both 150-action monkeys) and the 200-action texture view monkey

…bugs they found

New stage-2 cases under decorated-window-tao/src/test/.../headful:

- NativeViewMonkeyHeadfulCases: a BasicTextField, a Compose button, a real
  native text widget embedded through NativeView (GtkEntry / NSTextField /
  EDIT, handed out by new nativeDiag* bridge entry points) and a Compose
  button drawn over it. Alternating click storms, a right click on the
  embed, a resize storm (discrete, burst, animated, interactive corner
  drag with the real pointer) and a 150-action random walk, each driven
  both by in-process injection and by the AWT Robot. Invariants: Compose
  keeps counting clicks, one keyboard owner at a time, a typed letter and
  a caret key land where the focus says, the I-beam is requested and kept
  over the field, the embed sits on its Compose slot, no probe leaks, the
  main dispatcher keeps answering.
- TextureViewMonkeyHeadfulCases: four TextureViews fed by the platform
  test producers from their own threads plus a renderer drawing on the
  scene's own Skia context through rememberTaoGpuRenderContext; mount /
  unmount / swap / close-under-view / hide-show / minimize / DPI / bursts.
- MonkeySupport (shared watchdog, journal, seed), PointerDrivers,
  NativeProbe. The journal is echoed to stderr so a native abort still
  leaves the sequence; -Dnucleus.tao.headful.monkeyScript replays one;
  the suite filter accepts a|b.

What the monkeys found, all fixed:

- NativeView disposed the platform view before detaching it (SIGSEGV in
  nativeDetach on a freed GtkWidget); nativeViewHost() returned a fresh
  object per call, so every recomposition of the window root detached and
  re-attached every embed; a late setFrame after detach touched the widget.
- Linux: a right click forwarded to an embed lost its release to the
  widget's own context-menu grab, Compose held the button forever and every
  later Compose click was dead. The host now records buttons forwarded to
  an embed, reads GDK's live button mask on the next motion and releases
  the phantoms, and releases them before any new press.
- Linux: printable keys never reached a focused GTK embed (Tao's toplevel
  IME ate them and stopped propagation), and arrows moved GTK focus into
  the embed through GtkWindow's move-focus binding. Tao now lets GTK
  propagate keys when a foreign widget owns the focus, and stops the
  event once Compose has consumed it.
- Linux: GTK gave the map-time default focus to the embed. A focus sink
  now takes it, a Compose-kept press reclaims the keyboard, a press
  forwarded to the embed clears the Compose focus, and input boxes only
  grab focus when it is not already Compose's.
- Linux: the embed trailed its slot by a frame or more through a resize
  (queue_resize waits for the frame clock); nativeSetFrame now relayouts
  the overlay synchronously.
- Linux: a rememberTaoGpuRenderContext consumer crashed in Skia after a
  Wayland hide/show — the old TaoGlTextureHost bound the new EGL context
  for a closed DirectContext. The host is pinned to its own attachment.
- All platforms: a Dialog in a native popup layer closes its layer only
  when the disappearance animation ends; an owner window torn down before
  that leaked the layer's popup window, mapped for good and eating clicks.
  Hosts track their layers and close the survivors on detach.

NativeTaoBridge.setCursorIcon records the last requested cursor per window
for the suite; NucleusPlatformView.GtkWidget documents that the app owns a
g_object_ref_sink reference.
…d stop committing GTK's surface ourselves

The embed peeled off the Compose hole by a frame on every configure: an
embedded native view's sub-surface position is parent state that only
takes effect on GTK's toplevel commit, one GTK paint after Compose (which
tao renders after draining the draw signal) laid the new hole out and
swapped its own desync sub-surface. Through a resize burst with an embed
attached, the content sub-surface now runs in set_sync mode, so the
Compose buffer is applied atomically with the GTK commit that carries the
embed's new position; a toplevel draw is queued after each swap so that
commit always comes, including after the pointer stops. The burst's end
switches back to desync, which applies any cached state at once.

nativeSetContentOffset no longer issues an empty commit on GTK's toplevel
surface: GDK attaches its SHM buffer in end_paint and commits in
after_paint, and a commit of ours in between makes the compositor release
a buffer GDK still counts as staged — buffer_release_callback fails its
check and cairo aborts the process (reproduced by the texture monkey after
a dozen minimize/restore cycles). The caller queues a GTK toplevel draw
instead and GTK's own commit applies the offset.
The NativeView and TextureView monkeys had only ever run on Linux. On
Windows they found five ways an embedded child HWND takes something from
Compose and does not return it.

- A press forwarded to a child is replayed to the *owner* HWND a moment
  later — forwarding moves Win32 focus and the queue hands the message to
  the window that owns the pixels. Dispatched twice, Compose ended up
  holding a press with no release and every later click was dead. The host
  now recognises the replay of the overlay event it just handled and drops
  it, matched on button, position and recency so a genuine second click
  still gets through.
- The child captures the mouse on that press and keeps every later
  message: neither the scene nor its blending overlay would see the
  pointer again. The capture is handed straight back, and a release the
  child swallowed anyway is healed from Win32's own button state.
- Keys kept going to an embed clicked into earlier while Compose showed a
  focused text field. A press Compose keeps now takes Win32 focus back, as
  the macOS host does with makeFirstResponder, and a press handed to the
  embed clears the Compose focus.
- A child's handler may run a modal loop — an EDIT opens its context menu
  from WM_RBUTTONUP and does not return until it is dismissed. Everything
  but the press is posted rather than sent, so that loop never nests
  inside the Compose pointer dispatch, and the Tao loop drains the main
  dispatcher on a wake: Windows derives MainEventsCleared from a WM_PAINT
  a modal loop never generates, so the app's coroutines used to stop for
  as long as the menu was up.
- The overlay input replay repeated whatever it saw last regardless of
  what the caller was forwarding; it now only replays the matching event.

The suite: the robot driver types through the OS, the synthetic one
dispatches into the window, and on Win32 keys go to the focused HWND — so
only the robot can check that a letter reaches a focused embed.
…ces alive after close

NSTextField.mouseDown: runs trackMouse:untilMouseUp: on the Tao thread, so a
synthetic NativeView press never returned. Skip tracking/menu selectors,
makeFirstResponder, and hand keys plus Compose focus the way Linux/Windows do.

A TextureViewSource now retains its IOSurface, so closing the producer under a
live view cannot free the surface out from under a later remount.
…an embed

The robot monkey went dead at a checkpoint after clicking the embed, and
the synthetic one ended with a focused Compose field beside a focused
embed. Four separate faults, each visible to a user with a NativeView.

- The scene placed a click where the *last move it saw* left the pointer.
  Tao reports a button without a position and drops a WM_MOUSEMOVE whose
  coordinate equals the last one it saw, while every move over an embed
  goes to the blending overlay, which Tao knows nothing about. A pointer
  that left the embed and came back to a point Tao had seen before got no
  move at all, so the click landed on the embed the user had just left —
  and every click after it was dead. When the position came from the
  overlay, the host now asks Win32 where the pointer is and walks the
  scene there before dispatching.
- A native modal loop on the loop thread — an embedded EDIT's context
  menu — stops Tao producing MainEventsCleared, which is where the
  Windows redraws asked for during a batch were served. The window froze
  for as long as the menu was up: no frames, no recomposition, and the
  work queued behind a frame never ran. The wake that already drains the
  dispatcher inside such a loop now serves those redraws too.
- A press handed to an embed cleared the Compose focus through the frame
  queue, so if that very press was the one opening the embed's menu, the
  clear waited for a frame that could not come and the field kept its
  caret next to the embed's. It goes through the main dispatcher instead,
  which runs inside the modal loop.
- Handing an event to a child HWND runs its handler on this thread, and
  anything it pumps swallows the redraw the window had pending; the
  coalescing latch then suppressed every later request. Reset after a
  forwarded dispatch, like the other nested-pump callers.

Also: closing a native popup layer is idempotent now. The detach sweep
added with the layer leak fix closes surviving layers itself, and Compose
then disposes the same layer as the composition unwinds — the second pass
released the native panel twice and took the process down.

The caret assertion asks the field for its caret instead of inferring it
from where the next letter lands, and accepts a move from a caret the
value left past the end of the text.
@kdroidFilter
kdroidFilter marked this pull request as ready for review September 7, 2026 10:30
@kdroidFilter
kdroidFilter merged commit a43f910 into nucleus-2.6 Sep 7, 2026
18 of 23 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant