fix(tao): match AWT trackpad scrolling on macOS and surface Pan events - #656
Merged
Conversation
#652, #653, #654) Horizontal trackpad scrolling was reversed and precise deltas were scaled by the display factor, both relative to the AWT backend; trackpad gestures were indistinguishable from wheel notches. - vendored tao (patch 0007): stop negating scrollingDeltaX in the macOS scroll_wheel (AppKit already follows the MouseScrollDelta convention, as winit does) and carry the full AppKit phase / momentumPhase on WindowEvent::MouseWheel as `scroll_phase: ScrollPhase` - loop: hand SCROLL_PIXEL over in logical points (AWT never applies the display scale to preciseWheelRotation) and route phased steps to a new EventCallback.onScrollGesture - host: TaoTrackpadPanRouter turns the gesture stream into Compose PanStart / PanMove / PanEnd with panOffset = AWT delta x 10 dp, keeping the pan open across AppKit's momentum tail (deferred PanEnd) so Compose does not stack its own fling; wheel notches and phase-less precise scrolls stay Scroll events; NativeView forwards Pan to the native view - popups: appKitWheelToAwtScrollDelta flips both axes and drops the scale - headful e2e: MacOsTrackpadScrollHeadfulCases inject real scrollWheel: NSEvents into the tao content view (nativeDiagInjectScrollWheel), the #653 case flipping the display to its HiDPI twin; plus router, scene and wire unit tests
… step (review) Review follow-ups on #652 / #653 / #654: - TaoSceneScrollRouter: single front door for wheel / trackpad input shared by the window host and both NSPanel popup hosts; popup_panel.m now forwards the AppKit phase so a two-finger swipe over a popup list behaves like the window behind it. `-Dnucleus.tao.trackpadPanEvents=false` restores AWT-style Scroll events for handlers that only know PointerEventType.Scroll - tao patch 0007: PixelDelta carries AppKit's logical points, so the loop no longer locks WINDOWS per event nor round-trips through two scale caches - TaoTrackpadPanRouter: terminal steps with a delta still pan when no gesture is open; momentum grace 150 ms, tunable via -Dnucleus.tao.trackpadMomentumGraceMillis; isPanning removed - NativeView forwards only non-zero PanMove (PanStart / PanEnd replayed NSApp.currentEvent, stale for the deferred end) - host detach cancels the router's timer scope and ignores scrolls after the scene is gone - TaoWindow: one preciseScrollEvent() shaping helper, shared AWT constants (apiDump: the two leaked private-companion fields are gone) - headful cases: recorder reset per run, display-mode restore even when the window never reports the new scale; JNI descriptor drift guard for popup_panel.m; demo ScrollTestScreen logs Pan steps too
…tep (review 2) - TaoTrackpadPanRouter: an open pan always has an end timer (150 ms grace after the finger Ended, 1 s stall watchdog otherwise); MayBegin during the momentum tail closes the pan at once; finishNow() for clicks / wheel notches; TaoScrollGesturePhase is now an enum (wire codes attached), distinct from the public TaoTrackpadPhase, unknown codes degrade to a plain precise scroll - loop: the phase decides the route for the whole gesture — a step that arrives in lines is scaled to its point equivalent instead of being dropped as a stray wheel Scroll; dispatch_scroll_gesture loses its bogus dead_code allow; view.rs drops the dead ViewState binding - TaoSceneScrollRouter: pan position / modifiers come from gesture steps only (PanEnd must hit the node that got the PanMoves), a click or a wheel notch closes the pan, cancel() makes late events no-ops, the timer scope carries TaoFatalCoroutineExceptionHandler; TaoPopupSceneLayer reads the live layer density; TaoStandalonePopupHostMac drops the native callback before cancelling the router; first Pan routing logs one CONFIG line - NativeView consumes PanStart / PanEnd over an embedded view so the ancestor scrollable never opens a session of its own - popup_panel.m: NucleusScrollGesture enum instead of literals; TaoScrollWireDriftTest compares Rust, ObjC and Kotlin wire codes plus the JNI descriptors - nativeDiagInjectScrollWheel is inert unless NUCLEUS_TAO_INPUT_INJECTION=1 (set by taoHeadfulTest) and main-thread only - README / CLAUDE.md document the Pan behaviour and -Dnucleus.tao.trackpadPanEvents=false
…m tail (review 3) - NativeView forwards PanStart / PanMove / PanEnd with a phase; native_view.m replays the live AppKit event only when its phase class matches and otherwise synthesises a phased scroll, so an embedded scroll view sees the gesture begin and end (rubber-band, scroller fade) — Windows / Linux ignore the phase - popups keep the gesture phase whatever hasPreciseScrollingDeltas says, like the Rust window path - TaoTrackpadPanRouter: momentum steps only continue an open pan; a tail arriving after the grace already closed it is dropped instead of stacked on Compose's fling - TaoSceneScrollRouter: lazily created timer scope with TaoNonFatalCoroutineExceptionHandler (a broken deferred PanEnd costs one gesture, not the app); TaoScrollGesturePhase.fromWire is a map lookup - TaoStandalonePopupHostMac: finishPan() runs inside the frame pump; the invalid-host dispose path also shuts the render executor down - nativeDiagInjectScrollWheel targets the NSView it is handed, is nil-safe on the primary screen, inlined; documented as whole-point only (CGEvent delta fields are integers — verified) and the headful momentum tail uses whole points - demo ScrollTestScreen closes a gesture on PanEnd / next PanStart and names the 10 dp factor; harness closes the pan after the cursor-move guard like the host; TaoScrollWireDriftTest resolves its sources per test with a readable failure; stray @Suppress dropped
…ry an orphaned tail (review 4) - native_view.m remembers the AppKit event it last handed to a child (NSApp.currentEvent is not cleared while the app idles, and one event can yield two Compose steps), so a deferred PanEnd or a PanStart+PanMove pair never applies a delta twice; the synthesised fallback keeps a per-view sub-point residue instead of rounding slow drags to zero - popup_panel.m and the vendored view.rs test NSEventPhase bits instead of switching on the NS_OPTIONS mask - TaoSceneScrollRouter leaves the pan position alone for MayBegin (its PanEnd belongs to the previous gesture); an orphaned momentum tail is delivered as AWT-shaped Scroll instead of dropped; once-only CAS behind a plain read; scrollRouter declared with the rest of the host state - TaoStandalonePopupHostMac.dispose() is idempotent on the invalid path too - TaoWindow keeps AWT_PIXEL_TO_ROTATION / MACOS_AWT_SCROLL_AMOUNT as aliases of the shared constants (public statics in the validated ABI) - demo ScrollTestScreen finalises a gesture inline on PanEnd / PanStart - TaoScrollWireDriftTest also guards the kNv* native-view wire and the 10-units-per-wheel factor in events.rs, native_view.m and the demo; TaoSceneScrollTest pins MacOSCocoaConfig's 10 dp per unit on macOS; harness routers get one timer slot each
…(review 5) - native_view.m: an AppKit event is marked spent only once its delta has actually travelled (replayed, or synthesised by the step that carries it), so an Ended-with-delta arriving with no pan open (PanStart + PanMove from one event) no longer loses the finger movement; per-child gesture state, keyed on the child handle rather than a raw NSView*, keeps a deferred PanEnd from sending a second terminal phase and resets the sub-point residue at gesture boundaries - NativeView hands pan offsets over in scene px; the macOS host converts them back with its own scale, so an app-level LocalDensity override no longer skews the embedded view's distance. dispatchScrollToNative is back to its original signature (Windows / Linux untouched) - TaoTrackpadPanRouter: the pan end is a deadline, one timer in flight; finger and momentum steps only move the deadline, the grace re-schedules once (earlier), the timer re-arms for the remainder — no coroutine, wake and cancel per 120 Hz step - TaoSceneScrollRouter: a zero-delta orphaned tail end is not turned into a Scroll (AWT drops zero deltas); once-only announce on a plain volatile; the redundant sceneBundle guard in onPointerScroll is gone (the router's cancelled flag is the contract) - TaoPopupSceneLayer documents why the pan uses the layer density while the surface uses host.scale; TaoWindow uses import aliases for the ABI alias constants; the injector checks the main thread before its static init - headful: shared ScrollableColumn / ScrollableRow fixtures and a harness awaitUntilOrTimeout replace the copies; the wheel-notch baseline is taken right before the injection; the library drift test no longer reads the demo source
Collaborator
Author
|
@sargunv can you check if it's ok for you ? use publishDevToMavenLocal |
One screen in nucleus-demo to test the macOS trackpad work by hand: a root inspector (every Scroll / PanStart / PanMove / PanEnd with gaps, counters and one summary per gesture incl. how long after the last move the PanEnd came), vertical and horizontal strips for sign and magnitude, a map canvas that pans on Pan and zooms on Scroll, a scrollable DropdownMenu (inline, or NSPanel via "Open with native popup layers"), and an embedded WKWebView whose page draws its own scrollY / wheel HUD. The header shows density, px per wheel unit and the trackpadPanEvents flag. NUCLEUS_DEMO_TAB=<tab> opens the demo straight on a tab. Adds the composewebview dependency to nucleus-demo (in-tree Nucleus modules excluded, as in tao-demo).
kdroidFilter
marked this pull request as ready for review
September 6, 2026 13:39
|
Works great with maplibre/maplibre-compose#1290; except trackpad panning ended up inverted. I'm still investigating the intended convention in Compose (from Android) and whether that's my bug or yours |
|
Okay, confirmed the convention here matches Android's Compose implementation, so this lgtm. Works great once I flipped the sign on my end |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #652, closes #653, closes #654.
Root causes
scrollingDeltaXin the macOSscroll_wheel("macOS is the inverse of tao"), which is not the case: AppKit already follows theMouseScrollDeltaconvention (winit does not negate). The Kotlin side then negated both axes for the AWT convention, so X reached Compose reversed.PixelDeltato physical pixels andTaoWindowdivided by 10. AWT's conversion is-[event deltaX/Y](the legacy delta of a precise event isscrollingDelta × 0.1in points) with no display scale, hence 2× on Retina.Scroll.Changes
PixelDeltacarries AppKit's logical points (no backing-scale round trip, no per-event lock in the loop);WindowEvent::MouseWheelcarriesscroll_phase: ScrollPhase— the full AppKitphase/momentumPhase(Nonefor a wheel and on Windows / Linux, whichTouchPhasecannot express), derived by testing the NS_OPTIONS bits.EventCallback.onScrollGesture(GraalVM metadata added). The phase decides the route for the whole gesture; a step that arrives in lines is scaled to its point equivalent rather than leaking as a stray wheelScroll.TaoSceneScrollRouter(new): the single front door for wheel / trackpad input, used by the window host and both NSPanel popup hosts (popup_panel.mforwards the phase as a 6thonScrollarg, kept whateverhasPreciseScrollingDeltassays) so a two-finger swipe over a popup list behaves like the window behind it. Wheel notches and phase-less precise scrolls → AWT-shapedScroll; trackpad gesture steps →TaoTrackpadPanRouter→ ComposePanStart/PanMove/PanEnd,panOffset = AWT delta × 10 dp(theMacOSCocoaConfigwheel factor, so a pan and a notch move content equally; popups use their layer's own density, the one their content andMacOSCocoaConfigmeasure with).Endedpulls the deadline in to 150 ms (-Dnucleus.tao.trackpadMomentumGraceMillis) so AppKit's momentum tail continues the same pan instead of stacking on Compose's own fling; every other step pushes it out to a 1 s stall window without re-scheduling anything;MomentumEnded/Cancelled,MayBegin(fingers resting on the glass), a click and a wheel notch close it at once. Momentum steps only ever continue an open pan; a tail that arrives after the grace already closed it is delivered as the AWT-shapedScrollit would have been under AWT (zero-delta ends skipped, as AWT does) — neither stacked nor dropped. A finger step still carrying a delta pans even when no gesture is open.PanEndmust reach the node that got thePanMoves;MayBegin(which belongs to the next gesture) does not move it.PanEndruns on a lazily created scope withTaoNonFatalCoroutineExceptionHandler: a broken PanEnd costs one gesture, not the app (same as the synchronous popup path, wherepopup_panel.mclears the JNI exception).PointerEventType.Scrollno longer sees trackpad input on this backend (AWT has no Pan events).-Dnucleus.tao.trackpadPanEvents=falserestores the AWT-style behaviour where every gesture step is aScroll; documented in README and CLAUDE.md, and the first routed gesture logs one CONFIG line naming the property. A "re-dispatch unconsumed pans as Scroll" fallback is not possible from outside compose-ui (PointerEventResult.anyChangeConsumedis internal). The demoScrollTestScreenlogs Pan steps and finalises a gesture inline onPanEnd/ the nextPanStart. Default is on — flipping it to opt-in for the 2.5 line is a one-liner if preferred.TaoScrollGesturePhaseis an enum with its wire code, distinct from the publicTaoTrackpadPhase(magnify / rotate); unknown wire codes degrade to a plain precise scroll.NativeViewforwards the whole pan —PanStart/PanMove/PanEnd, offsets in scene px that the host converts back with its own scale — and consumes it.native_view.mreplays a fresh AppKit event whole when its phase class matches, otherwise synthesises a phased scroll carrying a per-child sub-point residue; an event's delta is marked spent only once it has actually travelled, so neither a deferredPanEndnor the second Compose step derived from one event applies it twice, and per-child gesture state keeps aPanEndfrom sending a second terminal phase. An embedded WKWebView / NSScrollView thus sees the gesture begin and end (rubber-band snap-back, scroller fade). Windows / Linux are untouched (dispatchScrollToNativekeeps its signature;dispatchPanToNativeis a no-op there).appKitWheelToAwtScrollDeltaflips both axes and no longer applies the scale;TaoStandalonePopupHostMacdrops the native callback before cancelling the router, runsfinishPan()inside its frame pump, and itsdispose()is idempotent and releases the render executor on the invalid-host path too.TaoWindow: onepreciseScrollEvent()shaping helper;AWT_PIXEL_TO_ROTATION/MACOS_AWT_SCROLL_AMOUNTstay onTaoWindowas aliases of the shared constants because they are public statics in the validated ABI (apiCheckunchanged vsmain).Tests
Headful e2e (
MacOsTrackpadScrollHeadfulCases, macOS only) written first — 4/4 red before the fix, 4/4 green after. They inject realscrollWheel:NSEvents into the tao NSView viaNativeMetalBridge.nativeDiagInjectScrollWheel(same idea asnativeLinuxInjectGdkScroll: no WindowServer, no Accessibility grant). The injector is inert unlessNUCLEUS_TAO_INPUT_INJECTION=1(set by thetaoHeadfulTesttask), main-thread only, and whole-point only — the CGEvent delta fields are integers andNSEvent(cgEvent:)derivesscrollingDelta*from them (verified;CGEventSetDoubleValueFieldon the point / fixed-point fields only changes the legacydeltaX/Y).#652two-finger swipe left scrolls a horizontal row forward#653precise deltas match AWTpreciseWheelRotation(flips the display to its HiDPI twin for the case, restored infinally)#654a gesture arrives asPanStart/PanMove/PanEnd(singlePanEnd, after the momentum tail, noScroll, no zero-offsetPanMove), a wheel notch staysScroll#654the pan drives averticalScrollcolumn through foundation'sTrackpadScrollingLogicUnit:
TaoTrackpadPanRouterTest(grace, stall deadline, one timer per gesture, MayBegin, orphaned momentum handed back, finishNow, -0.0),TaoSceneTrackpadPanTest(routed gesture path,trackpadPanEvents=falsepath, orphaned tail falling back to Scroll — all through a realTaoSceneScrollRouteron a manual clock),TaoSceneScrollTestpinsMacOSCocoaConfig's 10 dp per wheel unit on macOS, updatedTaoWindowScrollTest/MacOsWheelDeltaTest,TaoScrollWireDriftTest(RustSCROLL_GESTURE_*, ObjCNucleusScrollGesture*/kNv*and Kotlin wire codes, the hand-written JNI descriptors inpopup_panel.m, and the 10-units-per-wheel factor inevents.rsandnative_view.m); registered inTaoSceneTestBattery+ drift test.:decorated-window-tao:test detekt ktlintCheck apiCheckgreen on macOS (220 tests).Manual test rig
NUCLEUS_DEMO_TAB="Trackpad Lab" ./gradlew :examples:nucleus-demo:runopens the flagship demo on a new Trackpad Lab tab: a root inspector (everyScroll/PanStart/PanMove/PanEndwith gaps, counters, one summary per gesture — how long after the last move thePanEndcame tells momentum-closed from grace-closed), vertical / horizontal strips for sign and magnitude, a map canvas that pans on Pan and zooms on Scroll, a scrollableDropdownMenu(inline, or an NSPanel through "Open with native popup layers"), and an embedded WKWebView whose page draws its ownscrollY/ wheel-event HUD. Verified here with synthetic phased gestures: 7 steps →(0, 5.7)wheel units, onePanEnd19 ms after the last (momentum) move, the web view atscrollY 57with exactly 7 wheel events, the map panned by 57 px and zoomed only by the wheel notch.Not changed on purpose
delay()onTaoMainDispatchergoes through kotlinx's default delay executor (now one timer per gesture, not per step); teaching the dispatcherDelayneeds a native timer wake and is out of scope here.Not verified here
Windows / Linux only gain
scroll_phase: ScrollPhase::Nonein their tao constructors; no toolchain locally, relying onbuild-natives. Port tonucleus-2.6applies with--3way(only thetao-patches/README.mdtable row conflicts).