From ca79dff70e6a9811734785f8a0b76e5e8c6c92a2 Mon Sep 17 00:00:00 2001 From: Mohak Bajaj Date: Tue, 11 Aug 2026 22:37:52 +0530 Subject: [PATCH] canvas: carry click modifiers into press Msg arms `WidgetPointerEvent` already knows which modifiers were held at click time, but nothing forwarded them to app code: `UiApp.Options` has `on_key`, `on_pinch` and `on_drop` and no pointer hook, so cmd-click and shift-click were not expressible in either core language. Carry them on the existing press channel, the way `on-drag` already fills phase and geometry around an authored `sourceId`. A Msg arm that declares the four booleans `shift`/`control`/`alt`/`super` beside at most one authored payload field receives them; markup keeps its ordinary spelling, so `on-press="select:{row.id}"` fills `id` and the runtime fills the rest. An arm without all four booleans is an ordinary payload and takes exactly the path it takes today, so this is additive. - reflect: `declaredWidgetPressRecord` and `pressPayloadFieldName` - ui: `msgForPointerClickModified` plus the `withPressModifiers` injector, with the three-argument form kept as the unmodified entry point - both engines fill the authored field and leave the modifiers to dispatch - contract: the `press` payload class and `MsgTag.press_payload`, so a binding is kind-checked against the authored field; artifact version 7 - automation: `widget-click [cmd|shift|cmd+shift]`, so the channel is drivable from tests and smoke runs Co-Authored-By: Claude Opus 5 (1M context) --- docs/src/app/docs/native-ui/page.mdx | 2 + skill-data/automation/SKILL.md | 1 + skill-data/native-ui/SKILL.md | 2 + src/primitives/canvas/ui.zig | 36 ++++- src/primitives/canvas/ui_markup_compiled.zig | 17 ++ src/primitives/canvas/ui_markup_contract.zig | 46 +++++- src/primitives/canvas/ui_markup_reflect.zig | 45 ++++++ src/primitives/canvas/ui_markup_view.zig | 18 +++ .../canvas/ui_markup_view_tests.zig | 145 ++++++++++++++++++ src/runtime/automation_commands.zig | 49 ++++++ src/runtime/automation_widget_dispatch.zig | 13 +- src/runtime/flow.zig | 3 +- src/runtime/ui_app.zig | 10 +- tools/native-sdk/automation.zig | 8 +- 14 files changed, 384 insertions(+), 11 deletions(-) diff --git a/docs/src/app/docs/native-ui/page.mdx b/docs/src/app/docs/native-ui/page.mdx index a02250aa9..c01175742 100644 --- a/docs/src/app/docs/native-ui/page.mdx +++ b/docs/src/app/docs/native-ui/page.mdx @@ -238,6 +238,8 @@ pub fn draft(model: *const Model) []const u8 { `on-resize` (on the `split` element; `Ui.valueMsg(.tag)` on `on_resize` in Zig views) names a variant whose payload is the new first-pane fraction (`f32`): after every divider drag, keyboard adjustment, or assistive increment/decrement the runtime delivers the fraction it already applied and clamped — store it in the model and echo it back through the split's `value`, and rebuilds never fight live resizing. +`on-press` and `on-double-press` optionally carry the modifiers held at click time. An arm declaring the four booleans `shift`, `control`, `alt`, and `super` — alongside at most one authored payload field — receives them from the runtime, exactly the way `on-drag` fills geometry around an authored `sourceId`. Markup keeps its ordinary spelling (`on-press="select:{row.id}"` fills `id`; a bare `on-press="clear"` names the modifier-only form), so cmd-click and shift-click become expressible without a new attribute. An arm without all four booleans is an ordinary payload and resolves exactly as before. + `on-drag` makes any element a draggable spatial object and names the closed record `{ sourceId, phase, x, y, viewWidth, viewHeight }`. Markup supplies numeric `sourceId` from a binding such as `on-drag="card_dragged:{card.id}"`; the runtime supplies `phase` (a number able to represent 0, 1, and 2) plus floating-point view-local `x`, `y`, `viewWidth`, and `viewHeight` (`f32`/`f64` in Zig, `number` in TypeScript). Geometry stays floating-point because pointer capture can carry negative coordinates outside the view. Phase 0 means motion, 1 release, and 2 cancellation. The renderer lifts the actual source appearance under the pointer at full opacity and leaves its in-flow space blank. Apps that need precise insertion can keep committed data unchanged during phase 0 while returning a derived view that moves the same `global-key` into the candidate position. Its hidden in-flow rendering is the one card-sized reserved slot: it begins at the source, moves to each candidate, and never duplicates. Keyed draggable neighbors ease between candidate poses; release carries the floating item from its pointer position into that slot and commits the exact order. A plain Escape during the drag dispatches phase 2, consumes that key, and carries the item back to its source slot; pointer cancellation uses the same path. Reduced-motion appearances snap these reflows. `examples/kanban` demonstrates within-column and cross-column reordering with this pattern. `on-dismiss` (on the dismissible surfaces: `dialog`, `drawer`, `sheet`, `dropdown-menu`; `ElementOptions.on_dismiss` in Zig views) dispatches when Escape or a click outside dismisses the surface, so the model owns the close — clear the open flag in `update`. The engine hides the surface immediately as an optimistic echo; the next rebuild's source tree is truth. Escape works regardless of focus: it dismisses the nearest surface up the focused widget's chain, and when nothing relevant is focused — a menu opened from a plain-text trigger takes no focus — it falls back to the topmost mounted anchored surface. `on-hold` (any element; `ElementOptions.on_hold`) is press-and-hold: a pointer held ~350 ms dispatches the hold Msg and the release presses nothing, a quick click dispatches `on-press` as usual, and a right/ctrl-click with no context menu on its route dispatches the hold Msg immediately (a declared `` always wins the right-click) — the crumb-switcher shape (`on-press` selects, `on-hold` opens an anchored menu). Both legs are live-drivable through automation: `native automate widget-hold ` runs the pointer+timer gesture, `widget-context-press ` the secondary click. diff --git a/skill-data/automation/SKILL.md b/skill-data/automation/SKILL.md index 6d8a03742..6854176c6 100644 --- a/skill-data/automation/SKILL.md +++ b/skill-data/automation/SKILL.md @@ -65,6 +65,7 @@ native automate screenshot inbox-canvas native automate screenshot inbox-canvas 2 native automate widget-action canvas 2 press native automate widget-click canvas 3 +native automate widget-click canvas 3 cmd # modified click: cmd, shift, cmd+shift native automate widget-hold canvas 3 native automate widget-context-press canvas 3 native automate widget-drag canvas 4 0.25 0.82 diff --git a/skill-data/native-ui/SKILL.md b/skill-data/native-ui/SKILL.md index 734042200..1265e71fc 100644 --- a/skill-data/native-ui/SKILL.md +++ b/skill-data/native-ui/SKILL.md @@ -517,6 +517,8 @@ For ``, prefer an explicit boolean predicate method over numeric truthi Scroll offsets follow the same mirror discipline as text: the Msg carries the offset the runtime ALREADY applied, so store it in the model and echo it back through the scroll's `value` — the echoed source value equals the runtime offset, which the scroll reconcile rule treats as "unchanged", so rebuilds never stomp live scrolling. `on-scroll` is how long content pages or lazy-loads: keep a bounded window in the model and slide it from `offset` (near-end when `offset + viewport_extent` approaches `content_extent`). +A press arm may also ask for the MODIFIERS held at click time: declare the four booleans `shift`/`control`/`alt`/`super` beside at most one authored payload field, and the runtime fills them (markup still spells the binding the same way — `on-press="select:{r.id}"` fills `id`, a bare `on-press="clear"` names the modifier-only form). This is what makes cmd-click and shift-click expressible; an arm without all four booleans is an ordinary payload and behaves exactly as before. `native automate widget-click [cmd|shift|cmd+shift]` drives it. + `on-hover-enter` and `on-hover-leave` (any element; `ElementOptions.on_hover_enter` / `on_hover_leave` in Zig views) are the pointer-hover containment pair — Elm's onMouseEnter/onMouseLeave: enter dispatches once when the pointer enters the element's hit region, leave once when it exits — discrete edges, never per-move, so hover previews, prefetch, and hover cards are ordinary Msgs. Binding either makes the element hover-hittable the way a bound press makes it pressable — but never pressable: clicks fall through, no accessibility action, and NO hover wash (the wash is the visual channel of acting controls; a `quiet_hover` content tile that binds hover stays visually quiet while the model hears it). Nested bound elements track containment independently (entering a bound row inside a bound card never leaves the card); enters fire outermost-first, leaves innermost-first. Every enter is answered by exactly one eventual leave — the leave Msg is captured while the element stands (kept fresh across rebuilds, retained if unbound), so it still arrives when the exit is the element unmounting. Exits resolve exactly like the hover wash: moving off and the pointer leaving the window are direct edges; content scrolling or reflowing out from under a stationary pointer re-hit-tests the last pointer position; a dismissal removing the surface under the pointer delivers that surface's leaves immediately, and whatever it reveals is entered when the model's close rebuild re-hit-tests (pair dismissible surfaces with `on-dismiss`, as always); overlays occlude hover the way they occlude clicks. Mouse/trackpad only — containment advances on hover-phase motion, a pointer floating without contact that touch physically cannot produce, so touch never synthesizes hover and hover-revealed affordances need a second path. A handler or update error DEGRADES, it does not exit the app: dispatch catches it, records it in a bounded ring (`runtime.dispatchErrors()`, the `error event=... name=...` lines and `dispatch_errors=` count in automation snapshots, and a `dispatch.error` trace record at error level), and the app keeps running. Trace-sink capacity failures likewise never fail dispatch — dropped records are counted (`dropped_trace_records=`), not fatal. Design for it: an arm that can fail should still surface its own status in the model; the error ring is the safety net, not the UX. diff --git a/src/primitives/canvas/ui.zig b/src/primitives/canvas/ui.zig index fd958c0ce..4080a6015 100644 --- a/src/primitives/canvas/ui.zig +++ b/src/primitives/canvas/ui.zig @@ -1427,10 +1427,42 @@ pub fn Ui(comptime Msg: type) type { /// dispatcher the runtime pointer path uses; the two-argument /// form stays the single-click entry point. pub fn msgForPointerClick(self: Tree, target_id: ObjectId, phase: canvas.WidgetPointerPhase, click_count: u8) ?Msg { + return self.msgForPointerClickModified(target_id, phase, click_count, .{}); + } + + /// `msgForPointerClick` carrying the modifiers held at click + /// time. An arm declaring the press-record shape receives them; + /// every other arm resolves exactly as the three-argument form, + /// so this is the dispatcher the runtime pointer path uses. + pub fn msgForPointerClickModified( + self: Tree, + target_id: ObjectId, + phase: canvas.WidgetPointerPhase, + click_count: u8, + modifiers: canvas.WidgetKeyboardModifiers, + ) ?Msg { if (phase == .up and click_count == 2) { - if (self.msgFor(target_id, .double_press)) |msg| return msg; + if (self.msgFor(target_id, .double_press)) |msg| return withPressModifiers(msg, modifiers); } - return self.msgForPointer(target_id, phase); + const msg = self.msgForPointer(target_id, phase) orelse return null; + return withPressModifiers(msg, modifiers); + } + + /// Fill the four modifier booleans on an arm that declared them, + /// leaving every other arm untouched. The authored payload is + /// already in place: markup built it, this only adds what only + /// the runtime knows. + pub fn withPressModifiers(msg: Msg, modifiers: canvas.WidgetKeyboardModifiers) Msg { + return switch (msg) { + inline else => |payload, tag| if (comptime reflect.declaredWidgetPressRecord(@TypeOf(payload))) blk: { + var out = payload; + out.shift = modifiers.shift; + out.control = modifiers.control; + out.alt = modifiers.alt; + out.super = modifiers.super; + break :blk @unionInit(Msg, @tagName(tag), out); + } else msg, + }; } /// Typed dispatch for keyboard events: engine control intents diff --git a/src/primitives/canvas/ui_markup_compiled.zig b/src/primitives/canvas/ui_markup_compiled.zig index fa7a0df4b..e7c6a885a 100644 --- a/src/primitives/canvas/ui_markup_compiled.zig +++ b/src/primitives/canvas/ui_markup_compiled.zig @@ -2347,6 +2347,23 @@ fn CompiledMarkupEngine(comptime ModelT: type, comptime MsgT: type, comptime res } return @unionInit(MsgT, field.name, {}); } + // A press-modifier arm carries the four booleans the runtime + // fills at dispatch, so the authored binding sets its own field + // and the rest start false. + if (comptime interpreter.declaredWidgetPressRecord(field.type)) { + const payload_name = comptime interpreter.pressPayloadFieldName(field.type); + comptime { + if (payload_name == null and expression.payload.len > 0) fail(node, "message does not take a payload"); + if (payload_name != null and expression.payload.len == 0) fail(node, "message requires a payload"); + } + var payload: field.type = std.mem.zeroes(field.type); + if (comptime payload_name) |name| { + const press_variant = comptime pathVariant(node, entries, expression.payload, true); + const press_value = bindingValue(node, entries, expression.payload, ui, model, scope, true); + @field(payload, name) = coerce(@FieldType(field.type, name), node, press_variant, ui, press_value); + } + return @unionInit(MsgT, field.name, payload); + } comptime { if (expression.payload.len == 0) fail(node, "message requires a payload"); } diff --git a/src/primitives/canvas/ui_markup_contract.zig b/src/primitives/canvas/ui_markup_contract.zig index 7ece57467..c5a79bd25 100644 --- a/src/primitives/canvas/ui_markup_contract.zig +++ b/src/primitives/canvas/ui_markup_contract.zig @@ -61,7 +61,11 @@ pub const ValueKind = expr.ValueKind; /// preview insertion/reordering on change and restore on cancellation. /// Version 6: live drag geometry requires floating-point fields so captured /// out-of-view coordinates cannot trap an integer conversion at dispatch. -pub const format_version: u32 = 6; +/// Version 7: `press` classifies the modifier-carrying press record and +/// `MsgTag.press_payload` names the class of its authored field; a +/// format-6 artifact calls the record unsupported and would reject a valid +/// `on-press` binding. +pub const format_version: u32 = 7; /// Where the app's build step writes the artifact, relative to the app /// directory (a build product lives under zig-out, not in durable state). @@ -146,12 +150,17 @@ pub const Iterable = struct { /// cannot be built from markup at all. `legacy_scroll_state` is the /// RETIRED one-axis scroll record, recognized only so `on-scroll` can /// teach the two-axis migration by field name. -pub const PayloadClass = enum { none, string, integer, float, boolean, enum_tag, text_input, scroll_state, legacy_scroll_state, terminal_state, drag_drop, unsupported }; +pub const PayloadClass = enum { none, string, integer, float, boolean, enum_tag, text_input, scroll_state, legacy_scroll_state, terminal_state, drag_drop, press, unsupported }; pub const MsgTag = struct { name: []const u8, payload: PayloadClass = .none, payload_type: []const u8 = "", + /// For `.press` arms only: the class of the ONE authored field the + /// markup binding fills, or `.none` for the payload-less form. The + /// four modifier booleans are the runtime's to fill, so they never + /// appear here. + press_payload: PayloadClass = .none, }; pub const Contract = struct { @@ -327,6 +336,7 @@ fn describeMsgs(comptime Msg: type, comptime specials: Specials) []const MsgTag .name = field.name, .payload = payloadClassOf(field.type, specials), .payload_type = if (field.type == void) "" else @typeName(field.type), + .press_payload = pressPayloadClassOf(field.type, specials), }}; } return tags; @@ -356,6 +366,7 @@ fn payloadClassOf(comptime T: type, comptime specials: Specials) PayloadClass { // resolution as both engines' terminalConstructor. if (reflect.declaredTerminalStateRecord(T)) return .terminal_state; if (reflect.declaredWidgetDragDropRecord(T)) return .drag_drop; + if (reflect.declaredWidgetPressRecord(T)) return .press; return switch (@typeInfo(T)) { .int => .integer, .float => .float, @@ -366,6 +377,16 @@ fn payloadClassOf(comptime T: type, comptime specials: Specials) PayloadClass { }; } +/// The class of a press arm's authored field, so `on-press="tag:{path}"` +/// is kind-checked exactly like an ordinary payload. `.none` for any arm +/// that is not a press record, and for the payload-less press form. +fn pressPayloadClassOf(comptime T: type, comptime specials: Specials) PayloadClass { + if (T == void) return .none; + if (!reflect.declaredWidgetPressRecord(T)) return .none; + const name = reflect.pressPayloadFieldName(T) orelse return .none; + return payloadClassOf(@FieldType(T, name), specials); +} + fn optOutNames(comptime T: type) []const []const u8 { comptime { if (!@hasDecl(T, opt_out_decl)) return &.{}; @@ -934,6 +955,25 @@ const Checker = struct { } } const found = tag orelse return self.failNamed(node, unknown_tag_message, expression.tag, .{ .msgs = self.contract }); + // A press arm's four modifier booleans are the runtime's to fill; + // markup binds only the authored field, so the payload rules apply + // to THAT field's class rather than to the record. + if (found.payload == .press) { + if (found.press_payload == .none) { + if (expression.payload.len > 0) return self.failAttr(node, attribute, no_payload_message); + return; + } + if (expression.payload.len == 0) return self.failAttr(node, attribute, payload_required_message); + const press_resolved = try self.resolveBinding(node, expression.payload, true); + switch (found.press_payload) { + .integer => try self.requirePayloadKind(node, attribute, press_resolved, &.{.integer}, found), + .float => try self.requirePayloadKind(node, attribute, press_resolved, &.{ .float, .integer }, found), + .string, .enum_tag => try self.requirePayloadKind(node, attribute, press_resolved, &.{.string}, found), + .boolean => {}, + else => return self.failPayloadType(node, attribute, press_resolved, found), + } + return; + } if (found.payload == .none) { if (expression.payload.len > 0) return self.failAttr(node, attribute, no_payload_message); return; @@ -948,7 +988,7 @@ const Checker = struct { .boolean => {}, // These payloads cannot be constructed from a markup binding // (input/scroll payloads bind through their own events). - .text_input, .scroll_state, .legacy_scroll_state, .terminal_state, .drag_drop, .unsupported => return self.failPayloadType(node, attribute, resolved, found), + .text_input, .scroll_state, .legacy_scroll_state, .terminal_state, .drag_drop, .press, .unsupported => return self.failPayloadType(node, attribute, resolved, found), .none => unreachable, } } diff --git a/src/primitives/canvas/ui_markup_reflect.zig b/src/primitives/canvas/ui_markup_reflect.zig index 36fc9b094..46debe049 100644 --- a/src/primitives/canvas/ui_markup_reflect.zig +++ b/src/primitives/canvas/ui_markup_reflect.zig @@ -297,6 +297,51 @@ fn isDragPhaseNumber(comptime T: type) bool { }; } +/// The keyboard modifiers a press-carrying Msg arm declares. Named once +/// so the predicate, the injector, and the contract all agree on the +/// vocabulary. +pub const press_modifier_field_names = [_][]const u8{ "shift", "control", "alt", "super" }; + +/// A markup `on-press` / `on-double-press` Msg payload that also wants the +/// modifiers held at click time. The authored binding fills its own field +/// (`on-press="select:{row.id}"`); the runtime fills the four booleans, the +/// way `on-drag` fills phase and geometry around an authored `sourceId`. +/// +/// Shape: the four modifier booleans, plus AT MOST one other field — the +/// authored payload, whatever the app named it. Zero other fields is the +/// payload-less form (`on-press="clear"` on an arm that only wants to know +/// which modifiers were down). Requiring the four names keeps this +/// unambiguous against ordinary record payloads, and keeps a plain +/// `{ id: number }` arm on exactly the path it takes today. +pub fn declaredWidgetPressRecord(comptime T: type) bool { + const info = switch (@typeInfo(T)) { + .@"struct" => |s| s, + else => return false, + }; + if (info.fields.len != press_modifier_field_names.len and + info.fields.len != press_modifier_field_names.len + 1) return false; + inline for (press_modifier_field_names) |name| { + if (!@hasField(T, name)) return false; + if (@FieldType(T, name) != bool) return false; + } + return true; +} + +/// The authored payload field of a press record, or null for the +/// payload-less form. The press predicate has already established that at +/// most one field falls outside the modifier vocabulary. +pub fn pressPayloadFieldName(comptime T: type) ?[]const u8 { + const info = @typeInfo(T).@"struct"; + inline for (info.fields) |field| { + comptime var is_modifier = false; + inline for (press_modifier_field_names) |name| { + if (comptime std.mem.eql(u8, field.name, name)) is_modifier = true; + } + if (!is_modifier) return field.name; + } + return null; +} + /// A mirror of the RETIRED one-axis scroll state — `{offset, velocity, /// viewport_extent, content_extent}` in either spelling. Recognized only /// to fail with a teaching that names the new per-axis fields, so an app diff --git a/src/primitives/canvas/ui_markup_view.zig b/src/primitives/canvas/ui_markup_view.zig index 8c049d965..88145476f 100644 --- a/src/primitives/canvas/ui_markup_view.zig +++ b/src/primitives/canvas/ui_markup_view.zig @@ -2165,6 +2165,22 @@ pub fn MarkupView(comptime ModelT: type, comptime MsgT: type) type { } return @unionInit(MsgT, field.name, {}); } + // A press-modifier arm carries the four booleans the + // runtime fills at dispatch, so the authored binding + // sets its own field and the rest start false. + if (comptime reflect.declaredWidgetPressRecord(field.type)) { + var payload: field.type = std.mem.zeroes(field.type); + if (comptime reflect.pressPayloadFieldName(field.type)) |payload_name| { + if (expression.payload.len == 0) { + return self.failMsg(node, "message requires a payload"); + } + const bound = try self.evalBinding(scope, node, expression.payload, true); + @field(payload, payload_name) = try self.coerce(@FieldType(field.type, payload_name), node, bound); + } else if (expression.payload.len > 0) { + return self.failMsg(node, "message does not take a payload"); + } + return @unionInit(MsgT, field.name, payload); + } if (expression.payload.len == 0) { return self.failMsg(node, "message requires a payload"); } @@ -2647,6 +2663,8 @@ pub const declaredScrollStateRecord = reflect.declaredScrollStateRecord; pub const declaredTerminalStateRecord = reflect.declaredTerminalStateRecord; pub const declaredLegacyScrollStateRecord = reflect.declaredLegacyScrollStateRecord; pub const declaredWidgetDragDropRecord = reflect.declaredWidgetDragDropRecord; +pub const declaredWidgetPressRecord = reflect.declaredWidgetPressRecord; +pub const pressPayloadFieldName = reflect.pressPayloadFieldName; pub const valueArmClass = reflect.valueArmClass; pub const sliceElement = reflect.sliceElement; pub const isItemFn = reflect.isItemFn; diff --git a/src/primitives/canvas/ui_markup_view_tests.zig b/src/primitives/canvas/ui_markup_view_tests.zig index 29ffc6576..0dc9a57cc 100644 --- a/src/primitives/canvas/ui_markup_view_tests.zig +++ b/src/primitives/canvas/ui_markup_view_tests.zig @@ -4798,6 +4798,151 @@ test "declaredWidgetDragDropRecord accepts safe live drag fields and rejects uns })); } +const PressModel = struct { rowId: i64 = 7 }; + +const PressMsg = union(enum) { + // The authored payload plus the four booleans the runtime fills. + select: struct { id: i64, shift: bool, control: bool, alt: bool, super: bool }, + // The payload-less form. + clear: struct { shift: bool, control: bool, alt: bool, super: bool }, + // An ordinary arm, to prove the existing path is untouched. + plain: i64, +}; + +const press_markup_source = + \\ + \\ + \\ + \\ + \\ +; + +test "on-press fills a press arm's modifiers from the click and leaves other arms alone" { + var arena_state = std.heap.ArenaAllocator.init(testing.allocator); + defer arena_state.deinit(); + const arena = arena_state.allocator(); + const model = PressModel{}; + + const PressMarkup = markup_view.MarkupView(PressModel, PressMsg); + var view = try PressMarkup.init(arena, press_markup_source); + var ui = canvas.Ui(PressMsg).init(arena); + const tree = try ui.finalize(try view.build(&ui, &model)); + + const select_button = tree.root.children[0]; + const clear_button = tree.root.children[1]; + const plain_button = tree.root.children[2]; + + // The authored binding is in place and the modifiers start false, so a + // plain click behaves exactly as it did before this channel existed. + const plain_click = tree.msgForPointerClick(select_button.id, .up, 1).?; + try testing.expectEqual(@as(i64, 7), plain_click.select.id); + try testing.expect(!plain_click.select.shift); + try testing.expect(!plain_click.select.super); + + // A modified click fills exactly what was held, and never disturbs the + // authored payload. + const modified = tree.msgForPointerClickModified( + select_button.id, + .up, + 1, + .{ .shift = true, .super = true }, + ).?; + try testing.expectEqual(@as(i64, 7), modified.select.id); + try testing.expect(modified.select.shift); + try testing.expect(modified.select.super); + try testing.expect(!modified.select.control); + try testing.expect(!modified.select.alt); + + // The payload-less form carries modifiers alone. + const cleared = tree.msgForPointerClickModified( + clear_button.id, + .up, + 1, + .{ .control = true }, + ).?; + try testing.expect(cleared.clear.control); + try testing.expect(!cleared.clear.shift); + + // An ordinary arm is untouched by the injector. + const plain = tree.msgForPointerClickModified( + plain_button.id, + .up, + 1, + .{ .shift = true }, + ).?; + try testing.expectEqual(@as(i64, 7), plain.plain); +} + +test "declaredWidgetPressRecord accepts the modifier shapes and rejects near misses" { + // The payload-carrying form: four modifier booleans plus the one + // authored field, whatever the app named it. + try testing.expect(markup_view.declaredWidgetPressRecord(struct { + id: i64, + shift: bool, + control: bool, + alt: bool, + super: bool, + })); + // The payload-less form: modifiers only. + try testing.expect(markup_view.declaredWidgetPressRecord(struct { + shift: bool, + control: bool, + alt: bool, + super: bool, + })); + // A missing modifier is an ordinary record, not a press arm. + try testing.expect(!markup_view.declaredWidgetPressRecord(struct { + id: i64, + shift: bool, + control: bool, + alt: bool, + })); + // Modifiers must be booleans: a numeric near-miss would silently take + // the injection path and write the wrong type. + try testing.expect(!markup_view.declaredWidgetPressRecord(struct { + id: i64, + shift: u1, + control: bool, + alt: bool, + super: bool, + })); + // Two authored fields are ambiguous — which one does the binding fill? + try testing.expect(!markup_view.declaredWidgetPressRecord(struct { + id: i64, + other: i64, + shift: bool, + control: bool, + alt: bool, + super: bool, + })); + // The ordinary single-field payload keeps the path it has today. + try testing.expect(!markup_view.declaredWidgetPressRecord(struct { id: i64 })); +} + +test "pressPayloadFieldName names the authored field and nothing else" { + try testing.expectEqualStrings("id", markup_view.pressPayloadFieldName(struct { + id: i64, + shift: bool, + control: bool, + alt: bool, + super: bool, + }).?); + // Field order does not decide it; the modifier vocabulary does. + try testing.expectEqualStrings("rowKey", markup_view.pressPayloadFieldName(struct { + shift: bool, + control: bool, + rowKey: []const u8, + alt: bool, + super: bool, + }).?); + try testing.expect(markup_view.pressPayloadFieldName(struct { + shift: bool, + control: bool, + alt: bool, + super: bool, + }) == null); +} + test "valueArmClass classifies exactly the value-carrying arm shapes" { try testing.expect(markup_view.valueArmClass(f32) == .identity); try testing.expect(markup_view.valueArmClass(f64) == .float); diff --git a/src/runtime/automation_commands.zig b/src/runtime/automation_commands.zig index 5b3641a45..076c5a021 100644 --- a/src/runtime/automation_commands.zig +++ b/src/runtime/automation_commands.zig @@ -37,6 +37,15 @@ pub const AutomationWidgetTarget = struct { id: canvas.ObjectId, }; +/// `widget-click [modifiers]`, where the optional third token +/// spells a chord the way `widget-key` does (`cmd`, `shift`, `cmd+shift`). +/// The modifiers ride the synthetic press so a press-modifier Msg arm can +/// be driven from a test the same way a real click drives it. +pub const AutomationWidgetClick = struct { + target: AutomationWidgetTarget, + modifiers: AutomationKeyModifiers = .{}, +}; + pub const AutomationWidgetWheel = struct { target: AutomationWidgetTarget, delta_y: f32, @@ -223,6 +232,46 @@ pub const AutomationProvenanceTarget = struct { point: ?geometry.PointF = null, }; +/// The modifier-only form of a key chord (`cmd`, `cmd+shift`), for verbs +/// that carry modifiers without a key. Unknown names are refused so a +/// typo reads as an invalid command instead of a silently plain click. +fn parseAutomationModifierToken(token: []const u8) ?AutomationKeyModifiers { + var modifiers = AutomationKeyModifiers{}; + var rest = token; + while (rest.len > 0) { + const separator = std.mem.indexOfScalar(u8, rest, '+') orelse rest.len; + const part = rest[0..separator]; + if (part.len == 0) return null; + if (std.ascii.eqlIgnoreCase(part, "cmd") or std.ascii.eqlIgnoreCase(part, "meta") or std.ascii.eqlIgnoreCase(part, "super")) { + modifiers.command = true; + modifiers.primary = true; + } else if (std.ascii.eqlIgnoreCase(part, "ctrl") or std.ascii.eqlIgnoreCase(part, "control")) { + modifiers.control = true; + } else if (std.ascii.eqlIgnoreCase(part, "alt") or std.ascii.eqlIgnoreCase(part, "option")) { + modifiers.option = true; + } else if (std.ascii.eqlIgnoreCase(part, "shift")) { + modifiers.shift = true; + } else { + return null; + } + if (separator == rest.len) break; + rest = rest[separator + 1 ..]; + } + return modifiers; +} + +pub fn parseAutomationWidgetClick(value: []const u8) !AutomationWidgetClick { + const view = takeAutomationToken(value) orelse return error.InvalidCommand; + const id_part = takeAutomationToken(view.rest) orelse return error.InvalidCommand; + const id = std.fmt.parseInt(canvas.ObjectId, id_part.token, 10) catch return error.InvalidCommand; + if (id == 0) return error.InvalidCommand; + const target = AutomationWidgetTarget{ .view_label = view.token, .id = id }; + const modifier_part = takeAutomationToken(id_part.rest) orelse return .{ .target = target }; + if (takeAutomationToken(modifier_part.rest) != null) return error.InvalidCommand; + const modifiers = parseAutomationModifierToken(modifier_part.token) orelse return error.InvalidCommand; + return .{ .target = target, .modifiers = modifiers }; +} + pub fn parseAutomationProvenanceTarget(value: []const u8) !AutomationProvenanceTarget { const view = takeAutomationToken(value) orelse return error.InvalidCommand; const first = takeAutomationToken(view.rest) orelse return error.InvalidCommand; diff --git a/src/runtime/automation_widget_dispatch.zig b/src/runtime/automation_widget_dispatch.zig index 9209cdf0d..0384d0c0c 100644 --- a/src/runtime/automation_widget_dispatch.zig +++ b/src/runtime/automation_widget_dispatch.zig @@ -13,6 +13,7 @@ const runtime_canvas_widget_context_menu = @import("canvas_widget_context_menu.z const AutomationWidgetAction = automation_commands.AutomationWidgetAction; const AutomationWidgetTarget = automation_commands.AutomationWidgetTarget; +const AutomationWidgetClick = automation_commands.AutomationWidgetClick; const AutomationProvenanceTarget = automation_commands.AutomationProvenanceTarget; const AutomationWidgetWheel = automation_commands.AutomationWidgetWheel; const AutomationWidgetKey = automation_commands.AutomationWidgetKey; @@ -63,7 +64,15 @@ pub fn RuntimeAutomationWidgetDispatch(comptime Runtime: type) type { }; } - pub fn dispatchAutomationWidgetClick(self: *Runtime, app: runtime_api.App(Runtime), target: AutomationWidgetTarget) anyerror!void { + pub fn dispatchAutomationWidgetClick(self: *Runtime, app: runtime_api.App(Runtime), click: AutomationWidgetClick) anyerror!void { + const target = click.target; + const modifiers = platform.ShortcutModifiers{ + .shift = click.modifiers.shift, + .control = click.modifiers.control, + .option = click.modifiers.option, + .command = click.modifiers.command, + .primary = click.modifiers.primary, + }; const view_index = try automationWidgetTargetViewIndex(self, target); const point = try automationWidgetAimPoint(self, view_index, target.id); const window_id = self.views[view_index].window_id; @@ -94,6 +103,7 @@ pub fn RuntimeAutomationWidgetDispatch(comptime Runtime: type) type { .x = point.x, .y = point.y, .button = 0, + .modifiers = modifiers, } }); try self.dispatchPlatformEvent(app, .{ .gpu_surface_input = .{ .window_id = window_id, @@ -103,6 +113,7 @@ pub fn RuntimeAutomationWidgetDispatch(comptime Runtime: type) type { .x = point.x, .y = point.y, .button = 0, + .modifiers = modifiers, } }); try CanvasWidgetDisplayMethods().endCanvasWidgetDisplayListRefreshBatch(self); click_batch_active = false; diff --git a/src/runtime/flow.zig b/src/runtime/flow.zig index cf884b199..89e63bd8c 100644 --- a/src/runtime/flow.zig +++ b/src/runtime/flow.zig @@ -35,6 +35,7 @@ const nowNanoseconds = runtime_clock.nowNanoseconds; const canvasWidgetAccessibilityActionKindFromPlatform = widget_bridge.canvasWidgetAccessibilityActionKindFromPlatform; const parseAutomationCommandName = automation_commands.parseAutomationCommandName; const parseAutomationViewLabel = automation_commands.parseAutomationViewLabel; +const parseAutomationWidgetClick = automation_commands.parseAutomationWidgetClick; const parseAutomationNativeCommand = automation_commands.parseAutomationNativeCommand; const parseAutomationWidgetAction = automation_commands.parseAutomationWidgetAction; const parseAutomationWidgetTarget = automation_commands.parseAutomationWidgetTarget; @@ -959,7 +960,7 @@ pub fn RuntimeFlow(comptime Runtime: type) type { try AutomationWidgetMethods().dispatchAutomationWidgetAction(self, app, try parseAutomationWidgetAction(command.value)); }, .widget_click => { - try AutomationWidgetMethods().dispatchAutomationWidgetClick(self, app, try parseAutomationWidgetTarget(command.value)); + try AutomationWidgetMethods().dispatchAutomationWidgetClick(self, app, try parseAutomationWidgetClick(command.value)); }, .widget_hold => { try AutomationWidgetMethods().dispatchAutomationWidgetHold(self, app, try parseAutomationWidgetTarget(command.value)); diff --git a/src/runtime/ui_app.zig b/src/runtime/ui_app.zig index 69d4e8b3f..35cc9fd91 100644 --- a/src/runtime/ui_app.zig +++ b/src/runtime/ui_app.zig @@ -4739,7 +4739,15 @@ pub fn UiAppWithFeatures(comptime ModelT: type, comptime MsgT: type, comptime fe // `on_double_press` handler (falling back to the ordinary // press), while its first release already dispatched the // single press — select-then-act, the list convention. - if (tree.msgForPointerClick(target.id, pointer_event.pointer.phase, pointer_event.pointer.click_count)) |msg| { + // The modifiers ride along too: an arm declaring the + // press-record shape hears what was held at click time, which + // is what makes cmd-click and shift-click expressible. + if (tree.msgForPointerClickModified( + target.id, + pointer_event.pointer.phase, + pointer_event.pointer.click_count, + pointer_event.pointer.modifiers, + )) |msg| { try self.dispatch(runtime, pointer_event.window_id, msg); } } diff --git a/tools/native-sdk/automation.zig b/tools/native-sdk/automation.zig index 0744ebed6..8f0999559 100644 --- a/tools/native-sdk/automation.zig +++ b/tools/native-sdk/automation.zig @@ -62,8 +62,10 @@ pub fn run(allocator: std.mem.Allocator, io: std.Io, environ_map: *std.process.E defer allocator.free(value); try sendCommand(allocator, io, "widget-action", value); } else if (std.mem.eql(u8, command, "widget-click")) { - if (args.len != 3) return usage(); - const value = try std.fmt.allocPrint(allocator, "{s} {s}", .{ args[1], args[2] }); + // The optional fourth argument is a modifier chord (`cmd`, + // `shift`, `cmd+shift`) held for the synthetic click. + if (args.len != 3 and args.len != 4) return usage(); + const value = try std.mem.join(allocator, " ", args[1..]); defer allocator.free(value); try sendCommand(allocator, io, "widget-click", value); } else if (std.mem.eql(u8, command, "widget-hold")) { @@ -447,7 +449,7 @@ fn printUsage() void { \\ menu-command \\ native-command [view-label] \\ widget-action [value] - \\ widget-click (ids are the bare number; snapshots print #id) + \\ widget-click [modifiers] (ids are the bare number; snapshots print #id; modifiers spell a chord: cmd, shift, cmd+shift) \\ widget-hold (press-and-hold: arms and fires the on_hold timer, release suppressed) \\ widget-context-press (right-click: context menu, or on_hold when the route has none) \\ widget-context-menu (invoke a declared context-menu item; snapshots list them as context_menu=[...])