Treat interaction targets behind the visible keyboard as occluded/off-screen (parity with scroll) #2589

Purpose

When the software keyboard is up, elements it visually covers still report hittable: true with an on-screen rect, so press/click/fill/longpress tap through the keyboard and silently activate whatever the keyboard's own hit-testing routes the point to (or nothing). Keyboard occlusion is a normal, expected iOS/Android pattern — apps are not going to move their tab bar above the IME — so the fix belongs in agent-device: an interaction target whose tap point falls under the visible keyboard should be treated as occluded/off-screen and refused with a typed reason and a recovery hint, exactly the way scroll already does.

This came out of dogfooding MaintainX on a physical iPhone (thymikee-iphone, v5.39-311017).

Context / evidence

On the Messages tab of MaintainX, focusing the search field raised the keyboard over the bottom tab bar. The tab-bar nodes stayed in the app snapshot with their layout geometry and were still hittable:

Button "More"     center=(340,822)  rect={"x":303,"y":795,"width":74,"height":54}
Button "Messages" center=(274,822)  ...

The keyboard occupies roughly y≈540..874, so the More tab is fully behind it. press 340 822 (and press @more) did not navigate; the touch landed on the keyboard. The AX overlay below shows the tab-bar refs drawn under the keys:

The annotated overlay (tab-bar refs @e40–@e46 drawn under the keys) and a reproduction video are in the dogfood evidence bundle (bug3-keyboard-covers-tabbar.png, bug3-keyboard-tabbar.mp4); they need a manual web upload to attach here.

(Reproduction video: local bug3-keyboard-tabbar.mp4 — More tab works with no keyboard, then is dead once the search field is focused. Same class of silent misfire on Android, whose IME is also a separate window.)

Why the current guards miss it

• occlusion (ADR 0011) refuses targets "covered by another visible element" — but occlusion is computed from same-window drawing order in the snapshot tree. The keyboard is a separate window (a system surface), so it never appears as a covering sibling of app elements.
• offscreen refuses targets whose tap point is outside the root Application/Window viewport (isTapPointInsideViewport / isVisibleOnScreen in packages/contracts/src/snapshot-visibility.ts). The covered tab bar is still inside the app window rect, so it passes.

scroll already has this exact notion and does the right thing: SCROLL_KEYBOARD_OCCLUDES_SURFACE_REASON (packages/contracts/src/scroll-command.ts, packages/contracts/src/scroll-gesture.ts, golden table contracts/fixtures/scroll-keyboard-policy.json) and the documented behavior in src/commands/interaction/metadata.ts:

A visible keyboard shortens the swiped band instead of being dismissed; when too little is left, the command refuses with scroll_keyboard_occludes_surface.

Interaction tap paths have no equivalent, which is the asymmetry this issue closes.

Required behavior

• On press/click/fill/longpress (and their --settle/native-ref fast paths), when a software keyboard is visible, compute the keyboard's occluding frame and treat any target whose tap point (rect center) falls inside it as occluded/off-screen.
• Fail with a typed reason on that interaction guarantee (mirroring scroll_keyboard_occludes_surface), plus a hint to dismiss the keyboard first (reference keyboard dismiss, and note iOS only dismisses via the keyboard's own control per fix(ios): give keyboard dismiss a safe-area-tap fallback (#1598) #1606 / mechanism: 'dismissKey' in packages/contracts/src/keyboard.ts).
• Do NOT auto-dismiss the keyboard as a side effect — dismissal has semantic consequences (e.g. commits/cancels edit state); refuse and let the caller decide, consistent with the scroll policy.
• Apply to both platforms: iOS software keyboard and Android IME (focusedPackage/inputMethodPackage path already surfaced in KeyboardCommandResult). Coordinate-only taps must be guarded too, since that is how agents hit-tested past the tree here.
• Keyboard-frame source: derive from the already-captured snapshot (inputView / keyboard nodes are present in the iOS tree) and/or the runner's keyboard-frame API; on Android from the visible IME / visible-to-user inset. If the keyboard frame is unavailable on a backend, fail open (tap proceeds) to avoid regressing current behavior, but record the miss on a typed reason/detail rather than guessing.

Completion conditions

• A coordinate or @ref tap on an app element fully behind the visible keyboard is refused with the new typed reason + dismiss hint, and a follow-up after keyboard dismiss succeeds.
• An element only partially under the keyboard whose tap point (center) is still above it continues to tap normally (center rule, consistent with isTapPointInsideViewport).
• Golden/parity coverage: a keyboard-occlusion fixture under contracts/fixtures/ (like scroll-keyboard-policy.json) and an entry for the new path/guarantee stance in packages/contracts/src/interaction-guarantees.ts for occlusion and/or offscreen.
• Versioned CLI help / metadata updated to name the new refusal reason.

API / CLI shape

# before: silently no-ops (tap hits the keyboard)
agent-device press @more --settle

# after:
Error (COMMAND_FAILED): Target @more is behind the visible keyboard and cannot be tapped
Hint: Run `agent-device keyboard dismiss` (iOS taps the keyboard's own dismiss control) and retry.
details: { reason: 'tap_keyboard_occludes_target', keyboardFrame: {...}, targetRect: {...} }

Dependencies / open questions

• Needs a reliable keyboard-frame provider on the iOS runner (incl. physical devices) and Android; confirm both are exposed at snapshot time before choosing snapshot-derived vs. separate API.
• Decide whether this lands as a new guarantee cell or extends the existing offscreen enforcement set; if it can be computed purely from the captured snapshot node, it likely belongs in the shared runtime preflight (preflightNativeRefInteraction, Interaction guarantee gaps (ADR 0011 umbrella) #1081) so the fast path can't silently succeed.