diff --git a/docs/superpowers/specs/2026-07-04-mobile-terminal-ux-design.md b/docs/superpowers/specs/2026-07-04-mobile-terminal-ux-design.md new file mode 100644 index 0000000000..4232569e49 --- /dev/null +++ b/docs/superpowers/specs/2026-07-04-mobile-terminal-ux-design.md @@ -0,0 +1,174 @@ +# Mobile Terminal UX — Design + +**Date:** 2026-07-04 +**Status:** Approved (decisions settled interactively with Miguel; he is away during implementation) +**Builds on:** PR #22 (touch→wheel scroll bridge, mobile Copy/Paste/Keyboard controls, OSC 52 clipboard) + +## Problem + +The terminal console (xterm.js hosting the tmux-backed `claude` CLI session, plus +plain shell side terminals) is still close to unusable on phones: + +1. When the on-screen keyboard opens, nothing resizes or scrolls — the keyboard + covers the bottom half of the terminal, including claude's input line. +2. There is no way to select text by touch (xterm.js has no touch selection). +3. There is no way to send Esc, Tab, Shift+Tab, Ctrl combos, or arrow keys — + mobile soft keyboards simply don't have them. + +**Platforms:** iOS Safari (primary, plain web + home-screen PWA), Android Chrome +(secondary). Desktop behavior must remain byte-for-byte unchanged: everything +below is gated on touch capability (`useIsTouchDevice` / `isTouchDevice`, same +gates PR #22 used). + +**Scope:** `XTermInstance` and its satellites in `packages/web-core` — so it +applies to both the CLI main pane and shell side terminals automatically. + +## Decisions (settled with Miguel) + +| Topic | Decision | +|---|---| +| Keyboard handling | Shrink + refit: container tracks `visualViewport`, xterm refits, PTY/tmux resizes; input line always visible | +| Key bar | `esc ⇥ ⇧⇥ ^C ctrl ← ↓ ↑ → ⏎` with sticky/latching ctrl; fixed set, no settings UI | +| Gestures | Long-press + drag = arrow D-pad (3 speed tiers); double-tap = Tab; three-finger tap = paste. **No pinch zoom.** | +| Text selection | Explicit select-mode toggle button in the mobile controls (gestures/scroll bridge pause while active) | +| Font size | A+ / A− stepper buttons in mobile controls, persisted in localStorage | +| Delivery | PR + merge via ship-it light; **no /opt deploy** | + +## Design + +### 1. Keyboard-aware viewport (`useVisualViewportHeight` + layout wiring) + +New hook in `packages/web-core/src/shared/hooks/`: subscribes to +`window.visualViewport` `resize` + `scroll` events (SSR-safe, `null` / +no-op when `visualViewport` is unavailable or the device is not +touch-capable). It exposes the current visible height. + +Wiring: the terminal wrapper (in `XTermInstance`'s outer container's ancestor — +the workspace mobile layout) caps its height to the visual viewport when the +keyboard is open, instead of `h-screen`'s layout-viewport height. The existing +`ResizeObserver` + debounced `fitTerminal` in `XTermInstance` then does the +refit + WS resize automatically — no new resize plumbing. + +Details: + +- `interactive-widget=resizes-content` added to the viewport meta in both + `packages/local-web/index.html` and `packages/remote-web/index.html` — + Android Chrome then resizes the layout viewport natively; the hook remains + the iOS path (and a no-op safety net on Android). +- On keyboard open (viewport shrink) and on terminal focus: + `term.scrollToBottom()` so the prompt is visible immediately. +- Compensate iOS Safari's scroll-the-page-on-focus behavior: when + `visualViewport.offsetTop > 0`, scroll the window back so the app chrome + stays pinned (best-effort; keep it simple and defensive). +- The key bar (below) renders inside this sized container, bottom-anchored, so + it naturally sits directly above the system keyboard. + +### 2. Key bar (`TerminalKeyBar.tsx`) + +Touch-only component (returns `null` off-touch), rendered by `XTermInstance` +under the terminal viewport, always visible on touch devices (no fragile +keyboard-open detection; it doubles as the tap-target to focus the terminal). +Horizontally scrollable single row, 44px-high touch targets: + +`esc` `⇥` `⇧⇥` `^C` `ctrl` `←` `↓` `↑` `→` `⏎` + +Sequences sent through the existing single data pipe (`terminal.input(...)`, +which flows through `onData` → `conn.send`): + +- `esc` → `\x1b`; `⇥` → `\t`; `⇧⇥` → `\x1b[Z` (CSI Z); `^C` → `\x03`; + `⏎` → `\r`. +- Arrows respect application cursor keys mode + (`terminal.modes.applicationCursorKeysMode`): `\x1bOA…` vs `\x1b[A…`. +- Buttons must not steal focus from the terminal (`preventDefault` on + pointer/mouse down) so the system keyboard stays open. + +**Sticky ctrl:** tapping `ctrl` latches it (visually highlighted). The next +single printable character arriving on the data pipe is transformed to its +control code (`char & 0x1f`), then ctrl unlatches. Tapping `ctrl` again while +latched cancels. Implemented as a pure transform function +(`applyStickyCtrl(data, latched) → {out, stillLatched}`) inserted in the +`onData` → `send` path in `XTermInstance`, driven by shared state +(ref + subscription) owned by the key bar. Multi-char bursts (paste, IME +commits) pass through untouched and unlatch — only a single-char chunk is +transformed. Unit-tested. + +### 3. Gesture layer (`terminalTouchGestures.ts`) + +Same architecture as `terminalTouchScroll.ts`: a pure, unit-testable gesture +controller (state machine consuming `{type, touches, timestamp}` events and +emitting actions) plus a thin `install` function that binds real touch events +and executes actions. Coordinated with the existing scroll bridge: + +- **Long-press + drag = D-pad.** Finger down and held ≥ ~350 ms within a small + slop radius enters D-pad mode (a subtle overlay indicator appears). Dragging + past a threshold sends repeated arrow keys in the dominant axis direction; + repeat rate has 3 tiers by drag distance (Termius-style "gears"; roughly + ~110 ms / ~55 ms / ~25 ms per keystroke). Lifting the finger exits. While in + D-pad mode the scroll bridge is suppressed. +- **Movement before the long-press threshold** = scroll, handled by the + existing touch-scroll bridge exactly as today. +- **Double-tap = Tab.** Two taps within ~250 ms and tap-slop distance send + `\t`. Single taps still focus the terminal as before (tap behavior itself is + unchanged — we only add the second-tap interpretation). +- **Three-finger tap = paste.** Same clipboard read path + feedback as the + Paste button (including the availability guards from PR #22). +- Arrow sequences honor application cursor keys mode (shared helper with the + key bar). +- Any multi-touch that isn't a clean three-finger tap keeps today's "ignore + until all fingers lift" rule. + +### 4. Select mode (toggle in `TerminalMobileControls`) + +New toggle button (e.g. text-cursor icon) in the existing mobile controls: + +- **On:** gesture layer + touch-scroll bridge are disabled. Touch drag maps to + xterm buffer selection: translate touch coords → cell coords using the + `.xterm-screen` bounding box and `cols`/`rows`, drive + `terminal.select(col, row, length)` across the drag (line-spanning + selections included). The existing `onSelectionChange` handler already + copies selections to the clipboard; the Copy button also picks them up. + A status flash ("Select mode — drag to select") explains the state. +- **Off (default):** everything behaves as designed above. Toggling off clears + the selection. +- No synthetic mouse events are forwarded to tmux — selection is purely + client-side over the visible buffer, which sidesteps mouse-tracking entirely. +- This is the riskiest item (Miguel: skippable if it breaks things). It is + last in the build order and isolated behind its toggle so it can be dropped + without touching the rest. + +### 5. Font size steppers + +`A−` / `A+` buttons in `TerminalMobileControls`: adjust +`terminal.options.fontSize` within 8–20 (default 12), call `fitTerminal`-style +refit (fit + WS resize), persist to `localStorage` +(`vk-terminal-mobile-font-size`). On touch devices, terminal creation reads the +persisted value; desktop keeps the hardcoded 12. + +## Error handling + +- All new listeners/timers install once per created terminal (PR #22 pattern: + live with the element, die on dispose) and must not leak across reattach. +- Clipboard paths reuse PR #22's guarded helpers (distinct + unavailable/blocked/empty feedback). +- `visualViewport` absent → hook returns null → layout falls back to current + behavior (no crash, desktop unchanged). +- Repeat timers (D-pad) must be cancelled on touchend/touchcancel/dispose. + +## Testing + +- Vitest (run via `npx vitest`, per repo convention) for the pure parts: + gesture controller state machine (long-press vs scroll vs double-tap vs + three-finger disambiguation, speed tiers, cancellation), sticky-ctrl + transform, arrow-sequence selection by cursor mode, touch→cell coordinate + mapping, font-size clamp/persistence logic. +- `pnpm run check` + `pnpm run lint` green. +- Manual smoke on device after merge (Miguel; /opt deploy handled separately + via ship-it, not in this change). + +## Build order + +1. Keyboard-aware viewport + meta tag (issue #1 — biggest pain, standalone). +2. Key bar + sticky ctrl (issue #3). +3. Gesture layer (D-pad, double-tap Tab, three-finger paste). +4. Font steppers. +5. Select mode (riskiest, optional, isolated). diff --git a/packages/local-web/index.html b/packages/local-web/index.html index 543f99629b..41e6c16271 100644 --- a/packages/local-web/index.html +++ b/packages/local-web/index.html @@ -7,7 +7,7 @@ - + diff --git a/packages/remote-web/index.html b/packages/remote-web/index.html index 228f40d6df..9d788e5a49 100644 --- a/packages/remote-web/index.html +++ b/packages/remote-web/index.html @@ -2,7 +2,7 @@ - + BetterCoding Remote diff --git a/packages/web-core/src/shared/components/TerminalKeyBar.tsx b/packages/web-core/src/shared/components/TerminalKeyBar.tsx new file mode 100644 index 0000000000..6a16ff8685 --- /dev/null +++ b/packages/web-core/src/shared/components/TerminalKeyBar.tsx @@ -0,0 +1,136 @@ +import type { Terminal } from '@xterm/xterm'; +import type { Icon } from '@phosphor-icons/react'; +import { + ArrowDownIcon, + ArrowElbowDownLeftIcon, + ArrowLeftIcon, + ArrowLineLeftIcon, + ArrowLineRightIcon, + ArrowRightIcon, + ArrowUpIcon, +} from '@phosphor-icons/react'; + +import { cn } from '@/shared/lib/utils'; +import { useIsTouchDevice } from '@/shared/hooks/useIsMobile'; +import { keySequence, type BarKey } from '@/shared/lib/terminalKeySequences'; +import { + getTerminalMobileState, + patchTerminalMobileState, + useTerminalMobileState, +} from '@/shared/lib/terminalMobileState'; + +interface TerminalKeyBarProps { + /** Live terminal (null until mounted) — used for cursor-mode + ctrl latch state. */ + terminal: Terminal | null; + /** Send raw bytes to the PTY (bypasses the sticky-Ctrl transform on typed input). */ + onSendKey: (data: string) => void; +} + +// Icons instead of key-glyph characters (⇥ ⇧⇥ ⏎): those codepoints are +// missing from Android/Linux system fonts and render as tofu boxes; the +// phosphor icons render identically everywhere. 'ctrl' is the latching +// modifier button, rendered specially at its position in this list. +const KEYS: ReadonlyArray<{ + key: BarKey | 'ctrl'; + label?: string; + Icon?: Icon; + aria: string; +}> = [ + { key: 'esc', label: 'esc', aria: 'Escape' }, + { key: 'tab', Icon: ArrowLineRightIcon, aria: 'Tab' }, + { key: 'shift-tab', Icon: ArrowLineLeftIcon, aria: 'Shift+Tab' }, + { key: 'ctrl-c', label: '^C', aria: 'Control+C (interrupt)' }, + { + key: 'ctrl', + label: 'ctrl', + aria: 'Control (sticky — next letter sends the combo)', + }, + { key: 'left', Icon: ArrowLeftIcon, aria: 'Arrow left' }, + { key: 'down', Icon: ArrowDownIcon, aria: 'Arrow down' }, + { key: 'up', Icon: ArrowUpIcon, aria: 'Arrow up' }, + { key: 'right', Icon: ArrowRightIcon, aria: 'Arrow right' }, + { key: 'enter', Icon: ArrowElbowDownLeftIcon, aria: 'Enter' }, +]; + +const KEY_CLASS = + 'flex items-center justify-center min-w-11 h-9 px-2 rounded-md shrink-0 ' + + 'bg-secondary border text-sm text-low active:bg-primary active:text-normal ' + + 'transition-colors select-none'; + +/** + * Termius-style hotkey row for keys mobile soft keyboards can't type: + * Esc / Tab / Shift+Tab / ^C / sticky Ctrl / arrows / Enter. + * + * Touch-only (renders null otherwise) and always visible on touch devices — + * it sits at the bottom of the terminal pane, which the visual-viewport + * sizing keeps directly above the on-screen keyboard while typing. + * + * Buttons preventDefault on pointerdown so a tap never steals focus from + * xterm's textarea — the system keyboard stays open while sending keys. + */ +export function TerminalKeyBar({ terminal, onSendKey }: TerminalKeyBarProps) { + const isTouch = useIsTouchDevice(); + const { ctrlLatched } = useTerminalMobileState(terminal); + + if (!isTouch) return null; + + const keepFocus = (e: { preventDefault: () => void }) => e.preventDefault(); + + const sendKey = (key: BarKey) => { + const appCursor = terminal?.modes.applicationCursorKeysMode ?? false; + onSendKey(keySequence(key, appCursor)); + // A bar tap counts as the latch's "one keystroke" — otherwise ctrl, + // arrow, then a typed letter would surprise-fire a control combo. + if (terminal && getTerminalMobileState(terminal).ctrlLatched) { + patchTerminalMobileState(terminal, { ctrlLatched: false }); + } + terminal?.scrollToBottom(); + }; + + const toggleCtrl = () => { + if (!terminal) return; + patchTerminalMobileState(terminal, { + ctrlLatched: !getTerminalMobileState(terminal).ctrlLatched, + }); + }; + + const renderKey = ({ key, label, Icon, aria }: (typeof KEYS)[number]) => { + const isCtrl = key === 'ctrl'; + return ( + + ); + }; + + return ( + // preventDefault on pointer/mouse down at the toolbar level (events + // bubble) so no tap steals focus from xterm's textarea — the system + // keyboard stays open while sending keys. + // role="group", not "toolbar": the ARIA toolbar contract requires roving + // tabindex + arrow-key traversal, and arrow keys here are CONTENT. +
+ {KEYS.map(renderKey)} +
+ ); +} diff --git a/packages/web-core/src/shared/components/TerminalMobileControls.tsx b/packages/web-core/src/shared/components/TerminalMobileControls.tsx index 8a5ffa2f7c..9457adc703 100644 --- a/packages/web-core/src/shared/components/TerminalMobileControls.tsx +++ b/packages/web-core/src/shared/components/TerminalMobileControls.tsx @@ -1,8 +1,9 @@ -import { useEffect, useRef, useState } from 'react'; +import { useCallback, useEffect, useRef, useState } from 'react'; import type { Terminal } from '@xterm/xterm'; import { CopyIcon, ClipboardTextIcon, + CursorTextIcon, KeyboardIcon, CaretLeftIcon, CaretRightIcon, @@ -11,10 +12,24 @@ import { import { cn } from '@/shared/lib/utils'; import { useIsTouchDevice } from '@/shared/hooks/useIsMobile'; import { extractViewportText } from '@/shared/lib/terminalViewportText'; +import { + clampTerminalFontSize, + saveMobileTerminalFontSize, + TERMINAL_DEFAULT_FONT_SIZE, +} from '@/shared/lib/terminalFontSize'; +import { + getTerminalMobileState, + patchTerminalMobileState, + subscribeTerminalMobileFlash, + useTerminalMobileState, +} from '@/shared/lib/terminalMobileState'; +import { pasteIntoTerminal } from '@/shared/lib/terminalPaste'; interface TerminalMobileControlsProps { - /** Live terminal accessor — refs don't trigger renders, so read on demand. */ - getTerminal: () => Terminal | null; + /** Live terminal (null until the mount effect registers it). */ + terminal: Terminal | null; + /** Re-fit the grid + report the new size to the PTY (font-size steppers). */ + refit: () => void; } const STATUS_MS = 1600; @@ -24,23 +39,27 @@ const BUTTON_CLASS = 'text-low hover:text-normal active:bg-primary transition-colors'; /** - * Touch-only Copy / Paste / Keyboard affordances for the terminal. Desktop keeps - * its mouse/keyboard flow (drag-select, right-click, Ctrl/Cmd+V) untouched — this - * renders nothing unless the device is touch-capable. + * Touch-only Copy / Paste / Select / font-size / Keyboard affordances for the + * terminal. Desktop keeps its mouse/keyboard flow (drag-select, right-click, + * Ctrl/Cmd+V) untouched — this renders nothing unless the device is + * touch-capable. * * Mounted as a sibling of (NOT inside) the xterm element so taps never reach - * xterm's focus/selection handling. Collapsible and pinned top-right so it can't - * cover claude's bottom input. Every action gives explicit feedback (mobile - * clipboard calls fail silently otherwise). + * xterm's focus/selection handling. Collapsible and pinned top-right so it + * can't cover claude's bottom input. Every action gives explicit feedback + * (mobile clipboard calls fail silently otherwise). */ export function TerminalMobileControls({ - getTerminal, + terminal, + refit, }: TerminalMobileControlsProps) { const isTouch = useIsTouchDevice(); const [expanded, setExpanded] = useState(true); const [status, setStatus] = useState(null); const statusTimer = useRef | null>(null); + const { selectMode } = useTerminalMobileState(terminal); + useEffect( () => () => { if (statusTimer.current) clearTimeout(statusTimer.current); @@ -48,43 +67,32 @@ export function TerminalMobileControls({ [] ); - if (!isTouch) return null; - - const flash = (msg: string) => { + const flash = useCallback((msg: string) => { setStatus(msg); if (statusTimer.current) clearTimeout(statusTimer.current); statusTimer.current = setTimeout(() => setStatus(null), STATUS_MS); - }; + }, []); + + // Surface one-shot status messages emitted by the DOM-level installers + // (e.g. the three-finger paste gesture) in the same flash pill. + useEffect(() => { + if (!terminal) return; + return subscribeTerminalMobileFlash(terminal, flash); + }, [terminal, flash]); + + if (!isTouch) return null; const handleKeyboard = () => { - getTerminal()?.focus(); + terminal?.focus(); }; - const handlePaste = async () => { - const term = getTerminal(); - if (!term) return; - // Insecure contexts / some WebViews have no Clipboard API at all — optional - // chaining would otherwise resolve to undefined and look like "empty". - if (!navigator.clipboard?.readText) { - flash('Paste unavailable'); - return; - } - try { - const text = await navigator.clipboard.readText(); - if (!text) { - flash('Clipboard empty'); - return; - } - term.paste(text); - flash('Pasted'); - } catch { - flash('Paste blocked'); - } + const handlePaste = () => { + if (!terminal) return; + void pasteIntoTerminal(terminal, flash); }; const handleCopy = async () => { - const term = getTerminal(); - if (!term) return; + if (!terminal) return; // Guard the write API up front so we never flash "Copied" without copying. if (!navigator.clipboard?.writeText) { flash('Copy unavailable'); @@ -92,12 +100,12 @@ export function TerminalMobileControls({ } let text: string; let label: string; - if (term.hasSelection()) { - text = term.getSelection(); + if (terminal.hasSelection()) { + text = terminal.getSelection(); label = 'Copied selection'; } else { - const buf = term.buffer.active; - text = extractViewportText(buf, buf.viewportY, term.rows); + const buf = terminal.buffer.active; + text = extractViewportText(buf, buf.viewportY, terminal.rows); label = 'Copied screen'; } if (!text) { @@ -112,6 +120,28 @@ export function TerminalMobileControls({ } }; + const handleSelectMode = () => { + if (!terminal) return; + const next = !getTerminalMobileState(terminal).selectMode; + patchTerminalMobileState(terminal, { selectMode: next }); + if (!next) terminal.clearSelection(); + flash(next ? 'Select mode — drag to select' : 'Select mode off'); + }; + + const stepFontSize = (delta: number) => { + if (!terminal) return; + const current = terminal.options.fontSize ?? TERMINAL_DEFAULT_FONT_SIZE; + const next = clampTerminalFontSize(current + delta); + if (next === current) { + flash(`Font ${current}px (limit)`); + return; + } + terminal.options.fontSize = next; + saveMobileTerminalFontSize(next); + refit(); + flash(`Font ${next}px`); + }; + const actions = [ { label: 'Copy from terminal', Icon: CopyIcon, onClick: handleCopy }, { @@ -146,18 +176,50 @@ export function TerminalMobileControls({ {status} )} - {expanded && - actions.map(({ label, Icon, onClick }) => ( + {expanded && ( + <> + {actions.map(({ label, Icon, onClick }) => ( + + ))} + - ))} + + + )}