diff --git a/README.md b/README.md index b3f52cf..b399914 100644 --- a/README.md +++ b/README.md @@ -61,36 +61,52 @@ lives in **VsCodeOsCore** — a VS Code extension that ships *inside* the editor rather than being installed from the Marketplace. Its source is in [`extension/`](extension/), and it adds: +- **An all-apps button**, in the bottom-left corner — a searchable grid of every + program on the machine, the way a start menu works. - **A tray**, at the right end of the status bar. Left to right: now playing, - battery, volume, network, the clock and date, and the power button in the - corner. Each one opens a flyout in the bottom panel — a Windows-style card - that rises directly above the item you clicked. + battery, volume, network, Bluetooth, the clock and date, and the power button + in the corner. Each one opens a card in the side bar. (VS Code has no API for + a popup anchored to a status bar item; the side bar is the closest thing that + does not take the terminal panel away from you. `vscodeos.flyout.location` + puts the cards back in the bottom panel if you prefer them there.) - **Power** — sleep, restart, shut down and log out, with a confirmation. - **Calendar** — a month grid with today highlighted, on the clock. -- **Quick settings** — Wi-Fi, Bluetooth, airplane mode, energy saver, night - light and accessibility, plus brightness and volume sliders. Tiles hide - themselves when the hardware is not there, rather than showing a dead switch. -- **Network** — scan, connect with a password, and switch between saved - connections. +- **Power settings**, on the battery — energy saver, a brightness slider whose + moon button toggles night light, and the charge left. Nothing else: the radio + switches live on the cards that own them. +- **Network** — scan, connect with a password, switch between saved connections, + and airplane mode. +- **Bluetooth** — turn the adapter on, scan for devices, pair, connect and + forget. The button hides itself when there is no adapter. - **Task Manager**, in the activity bar — processes with CPU and memory, per-core meters, load average, uptime and CPU temperature, sortable and filterable, with End task. - **Files** — a graphical file explorer with a places sidebar, grid and list - views, rename, trash, copy and paste. Text opens in the editor; everything else - goes to `xdg-open`. + views, rename, trash, copy and paste. **Everything opens in the editor**: text + in the text editor, images in the built-in preview, video and audio in the + media player. +- **Browser** — a real browser rendered *inside* an editor tab, with tabs, an + address bar and history. It drives a headless Chromium and streams its picture + back, which is the only way to show sites that refuse to be framed. "Open in + browser" hands the page to a real window when that is the better answer. +- **Media Player** — video and audio in a tab, with a playlist of whatever else + is in the folder. - **Music** — transport controls for whatever is playing, over MPRIS, plus - one-click launchers that open Spotify Web and YouTube Music as their own - browser windows. -- **Apps** — Calculator, Notepad, Paint, Screenshot and Voice Recorder. `VS Code - OS: All Apps…` in the command palette (**Ctrl** + **Shift** + **P**) lists - everything. + one-click launchers for Spotify Web and YouTube Music. +- **Updater** — a GUI for the two update paths below, with live output and a + restart prompt when one is needed. +- **Apps** — Calculator, Paint, Screenshot and Voice Recorder. Pressing + **Print Screen** jumps straight to a region capture. + +There is deliberately no Notepad: the editor is a better one, and the file +manager now opens text in it. Two honest limits, both imposed by VS Code rather than by this project: **Spotify audio cannot play inside the editor** (VS Code's Electron ships no Widevine, so the Web Playback SDK cannot work — which is exactly why the player controls a real browser window instead), and **Microsoft Edge is not on either image** (it is AUR-only on Arch, and Microsoft publishes no ARM64 Linux build at -all, so the Pi could never have matched). The browser launcher prefers +all, so the Pi could never have matched). Everything browser-shaped prefers `microsoft-edge-stable` if you install it yourself, and falls back to Chromium. Every part of the shell can be turned off individually in settings under @@ -205,6 +221,9 @@ VSCODEOS_RESPAWN=1 # 0 = do not relaunch when VS Code exits ## Day-to-day +The **Updater** app does all of this with buttons — open it from the all-apps +button in the bottom-left corner. By hand: + ```bash sudo pacman -Syu # update the Arch base sudo vscodeos-update-code # update VS Code itself (it is not a pacman package) @@ -212,6 +231,12 @@ nmtui # join a Wi-Fi network (or use the tray) code ~/Projects/thing # open something in the running editor ``` +The Updater runs the same work through `/usr/local/bin/vscodeos-update`, which +it launches with `pkexec`. polkit is configured to allow that one program +without a password (`/etc/polkit-1/rules.d/49-vscodeos.rules`) because there is +no authentication agent in a session whose entire UI is the editor — a password +prompt would have nowhere to appear. + Extensions, settings sync and Marketplace sign-in all work normally; `gnome-keyring` is started by the session so credentials persist. diff --git a/archiso/packages.x86_64 b/archiso/packages.x86_64 index 1a3a924..db07bb8 100644 --- a/archiso/packages.x86_64 +++ b/archiso/packages.x86_64 @@ -27,6 +27,11 @@ mkinitcpio-nfs-utils sudo polkit dbus +# checkupdates, for the Updater app. It lists pending updates against a copy of +# the sync database, so the GUI never has to run `pacman -Sy` - which would +# leave the real database half-synced and the machine one command away from a +# partial upgrade. +pacman-contrib # Live medium / installer tooling arch-install-scripts diff --git a/archiso/profiledef.sh b/archiso/profiledef.sh index ee64cd2..4cbf298 100755 --- a/archiso/profiledef.sh +++ b/archiso/profiledef.sh @@ -26,6 +26,11 @@ file_permissions=( ["/usr/local/bin/vscodeos-kiosk"]="0:0:755" ["/usr/local/bin/vscodeos-update-code"]="0:0:755" ["/usr/local/bin/vscodeos-install-extensions"]="0:0:755" + # This one is load-bearing for security, not just for tidiness: polkit grants + # `pkexec /usr/local/bin/vscodeos-update` without a password, so a copy the + # kiosk user could write to would be a one-line path to root. + ["/usr/local/bin/vscodeos-update"]="0:0:755" + ["/usr/local/bin/vscodeos-screenshot"]="0:0:755" ["/usr/local/bin/code"]="0:0:755" ["/etc/sudoers.d/vscodeos"]="0:0:0440" ) diff --git a/extension/README.md b/extension/README.md index 5d7d787..2fc0e43 100644 --- a/extension/README.md +++ b/extension/README.md @@ -12,16 +12,44 @@ accidentally uninstalled. | | | | --- | --- | -| **Tray** | Power button, clock and date, battery, volume, network and now-playing, at the right end of the status bar | -| **Flyouts** | Power, calendar, quick settings, volume mixer, network picker and music player | +| **Launcher** | An all-apps button in the bottom-left corner: a searchable grid of every program | +| **Tray** | Now-playing, battery, volume, network, Bluetooth, clock and date, and the power button, at the right end of the status bar | +| **Flyouts** | Apps, power, calendar, power settings, volume mixer, network picker, Bluetooth and music player | | **Task Manager** | Processes with CPU/RAM, per-core meters, load, uptime and thermals, in the activity bar | -| **Files** | A graphical file explorer: places sidebar, grid/list, rename, trash, copy/paste, open with `xdg-open` | +| **Files** | A graphical file explorer: places sidebar, grid/list, rename, trash, copy/paste. Everything opens in the editor | +| **Browser** | A headless Chromium streamed into an editor tab, with tabs, an address bar and history | +| **Media Player** | Video and audio in a tab, with a folder playlist | | **Music** | MPRIS transport for whatever is playing, plus launchers for Spotify Web and YouTube Music | -| **Browser** | Launches Edge → Chromium → Firefox, whichever is installed | -| **Apps** | Calculator, Notepad, Paint, Screenshot and Voice Recorder | +| **Updater** | pacman, VS Code and the shell itself, with live output | +| **Apps** | Calculator, Paint, Screenshot and Voice Recorder | Every feature is behind a `vscodeos..enabled` setting, all defaulting to -on. `VS Code OS: All Apps…` in the command palette lists everything. +on. The all-apps button, or `VS Code OS: All Apps…` in the command palette, +lists everything. + +### Three things worth knowing + +**The flyouts are not popups.** VS Code has no API to anchor one to a status bar +item, and the only floating-window route — moving an editor to an auxiliary +window — brings editor tab chrome with it and cannot be sized or placed. So they +are a webview view, in the side bar by default. The bottom panel, where they used +to live, is where the terminal is; `vscodeos.flyout.location` puts them back. + +**The browser is a screencast.** Every site worth visiting sends +`X-Frame-Options`, which is why VS Code's own Simple Browser shows a blank +rectangle on most of them. `puppeteer-core` drives a headless Chromium, CDP's +`Page.startScreencast` pushes JPEG frames into an ``, and pointer and key +events go back through `Input.dispatch*`. That costs an encode and a decode per +frame, so `vscodeos.browser.frameRate` and `vscodeos.browser.quality` are +settings — turn them down on a Pi — the stream stops when the tab is hidden, and +"Open in browser" is always there. `puppeteer-core` is the extension's one +runtime dependency and is bundled into `dist/extension.js` like everything else. + +**Print Screen is bound by the window manager, not here.** On X11 the Print key +never reaches Electron as a keydown, so a contributed keybinding cannot see it. +Openbox runs `/usr/local/bin/vscodeos-screenshot`, which hands +`vscode://vscodeos.vscodeos-core/screenshot?mode=region` to the running editor, +where a `registerUriHandler` picks it up. ## Developing @@ -57,8 +85,9 @@ src/ extension.ts activate(): wires everything, one DisposableStore sys/ the only code that touches the machine statusbar/ the tray, and the priority ladder that orders it - views/ flyout (panel) and task manager (activity bar) providers - apps/ file explorer, mini-apps, panel plumbing + views/ flyout (side bar) and task manager (activity bar) providers + apps/ registry, file explorer, browser, media player, updater, + mini-apps, panel plumbing webview/ HTML shell + the host↔webview message types media/ src/ one TypeScript entry point per page, shared code in src/lib diff --git a/extension/esbuild.mjs b/extension/esbuild.mjs index 497b6ce..3c2e4ce 100644 --- a/extension/esbuild.mjs +++ b/extension/esbuild.mjs @@ -6,7 +6,9 @@ // media/src/*.ts -> media/dist/*.js browser/IIFE, runs inside each webview // // Nothing is left in node_modules: the shipped extension is package.json, dist/ -// and media/, which keeps the payload that goes into every image small. +// and media/, which keeps the payload that goes into every image small. That +// applies to puppeteer-core too - the browser app's only runtime dependency is +// bundled into dist/extension.js like everything else. import * as esbuild from 'esbuild'; import { readdirSync } from 'node:fs'; @@ -40,8 +42,16 @@ const configs = [ platform: 'node', target: 'node18', format: 'cjs', - // Supplied by the extension host at run time, never bundled. - external: ['vscode'], + external: [ + // Supplied by the extension host at run time, never bundled. + 'vscode', + // Optional native accelerators that `ws` (a puppeteer-core dependency) + // requires inside a try/catch. esbuild resolves them at build time + // regardless of the guard, so they have to be named here or the + // bundle fails on a machine that never installed them. + 'bufferutil', + 'utf-8-validate', + ], }, { ...common, diff --git a/extension/media/css/vscodeos.css b/extension/media/css/vscodeos.css index 8025b13..fb43859 100644 --- a/extension/media/css/vscodeos.css +++ b/extension/media/css/vscodeos.css @@ -92,6 +92,21 @@ select { width: min(560px, 100%); } +/* In the side bar the container is already about the card's width, so the + corner-anchoring and the drop shadow have nothing to do and the borders just + waste horizontal space. The card fills the view instead. */ +@media (max-width: 460px) { + .flyout-host { padding: 8px; } + .flyout, + .flyout.wide { + width: 100%; + border: none; + box-shadow: none; + background: transparent; + padding: 6px 2px; + } +} + @keyframes flyout-in { from { opacity: 0; transform: translateY(6px); } to { opacity: 1; transform: none; } @@ -434,6 +449,91 @@ input[type='range']::-webkit-slider-thumb { font-variant-numeric: tabular-nums; } +/* --------------------------------------------------------- app launcher */ + +.app-search-row { + display: flex; + align-items: center; + gap: 8px; + padding: 6px 10px; + margin-bottom: 10px; + border-radius: var(--vscodeos-radius); + background: var(--vscode-input-background, rgba(127, 127, 127, 0.12)); + border: 1px solid var(--vscode-input-border, transparent); +} + +.app-search-icon { + display: flex; + color: var(--vscodeos-muted); +} + +.app-search { + flex: 1; + min-width: 0; + border: none; + background: transparent; + color: var(--vscode-input-foreground, inherit); + font: inherit; + outline: none; +} + +.app-grid { + display: grid; + grid-template-columns: repeat(auto-fill, minmax(96px, 1fr)); + gap: 8px; +} + +.app-tile { + display: flex; + flex-direction: column; + align-items: center; + justify-content: flex-start; + gap: 8px; + padding: 14px 6px 12px; + border-radius: var(--vscodeos-radius); + background: var(--vscodeos-tile-bg); + color: var(--vscodeos-tile-fg); + text-align: center; +} + +.app-tile:hover { + background: var(--vscode-button-secondaryHoverBackground, rgba(127, 127, 127, 0.24)); +} + +.app-tile:focus-visible { + outline: 1px solid var(--vscode-focusBorder); + outline-offset: -1px; +} + +.app-tile-icon { + display: flex; + color: var(--vscodeos-accent); +} + +.app-tile-label { + font-size: 0.88em; + line-height: 1.25; +} + +/* An icon-button that is a toggle rather than an action. */ +.icon-button.on { + background: var(--vscodeos-accent); + color: var(--vscodeos-accent-fg); +} + +.battery-readout { + display: flex; + align-items: baseline; + gap: 10px; + padding: 12px 4px 2px; +} + +.battery-level { + font-size: 1.9em; + font-weight: 300; + font-variant-numeric: tabular-nums; +} + /* ----------------------------------------------------------- app chrome */ .app { @@ -987,3 +1087,243 @@ table.processes tbody tr.selected { audio { width: 100%; } + +/* --------------------------------------------------------- media player */ + +.player-body { + display: flex; + gap: 0; + overflow: hidden; +} + +.player-stage { + flex: 1; + min-width: 0; + display: flex; + align-items: center; + justify-content: center; + position: relative; + background: #000000; +} + +.player-surface { + width: 100%; + height: 100%; + max-height: 100%; + object-fit: contain; + background: #000000; +} + +/* Audio has nothing to look at, so the element is left visible for its + controls and a glyph is floated over the empty picture area. */ +.player-art { + position: absolute; + inset: 0; + display: flex; + align-items: center; + justify-content: center; + pointer-events: none; + color: var(--vscodeos-muted); +} + +.player-art[hidden] { + display: none; +} + +.player-title { + font-weight: 600; + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; + max-width: 45%; +} + +.player-list { + flex: 0 0 240px; + overflow: auto; + border-left: 1px solid var(--vscode-panel-border, transparent); + padding-bottom: 10px; +} + +@media (max-width: 720px) { + .player-body { flex-direction: column; } + .player-list { flex: 0 0 auto; max-height: 34%; border-left: none; border-top: 1px solid var(--vscode-panel-border, transparent); } +} + +/* ------------------------------------------------------------- browser */ + +.browser-tabs { + display: flex; + align-items: stretch; + gap: 2px; + padding: 4px 6px 0; + overflow-x: auto; + flex: 0 0 auto; +} + +.browser-tab { + display: flex; + align-items: center; + gap: 6px; + max-width: 190px; + padding: 5px 8px; + border-radius: 5px 5px 0 0; + background: var(--vscodeos-tile-bg); + cursor: pointer; + font-size: 0.9em; +} + +.browser-tab.active { + background: var(--vscode-editor-background); + color: var(--vscode-foreground); +} + +.browser-tab-title { + overflow: hidden; + text-overflow: ellipsis; + white-space: nowrap; +} + +.browser-tab-close, +.browser-tab-new { + display: inline-flex; + align-items: center; + justify-content: center; + width: 20px; + height: 20px; + border-radius: 4px; + flex: 0 0 auto; +} + +.browser-tab-close:hover, +.browser-tab-new:hover { + background: var(--vscode-toolbar-hoverBackground, rgba(127, 127, 127, 0.25)); +} + +.browser-toolbar { + flex-wrap: nowrap; +} + +.browser-address { + flex: 1; + min-width: 80px; +} + +.browser-body { + display: flex; + overflow: hidden; +} + +/* The stage takes focus so the keyboard can be forwarded; it must not draw a + focus ring around the whole page while doing it. */ +.browser-stage { + flex: 1; + min-width: 0; + position: relative; + background: #ffffff; + outline: none; +} + +.browser-screen { + position: absolute; + inset: 0; + width: 100%; + height: 100%; + object-fit: contain; + /* The frames are already blurry from JPEG; smoothing them further just + costs time on a Pi. */ + image-rendering: auto; + user-select: none; + -webkit-user-drag: none; +} + +/* ------------------------------------------------------------- updater */ + +.update-rows { + display: flex; + flex-direction: column; + gap: 8px; + padding: 12px; +} + +.update-row { + display: flex; + align-items: flex-start; + gap: 12px; + padding: 12px; + border-radius: var(--vscodeos-radius); + background: var(--vscodeos-tile-bg); +} + +.update-icon { + display: flex; + color: var(--vscodeos-accent); + padding-top: 2px; +} + +.update-main { + flex: 1; + min-width: 0; +} + +.update-title { + font-weight: 600; +} + +.update-side { + display: flex; + flex-direction: column; + align-items: flex-end; + gap: 8px; + flex: 0 0 auto; +} + +.update-badge { + font-size: 0.85em; + padding: 2px 8px; + border-radius: 10px; + background: var(--vscode-badge-background, rgba(127, 127, 127, 0.3)); + color: var(--vscode-badge-foreground, inherit); + white-space: nowrap; +} + +.update-badge.available { + background: var(--vscodeos-accent); + color: var(--vscodeos-accent-fg); +} + +.update-badge.current { + background: transparent; + color: var(--vscodeos-muted); +} + +.update-detail { + margin: 8px 0 0; + padding: 8px; + max-height: 150px; + overflow: auto; + border-radius: 4px; + background: var(--vscode-textCodeBlock-background, rgba(127, 127, 127, 0.12)); + font-family: var(--vscode-editor-font-family, monospace); + font-size: 0.85em; + white-space: pre-wrap; +} + +.update-restart { + padding: 12px; + border-radius: var(--vscodeos-radius); + border: 1px solid var(--vscodeos-accent); +} + +.update-log { + margin: 0 12px 12px; + padding: 10px; + max-height: 340px; + overflow: auto; + border-radius: var(--vscodeos-radius); + background: var(--vscode-terminal-background, var(--vscode-editor-background)); + font-family: var(--vscode-editor-font-family, monospace); + font-size: 0.86em; + white-space: pre-wrap; + word-break: break-word; +} diff --git a/extension/media/icons/browser.svg b/extension/media/icons/browser.svg new file mode 100644 index 0000000..00a2005 --- /dev/null +++ b/extension/media/icons/browser.svg @@ -0,0 +1,3 @@ + + + diff --git a/extension/media/icons/player.svg b/extension/media/icons/player.svg new file mode 100644 index 0000000..684c431 --- /dev/null +++ b/extension/media/icons/player.svg @@ -0,0 +1,3 @@ + + + diff --git a/extension/media/icons/notepad.svg b/extension/media/icons/updater.svg similarity index 63% rename from extension/media/icons/notepad.svg rename to extension/media/icons/updater.svg index 89f59e4..09ca560 100644 --- a/extension/media/icons/notepad.svg +++ b/extension/media/icons/updater.svg @@ -1,3 +1,3 @@ - + diff --git a/extension/media/src/browser.ts b/extension/media/src/browser.ts new file mode 100644 index 0000000..07e3b07 --- /dev/null +++ b/extension/media/src/browser.ts @@ -0,0 +1,240 @@ +// Web browser front end. +// +// The page itself is an that a headless Chromium repaints many times a +// second. Everything below it exists to make that image behave like a browser: +// the pointer and the keyboard are forwarded to the real page, and coordinates +// are scaled because the is almost never the size of the viewport it is +// showing. + +import { clear, h, onMessage, post, root, throttle } from './lib/dom'; +import { icon } from './lib/icons'; +import type { BrowserInput, BrowserState, HostMessage } from '../../src/webview/protocol'; + +let state: BrowserState = { + url: '', title: '', loading: false, canGoBack: false, canGoForward: false, tabs: [], +}; +/** The size of the remote viewport, as reported with the last frame. */ +let frameSize = { width: 1280, height: 800 }; +let addressFocused = false; + +const screen = h('img', { class: 'browser-screen', alt: '' }); +const stage = h('div', { class: 'browser-stage', tabIndex: 0 }, screen); +const tabsRow = h('div', { class: 'browser-tabs' }); +const banner = h('div', { class: 'error-banner', hidden: true }); + +const address = h('input', { + class: 'browser-address', + type: 'text', + placeholder: 'Search or enter an address', + 'aria-label': 'Address', + on: { + focus: () => { addressFocused = true; }, + blur: () => { addressFocused = false; }, + keydown: (event: KeyboardEvent) => { + if (event.key === 'Enter') { + post({ type: 'browserNavigate', url: (event.target as HTMLInputElement).value }); + stage.focus(); + } + }, + }, +}); + +const navButton = (glyph: string, title: string, action: 'back' | 'forward' | 'reload' | 'home'): HTMLButtonElement => + h('button', { + class: 'icon-button', + title, + html: icon(glyph, 16), + on: { click: () => post({ type: 'browserGo', action }) }, + }); + +const backButton = navButton('chevronLeft', 'Back', 'back'); +const forwardButton = navButton('chevronRight', 'Forward', 'forward'); + +clear(root()).append(h('div', { class: 'app' }, + tabsRow, + h('div', { class: 'toolbar browser-toolbar' }, + backButton, + forwardButton, + navButton('refresh', 'Reload', 'reload'), + navButton('home', 'Homepage', 'home'), + address, + h('button', { + class: 'button', + title: 'Open this page in a real browser window', + on: { click: () => post({ type: 'browserExternal' }) }, + }, h('span', { html: icon('open', 15) }), 'Open in browser'), + ), + banner, + h('div', { class: 'body browser-body' }, stage), +)); + +// ------------------------------------------------------------------ frames + +onMessage((message) => { + if (message.type === 'browserFrame') { + frameSize = { width: message.width, height: message.height }; + screen.src = `data:image/jpeg;base64,${message.data}`; + return; + } + + if (message.type === 'browserState') { + state = message.state; + // Never fight the user for the address bar while they are typing in it. + if (!addressFocused) { + address.value = state.url === 'about:blank' ? '' : state.url; + } + backButton.disabled = !state.canGoBack; + forwardButton.disabled = !state.canGoForward; + renderTabs(); + return; + } + + if (message.type === 'browserError') { + clear(banner).append(h('div', {}, message.message)); + banner.hidden = false; + if (!message.fatal) { + setTimeout(() => { banner.hidden = true; }, 6000); + } + } +}); + +function renderTabs(): void { + clear(tabsRow); + for (const tab of state.tabs) { + tabsRow.append(h('div', { + class: `browser-tab${tab.active ? ' active' : ''}`, + title: tab.title, + on: { click: () => post({ type: 'browserTab', action: 'select', id: tab.id }) }, + }, + h('span', { class: 'browser-tab-title' }, tab.title || 'New tab'), + h('button', { + class: 'browser-tab-close', + title: 'Close tab', + html: icon('close', 12), + on: { + click: (event: MouseEvent) => { + event.stopPropagation(); + post({ type: 'browserTab', action: 'close', id: tab.id }); + }, + }, + }), + )); + } + tabsRow.append(h('button', { + class: 'browser-tab-new', + title: 'New tab', + html: icon('plus', 14), + on: { click: () => post({ type: 'browserTab', action: 'new' }) }, + })); +} + +// ------------------------------------------------------------------- sizing + +const sendSize = throttle(() => { + const rect = stage.getBoundingClientRect(); + if (rect.width > 0 && rect.height > 0) { + post({ type: 'browserResize', width: Math.round(rect.width), height: Math.round(rect.height) }); + } +}, 250); + +new ResizeObserver(() => sendSize()).observe(stage); + +// ------------------------------------------------------------------- input + +/** + * Where a click in the lands in the real page. The image is letterboxed by + * `object-fit: contain`, so both the scale and the offset have to come out. + */ +function toPage(event: MouseEvent): { x: number; y: number } { + const rect = screen.getBoundingClientRect(); + const scale = Math.min(rect.width / frameSize.width, rect.height / frameSize.height) || 1; + const drawnWidth = frameSize.width * scale; + const drawnHeight = frameSize.height * scale; + const offsetX = (rect.width - drawnWidth) / 2; + const offsetY = (rect.height - drawnHeight) / 2; + return { + x: Math.round((event.clientX - rect.left - offsetX) / scale), + y: Math.round((event.clientY - rect.top - offsetY) / scale), + }; +} + +/** CDP's modifier bitmask: alt 1, ctrl 2, meta 4, shift 8. */ +function modifiers(event: MouseEvent | KeyboardEvent | WheelEvent): number { + return (event.altKey ? 1 : 0) | (event.ctrlKey ? 2 : 0) | (event.metaKey ? 4 : 0) | (event.shiftKey ? 8 : 0); +} + +const BUTTONS = ['left', 'middle', 'right'] as const; + +function send(input: BrowserInput): void { + post({ type: 'browserInput', input }); +} + +function mouse(type: 'mousePressed' | 'mouseReleased' | 'mouseMoved', event: MouseEvent): void { + const { x, y } = toPage(event); + send({ + kind: 'mouse', + type, + x, + y, + button: type === 'mouseMoved' ? 'none' : (BUTTONS[event.button] ?? 'left'), + buttons: event.buttons, + clickCount: type === 'mouseMoved' ? 0 : event.detail || 1, + modifiers: modifiers(event), + }); +} + +stage.addEventListener('mousedown', (event) => { + event.preventDefault(); + stage.focus(); + mouse('mousePressed', event); +}); +stage.addEventListener('mouseup', (event) => mouse('mouseReleased', event)); +// Moves are the noisiest events by far and the page only needs enough of them +// for hover and drag to feel continuous. +stage.addEventListener('mousemove', throttle((event: MouseEvent) => mouse('mouseMoved', event), 40)); +stage.addEventListener('contextmenu', (event) => event.preventDefault()); + +stage.addEventListener('wheel', (event) => { + event.preventDefault(); + const { x, y } = toPage(event); + send({ kind: 'wheel', x, y, deltaX: event.deltaX, deltaY: event.deltaY, modifiers: modifiers(event) }); +}, { passive: false }); + +/** + * Keys that produce a character get a `char` event as well, which is what + * actually types into a text field; the rest only need down and up. + */ +stage.addEventListener('keydown', (event) => { + if (event.key === 'F5') { + event.preventDefault(); + post({ type: 'browserGo', action: 'reload' }); + return; + } + event.preventDefault(); + const base = { + key: event.key, + code: event.code, + windowsVirtualKeyCode: event.keyCode, + modifiers: modifiers(event), + }; + send({ kind: 'key', type: 'keyDown', ...base, text: event.key.length === 1 ? event.key : undefined }); + if (event.key.length === 1 && !event.ctrlKey && !event.metaKey) { + send({ kind: 'key', type: 'char', ...base, text: event.key }); + } +}); + +stage.addEventListener('keyup', (event) => { + event.preventDefault(); + send({ + kind: 'key', + type: 'keyUp', + key: event.key, + code: event.code, + windowsVirtualKeyCode: event.keyCode, + modifiers: modifiers(event), + }); +}); + +renderTabs(); +sendSize(); +post({ type: 'ready' }); diff --git a/extension/media/src/flyout.ts b/extension/media/src/flyout.ts index 5bb474f..af0adfe 100644 --- a/extension/media/src/flyout.ts +++ b/extension/media/src/flyout.ts @@ -1,18 +1,21 @@ -// The six tray flyouts, drawn as one right-anchored card. +// The eight tray flyouts, drawn as one card. // -// The panel is full width, so the card pins itself to the bottom-right corner: -// that is what makes it read as a flyout rising out of the tray item that was -// clicked, rather than as a docked panel. +// In the bottom panel the host is full width, so the card pins itself to the +// bottom-right corner and reads as a flyout rising out of the tray item that was +// clicked. In the side bar - the default - the container is already the right +// width and the anchoring does nothing, which is exactly what is wanted there. import { append, clear, h, onMessage, post, root, throttle, formatDuration } from './lib/dom'; import { icon, signalIcon } from './lib/icons'; -import type { FlyoutKind, FlyoutState, HostMessage } from '../../src/webview/protocol'; +import type { AppEntry, BluetoothDevice, FlyoutKind, FlyoutState, HostMessage } from '../../src/webview/protocol'; -let kind: FlyoutKind = 'quicksettings'; +let kind: FlyoutKind = 'apps'; let state: FlyoutState | undefined; let busy: string | undefined; /** Month the calendar is showing; the clock keeps ticking regardless. */ let calendarMonth = new Date(); +/** What has been typed into the launcher's search box. */ +let appQuery = ''; const host = h('div', { class: 'flyout-host' }); const card = h('div', { class: 'flyout' }); @@ -25,6 +28,7 @@ onMessage((message) => { if (message.kind !== kind) { kind = message.kind; calendarMonth = new Date(); + appQuery = ''; busy = undefined; } render(); @@ -58,15 +62,17 @@ setInterval(() => { }, 1000); function render(): void { - card.classList.toggle('wide', kind === 'network' || kind === 'music'); + card.classList.toggle('wide', kind === 'network' || kind === 'music' || kind === 'apps' || kind === 'bluetooth'); clear(card); switch (kind) { + case 'apps': return renderApps(); case 'power': return renderPower(); case 'calendar': return renderCalendar(); case 'volume': return renderVolume(); case 'network': return renderNetwork(); + case 'bluetooth': return renderBluetooth(); case 'music': return renderMusic(); - default: return renderQuickSettings(); + default: return renderPowerSettings(); } } @@ -74,6 +80,78 @@ function title(text: string): HTMLElement { return h('h2', { class: 'flyout-title' }, text); } +// -------------------------------------------------------------- app launcher + +function matches(app: AppEntry, query: string): boolean { + if (!query) { + return true; + } + const haystack = [app.title, app.description, ...(app.keywords ?? [])].join(' ').toLowerCase(); + // Every word has to appear somewhere, so "voice rec" finds the recorder but + // "voice paint" finds nothing. + return query.toLowerCase().split(/\s+/).filter(Boolean).every((word) => haystack.includes(word)); +} + +function renderApps(): void { + const open = (app: AppEntry): void => post({ type: 'command', command: app.command }); + + const search = h('input', { + class: 'app-search', + type: 'text', + placeholder: 'Search apps', + value: appQuery, + 'aria-label': 'Search apps', + on: { + input: (event: Event) => { + appQuery = (event.target as HTMLInputElement).value; + renderGrid(); + }, + keydown: (event: KeyboardEvent) => { + if (event.key === 'Enter') { + const first = (state?.apps ?? []).filter((app) => matches(app, appQuery))[0]; + if (first) { + open(first); + } + } + }, + }, + }); + + const grid = h('div', { class: 'app-grid' }); + + function renderGrid(): void { + const list = (state?.apps ?? []).filter((app) => matches(app, appQuery)); + clear(grid); + if (list.length === 0) { + grid.append(h('div', { class: 'empty' }, 'No app matches that.')); + return; + } + for (const app of list) { + grid.append(h('button', { + class: 'app-tile', + title: app.description, + on: { click: () => open(app) }, + }, + h('span', { class: 'app-tile-icon', html: icon(app.icon, 26) }), + h('span', { class: 'app-tile-label' }, app.title), + )); + } + } + + card.append( + h('div', { class: 'app-search-row' }, + h('span', { class: 'app-search-icon', html: icon('search', 16) }), + search, + ), + grid, + ); + renderGrid(); + + // Typing should start immediately, the way a start menu does. The view is + // focused by the host unless it was opened with preserveFocus. + setTimeout(() => search.focus(), 0); +} + // ------------------------------------------------------------------- power type PowerName = 'poweroff' | 'reboot' | 'suspend' | 'logout'; @@ -162,107 +240,55 @@ function renderCalendar(): void { ); } -// ----------------------------------------------------------- quick settings - -function renderQuickSettings(): void { - const tiles = h('div', { class: 'tiles' }); +// ----------------------------------------------------------- power settings - const tile = (options: { - on: boolean; - glyph: string; - label: string; - sub?: string; - onClick: () => void; - disabled?: boolean; - }): HTMLElement => - h('button', { - class: `tile${options.on ? ' on' : ''}`, - disabled: options.disabled, - on: { click: options.onClick }, - }, - h('span', { html: options.glyph }), - h('span', { class: 'tile-label' }, - options.label, - options.sub ? h('div', { class: 'tile-sub' }, options.sub) : null, - )); +/** + * The battery tile's menu. Three things and no more: how much power the machine + * is using, how bright the screen is, and how much charge is left. Wi-Fi, + * Bluetooth and airplane mode used to be here too and are now on the tray items + * and cards that own them. + */ +function renderPowerSettings(): void { + const battery = state?.battery; - const network = state?.network; - if (network?.wifiHardware) { - const active = network.active.find((c) => c.type.includes('wireless')); - tiles.append(tile({ - on: network.wifiEnabled, - // A real signal strength only exists per access point, and the tile - // has no room for one - so it says on/off rather than inventing bars. - glyph: icon(network.wifiEnabled ? 'wifi' : 'wifiOff', 20), - label: 'Wi-Fi', - sub: network.wifiEnabled ? (active?.name ?? 'Not connected') : 'Off', - onClick: () => post({ type: 'wifi', enabled: !network.wifiEnabled }), - })); - } - - if (state?.bluetooth?.available) { - const connected = state.bluetooth.devices.find((d) => d.connected); - tiles.append(tile({ - on: state.bluetooth.powered, - glyph: icon('bluetooth', 20), - label: 'Bluetooth', - sub: state.bluetooth.powered ? (connected?.name ?? 'Not connected') : 'Off', - onClick: () => post({ type: 'bluetooth', enabled: !state?.bluetooth?.powered }), - })); - } - - tiles.append(tile({ - on: state?.airplaneMode ?? false, - glyph: icon('airplane', 20), - label: 'Airplane mode', - onClick: () => post({ type: 'airplane', enabled: !state?.airplaneMode }), - })); - - tiles.append(tile({ - on: state?.energySaver ?? false, - glyph: icon('battery', 20), - label: 'Energy saver', - onClick: () => post({ type: 'energySaver', enabled: !state?.energySaver }), - })); - - tiles.append(tile({ - on: state?.nightLight ?? false, - glyph: icon('moon', 20), - label: 'Night light', - onClick: () => post({ type: 'nightLight', enabled: !state?.nightLight }), - })); - - tiles.append(tile({ - on: false, - glyph: icon('accessibility', 20), - label: 'Accessibility', - onClick: () => post({ type: 'accessibility' }), - })); - - card.append(title('Quick settings'), tiles); + card.append( + title('Power'), + h('div', { class: 'tiles', style: { gridTemplateColumns: '1fr' } }, + h('button', { + class: `tile${state?.energySaver ? ' on' : ''}`, + on: { click: () => post({ type: 'energySaver', enabled: !state?.energySaver }) }, + }, + h('span', { html: icon('bolt', 20) }), + h('span', { class: 'tile-label' }, + 'Energy saver', + h('div', { class: 'tile-sub' }, + state?.energySaver ? 'On — dimmed, screen sleeps sooner' : 'Off — full performance'), + )), + ), + ); if (state?.brightness !== undefined) { - card.append(slider('sun', state.brightness, 1, 100, (value) => post({ type: 'brightness', value }))); - } - - if (state?.audio?.available) { - const audio = state.audio; + // The moon button toggles night light: it is a property of the screen's + // light, so it belongs on the brightness row rather than in a tile. card.append(slider( - audio.muted ? 'volumeMute' : 'volumeHigh', - audio.muted ? 0 : audio.volume, - 0, + state.nightLight ? 'moon' : 'sun', + state.brightness, + 1, 100, - (value) => post({ type: 'volume', value }), - () => post({ type: 'mute' }), + (value) => post({ type: 'brightness', value }), + () => post({ type: 'nightLight', enabled: !state?.nightLight }), + state.nightLight ? 'Turn night light off' : 'Turn night light on', )); } - const battery = state?.battery; - if (battery) { - card.append(h('p', { class: 'flyout-note' }, - battery.present - ? `Battery ${battery.level}% · ${battery.charging ? 'charging' : battery.onAc ? 'plugged in' : 'on battery'}` - : 'Running on mains power')); + if (battery?.present) { + card.append(h('div', { class: 'battery-readout' }, + h('span', { class: 'battery-level' }, `${battery.level}%`), + h('span', { class: 'list-sub' }, + battery.charging ? 'Charging' : battery.onAc ? 'Plugged in' : 'On battery'), + )); + } else if (battery) { + card.append(h('p', { class: 'flyout-note' }, 'Running on mains power.')); } } @@ -273,12 +299,14 @@ function slider( max: number, onInput: (value: number) => void, onIconClick?: () => void, + iconTitle?: string, ): HTMLElement { const readout = h('span', { class: 'slider-value' }, `${Math.round(value)}%`); const send = throttle(onInput, 120); return h('div', { class: 'slider-row' }, h(onIconClick ? 'button' : 'span', { class: onIconClick ? 'icon-button' : '', + title: iconTitle, html: icon(glyph, 18), on: onIconClick ? { click: onIconClick } : {}, }), @@ -371,9 +399,17 @@ function renderNetwork(): void { h('div', { class: 'section-head' }, h('span', {}, network.wifiHardware ? 'Wi-Fi' : 'Connections'), h('span', { style: { display: 'flex', gap: '2px' } }, + // Airplane mode lives here rather than in the power menu: it is a + // radio switch, and this is the card about radios. + h('button', { + class: `icon-button${state?.airplaneMode ? ' on' : ''}`, + title: state?.airplaneMode ? 'Turn airplane mode off' : 'Turn airplane mode on', + html: icon('airplane', 15), + on: { click: () => post({ type: 'airplane', enabled: !state?.airplaneMode }) }, + }), network.wifiHardware ? h('button', { - class: 'icon-button', + class: `icon-button${network.wifiEnabled ? ' on' : ''}`, title: network.wifiEnabled ? 'Turn Wi-Fi off' : 'Turn Wi-Fi on', html: icon(network.wifiEnabled ? 'wifi' : 'wifiOff', 15), on: { click: () => post({ type: 'wifi', enabled: !network.wifiEnabled }) }, @@ -392,6 +428,10 @@ function renderNetwork(): void { list, ); + if (state?.airplaneMode) { + card.append(h('p', { class: 'flyout-note' }, 'Airplane mode is on — every radio is off.')); + } + if (busy) { card.append(h('div', { class: 'empty' }, busy)); return; @@ -437,6 +477,101 @@ function renderNetwork(): void { card.append(wifiList); } +// --------------------------------------------------------------- bluetooth + +function renderBluetooth(): void { + const bluetooth = state?.bluetooth; + card.append(title('Bluetooth')); + + if (!bluetooth?.available) { + card.append(h('div', { class: 'empty' }, 'No Bluetooth adapter on this machine.')); + return; + } + + card.append(h('div', { class: 'section-head' }, + h('span', {}, bluetooth.powered ? 'On' : 'Off'), + h('span', { style: { display: 'flex', gap: '2px' } }, + h('button', { + class: `icon-button${bluetooth.powered ? ' on' : ''}`, + title: bluetooth.powered ? 'Turn Bluetooth off' : 'Turn Bluetooth on', + html: icon(bluetooth.powered ? 'bluetooth' : 'bluetoothOff', 15), + on: { click: () => post({ type: 'bluetooth', enabled: !bluetooth.powered }) }, + }), + bluetooth.powered + ? h('button', { + class: 'icon-button', + title: 'Look for nearby devices', + disabled: state?.bluetoothScanning, + html: icon('refresh', 15), + on: { click: () => post({ type: 'bluetoothScan' }) }, + }) + : null, + ), + )); + + if (!bluetooth.powered) { + card.append(h('div', { class: 'empty' }, 'Bluetooth is off.')); + return; + } + + if (busy) { + card.append(h('div', { class: 'empty' }, busy)); + return; + } + + const paired = bluetooth.devices.filter((device) => device.paired); + const nearby = bluetooth.devices.filter((device) => !device.paired); + + const row = (device: BluetoothDevice): HTMLElement => + h('div', { + class: `list-row${device.connected ? ' active' : ''}`, + title: device.mac, + }, + h('span', { html: icon('bluetooth', 18) }), + h('span', { class: 'list-main' }, + h('div', { class: 'list-name' }, device.name), + h('div', { class: 'list-sub' }, device.connected ? 'Connected' : device.paired ? 'Paired' : device.mac), + ), + h('button', { + class: 'button', + on: { + click: () => post(device.paired + ? { type: 'bluetoothDevice', mac: device.mac, connect: !device.connected } + : { type: 'bluetoothPair', mac: device.mac }), + }, + }, device.paired ? (device.connected ? 'Disconnect' : 'Connect') : 'Pair'), + device.paired + ? h('button', { + class: 'icon-button', + title: 'Forget this device', + html: icon('trash', 15), + on: { click: () => post({ type: 'bluetoothForget', mac: device.mac, name: device.name }) }, + }) + : null, + ); + + const pairedList = h('div', { class: 'list' }); + if (paired.length === 0) { + pairedList.append(h('div', { class: 'empty' }, 'Nothing paired yet.')); + } + for (const device of paired) { + pairedList.append(row(device)); + } + card.append(h('div', { class: 'section-head' }, 'My devices'), pairedList); + + card.append(h('div', { class: 'section-head' }, + state?.bluetoothScanning ? 'Looking for devices…' : 'Nearby')); + const nearbyList = h('div', { class: 'list' }); + if (nearby.length === 0) { + nearbyList.append(h('div', { class: 'empty' }, + state?.bluetoothScanning ? 'Scanning…' : 'Press the refresh button to look for devices.')); + } + for (const device of nearby) { + nearbyList.append(row(device)); + } + card.append(nearbyList); +} + // ------------------------------------------------------------------- music function renderMusic(): void { @@ -521,9 +656,11 @@ function renderMusic(): void { } } -// Escape closes the flyout, the way a real one does. +// Escape closes the flyout, the way a real one does. Which container it lives in +// decides whether that means closing the side bar or the panel, and only the +// host knows - closing both here would take the terminal down with it. window.addEventListener('keydown', (event) => { if (event.key === 'Escape') { - post({ type: 'command', command: 'workbench.action.closePanel' }); + post({ type: 'closeFlyout' }); } }); diff --git a/extension/media/src/lib/icons.ts b/extension/media/src/lib/icons.ts index 6b34815..997bbbb 100644 --- a/extension/media/src/lib/icons.ts +++ b/extension/media/src/lib/icons.ts @@ -12,6 +12,7 @@ const PATHS: Record = { wifi: 'M2.5 9a15 15 0 0 1 19 0M5.5 12.5a10 10 0 0 1 13 0M8.5 16a5.5 5.5 0 0 1 7 0M12 19.5h.01', wifiOff: 'M2.5 9a15 15 0 0 1 6-3.6M15 5.6A15 15 0 0 1 21.5 9M8.5 16a5.5 5.5 0 0 1 7 0M12 19.5h.01M3 3l18 18', bluetooth: 'M7 7l10 10-5 4V3l5 4L7 17', + bluetoothOff: 'M7 7l10 10-5 4V3l5 4L7 17M3 3l18 18', airplane: 'M21 15l-9-4V5.5a1.5 1.5 0 0 0-3 0V11l-9 4v2l9-2.5V19l-2.5 1.5V22l4-1 4 1v-1.5L12 19v-4.5L21 17z', battery: 'M3 8h14a1 1 0 0 1 1 1v6a1 1 0 0 1-1 1H3a1 1 0 0 1-1-1V9a1 1 0 0 1 1-1zM21 11v2', bolt: 'M13 2L4 14h7l-1 8 9-12h-7l1-8z', @@ -60,6 +61,14 @@ const PATHS: Record = { memory: 'M3 7h18v10H3V7zM7 17v3M12 17v3M17 17v3M7 11v2M12 11v2M17 11v2', search: 'M11 18a7 7 0 1 0 0-14 7 7 0 0 0 0 14zM16 16l5 5', warning: 'M12 3l9.5 17H2.5L12 3zM12 10v5M12 18h.01', + plug: 'M9 3v6M15 3v6M6 9h12v3a6 6 0 0 1-12 0V9zM12 18v3', + // A download arrow inside the refresh ring: "there is a newer one". + update: 'M20 12a8 8 0 1 1-2.3-5.7M20 4v5h-5M12 9v6M9.5 12.5L12 15l2.5-2.5', + gear: 'M12 15.5a3.5 3.5 0 1 0 0-7 3.5 3.5 0 0 0 0 7zM12 2.5l1.4 2.6 2.9-.5.6 2.9 2.6 1.4-1.4 2.6 1.4 2.6-2.6 1.4-.6 2.9-2.9-.5L12 21.5l-1.4-2.6-2.9.5-.6-2.9-2.6-1.4L5.9 12 4.5 9.4l2.6-1.4.6-2.9 2.9.5L12 2.5z', + calculator: 'M5 3h14v18H5V3zM7.5 7h9M7.5 11h2M11 11h2M14.5 11h2M7.5 14.5h2M11 14.5h2M14.5 14.5h2M7.5 18h2M11 18h5.5', + brush: 'M15.5 3.5l5 5-8 8-5-5 8-8zM7.5 11.5L4 15c-1 1-1 4-1 5 1 0 4 0 5-1l3.5-3.5', + tab: 'M3 6h7l2 2h9v10H3V6z', + fullscreen: 'M4 9V4h5M20 9V4h-5M4 15v5h5M20 15v5h-5', }; /** Solid glyphs, drawn with a fill instead of a stroke. */ diff --git a/extension/media/src/notepad.ts b/extension/media/src/notepad.ts deleted file mode 100644 index 6fe8270..0000000 --- a/extension/media/src/notepad.ts +++ /dev/null @@ -1,80 +0,0 @@ -// Notepad: a plain-text scratchpad that saves to a real file. -// -// Deliberately not a second editor - anything that wants syntax highlighting or -// a diff belongs in VS Code proper, and "Open in editor" is one click away. - -import { clear, h, onMessage, post, root } from './lib/dom'; -import { icon } from './lib/icons'; -import type { HostMessage } from '../../src/webview/protocol'; - -let path: string | undefined; -let dirty = false; - -const textarea = h('textarea', { - id: 'notepad-text', - placeholder: 'Start typing…', - on: { - input: () => { - dirty = true; - renderStatus(); - }, - }, -}); - -const status = h('div', { class: 'status' }); - -clear(root()).append(h('div', { class: 'app' }, - h('div', { class: 'toolbar' }, - button('New', 'file', () => { - if (!dirty || confirm('Discard unsaved changes?')) { - post({ type: 'newNote' }); - } - }), - button('Open', 'open', () => post({ type: 'openNote' })), - button('Save', 'save', () => post({ type: 'saveNote', text: textarea.value, path }), true), - button('Save as', 'save', () => post({ type: 'saveNote', text: textarea.value, path, saveAs: true })), - h('span', { class: 'spacer' }), - button('Open in editor', 'editor', () => post({ type: 'noteToEditor', text: textarea.value })), - ), - h('div', { class: 'body', style: { display: 'flex' } }, textarea), - status, -)); - -function button(label: string, glyph: string, onClick: () => void, primary = false): HTMLElement { - return h('button', { class: `button${primary ? ' primary' : ''}`, on: { click: onClick } }, - h('span', { html: icon(glyph, 15) }), label); -} - -onMessage((message) => { - if (message.type !== 'note') { - return; - } - path = message.path; - textarea.value = message.text; - dirty = message.dirty; - renderStatus(); -}); - -function renderStatus(): void { - const text = textarea.value; - const words = text.trim() ? text.trim().split(/\s+/).length : 0; - clear(status).append( - h('span', {}, path ? `${path}${dirty ? ' •' : ''}` : dirty ? 'Unsaved note •' : 'Unsaved note'), - h('span', {}, `${text.length} characters`), - h('span', {}, `${words} words`), - h('span', {}, `${text ? text.split('\n').length : 0} lines`), - ); -} - -document.addEventListener('keydown', (event) => { - if ((event.ctrlKey || event.metaKey) && event.key === 's') { - event.preventDefault(); - post({ type: 'saveNote', text: textarea.value, path, saveAs: event.shiftKey }); - } else if ((event.ctrlKey || event.metaKey) && event.key === 'o') { - event.preventDefault(); - post({ type: 'openNote' }); - } -}); - -renderStatus(); -post({ type: 'ready' }); diff --git a/extension/media/src/player.ts b/extension/media/src/player.ts new file mode 100644 index 0000000..274b29c --- /dev/null +++ b/extension/media/src/player.ts @@ -0,0 +1,205 @@ +// Media player. +// +// One