Follow your coding agents, check AI usage and control a small desktop display from a local web app.
Companion is tested on the Waveshare ESP32-S3-Touch-AMOLED-1.75-B. Have a different board? Explore potential ports or contribute a board profile. You can also use the browser preview without a device.
To refresh the README images or build the website, see landing page and screenshot commands.
Choose from 7 modules:
| Module | What it shows | What you need |
|---|---|---|
| Herdr Face | Agent status, session names and animated expressions | Herdr running on your computer |
| CodexBar | Usage balances and reset times | An installed, configured CodexBar CLI |
| HEY | Senders and subjects from your chosen mailbox | An installed, authenticated HEY CLI |
| Clock | Time and an optional weekday | No additional service |
| Speed Dial | Custom buttons to launch apps, links, Shortcuts and commands | Actions run on your Mac |
| Audio | Mac input/output device, volume and mute | macOS; controls depend on the audio device |
| Now Playing | Album artwork, track details and playback controls for Roon, Spotify, Apple Music and macOS | Enable each player separately |
The app detects installed CodexBar and HEY CLIs. Herdr Face, CodexBar, HEY and Clock start enabled.
Enable the modules you want on the Modules page. Swipe left or right to switch between them. You can change their order.
Now Playing shares one design across four players. Enable Roon, Spotify, Apple Music and macOS Now Playing separately, then swipe up or down to switch between them. Switching the display does not change playback. Tap the app icon at the top-left of the artwork to open Spotify, Apple Music or Roon on your Mac. The system player has no badge.
Spotify and Apple Music use their installed Mac apps. Apple Music starts disabled and controls Music independently of the active system player. Allow Companion access in System Settings > Privacy and Security > Automation when prompted. No developer account is required. macOS Now Playing follows the active Mac media app, which can be the same track shown by Spotify or Roon. Its bundled adapter uses private macOS APIs, so compatibility can change with system updates. Roon still uses its local server extension and selected listening zone.
The bottom-right heart is reserved for sources that support changing likes. The current local integrations do not expose that capability, so it stays hidden.
Audio starts disabled. Enable it in Modules to choose Mac input and output devices. Swipe up or down to switch between input and output, and tap the title, percentage or device name to open the device list. Swipe up or down to browse pages. The active device is first on page 1, with a check on the left; selecting any device returns to the audio controls. Volume and mute are available only when the selected device supports them. Audio controls the Mac default devices; it does not change a separate Roon network endpoint. You can adjust the layout in Designer.
Enable Speed Dial in Modules, then select Add button. Choose an emoji, a built-in icon or your own PNG or SVG. Give it a label and add one or more actions. Save the button, then use Test to check it.
Buttons can open apps, URLs, files or folders, run macOS Shortcuts, or run shell commands. A sequence runs in order and stops if an action fails. Saving or editing never runs an action. Each action has a 30-second limit; up to four buttons can run at once. Results and command output appear in the editor.
Automatic grids pack buttons into staggered rows on round displays and straight rows on rectangular displays. The default design fits seven labelled buttons on the round screen or nine on a square canvas. Hide labels for a honeycomb cluster: nine large buttons, including one at the top and bottom, and four smaller side buttons. Pages hold at most thirteen buttons. Button sizes adjust to keep the cluster and page counter clear. Fixed grids of four or six buttons and lists of three or four rows are also available. Automatic grids fill from the centre outward so partial pages stay balanced. Reorder, duplicate or disable buttons in Modules. Use Designer to adjust button size, icon size, text and spacing. Swipe up or down on the device to browse pages; swipe left or right to change modules. Tap a button once to run it. A running button cannot be started again until it finishes.
Emoji search works offline using bundled Frimousse data. Icons are prepared on the Mac and sent over USB, so changing an icon does not require reflashing. Install firmware built with Speed Dial support before using it on the device. See Speed Dial configuration and actions.
Attention lets an agent temporarily show a face, a message and a choice over your current module. Tap the face to read the detail and choose an action. Actions stack vertically. Double-tap the face or detail view to dismiss without choosing an action. When it closes, your previous module returns.
Notifications expire after 10 seconds of visible time. Decision requests stay until answered or cleared. Reading the detail pauses a notification's timer. Requests queue so agents cannot overwrite each other's messages.
Use the Attention page to send a preview, manage the queue or turn interruptions off. Open Settings > Command line to install the companion command, then Agent skills to install companion-attention.
The CLI and skills are bundled with the app and install offline. Settings shows installation status and offers Update and Uninstall. Existing files and edited installations are preserved.
companion skills
companion skills path
companion skills path companion-attention
companion skills install companion-attention
companion skills install --all
companion skills uninstall companion-attentionSkills default to ~/.agents/skills. Change the destination in Settings or use companion skills --directory /your/skills/path. The CLI installs to ~/.local/bin/companion; Settings tells you if that directory needs adding to your shell's PATH.
Add a reference to the installed SKILL.md in your agent instructions. The attention skill covers notifications, decisions, chained screens and waiting for a response. Keep Companion in its installed location. Development installs depend on this checkout and its Node runtime.
The queue and recent responses are held in memory. Restarting the bridge clears them. A timeout or missing request never counts as approval.
Agents can attach a generic command callback to receive the result as JSON on standard input. The skill includes a tip for using this to reply to an originating Herdr pane. Delivery is attempted once, with a 5-second timeout; failures remain visible in request history.
Use Node.js 22.12 or later and pnpm 11.19.0. Development and hardware checks have been tested on macOS with Node.js 24.
From the repository directory, run:
pnpm install
cp .env.example .env
pnpm build
pnpm startOpen Companion on your computer.
Leave ESP_SERIAL_PORT empty in .env to start in browser-only mode. After flashing, choose Automatic or a serial port on the Device page.
The app reads data through your local integrations. Follow the integration setup guide to connect them.
Use Animations to map expressions to Working, Needs your input, Ready, Idle, Unknown and Disconnected.
The library contains 5 original expressions: Working, Needs input, Ready, Idle and Sleeping.
Choose an animation to preview it in the browser. Select Send to device to show it on the hardware. Select Return to live to resume agent status.
Mappings change the expression, not the status text. One working agent shows Working; 2 or more show the count. Ready takes precedence over idle in the secondary summary.
Tap the physical face to cycle through agents. A swipe changes modules instead.
Use Designer to change each module's layout with a live preview. Adjust text sizes, spacing, positions, card dimensions and playback controls.
Live updates save changes as you make them. Turn them off to experiment, then select Apply design. Use Undo or Reset module to restore earlier values.
You can export and import designs. Sample content changes the preview without replacing live data on the device.
Now Playing artwork expands instantly by default. You can enable the expansion animation and artwork spin in Designer.
Device appearance is separate from the dashboard theme. Choose Light, Dark or System on the Device page.
Each device mode has its own editable colours for the background, text, cards, progress bars and status accents. Album artwork keeps its original colours.
System follows the connected computer's appearance, even when the browser is closed. The bridge checks macOS, Windows or GNOME preferences every 3 seconds. Failed queries retain the last reading.
Physical rotation supports 0 to 359 degrees in 1-degree steps. Touch coordinates follow the rotation. The browser preview stays upright.
Custom angles require more rendering work than right angles. The renderer skips unchanged pixels and limits updates to changed areas. Performance depends on the animation and module.
Mouse following is available on macOS. The face moves smoothly towards sampled cursor positions. Disable it or change the sampling interval on the Device page.
The supported Waveshare board on Amazon has a round 466 x 466 CO5300 AMOLED display, CST9217 touch controller, 16 MB flash and 8 MB PSRAM.
The device renders native LVGL graphics; it does not run the web app.
Tested support means Companion has run on the physical board. Theoretical compatibility means the published hardware looks suitable for a port, not that the current firmware can be flashed and used unchanged. Only the 1.75-B has been tested.
| Board | Screen | Companion status |
|---|---|---|
| ESP32-S3-Touch-AMOLED-1.75-B | Round, 466 x 466 | Tested. Current firmware and layouts target this board. |
| Other 1.75 variants and 1.75C | Round, 466 x 466 | Theoretical compatibility. Shared CO5300 display and CST9217 touch controllers make these close candidates. Wiring, power setup and board revisions still need checking and testing. |
| ESP32-S3-Touch-AMOLED-1.8 V2 | Rectangular, 368 x 448 | Potential port. Shares the CO5300 display controller, but needs board-specific touch support and layouts. V1 and V2 hardware differ. |
| ESP32-S3-Touch-AMOLED-2.06 | Watch-style, 410 x 502 | Potential port. Uses CO5300 with FT3168 touch; needs a board profile and layouts for its screen. |
| ESP32-S3-Touch-AMOLED-2.16 | Square resolution, 480 x 480 | Potential port. Uses CO5300 with CST9220 touch; needs a board profile and layout validation. |
These candidates are based on Waveshare's linked specifications, not compatibility tests. A square, watch or stopwatch-style enclosure does not identify the electronics inside. Check the exact model and revision. Sharing the ESP32-S3 chip, or even the display controller, does not guarantee compatible pins, touch, power management or memory.
The bridge uses an explicit board profile registry. Firmware reports its board ID and screen geometry; the dashboard and Designer use the registered screen shape and dimensions. Older 1.75-B firmware remains compatible.
Module layouts still use a 466 x 466 logical canvas. A different screen can show a fitted preview, but needs matching firmware layout and touch work before it is supported. Changing Designer settings does not add another display driver.
Start with the board contribution guide. It explains the profile format, firmware boundary and physical checks required for tested support. Ports begin as experimental. The current board remains the only tested profile.
Install the GitHub CLI, Git and Python before setting up the firmware toolchain. From the repository root, run:
pnpm firmware:setup
pnpm firmware:buildSetup downloads the pinned ESP-IDF toolchain into .tools. The build prints a flash command. See the firmware build and flash instructions for the complete steps.
Stop the bridge before flashing or opening the serial port with another tool. After flashing, run pnpm start and select a connection on the Device page.
To keep a copy of the original firmware, install esptool using requirements-device.txt. Replace the port below with your board's port:
export ESP_PORT=/dev/your-serial-port
python -m esptool --chip esp32s3 --port "$ESP_PORT" \
read-flash 0 0x1000000 original-flash.binStore the backup outside the repository. To restore it with the bridge stopped, run:
python -m esptool --chip esp32s3 --port "$ESP_PORT" \
write-flash 0 original-flash.binUSB carries power and data. Firmware updates use USB; there is no over-the-air update feature.
On the Device page, choose a connection mode:
- automatic finds a compatible Espressif USB device and remembers its serial number
- manual lets you choose an available serial port
- browser only disconnects the hardware without stopping the app
A remembered device can reconnect after moving to another USB port. If several compatible devices are found, choose one manually first.
Refresh scans the available ports. Reconnect closes and reopens the selected connection. The bridge waits for Companion firmware before sending display data.
Connection choices are saved separately in .cache/device-connection.json. They take precedence over ESP_SERIAL_PORT, which supplies the initial choice when no saved connection exists.
The bridge listens on 127.0.0.1. Settings are saved in .cache/studio-settings.json. Export settings from the Device page to keep a copy.
Enabled integrations can send agent names, usage values, mail senders and subjects, or music details to the local preview and device.
The HEY module does not send messages or fetch message bodies for display. Tapping a supported mail or usage card opens its link on your computer.
The integration CLIs handle their own authentication and network requests. Mouse following keeps no cursor history. Local settings, credentials and build files are excluded by .gitignore.
On macOS 13 or later, build the web app and menu bar launcher:
pnpm build
pnpm menubar:build
mkdir -p ~/Applications
ditto ".tools/Companion.app" "$HOME/Applications/Companion.app"
open "$HOME/Applications/Companion.app"The menu shows device status and provides Dashboard, Settings, Logs, Restart Companion, About, Check for Updates and Quit. Closing the browser leaves the bridge running. Quitting stops the bridge started by the app. An existing externally managed bridge is left alone.
Logs opens the dashboard with live diagnostics, search, level filters, pause/resume, and copy/download for the filtered events. The page retains the latest 500 events from the current bridge session. If the bridge is unavailable, the menu opens the local log file instead.
Launch at login is off by default. Enable it in the dashboard under App settings after copying the app to Applications. This page also shows the installed version and Sparkle update preferences. Native settings require the menu bar app to manage the bridge; update controls require a release build.
Building requires the Xcode command line tools. Quit an existing copy before replacing it.
This is a local launcher, not a standalone distribution. Keep this checkout, its installed dependencies and the Node runtime in place. The build records their current paths. Rebuild the launcher if you move the checkout or change the Node installation.
Signed releases bundle Node and the dashboard and use Sparkle for updates. See Release Companion for macOS for signing, notarization, update hosting and the staged GitHub workflow.
Stop any running bridge before starting development mode:
pnpm devThis starts the bridge and Vite. Open the development app.
Format frontend source and organise imports:
pnpm formatRun formatting checks, linting, tests, type checks and the production build:
pnpm checkpnpm format:check checks formatting and imports without changing files. pnpm lint runs lint checks alone.
Biome covers app-owned frontend files, including UI primitives, and leaves vendored code unchanged.
CSS overrides, non-null assertions and existing ARIA containers remain allowed. Local hook exceptions explain intentional timing and revision guards.
Native rendering tests require a C11 compiler available as cc.
The app uses React, TypeScript, Vite, Tailwind CSS and shadcn/ui. Firmware uses ESP-IDF and LVGL. Lockfiles record dependency versions.
To regenerate face profiles and font assets, run:
pnpm faces:generate
pnpm fonts:generateOriginal project code is available under the MIT licence.
Third-party code and assets retain their own licences:
| Source | Use | Licence and notice |
|---|---|---|
| Bloub | Original face expressions and motion | MIT licence, notice |
| Geist | Text and pixel fonts | SIL Open Font License, font sources |
| Frimousse and Emojibase | Offline emoji picker and search metadata | Frimousse MIT licence, Emojibase MIT licence |
| Reicon | Previous, next and play icons | MIT licence, icon sources |
The bundled MediaRemote Adapter retains its BSD-3-Clause licence. Its pinned source and attribution are in vendor/mediaremote-adapter. The Roon client packages retain their Apache-2.0 licences. Other dependencies retain the licences distributed with their packages.