feat(macos): show the shell rail and the model conversation for computer-use runs - #370
Merged
Conversation
…round runs A window-scope run drives one app window and paints nothing on the desktop, so everything it did was compressed into a single menu bar caption line: no shell commands, no reasoning, no way to read a step that had already scrolled past. Foreground runs show all of it (capsule, cursor, shell rail); background runs are exactly the ones the operator is not watching, and had the least. Status mode now carries three surfaces: - The same click-through shell rail under the menu bar, in its own borderless non-activating panel. It ignores mouse events, and window-scope capture sees only the target window, so neither the operator's work nor the model's view is disturbed -- a command running on this Mac no longer requires opening a window to notice. - The activity window (menu: Show activity): the live frame of the driven window above the conversation with the model -- the task, its thinking, every action, and every shell command -- in the playground chat's vocabulary, transcribed from Tailwind to plain CSS on a new page the host loads in a floating, non-activating, resizable panel. Rows are addressed by id, so a shell command is revised in place as it moves from running to its exit code. The page is built up front and hidden, so the transcript starts at step one no matter when it is opened; rows that arrive while it loads are buffered. - The menu keeps the last four steps rather than only the newest. The transcript opens with the task itself: `run()` and `resume()` now present a `task` event, so a presentation shows the conversation from its first message. Background runs also record shell telemetry, which only the overlay path did. Both status-mode pages are extras next to a menu bar item that still carries Stop, so one that fails to load is dropped rather than taken as fatal. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Which surfaces exist should not depend on how a run drives the Mac. The desktop overlay says what is happening right now; the activity window is where the operator reads back what happened -- the task, the model's thinking, every action and shell command -- and a foreground run has exactly as much of that to show as a background one. The host now builds the window in both modes and puts Show activity in the overlay run's menu bar menu, and the controller feeds the transcript from the one `present()` path both modes share. Two consequences follow from foreground capture being the whole desktop: the window is hidden with the overlay for every capture, so the model never sees it or clicks into it, and it shows no frame pane, because the operator is already looking at what the model sees -- the pane appears only when a run streams frames of its own. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 61f3cfb. Configure here.
The menu grew to the last four steps back when it was the only place a background run's history existed. The activity window keeps that history now, and better, so the menu goes back to what it is good at: one glanceable line about the latest step, above the frame and Stop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 3, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why
The conversation with the model was only ever readable where it happened to be painted. A foreground run showed reasoning in the cursor capsule for as long as a step lasted and nothing after; a background (window-scope) run drives one app window and paints nothing on the desktop, so everything it did was compressed into a single menu bar caption line — no shell commands, no reasoning, no way to read a step that had already scrolled past.
What runs show now
navigator-activity.{html,css,js}). Rows are addressed by id, so a shell command is revised in place as it moves from running to its exit code. The page is built up front and hidden, so the transcript starts at step one no matter when it is opened; rows arriving while it loads are buffered. A window-scope run puts the driven window's live frame above the conversation; a foreground run shows no frame pane (the operator is already looking at what the model sees) and the window is hidden with the overlay for every capture, so the model never sees it.The status menu is unchanged apart from the new Show activity item: one caption line about the latest step, above the frame thumbnail and Stop. The window keeps the history now, so the menu does not have to.
The transcript opens with the task itself:
run()andresume()present ataskevent, so a presentation shows the conversation from its first message. Background runs also record shell telemetry, which only the overlay path did.Both status-mode pages are extras next to a menu bar item that still carries Stop, so one that fails to load is dropped rather than treated as fatal (the overlay page stays fatal — it is the run's only surface).
Verification
pytest -m "not slow": 862 passed. New coverage for the transcript stream in both modes, in-place shell rows, the status-mode rail and its counts, entry clamping, and the activity page's integrity check (it is hashed and verified like every other packaged asset).captureHidein flight — the desktop with the window correctly absent from what the model would capture.availablewith no degradation; the second streamed 240 preview frames into the activity window while it ran.Downstream
yutori-mcp needs a pin bump once this releases, plus wording updates in
computer_use/result.py("Menu bar status", "streamed to the menu bar item"). Users pick the new assets up by re-runningcomputer-use setup, which recompiles the host.🤖 Generated with Claude Code
Note
Medium Risk
Touches macOS overlay host, capture hide/show, and presentation paths used during live agent runs; failures are mostly fail-soft for auxiliary UI, but incorrect capture hiding could leak UI into model screenshots.
Overview
macOS computer-use presentation now surfaces the full model conversation (task, reasoning, actions, shell, errors, final answer) in a new activity window (Show activity), shared by foreground overlay runs and window-scope status mode—with a live driven-window frame above the transcript in background runs only.
Window-scope (
status) runs gain the same click-through shell rail under the menu bar as foreground runs (dedicated borderless panel), so shell work is visible without opening the menu. The old floating “live view” is replaced by this activity UI; preview streaming keys offactivityOpeninstead ofliveView.Python/Swift wiring:
MacOSPresentationControllermaps events totranscripthost commands (_transcript_entry, shell rows updated in place by id);N2ComputerAgent.run/resumeemit an initialtaskevent. New packaged assetsnavigator-activity.{html,css,js}are built, integrity-checked, and passed viaactivityHtmlconfig. Non-fatal load failures for rail/activity pages in status mode degrade gracefully; overlay page failure remains fatal.Docs and tests update
api.mdand add coverage for transcript streaming, status-mode rail counts, activity page tampering, and preview demand.Reviewed by Cursor Bugbot for commit cc9a54f. Bugbot is set up for automated code reviews on this repo. Configure here.