Skip to content

Windows browser, image previews in tool calls, selectable tool output - #813

Merged
wingleeio merged 3 commits into
zeronsh:mainfrom
Master0fFate:windows-browser-annotations
Oct 8, 2026
Merged

wingleeio merged 3 commits into
zeronsh:mainfrom
Master0fFate:windows-browser-annotations

Conversation

@Master0fFate

@Master0fFate Master0fFate commented Oct 5, 2026 •

Copy link
Copy Markdown
Contributor

Browser tabs now render pages inside Zeron on Windows (WebView2) instead of opening the default browser. Menus draw above the page and hidden tabs are throttled.

Expanded tool calls preview images they read or write and free them when collapsed. Right-click a transcript image to copy its path or the image.

Text in expanded tool calls (paths, commands, output) can be selected and copied.

Fixed multi-line answers to agent questions spilling below the question card.

Depends on zeronsh/zui#15; Cargo.toml patches zui to that branch until it merges.

@Master0fFate
Master0fFate force-pushed the windows-browser-annotations branch from 26031ee to e2087e9 Compare October 5, 2026 20:50
- Browser tabs render pages inside Zeron on Windows (WebView2), matching macOS and Linux.
- Expanded tool calls preview the images they touch and free them on collapse.
  Right-click any transcript image to copy its path or the image itself.
- Paths, commands and output in expanded tool calls can be selected and copied.
- Fix: answering an agent question with multiple lines now grows the card instead
  of spilling text below it.
@Master0fFate
Master0fFate force-pushed the windows-browser-annotations branch from e2087e9 to 8676ce5 Compare October 5, 2026 22:07
@Master0fFate Master0fFate changed the title Windows browser, element annotations, and tool call improvements Windows browser, image previews in tool calls, selectable tool output Oct 5, 2026
wingleeio and others added 2 commits October 7, 2026 22:29
…eviews

- Depend on zeronsh/zui dce5c1f (zui#15 merged) and gpui-component
  47ccd07 instead of patching every gpui crate to a personal fork.
- Tool image previews: refuse UNC/device/drive-relative paths in the UI
  and, before any filesystem call, lexically jail requested paths in the
  engine (canonicalizing \\host\share opened an SMB session and leaked
  the user's NTLM hash); read chunks on the blocking pool; bound decodes
  with the shared raster limits; release keys a chip stops showing;
  decode base64 off the UI thread; copy sanitized SVG; cap output lines.
- Windows browser: fail closed on unreadable navigation targets; police
  subframes and cancel external-scheme launches; open tabs only for
  user-initiated popups (handled first); deny permission requests;
  disable web messages and host objects; cancel downloads silently
  unless the page itself is the file; recover from a browser-process
  exit on Reload; give each BrowserData its own InPrivate profile (no
  persistent fallback); sweep only folders of exited processes; never
  remount from an older composition generation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@wingleeio

Copy link
Copy Markdown
Contributor

Thanks — security-audited and reviewed; pushed fixes (5b49bc3, plus a merge of main).

Dependency: the [patch] pointing every gpui crate at a personal fork couldn't merge (every platform's release build would compile it). zui#15 is reviewed and merged upstream (with a fix: device-lost recovery now restores the overlay across retries, and NativeComposition handles are documented as valid per generation), and this PR now pins zeronsh/zui@dce5c1f + gpui-component@47ccd07 with no patch.

Tool image previews (high): paths come from agent output, and the engine canonicalized them before its jail check — on Windows, expanding a chip showing \\host\share\a.png opened an SMB session and leaked the user's NTLM hash. Fixed in both layers: the UI drops UNC/device/drive-relative tokens, and Uploads::inspect now jails lexically before any filesystem call (then canonicalizes and re-checks); chunk reads run on the blocking pool. Also: decodes use the shared raster limits (4096px / 64 MiB), keys a chip stops showing are released, base64 decodes off the UI thread, "Copy image" copies sanitized SVG, output lines are capped at 8k chars, and docs/image-preview.md describes the real jail.

Windows browser: navigation fails closed on an unreadable URI; subframes are policed and external-scheme launches cancelled; popups open a tab only when user-initiated (SetHandled first — WebView2 has no popup blocker); permission requests denied; web messages/host objects disabled; downloads cancel silently unless the page itself is the file; a browser-process exit is recoverable via Reload (other process failures no longer kill the page); each BrowserData gets its own InPrivate profile (no non-InPrivate fallback); the folder sweep only removes folders of exited processes; never remount from an older composition generation.

Checked: Windows cargo check of zeron-ui, zeron-ui 1577 tests, zeron-engine 662 (one queued-attachments timing flake under full-suite load, unrelated path; 40/40 on rerun).

Follow-ups (not blocking): the overlay duplicates per-window path/MSAA intermediates (~260 MB at 4K once a browser opens); favicon fetch doesn't re-check redirects into loopback (shared with macOS).

@wingleeio wingleeio left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Security-audited and reviewed; fixes pushed (see comment above), fork patch replaced by upstream zui dce5c1f. CI green incl. Windows.

@wingleeio
wingleeio merged commit e5466c5 into zeronsh:main Oct 8, 2026
23 checks passed
@Master0fFate
Master0fFate deleted the windows-browser-annotations branch October 8, 2026 12:42
kanishkaverma added a commit to kanishkaverma/heron that referenced this pull request Oct 9, 2026
zeronsh#522 and zeronsh#813 landed in parallel; the fixture zeronsh#522 added predates the
field, so the zeron-ui test target stopped compiling.
wing-anara added a commit to gaelcado/comet that referenced this pull request Oct 9, 2026
main's UI test build fails after zeronsh#522 and zeronsh#813 met: the fixture predates
ToolItem::images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
wing-anara added a commit that referenced this pull request Oct 9, 2026
* feat(rpc): add pull request contracts and cancellable requests

Co-authored-by: José Gurruchaga <jsgrrchg@gmail.com>

* feat(engine): query and manage repository pull requests via GitHub CLI

Co-authored-by: José Gurruchaga <jsgrrchg@gmail.com>

* feat(ui): add shared rendering support for pull request views

* feat(ui): add native pull request board and detail workspace

Co-authored-by: José Gurruchaga <jsgrrchg@gmail.com>

* test(pr): add dashboard demo and deterministic GitHub fixtures

Co-authored-by: José Gurruchaga <jsgrrchg@gmail.com>

* Improve PR code review with persistent file navigation and fitted gutters

* Keep the PR mention hint beside the composer placeholder

* feat(ui): add interruptible selection indicator motion

* refactor(pr): use a sliding indicator in floating navigation

* fix(pr): keep file search at toolbar height

* feat(pr): add split diff layout using shared Changes rendering

* feat(pr): review the Code tab as one stream with a file tree

Show every changed file in one virtualized stream with foldable,
sticky file headers, a directory-grouped tree that follows the scroll
position, and a compact picker on narrow windows. The tab bar gains
counts, neutral selection, and tablist keyboard navigation.

* feat(engine): page repository pull request listings

Add ListChangeRequestPage, which follows GitHub search cursors 50 at a
time and reports the total match count. Cursors are validated before
they reach gh, and the single-page listing keeps serving older clients.

* feat(pr): load more pull requests on the board

The board shows GitHub's total count and appends further pages on
request, without repeating a PR that moved between pages. Engines
without paging fall back to the single-page listing.

* feat(pr): hand a pull request off to an agent session

The Summary tab offers Review, Address feedback and Fix failing checks.
Each opens a new session with a prompt carrying the PR identity, branch,
failing checks and gh commands, staged in the composer for review. A
copyable checkout command sits alongside.

* fix(pr): stack Summary cards without floating titles

Summary cards now carry their own headings and sit at one 12px rhythm;
freshness joins the header's metadata line. The titlebar and metadata
use the PR icon instead of a hash. The changed-file tree runs to the
window edge with scroll fades while the diff keeps clear of the tab bar.

* fix(pr): stop a jumped-to file header doubling the diff border

Jumping to a file landed its header's top divider directly under the
stream card's border. Jumps now land just past the divider, and the
pinned header stays hidden until the real one scrolls away.

* test(pr): capture the board and detail offscreen for visual QA

The pull-request-fixture example renders the production Shell in an
offscreen window that never takes focus, against a real engine and gh.
A delaying gh shim holds loading states long enough to capture them.

* fix(pr): keep layouts stable while loading and align card content

Loading now renders the loaded layout with pulsing placeholders: board
rows, the detail header with Summary cards, and the Code workspace with
tree and diff. Overview rows share one label column, the handoff card
sits on the card's content edge, long titles wrap, long branches
ellipsize, and Activity keeps the Summary's 24px header gap.

* fix(pr): lay out the agent handoff as a settings-style row

The handoff card now reads like the Overview: its purpose on the left,
filled actions trailing on one line (wrapping below when narrow) with
hover summaries, then the checkout row on the shared label column.

* fix(pr): match the pinned file header to the header rows

On frosted surfaces the pinned header used the Changes pane's near-opaque
content tint, so it read lighter than the rows and its square blur
overhung the stream card's rounded corners. It now keeps the rows' own
wash over a stronger blur with a faint veil, and follows the card's
inner radius. The visual fixture captures it in frosted light and dark.

* fix(ui): keep the Pull requests entry in the Settings footer

Chat and Settings now render one shared footer row (account, settings,
Pull requests), so the entry holds its place across the swap and opens
the board straight from Settings.

* fix(pr): send PR comments with the composer's send control

The comment composer drops its Markdown caption and labelled Comment
button. The input now carries the chat composer's filled send circle on
its last line, shared through composer::send_circle.

* fix(pr): run distinct GitHub reads side by side and skip unreadable results

The PR cache held its lock across each gh call, so a diff, a detail and a
board page queued behind one another; through the relay two back-to-back
reads could exceed the 30s forward deadline. Identical reads still share one
call; distinct reads now run concurrently, and a comment post also drops an
in-flight detail read that may predate it.

GitHub returns null search nodes for results the viewer cannot read; skip
them instead of failing the whole page.

* fix(pr): load PR images only from public HTTPS hosts

Images come from untrusted PR text and load on sight. Refuse plain HTTP,
credentials, loopback, private, link-local, CGNAT/Tailscale and local
names, in the URL filter, the transport, and every redirect hop.

* fix(pr): refresh the diff after Refresh and explain failures in words

Refresh away from the Code tab only dropped the diff, so the next visit
could reuse a diff up to five minutes old against fresh details. The next
diff load now bypasses the cache.

Detail, diff and comment errors showed raw RPC text such as
"capability error: pull_requests.authentication"; they now say what went
wrong and what to do. Also drop an unused binding and two unused imports.

* fix(rpc): resolve unary calls whose reply is null

ServerFrame decoded "ok": null as a missing ok, so a reply of null (an
absent Option, such as a checkout with no GitHub remote) never reached the
caller, which waited for its own timeout. Keep an explicit null as a reply.

* feat(pr): open the board on the repository it is opened from

Resolve the repository without asking: the sidebar's project, else the open
session's project, else the repository viewed last, else the most recently
active local Git project that is on GitHub. Reopening the board from another
project follows it; an explicit choice stands until then. The project menu
lists Git projects by name with their path muted.

* fix(pr): dock the comment composer as the chat composer's pill

The Activity composer floated over the thread above the floating tab bar,
with text showing between the two layers, and used its own surface. It now
sits in the flow between the thread and the navigation (the thread ends and
fades above it), with the chat composer's surface, radius, 49px compact
height, insets and send placement.

* fix(client): map capability RPC errors for the mobile core

RpcError::Capability left zeron-client's exhaustive match incomplete, so the
iOS job's Rust mobile core tests failed to compile.

* refactor(pr): use the shared text tooltip on PR buttons

Main standardized icon-button tooltips on widgets::text_tooltip; drop the
PR board's two private tooltip views in favor of it.

* style: rustfmt the pull request board changes

* fix(pr): finish abandoned GitHub reads and trim the RPC surface

Engine
- A read whose only caller left stayed registered forever; later requests
  joined it and got a false timeout or stale data. Reads now run detached:
  they complete and are cached whoever is still waiting.
- An explicit refresh retries a cached failure instead of replaying it.
- gh exits non-zero when a search carries partial errors (results the
  viewer cannot read); keep the readable results once the repository
  resolved.
- Comment text goes to gh on stdin, so long comments fit on Windows.

RPC
- Drop cancel-on-drop for unary calls: it applied to every call site and
  aborted work such as harness installs when a view let go of its task.
- Keep one listing method, ListChangeRequestPage; the three earlier
  listing methods never shipped.
- The board stops falling back to the unshipped single-page listing.

Tests cover an abandoned read, refresh after failure, partial errors, stdin
delivery, the evicted cache key, and the comment RPC end to end.

* fix(pr): reuse the board across repository casing and keep later pages fresh

- Snapshots are matched without regard to case, so adopting GitHub's
  casing no longer forces a reload when the repository is reselected.
- Pages loaded after a refreshed first page bypass the engine cache too.
- Replace the trivial stale-response test with one that leaves a filter
  mid-load and checks the late reply is ignored.

* fix(pr): keep comments, drafts and images safe in the pull request view

- A comment keeps posting when the view closes; a failed one returns as a
  draft with the reason, and a posted one invalidates the cached thread.
- PR badges for hosts other than github.com open in the browser.
- Handoff prompts carry pull request text as one plain line, branch names
  reduced to shell-inert characters, only github.com check links, and a
  note that this text is data.
- Images on hosts other than GitHub's load on click; the transport also
  refuses names that resolve to private addresses.

* fix(pr): hide the chat side pane on the Pull requests route and drop leftovers

- The Pull requests route shared the chat layout with its side pane still
  mounted, which could take the board's whole width.
- A handoff keeps a draft already on the new-session canvas above its prompt.
- Remove unreachable Settings arms, the stale fake-gh demo, the unused
  "browser" settings alias and visibility change, the shared page-header
  alignment change, and formatting-only edits to code outside this feature.

* fix(pr): make the pull request screens keyboard-complete and clear up their copy

- Every control uses the app's 2px accent focus ring; the send button and
  project rows had none, and tint-only focus matched hover.
- Board search, file search and the comment box join the Tab order. Menus
  return focus to their trigger, sort and device menus opened from the
  keyboard start on the current option, and the file tree is one stop moved
  with the arrow keys. Board rows lose their duplicate badge stop.
- Branch and check names wrap instead of hiding the base branch; offline
  devices are labelled; section-tab counts keep full contrast.
- A failed image opens in the browser; errors suggest the repository name or
  link and offer Try again; the header says Loaded, not Updated; "1 file".
- A comment shows a spinner while it posts; comment authors line up with the
  title; board text follows the interface text size; one name per screen.

* Clean up pull request state and workflow coverage

* Improve pull request discovery and loading states

* fix(ui): match PR stream corner guards to shell surface

* fix(ui): keep PR titlebar clear of the new-session control

* Center the empty pull request activity state

* Add PR assessments, CI summaries, stars, and revision handoffs

* Polish the PR repository picker and cancel abandoned selections

* Remove PR picker eyebrow and add overflow-aware menu fades

* Keep PR diff content inside the rounded frame without painted corner masks

* Preserve newer PR drafts and pending diff refreshes across reopening

* Align PR typography and give empty states and verification controls room

* Fix PR reopening, handoff context, and inline review feedback

* Fix PR refresh races, fork handoffs, and Markdown selection

* Fix remote sends stranded by mobile suspension and lost host wakes (#777)

* Fix durable remote send and host wake recovery

* Couple accepted chat rows to durable server host wakes

* Make Linux browser captures wait for mapped and presented frames

---------

Co-authored-by: Wing <contact@winglee.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Explorer: group finished subagents, show running counts (#647)

* Explorer: group finished subagents, show running counts

The Subagents section lists running subagents as plain rows, then folds
everything finished into a Finished dropdown (closed by default) with
Completed and Failed lists inside it. Each list collapses on its own and
pages at ten rows.

A "● N" pill shows how many subagents are running: on sidebar chat rows
(in place of the working spinner), under the titlebar's explorer button,
and beside the Subagents title. To badge chats that are not open, the
session row now carries `running_subagents`, published by the engine as
subagents open and settle. It is optional on the wire and stale rows
count as zero, like the working indicator.

* Explorer button: pulse and count badge for running subagents

The pill hung under the button and crossed the titlebar hairline. Mark the
button itself instead: its face breathes in the activity tone and a small
count badge sits on its corner, both drawn over the button so titlebar
layout is unchanged.

* Sidebar: running-subagents pill takes the time stamp's place

The pill no longer replaces the row's own activity indicator, so compact
titles stop shifting. In compact rows it stands where the time stamp is
while subagents run; in regular rows it leads the status label.

* Running-count pill and badge: centre the digits

The titlebar badge had no explicit line height and used the UI font, so its
digit sat about a pixel high; the sidebar pill's digits sat half a pixel low.
Both now draw their count through one helper: mono digits in a box exactly as
tall as the pill or badge, with a small optical lift (and, for the badge, a
rightward shift) measured from rendered captures at one, two and three
characters.

* Running count: centre digits by font metrics, drop hand-tuned offsets

Geist Mono's digits are centred on the line box and each digit's ink is
centred in its advance (checked from the font tables), so with the line height
equal to the box height nothing needs nudging. The earlier optical offsets
were fitted to two monitor scales and gave the same result there as none; at
a whole-number scale they would only add error. Residual misalignment is
device-pixel rounding of the baseline, at most half a device pixel.

* Running-count pill: hang dot and digits off one baseline; balance the 99+ badge

The pill's dot was a box centred in the pill while the digits' vertical
position followed the text engine's own metrics and rounding, so at
fractional display scales the two could differ by a device pixel. The pill
now paints its background, dot and digits in one canvas: the digits' baseline
is snapped to a device pixel and the dot is centred on the digits' middle
from that baseline, so they share a centre at any scale.

A capped label ends in '+', whose thin arms leave the eye a wider gap than a
digit does; trim the padding after it by a pixel in both the pill and the
titlebar badge.

* Titlebar count badge: paint it so the digit is centred exactly

The badge was a laid-out box, whose text landed on a rounded device pixel and
read as the digit sitting left of centre. Paint the badge and its digits in
one canvas instead: the digits are centred on the badge's real middle, with
the baseline placed as in the sidebar pill. Measured for every digit 1-9 on
1.25x and 1.875x captures: centred to within half a device pixel, exactly for
most.

* Add running_subagents to Session fixtures added on main

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: wing-anara <wing@anara.com>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

* Open tool-call files in the files panel, and in the default editor (#522)

* feat(transcript): open tool-call files in the files panel

Clicking the file badge on a Read/Write/Edit/Patch tool row now opens
that file in the right-hand files panel, instead of only expanding the
row to show its path. The badge goes through the transcript's existing
workspace-link handler, the same route Markdown file links take, so it
resolves against the same roots (the chat's checkout, its parent's, and
this device's projects) and opens outside absolute paths read-only.

The badge stops propagation, so clicking it does not also toggle the
row's accordion; the rest of the header still expands the output or
diff. Paths the link handler cannot open keep an inert badge rather
than looking clickable and doing nothing.

* feat(files): open the active file in the default editor

Add an "Open in default editor" button to the file viewer's breadcrumb
toolbar, next to "Reveal file in tree". It hands the file to the
platform's default application (xdg-open, open, ShellExecute) via
gpui's open_with_system.

The button only appears for workspaces on this device: a remote
space's files live on another machine, where the local default app
cannot reach them.

* Pull requests: harden untrusted content, cache the board, polish the views

Security
- Images in pull request text load through the app's bounded decoder
  (4096 px per side, 64 MiB, one frame, sanitized SVG), at most four at
  once, instead of GPUI's unbounded image loader.
- The image transport refuses NAT64, 6to4, Teredo, IPv4-compatible,
  multicast, reserved and benchmarking addresses, never uses a proxy
  (which would resolve names past the public-address check), and sends
  no Referer on redirects.
- Check links open only https pages; handoff prompts drop invisible
  bidi, zero-width and tag characters; the handoff names the repository
  from the engine's URL; GetChangeRequestRepository accepts only chat or
  project checkouts on the device.

Board
- Virtualized results list (the transcript's ListState pattern), so long
  boards scroll smoothly; scroll position holds across paging.
- The engine keeps the board's first pages in SQLite, one row per pull
  request per gh account plus each query's ordered results, so the board
  opens instantly and a pull request reads the same in every filter. An
  aged page shows at once and refreshes behind it; Refresh always reads
  GitHub.
- Filters read as controls; header controls share one height, refresh
  sits left of the repository; Load more is a filled button.
- Repository identity follows renames and SSH host aliases.

Pull request view
- Back/forward history covers the board and each pull request; the
  header is a plain title like the session header.
- The code viewer clips to its rounded card per pixel
  (zui 361ff20, Window::with_rounded_clip).
- Your reviews and comments read as yours before you comment (the
  engine reports the viewer's login); quieter posting and inline-thread
  rows; Cmd/Ctrl+C copies selected text.
- Summary/Code/Activity pill uses the composer's material with spaced
  segments; the Pull requests entry sits beside Settings; the copy icon
  no longer overlaps itself.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* Fix the ReadFile tool fixture for the images field

main's UI test build fails after #522 and #813 met: the fixture predates
ToolItem::images.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: José Gurruchaga <jsgrrchg@gmail.com>
Co-authored-by: Daniel Katulevskiy <84909978+katulevskiy@users.noreply.github.com>
Co-authored-by: Wing <contact@winglee.io>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: wing-anara <wing@anara.com>
hoangvu12 added a commit to hoangvu12/roboco that referenced this pull request Oct 11, 2026
…acklog window

Source: .scratch/upstream-drift/2026-10-11.md. Tickets 01-18 actionable
(quick ports, chips supersedes wave-6 03, the sidebar series 06-09, the
older missed fixes 13-16, parity verification 17-18); 19 is the GPUI pin
sync gating 20 (zeronsh#856) and 22 (zeronsh#813 Windows browser); 21 (zeronsh#484 PR board)
PARKED on a product decision.
hoangvu12 added a commit to hoangvu12/roboco that referenced this pull request Oct 11, 2026
…g read chunks

Upstream e5466c5 (wave7 ticket 01, upstream zeronsh#813 engine half; the
Windows-browser UI half is ticket 22):

- uploads: jail requested paths lexically BEFORE any filesystem call —
  canonicalizing \\host\share opened an SMB session and leaked the
  user's NTLM hash. lexical_components resolves ./.. without touching
  the filesystem, refuses relative, drive-relative (C:x.png), UNC,
  \\.\ device and verbatim non-disk prefixes, and compares
  case-insensitively on Windows; roots match in raw and canonical form
  (/var vs /private/var, symlinked cwds) and canonicalization now only
  confirms what the lexical jail already admitted.
- rpc: READ_ATTACHMENT_CHUNK reads on the blocking pool (Uploads is an
  Arc-clone).
- tests: network/escaping paths refused before any filesystem call
  (including a symlink from outside every root into one), and
  lexical_components dot/prefix semantics.

Deliberately excluded: the WebView2 browser host, tool image previews,
selectable tool output (ticket 22 / UI half), raster-limit decodes and
base64-off-UI-thread (UI side), apps/ios and crates/client hunks.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants