Skip to content

Browser element annotations - #30

Open
katulevskiy wants to merge 4 commits into
mirror/upstream-mainfrom
mirror/upstream-pr/814
Open

katulevskiy wants to merge 4 commits into
mirror/upstream-mainfrom
mirror/upstream-pr/814

Conversation

@katulevskiy

Copy link
Copy Markdown
Owner

Upstream mirror

Source: zeronsh#814
Original author: @Master0fFate
Original revision: f65275573af0eabc1302ce83c90aac79af4152d3

This PR is an isolated Contribution Manager pilot mirror. Its reviews, checks and merge state are separate from upstream. An empty [skip ci] commit suppresses automatic bulk CI; the code tree matches the imported upstream revision.


Adds an annotate button to the browser toolbar. Click an element on the page and it goes into the composer as an "Annotation N" chip; the element keeps a numbered outline that follows scroll until the chip is removed or sent. Hovering a chip shows the element's code.

When sent, every harness gets the element as plain text (selector, path, box, styles, HTML), so it works with models that can't read images.

Annotations in the browser and composer

Builds on zeronsh#813, so only the last commit is new here. The macOS/Linux picker code wasn't compiled here, so it needs a build on each.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

Master0fFate and others added 3 commits October 6, 2026 00:07
- Browser tabs render pages inside Zeron on Windows (WebView2), matching macOS and Linux.
- Expanded tool calls preview the images they touch and free them on collapse.
  Right-click any transcript image to copy its path or the image itself.
- Paths, commands and output in expanded tool calls can be selected and copied.
- Fix: answering an agent question with multiple lines now grows the card instead
  of spilling text below it.
The browser toolbar gets an annotate button. Click an element on the page and
it lands in the composer as an "Annotation N" chip, with a numbered outline on
the page that follows scroll until the chip is removed or sent. Harnesses get
each element as plain text (selector, path, box, styles, HTML), so it works with
models that can't read images.
@contribution-manager-katulevskiy contribution-manager-katulevskiy Bot added the review:needs-triage Managed by Contribution Manager label Oct 6, 2026
@contribution-manager-katulevskiy

contribution-manager-katulevskiy Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Contribution Manager

needs triage · medium risk · untriaged

Open review workspace · Revision b85dd3f1

Next actions:

  • Direction decision needed
  • Scope, risk, and platform triage needed
  • Independent code review needed
  • Required check: core-tests (missing)

This summary is maintained by the review app. Decisions and test evidence are recorded in the workspace; the app does not merge PRs.

@contribution-manager-katulevskiy contribution-manager-katulevskiy Bot added cm:inbox Contribution Manager: waiting for triage and removed review:needs-triage Managed by Contribution Manager labels Oct 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cm:inbox Contribution Manager: waiting for triage

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants