Annotate live UI pages and deliver structured feedback to AI coding agents.
Frontendnotator opens any URL in a real browser with an interactive annotation overlay. Click elements to mark them with comments, issues, suggestions, or questions — then submit the feedback as markdown or JSON for AI agents to consume.
- CLI launches a local server and opens the target URL in Playwright-controlled Chrome
- An annotation overlay (Shadow DOM) is injected into the page via
addScriptTag() - You click elements, pick an annotation type, and add comments
- On submit, annotations are sent back to the CLI via
exposeBinding() - The CLI outputs structured feedback (markdown or JSON) and exits
The agent calls frontendnotator, which blocks until the user submits feedback. The output goes to stdout.
frontendnotator --url https://myapp.comThe CLI blocks, the user annotates in the browser, and when they click "Submit Feedback" the structured markdown is printed to stdout.
Run the tool yourself, annotate, and export the result:
# Export to file
frontendnotator --url https://myapp.com --output feedback.md
# Copy to clipboard
frontendnotator --url https://myapp.com --clipboard
# JSON format
frontendnotator --url https://myapp.com --output feedback.json --format jsonFor non-web UIs (native apps, Fyne, etc.), annotate a static screenshot:
frontendnotator --file screenshot.png --output feedback.mdBy default, frontendnotator launches a new Chrome window. To connect to your already-running Chrome instead:
# 1. Start Chrome with remote debugging enabled
/Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome --remote-debugging-port=9222
# 2. Point frontendnotator to it
frontendnotator --url http://localhost:17608/dashboard/ --debugger-port 9222This opens a new tab in your existing Chrome session instead of a separate window.
# Clone and install
git clone <repo-url> && cd frontendnotator
bun install
# Install Playwright browser
bunx playwright install chromium
# Build the overlay bundle
bun run buildfrontendnotator --url <url> [options]
Options:
-u, --url URL to annotate (opens in browser)
-f, --file Screenshot/image file to annotate
-o, --output Write feedback to file
-F, --format Output format: markdown (default) or json
-c, --clipboard Copy feedback to clipboard
--channel Browser: chrome (default) or chromium
--viewport Set fixed viewport, e.g. 1280x800 (default: no fixed viewport)
--debugger-port Connect to running Chrome via CDP port (e.g. 9222)
-h, --help Show help
| Type | Color | Key | Description |
|---|---|---|---|
| Comment | Blue | C | General feedback |
| Issue | Red | I | Bug or problem |
| Suggestion | Yellow | S | Improvement idea |
| Question | Purple | Q | Clarification needed |
| Key | Action |
|---|---|
| E | Toggle between Pick mode and Review mode |
| H | Hide / show the sidebar |
| C | Add a Comment annotation |
| I | Mark as Issue |
| S | Mark as Suggestion |
| Q | Mark as Question |
| Arrow Up | Select parent element |
| Arrow Down | Select child element |
| Escape | Cancel current selection |
| Cmd/Ctrl + Enter | Submit comment |
# UI Annotations
## 1. Comment on "Submit button"
- **Page**: Dashboard (https://myapp.com/dashboard)
- **Element**: `<button>` | role: button | aria-label: "Submit"
- **Selector (strict)**: `#main > form > button.btn-primary`
- **Selector (resilient)**: `[aria-label="Submit"]`
- **Position**: (120, 340) 80×32px
- > The submit button should be more prominent{
"format": "frontendnotator-feedback-v1",
"url": "https://myapp.com/dashboard",
"submittedAt": 1745100000000,
"annotations": [
{
"id": "ann_1745100_abc123",
"type": "comment",
"comment": "The submit button should be more prominent",
"target": {
"strictSelector": "#main > form > button.btn-primary",
"relaxedSelector": "[aria-label=\"Submit\"]",
"rect": { "x": 120, "y": 340, "w": 80, "h": 32, "dpr": 2, "scrollX": 0, "scrollY": 0 },
"tagName": "button",
"role": "button",
"ariaLabel": "Submit"
},
"page": { "url": "https://myapp.com/dashboard", "title": "Dashboard" },
"createdAt": 1745100000000,
"status": "active"
}
]
}From an agent tool hook, call frontendnotator --url <url> and read stdout:
# Agent calls this and blocks until user submits
feedback=$(frontendnotator --url https://myapp.com)# Write annotations to a file, agent reads it later
frontendnotator --url https://myapp.com --output .plannotator/ui-feedback.mdAgents can be instructed to check for .plannotator/ui-feedback.md periodically.
CLI (Bun)
└── startAnnotateServer() → Bun.serve on random port
└── launchBrowser() → Playwright visible Chrome
└── injectOverlay() → page.addScriptTag() → Shadow DOM overlay
└── await waitForDecision() → blocks until user submits
└── window.onAnnotationSubmit() ← exposeBinding() callback
The overlay is a React app bundled into a single JS file, injected into the page via Playwright's addScriptTag(). It mounts inside a Shadow DOM for CSS isolation. Bidirectional communication uses exposeBinding() — the overlay calls window.onAnnotationSubmit() which resolves the server's decision promise.
# Run in dev mode (auto-rebuild overlay on change not yet supported)
bun run dev --url https://example.com
# Build overlay bundle
bun run build
# Type check
bun run typecheckbin/frontendnotator.ts CLI entry point
src/
types.ts Shared types (UIAnnotation, ElementTarget, etc.)
selector.ts CSS selector generation (@medv/finder + custom)
export.ts Markdown and JSON export
server.ts Local HTTP server + waitForDecision()
browser.ts Playwright browser launch + overlay injection
overlay/
index.tsx Shadow DOM mount + CSS styles
App.tsx Main app (mode state, annotation CRUD, submit)
ElementPicker.tsx Hover-highlight + click-select
AnnotationToolbar.tsx Action buttons (Comment/Issue/Suggest/Screenshot/Cancel)
CommentPopover.tsx Textarea for comment, Cmd+Enter to save
HighlightManager.tsx Colored outlines on annotated elements
AnnotationSidebar.tsx Right sidebar listing annotations
MIT