diff --git a/.cursor/rules/project.mdc b/.cursor/rules/project.mdc index d4d3817..923df13 100644 --- a/.cursor/rules/project.mdc +++ b/.cursor/rules/project.mdc @@ -1,67 +1,75 @@ --- -description: Core project context for real-browser-mcp +description: Core project context for browser-controller alwaysApply: true --- -# real-browser-mcp +# browser-controller -MCP server (TypeScript) + Chrome extension (plain JS) that talk over WebSocket on localhost:7225. +Multi-client MCP server (TypeScript) + Chrome extension (plain JS). Renamed from `real-browser-mcp` (v2.0.0). npm `browser-controller`, MCP id `io.github.noiemany/browser-controller`. -## Versioning +## Three-piece architecture -Bump minor versions: 1.0.0 -> 1.1.0 -> 1.2.0. Three places must stay in sync: -1. `package.json` "version" -2. `extension/manifest.json` "version" -3. `mcp-server/src/index.ts` SERVER_VERSION +``` +Agent ─stdio─ thin MCP client (index.ts) ─IPC─► daemon (daemon.ts, owns :7225) ─WS─► extension (background.js) +``` -## Adding a Tool +- The daemon owns the WS server on 7225 AND an HTTP server on the **same port** (`/pair` returns the token, `/status` returns agents). The extension auto-fetches the token via `/pair` (MV3 can't read the filesystem). +- Bind host is **`127.0.0.1`** (NOT 'localhost' — macOS resolves localhost to IPv6 ::1 first and refuses IPv4). Both WS connect and HTTP fetch use 127.0.0.1. +- Thin client auto-spawns the daemon if absent. State in `~/.browser-controller/`. -Four files to touch: -1. `mcp-server/src/tools/.ts` - ToolDefinition with Zod schema -2. `mcp-server/src/tools/index.ts` - add to allTools array -3. `extension/background.js` - add handler in dispatch() map + handler function -4. `tests/tools/` - add test +## The two invariants -The schema type MUST be `z.ZodObject`, not `z.ZodType`. The MCP SDK calls `.shape` on it. +1. **`tabId` is mandatory** for every page-interaction tool. `navigate` is the ONE exception (optional, active-tab fallback). `tabs` create/close/focus/list take no target tabId. Enforced via `requireTabId()` in `tools/types.ts`. +2. **background.js never acts on "the active tab"** implicitly — `resolveTab(tabId)` is the entry point. The active-tab fallback survives ONLY inside `handleNavigate`. -## Tool inventory (18 tools) +## Element identification (3 layers + auto-recovery) -Navigation: browser_navigate, browser_tabs -Interaction: browser_click (ref OR selector), browser_click_text (by visible text, CSP-safe), browser_type, browser_press_key, browser_scroll, browser_hover, browser_select -Reading: browser_snapshot (compact mode default), browser_screenshot, browser_text, browser_find -JS execution: browser_evaluate (via chrome.debugger CDP - bypasses CSP but shows debugger banner) -Dialogs: browser_handle_dialog (override alert/confirm/prompt) -Wait: browser_wait -Debug: browser_console (read-only), browser_network +1. `data-mcp-ref="e5"` (fast, breaks on re-render) +2. robust CSS selector via `smart-selector.js` (climbs ≤3 ancestors, ignores React/generated classes+ids) +3. text+role+tag scan (survives total class/id churn) — resolves the **correct** sibling via `nth` ordinal when several elements share text+role (e.g. 3 "Like" buttons) +- `autoReSnapshot`: on ref failure (virtualized feed), snapshot the tab and embed `freshRefs` in the response so the agent retries in one step. Do NOT auto-retry the click (non-idempotent). `browser_scroll` returns `refsMayBeStale: true` as a hint. +- `isNew`: each snapshot tracks `role|name` fingerprints per tab; elements appearing since the last snapshot are tagged `isNew: true` so the agent can focus on what changed (big token saver after an overlay/dropdown opens). State lives in `lastSnapshotFingerprints` Map, cleared on tab close. -## CSP and browser_evaluate +## Concurrency (extension/lib/tab-concurrency.js — pure, unit-tested) -`browser_evaluate` uses chrome.debugger API (CDP Runtime.evaluate). This bypasses CSP but: -- Shows "is being debugged" infobar -- Attaching/detaching the debugger can steal focus and close open popups/dropdowns -- Requires `debugger` permission in manifest.json +- `TabMutexMap`: same-tab serializes, cross-tab parallelizes. +- `TabLockMap`: per-agent tab ownership via `browser_tabs {action:'lock'}`; non-owner calls queue. +- `isIdempotent` (daemon-config.ts): read tools retry on timeout; **click/type/navigate/evaluate NEVER retry**. -For DOM interactions (clicking dropdown options, etc.), prefer `browser_click` with CSS selectors or `browser_click_text` instead. These use `chrome.scripting.executeScript` with compiled functions — no eval, no CSP issues. +## Tool inventory (22 tools) -Previous attempts using `new Function()` in MAIN world failed on GitHub/Google/strict-CSP sites. Sandbox/offscreen approaches can't access page DOM. chrome.debugger is the only universal eval but has the focus-stealing side effect. +Navigation: browser_navigate (tabId optional), browser_tabs (list/create/close/focus/**lock**/**unlock**) +Interaction (tabId required): browser_click, browser_click_text, browser_type, browser_press_key, browser_scroll, browser_hover, browser_select, browser_drag, browser_fill_form, browser_upload_file +Reading (tabId required): browser_snapshot, browser_screenshot, browser_text, browser_find +JS/Dialogs (tabId required): browser_evaluate (MAIN world, no banner), browser_handle_dialog, browser_run_action +Waiting: browser_wait +Debug (tabId required, per-tab buffers capped 200): browser_console, browser_network -## Snapshot modes +## browser_evaluate (changed in v2) -`compact: true` (default) - returns only interactive elements + landmarks, ~70% smaller output. No `tag` field. -`compact: false` - full accessibility tree with all visible elements. Includes `tag` for generic roles. +Now runs via `chrome.scripting` MAIN world (NOT chrome.debugger) — no "is being debugged" infobar, CSP-safe. `run_action`/`upload_file`/`drag` still use CDP (CDP-only operations: `DOM.setFileInputFiles`, `Input.dispatchMouseEvent`). -## Extension is plain JS +## chrome.scripting serialization gotcha -No build step. Reload in chrome://extensions after changes. Chrome kills service workers after 30s idle - the `chrome.alarms` keepalive handles this. Don't remove it. +`chrome.scripting.executeScript` args **cannot be functions** (recent MV3). The smart-selector fallback fns are passed as their `.toString()` source and rebuilt via `eval('(' + src + ')')` inside the page. If a tool suddenly fails with "Value is unserializable" at args index N, that's the cause. -## Element refs are ephemeral +## Adding a tool -`browser_snapshot` assigns `data-mcp-ref` attributes. These refs break after navigation, scroll, or DOM changes. Always re-snapshot before using refs. +1. `mcp-server/src/tools/.ts` — ToolDefinition with `requireTabId()` in the Zod schema +2. `mcp-server/src/tools/index.ts` — add to `allTools` +3. `extension/background.js` — add to `dispatch()` map + handler (use `resolveTab(tabId)` + `safeExec`) +4. If it has side effects, ensure `isIdempotent()` in `daemon-config.ts` does NOT list it. -## Port 7225 +## Build/test -Not negotiable without updating both server default (index.ts) AND extension default (background.js + popup.html). Configurable via `WS_PORT` env var on server side, popup input on extension side. +- `npm run build` (tsc), `npm test` (vitest), `npm run typecheck` +- Extension is plain JS (no build) — `node --check` each file after edits +- Daemon is built to `mcp-server/dist/`. Reload the extension at `chrome://extensions` after any extension edit; content scripts cache per-page-load so open tabs need a hard refresh. -## Tests need port isolation +## graphify -Each test gets a unique port (counter starting at 19230). WebSocket servers take time to unbind - tests will flake without this. +This repo is graphify-indexed (457 nodes / 727 edges / 28 communities). Use `graphify query "..."` to trace code paths, `graphify update .` for zero-cost code refresh, `graphify . --backend claude` for image extraction. See memory `mem:graphify-setup` for the Z.ai endpoint config. + +## Known benign console noise (already filtered) + +`ResizeObserver loop` and `message channel closed` are filtered in `content.js`. Both need extension reload + tab refresh to take effect (content scripts cache per page). diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index e5bc826..7d27fa2 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -16,7 +16,7 @@ jobs: steps: - uses: actions/checkout@v7 - - uses: actions/setup-node@v6 + - uses: actions/setup-node@v7 with: node-version: ${{ matrix.node-version }} diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 07732d1..9974d57 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -17,7 +17,7 @@ jobs: - uses: actions/checkout@v7 with: fetch-depth: 0 - - uses: actions/setup-node@v6 + - uses: actions/setup-node@v7 with: node-version: 22 cache: npm diff --git a/.github/workflows/scorecard.yml b/.github/workflows/scorecard.yml index 15f888c..a7dcb83 100644 --- a/.github/workflows/scorecard.yml +++ b/.github/workflows/scorecard.yml @@ -18,7 +18,7 @@ jobs: - uses: actions/checkout@v7 with: persist-credentials: false - - uses: ossf/scorecard-action@v2.4.3 + - uses: ossf/scorecard-action@v2.4.4 with: results_file: results.sarif results_format: sarif diff --git a/.gitignore b/.gitignore index 9d9c8e0..5eab5fc 100644 --- a/.gitignore +++ b/.gitignore @@ -2,3 +2,7 @@ node_modules/ mcp-server/dist/ *.tsbuildinfo .DS_Store +graphify-out/ +# daemon runtime state (task 1.0 / 3.1) +daemon.log +.serena/ diff --git a/.graphifyignore b/.graphifyignore new file mode 100644 index 0000000..daf7e36 --- /dev/null +++ b/.graphifyignore @@ -0,0 +1,3 @@ +# Images ARE extracted now via the Z.ai Anthropic endpoint (glm-4.6 vision). +# Keep generated graphify output out of itself. +graphify-out/ diff --git a/.zcode/plans/plan-sess_cb96dc0a-88a2-4f04-b749-9806799a675a.md b/.zcode/plans/plan-sess_cb96dc0a-88a2-4f04-b749-9806799a675a.md new file mode 100644 index 0000000..167e0d0 --- /dev/null +++ b/.zcode/plans/plan-sess_cb96dc0a-88a2-4f04-b749-9806799a675a.md @@ -0,0 +1,146 @@ +# إصلاح الوكلاء الأشباح (Zombie) + اسم الوكيل + إدارة التبويبات من الـ Popup + +أربعة أعراض، ثلاثة أسباب جذرية مثبتة بالكود وسجل الـ daemon الحيّ. كل تغيير مؤكد ضد `daemon.ts`, `bridge.ts`, `popup.js`, `background.js`, `manifest.json`. + +--- + +## السبب الجذري لكل عرض + +| العرض | السبب | الدليل | +|---|---|---| +| ١. الاسم "agent" بدلاً من "Cursor" | `deriveAgentName()` ترجع القيمة الافتراضية الحرفية `'agent'` عند فشل اكتشاف البيئة | `index.ts:218` | +| ٢. ١٤ وكيل أشباح | لا نبضة حياة (heartbeat) على مقابس IPC ← تأتي `close` متأخرة (دقائق). سجل الـ daemon يُظهر s8–s21 أُنشئت خلال ثوانٍ (IDE يعيد تشغيل MCP)، وفصلت بعد ٥ دقائق فقط | `daemon.ts:54,167,186`؛ السجل أسطر ٩٨-١٢٩ | +| ٣. أقفال التبويب "none" | **سلوك صحي** — الأقفال تُنشأ فقط بأداة `browser_tabs lock` الصريحة، لا أثناء التنفيذ العادي. المشكلة أنك لا تملك واجهة لقفل التبويب | `background.js:1066` | +| ٤. لا قائمة تبويبات ولا قفل فردي | الـ popup لا يستدعي `chrome.tabs.query` أبداً ولا يعرض الأقفال إلا للقراءة | `popup.js` | + +بالإضافة لذلك: سجل الـ daemon يُكتب في `~/.real-browser-mcp/daemon.log` (الاسم القديم) بينما الحالة في `~/.browser-controller/` (`index.ts:229`). + +--- + +## التغييرات (٥ ملفات) + +### ١. `mcp-server/src/daemon.ts` — نبضة الحياة + إزالة التكرار + `/kill` + +**أ. إزالة التكرار بالاسم + نبضة الحياة (يصلح الأشباح من المصدر):** +- عند تسجيل عميل جديد (`hello`، السطر ١٦٥)، إذا وجد عميل موجود بنفس `agentName`، استبدله (احذف القديم ثم أضف الجديد). هذا يمنع تراكم N سجل بنفس الاسم عندما يعيد IDE تشغيل الـ MCP. +- أضف `setInterval` كل ١٥ ثانية يرسل `kind:'ping'` لكل عميل عبر الـ IPC socket؛ العميلة ترسل `kind:'pong'`. إذا فات ٣ نبضات متتالية (~٤٥ ثانية)، اعتبر السوكيت ميتاً وألغِ الاتصال (`socket.destroy()` → يُطلق `close` → تنظيف موجود). +- مكّن `socket.setKeepAlive(true, 30_000)` لتسريع كشف السوكيت شبه المفتوح على مستوى النواة. + +**ب. نقطة نهاية `/kill` (GET):** في `handleHttp` (السطر ٢٤٦) أضف: +```ts +if (path === '/kill') { + const sid = url.searchParams.get('sessionId'); + const victim = Array.from(this.clients.values()).find(c => c.sessionId === sid); + if (!victim) return { ok: false, error: 'session not found' }; + victim.socket.destroy(); // → fires close → cleanup + return { ok: true, killed: sid }; +} +``` +ملاحظة: يستخدم GET لأن `bridge.ts:195` يرفض أي طلب غير GET — لا حاجة لتعديل الـ bridge. + +**ج. إضافة `kind:'ping'`/`'pong'` لبروتوكول IPC** في `daemon-config.ts`: +```ts +| { kind: 'ping' } | { kind: 'pong' } +``` +وأضف فرعاً في `handleNewClient` (السطر ١٧٨) يتجاهل `pong` بهدوء. + +### ٢. `mcp-server/src/index.ts` — اسم وكيل أفضل + نبضة pong + سجل صحيح + +**أ. اسم وكيل أقوى (`deriveAgentName` السطر ٢١٠):** أضف كشف العملية الأم قبل الـ fallback: +```ts +// كشف اسم العملية الأم (ppid) — أكثر دقة من متغيرات البيئة +try { + const ppid = process.ppid; + const parent = execSync(`ps -p ${ppid} -o comm=`, {encoding:'utf8'}).trim(); + if (/cursor/i.test(parent)) return 'Cursor'; + if (/claude/i.test(parent)) return 'Claude'; + if (/windsurf|code/i.test(parent)) return 'Windsurf'; + if (parent) return parent.split('/').pop(); // اسم العملية كبديل أخير +} catch {} +return 'agent'; // أبعد ما يكون +``` +يحافظ على ترتيب متغيرات البيئة الموجود (السطور ٢١٢-٢١٧) فوقه. + +**ب. نبضة pong (عميل IPC):** في `handleMessage` (السطر ١٢٨) أضف: +```ts +case 'ping': + this.socket?.write(JSON.stringify({ kind: 'pong' }) + '\n'); + break; +``` + +**ج. إصلاح مسار السجل (السطر ٢٢٩):** غيّر من `~/.real-browser-mcp/daemon.log` إلى `path.join(STATE_DIR, 'daemon.log')` (= `~/.browser-controller/daemon.log`). استورد `STATE_DIR` من `daemon-config`. + +### ٣. `extension/background.js` — قفل/فتح من الـ popup + قائمة التبويبات + +أضف معالجتي رسائل جديدتين في مستمع `chrome.runtime.onMessage` (السطر ١٤٤٢): + +```js +if (msg.type === 'lockTab') { + // { tabId, sessionId } — يربط تبويب بوكيل من الـ popup + tabLocks.lock(msg.tabId, msg.sessionId); + broadcastStatus(`Tab ${msg.tabId} pinned to ${msg.sessionId}`); + respond({ success: true }); + return false; +} +if (msg.type === 'unlockTab') { + // { tabId } — يفتح تبويباً واحداً (ليس الكل) + const was = tabLocks.owner(msg.tabId); + tabLocks.release(msg.tabId); + broadcastStatus(`Tab ${msg.tabId} unlocked (was ${was || '-'})`); + respond({ success: true, previousSession: was }); + return false; +} +``` + +وسّع `buildStatusPayload` (السطر ٢٤٨) ليشمل `tabLocks` **وقائمة التبويبات المفتوحة** (`tabs` مع `lockedBy`) حتى يعرضها الـ popup دفعة واحدة: +```js +tabs: await getOpenTabs(), // [{id, title, url, active, lockedBy}] +``` +حيث `getOpenTabs()` تغلف `chrome.tabs.query({currentWindow:true})` + `tabLocks.owner(t.id)` (نفس منطق `handleTabs` السطور ١٠٣٨-١٠٤٩ — **أعد استخدام النمط، لا تكرره**). + +### ٤. `extension/popup/popup.html` + `popup.js` — واجهة حقيقية + +**HTML:** أضف قسماً جديداً "Open Tabs" (قائمة التبويبات) **قبل** قسم "Connected Agents". كل صف تبويب يحتوي: +- عنوان/رابط التبويب (مقتطف) +- `` dropdowns | | `browser_wait` | Wait for elements to appear or disappear | +| `browser_fill_form` | Fill multiple form fields in one call | +| `browser_drag` | Drag element-to-element (uses CDP for reliability) | +| `browser_upload_file` | Upload files through `` (uses CDP) | **Navigate** | Tool | What it does | |------|-------------| -| `browser_navigate` | Go to a URL in the active tab | -| `browser_tabs` | List, create, close, or focus tabs | +| `browser_navigate` | Go to a URL in a tab (`tabId` optional, defaults to active) | +| `browser_tabs` | List / create / close / focus / **lock** / **unlock** tabs | -**Debug** +**Debug & Advanced** | Tool | What it does | |------|-------------| -| `browser_console` | Console output (log, warn, error) | -| `browser_network` | XHR/fetch requests with status codes | -| `browser_evaluate` | Run JavaScript via Chrome DevTools Protocol | +| `browser_console` | Console output (log, warn, error) — per-tab, capped at 200 entries | +| `browser_network` | XHR/fetch requests with status codes — per-tab | +| `browser_evaluate` | Run JavaScript in the page's MAIN world (no banner, CSP-safe) | | `browser_handle_dialog` | Handle alert/confirm/prompt dialogs | +| `browser_run_action` | Run a self-contained JS action object via CDP | + +--- + +## How Others Compare + +| | Browser Controller | Playwright MCP | Chrome DevTools MCP | +|---|---|---|---| +| Uses your existing browser | Yes | No, launches new | Partial, needs debug port | +| Sessions and cookies | Already there | Fresh profile | Manual setup | +| Works behind corporate SSO | Yes | No | Depends | +| Multiple agents, multiple tabs | Yes (v2) | No | No | +| Tab-targeting (won't hijack active tab) | Yes (v2) | N/A | No | +| Authenticated local connection | Yes (v2) | N/A | No | +| Setup | Extension + MCP config | Headless browser | Chrome with `--remote-debugging-port` | --- @@ -194,23 +338,42 @@ See [`agent-config/`](agent-config/) for manual installation or to customize the | Env var | Default | What it does | |---------|---------|-------------| -| `WS_PORT` | `7225` | WebSocket port for extension connection | +| `WS_PORT` | `7225` | WebSocket port the daemon uses for the extension connection | + +### Daemon state files -Connection drops are handled automatically with exponential backoff (1s to 30s), ping/pong health checks every 10s, and per-tool timeouts (5s for clicks, 60s for navigation). +The daemon keeps everything in `~/.browser-controller/` (Windows: `%USERPROFILE%\.browser-controller\`): + +| File | Purpose | +|------|---------| +| `token.json` | Auth token the extension must present (mode `0600`) | +| `daemon.sock` | The IPC socket thin clients connect to (AF_UNIX on mac/linux; named pipe on Windows) | +| `daemon.json` | Daemon metadata (pid, port, start time) — used to detect a running daemon | +| `daemon.log` | Daemon stdout/stderr when spawned by a client | + +To fully reset: stop your MCP clients, delete the folder, and the next run recreates it with a fresh token. + +### Reliability + +- The daemon is **auto-spawned** the first time any client runs and left running detached. +- Connection drops use exponential backoff (1s → 30s), ping/pong health checks every 10s. +- Per-tool timeouts (5s for clicks, 60s for navigation). +- **Read-only tools** (snapshot, screenshot, text, find, console, network) are retried on timeout; **side-effecting tools** (click, type, navigate, evaluate) are **never** retried — a click can't fire twice. +- If a previous daemon crashed while holding port 7225, the new one finds and evicts the stale PID (cross-platform: `lsof` on mac/linux, `Get-NetTCPConnection` on Windows).
Multiple Chrome profiles -Run two server instances on different ports: +Run two daemons on different ports by setting `WS_PORT` per client: ```json { "mcpServers": { "browser-work": { - "command": "npx", "args": ["-y", "real-browser-mcp"] + "command": "npx", "args": ["-y", "browser-controller"] }, "browser-personal": { - "command": "npx", "args": ["-y", "real-browser-mcp"], + "command": "npx", "args": ["-y", "browser-controller"], "env": { "WS_PORT": "9333" } } } @@ -226,21 +389,27 @@ Update the port in each extension popup to match.
Architecture -Everything stays on your machine. The extension connects to the MCP server via WebSocket on localhost. No cloud, no proxy, nothing leaves your browser. +Everything stays on your machine. The extension connects to the daemon via an authenticated WebSocket on localhost; MCP clients connect to the daemon via a local IPC socket. No cloud, no proxy, nothing leaves your browser. ``` -real-browser-mcp/ +browser-controller/ ├── mcp-server/ MCP server (npm package, TypeScript) -│ └── src/tools/ One file per tool, registry pattern +│ └── src/ +│ ├── daemon.ts Single multi-client daemon (owns WS :7225) +│ ├── daemon-config.ts IPC protocol, paths, auth token, idempotency set +│ ├── index.ts Thin stdio MCP client (spawns daemon, multiplexes) +│ ├── bridge.ts Extension WS server + cross-platform port probe +│ └── tools/ One file per tool (22), registry pattern ├── extension/ Chrome extension (Manifest V3, plain JS) -│ ├── background.js Service worker, WebSocket client, tool handlers -│ ├── content.js Console capture -│ └── popup/ Connection status UI +│ ├── background.js Service worker: tab resolution, per-tab mutex, locks +│ ├── lib/tab-concurrency.js Pure, unit-tested mutex + lock primitives +│ ├── content.js Console capture +│ └── popup/ Status, Open Tabs (pin/unpin), Connected Agents (disconnect), tab-lock viewer ├── agent-config/ Pre-built configs for Cursor + Claude Code -│ ├── cursor/ Rules and commands -│ ├── skills/ Browser automation skill -│ └── setup.mjs One-command installer -└── tests/ Bridge + registry tests +│ ├── cursor/ Rules and commands +│ ├── skills/ Browser automation skill +│ └── setup.mjs One-command installer +└── tests/ Bridge + registry + concurrency tests (45 passing) ``` **Stack:** TypeScript (strict) · MCP SDK · WebSocket · Chrome Extension Manifest V3 · Vitest @@ -251,21 +420,23 @@ real-browser-mcp/ Development ```bash -git clone https://github.com/ofershap/real-browser-mcp.git -cd real-browser-mcp +git clone https://github.com/compnew2006/browser-controller.git +cd browser-controller npm install npm run build npm test ``` | Command | What it does | -|---------|-------------| -| `npm run build` | Compile TypeScript | +|---------|---------| +| `npm run build` | Compile TypeScript → `mcp-server/dist/` | | `npm run dev` | Watch mode | -| `npm test` | Run tests | +| `npm test` | Run the full test suite (45 tests) | | `npm run typecheck` | Type check without emitting | | `npm run setup:cursor` | Install Cursor rule + command | +The test suite covers the WebSocket bridge (including token-auth rejection), the tool registry, and the per-tab concurrency primitives (same-tab serialization + cross-tab parallelism). The daemon's IPC auth and the WS token enforcement are also smoke-tested via the build artifacts in `mcp-server/dist/`. +
## FAQ @@ -273,35 +444,49 @@ npm test
Does it work with my logged-in sessions? -That's the whole point. The extension runs inside your actual Chrome - same cookies, same sessions, same local storage. No re-authentication needed. +That's the whole point. The extension runs inside your actual Chrome — same cookies, same sessions, same local storage. No re-authentication needed.
Does it send data anywhere? -No. The MCP server and extension talk over WebSocket on localhost. Nothing leaves your machine. There's no analytics, no telemetry, no cloud component. [Privacy policy.](PRIVACY.md) +No. The MCP clients, the daemon, and the extension all talk over localhost (IPC socket + WebSocket). Nothing leaves your machine. There's no analytics, no telemetry, no cloud component. [Privacy policy.](PRIVACY.md)
Which AI clients work? -Any MCP-compatible client. Cursor, Claude Desktop, Claude Code, Windsurf, Cline, and anything else that speaks the MCP protocol. +Any MCP-compatible client. Cursor, Claude Desktop, Claude Code, Windsurf, Cline, and anything else that speaks the MCP protocol. v2 lets several of them run at once against the same daemon. + +
+ +
+Can two agents really work at the same time without breaking each other? + +Yes — that's the main v2 improvement. Each agent connects to the shared daemon, gets its own `sessionId`, and targets a specific `tabId`. Actions on the same tab serialize through a per-tab mutex; actions on different tabs run in parallel. Optionally an agent can `lock` a tab to claim exclusive access; other agents queue behind the lock rather than failing.
-Can I use it with multiple Chrome profiles? +What if the agent acts on the wrong tab? -Yes. Run two MCP server instances on different ports. See [Configuration](#configuration) for the setup. +It can't — not silently. Every page-interaction tool requires a `tabId`, and if it's missing you get a clear `tabId is required` error. The agent can never accidentally act on the tab you happen to be looking at. (The one exception is `browser_navigate` without a `tabId`, which uses the active tab — but for multi-agent use you should always pass `tabId`.) + +
+ +
+Why is there an auth token? + +Without it, any local process on your machine could open a WebSocket to port 7225 and drive your authenticated browser sessions (your bank, your email, your company SSO). The daemon generates a secret token in `~/.browser-controller/token.json` and the extension must present it to connect.
How is this different from Playwright MCP or browser-use? -They launch a new browser instance from scratch - no state, no cookies, no sessions. You have to replay the full login flow every time. This connects to the browser you already have open with everything already loaded. +They launch a new browser instance from scratch — no state, no cookies, no sessions. You have to replay the full login flow every time. This connects to the browser you already have open with everything already loaded.
@@ -313,16 +498,8 @@ Bug reports, feature requests, and PRs welcome. Open an issue first for larger c ## Author -[![Made by ofershap](https://gitshow.dev/api/card/ofershap)](https://gitshow.dev/ofershap) - -[![LinkedIn](https://img.shields.io/badge/LinkedIn-Connect-0A66C2?style=flat&logo=linkedin&logoColor=white)](https://linkedin.com/in/ofershap) -[![GitHub](https://img.shields.io/badge/GitHub-Follow-181717?style=flat&logo=github&logoColor=white)](https://github.com/ofershap) - - ---- - -README built with [README Builder](https://ofershap.github.io/readme-builder/) +**noiemany** — [GitHub](https://github.com/noiemany) ## License -[MIT](LICENSE) © [Ofer Shapira](https://github.com/ofershap) +[MIT](LICENSE) © noiemany diff --git a/SECURITY.md b/SECURITY.md index 708e836..593eaf8 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -6,7 +6,7 @@ Report vulnerabilities through: -1. **GitHub Security Advisories (preferred):** [Create a private security advisory](https://github.com/ofershap/real-browser-mcp/security/advisories/new) +1. **GitHub Security Advisories (preferred):** [Create a private security advisory](https://github.com/compnew2006/browser-controller/security/advisories/new) ### What to Include diff --git a/agent-config/AGENTS.md b/agent-config/AGENTS.md index 6b2bc10..0394a20 100644 --- a/agent-config/AGENTS.md +++ b/agent-config/AGENTS.md @@ -1,22 +1,42 @@ -# Real Browser MCP - Agent Config +# Browser Controller - Agent Config ## Browser Control -This project has `real-browser-mcp` configured. Use it to interact with the user's real browser. +This project has `browser-controller` configured (renamed from `real-browser-mcp`). Use it to interact with the user's real browser. -The MCP server runs on `ws://localhost:7225` by default. The Chrome extension connects automatically. +The daemon runs on `127.0.0.1:7225` (WS + HTTP). The Chrome extension auto-pairs a token and connects automatically. Multiple agents can connect at once — they share a single daemon, each getting its own sessionId (visible in the popup alongside its agent name and uptime). Reconnecting with the same agent name replaces the old session (no duplicates), and dead connections are evicted by a heartbeat after ~45s. Name the agent explicitly with `--agent ` in the MCP config args, or `MCP_AGENT_NAME` env. -## Tools +## CRITICAL: Tab targeting (v2) -Navigation: `browser_navigate`, `browser_tabs` -Interaction: `browser_click`, `browser_type`, `browser_press_key`, `browser_scroll`, `browser_hover`, `browser_select` -Reading: `browser_snapshot`, `browser_screenshot`, `browser_text`, `browser_find` +This is a multi-client server. **Never assume which tab your actions hit.** You MUST target a tab explicitly with `tabId`. + +1. `browser_tabs` with `action: "list"` → get a `tabId` +2. Pass that **same `tabId`** to every page-interaction tool +3. Refs from a snapshot are valid **only for the tabId that produced them** +4. For exclusive access: `browser_tabs { action: "lock", tabId }` → `unlock` when done + +## Element refs + automatic recovery + +Refs (`e5`) can break on dynamic sites (React re-renders, virtualized feeds like Facebook/Instagram). The extension handles this automatically — you don't need to do anything special: + +- If a ref breaks but the element still exists → it's found via a robust CSS selector, then text+role scan. The response includes `via: "fallback"`. +- If the element is gone entirely (scrolled away) → the response includes `freshRefs: [...]` with a fresh snapshot so you can retry in ONE step with a new ref. Do not re-send the old ref. +- After `browser_scroll`, expect `refsMayBeStale: true` — re-snapshot before your next interaction on virtualized feeds. + +## Tools (22) + +Navigation: `browser_navigate` (tabId optional), `browser_tabs` (list/create/close/focus/lock/unlock) +Interaction (tabId required): `browser_click`, `browser_click_text`, `browser_type`, `browser_press_key`, `browser_scroll`, `browser_hover`, `browser_select`, `browser_drag`, `browser_fill_form`, `browser_upload_file` +Reading (tabId required): `browser_snapshot`, `browser_screenshot`, `browser_text`, `browser_find` +JS/Dialogs (tabId required): `browser_evaluate` (MAIN world, no banner), `browser_handle_dialog`, `browser_run_action` Waiting: `browser_wait` -Debug: `browser_console`, `browser_network` +Debug (tabId required, per-tab, capped 200 entries): `browser_console`, `browser_network` ## Pattern -1. `browser_snapshot` to see the page and get refs -2. Use refs with interaction tools -3. Snapshot again to verify -4. `browser_wait` before interacting with dynamic content +1. `browser_tabs { action: "list" }` → pick a `tabId` +2. `browser_snapshot { tabId }` to see the page and get refs (refs are tab-scoped) +3. Use `{ tabId, ref }` with interaction tools +4. Re-snapshot after navigation/DOM changes to refresh refs +5. `browser_wait { tabId, selector }` before interacting with dynamic content +6. On a `freshRefs` response, retry with one of the new refs immediately (no separate snapshot needed) diff --git a/agent-config/README.md b/agent-config/README.md index e9e9102..e82a9ec 100644 --- a/agent-config/README.md +++ b/agent-config/README.md @@ -6,13 +6,13 @@ Your AI agent needs to know these tools exist and how to use them. These configs ```bash # Cursor - installs rule + /check-browser command -npx real-browser-mcp --setup cursor +npx browser-controller --setup cursor # Claude Code - adds AGENTS.md -npx real-browser-mcp --setup claude +npx browser-controller --setup claude # Both -npx real-browser-mcp --setup all +npx browser-controller --setup all ``` ## What Gets Installed @@ -21,7 +21,7 @@ npx real-browser-mcp --setup all | File | Location | What it does | |------|----------|-------------| -| `real-browser-mcp.mdc` | `~/.cursor/rules/` | Global rule - teaches the agent how to use browser tools | +| `browser-controller.mdc` | `~/.cursor/rules/` | Global rule - teaches the agent how to use browser tools | | `check-browser.md` | `~/.cursor/commands/` | `/check-browser` command - quick way to say "look at my browser" | After install, type `/check-browser` in Cursor chat or just tell your agent: @@ -44,7 +44,7 @@ The `skills/browser-automation/SKILL.md` is a detailed workflow guide for browse Copy the files yourself if you prefer: -- **Cursor rule:** `cursor/rules/real-browser-mcp.mdc` -> `~/.cursor/rules/` +- **Cursor rule:** `cursor/rules/browser-controller.mdc` -> `~/.cursor/rules/` - **Cursor command:** `cursor/commands/check-browser.md` -> `~/.cursor/commands/` - **Claude Code:** `AGENTS.md` -> your project root - **Skill:** `skills/browser-automation/` -> your skills directory diff --git a/agent-config/cursor/commands/check-browser.md b/agent-config/cursor/commands/check-browser.md index 51b7a05..968e3ec 100644 --- a/agent-config/cursor/commands/check-browser.md +++ b/agent-config/cursor/commands/check-browser.md @@ -1,13 +1,14 @@ # check-browser -Use Real Browser MCP to check what's currently showing in my browser. +Use Browser Controller to check what's currently showing in my browser. ## Steps -1. Call `browser_snapshot` to see the current page structure -2. Based on what you find, take a `browser_screenshot` if visual verification is needed -3. Report back what you see - page title, key elements, any errors or unexpected state -4. If I asked you to verify something specific, focus on that +1. Call `browser_tabs { action: "list" }` to get a `tabId` +2. Call `browser_snapshot { tabId }` to see the current page structure (refs are tab-scoped) +3. Based on what you find, take a `browser_screenshot { tabId }` if visual verification is needed +4. Report back what you see - page title, key elements, any errors or unexpected state +5. If I asked you to verify something specific, focus on that ## When to use @@ -16,9 +17,16 @@ Use Real Browser MCP to check what's currently showing in my browser. - To read page content I'm looking at - To debug visual issues or check responsive layout +## Reading the snapshot efficiently + +- **`isNew: true`** on a ref means that element appeared since the last snapshot — focus on those when something just changed (e.g. an overlay opened after a click). You don't need to re-read the whole tree. +- After `browser_scroll`, expect `refsMayBeStale: true` — re-snapshot before interacting on feeds like Facebook/Instagram that recycle DOM nodes. +- If a click returns `freshRefs`, the element was scrolled away — retry with one of the new refs in the same response (no separate snapshot needed). + ## Important - This is my REAL browser. I'm already logged in everywhere. -- Don't close tabs I didn't ask you to close -- Don't navigate away from the current page unless I ask you to -- Use element refs from snapshots for any interactions +- Always pass `tabId` — never assume the active tab is the one to act on. +- Don't close tabs I didn't ask you to close. +- Don't navigate away from the current page unless I ask you to. +- Use element refs from snapshots for any interactions; refs are valid only for the tabId that produced them. diff --git a/agent-config/cursor/rules/browser-controller.mdc b/agent-config/cursor/rules/browser-controller.mdc new file mode 100644 index 0000000..0c360c6 --- /dev/null +++ b/agent-config/cursor/rules/browser-controller.mdc @@ -0,0 +1,84 @@ +--- +description: Instructions for using Browser Controller to control the user's actual browser +alwaysApply: true +--- + +# Browser Controller + +MCP server: `browser-controller`. Controls the user's REAL Chrome browser - not headless. All sessions, logins, cookies are already active. + +## CRITICAL: Tab-targeting model (v2) + +This is a **multi-client** server. Other agents may be working in other tabs right now. +**Never assume which tab your actions hit.** You MUST target a tab explicitly. + +### The required flow (instead of "snapshot the active tab") + +1. `browser_tabs` with `action: "list"` → get a `tabId` for the tab you want to work in +2. Pass that **same `tabId`** to EVERY page-interaction tool (`browser_snapshot`, `browser_click`, `browser_type`, …) +3. Refs from a snapshot are **only valid for the tabId that produced them**. A ref from tab 10 will never resolve on tab 20. +4. If you need exclusive access to a tab (another agent might touch it), `browser_tabs` with `action: "lock", tabId` first; `action: "unlock"` when done. + +If `tabId` is missing on a page tool, you'll get: *"tabId is required. Call browser_tabs list first."* + +## Rules + +- NEVER close tabs you didn't create +- NEVER navigate away from pages the user is working on without asking +- ALWAYS pass `tabId` — the "active tab" the user is looking at is NOT yours to act on +- Prefer `browser_snapshot` over `browser_screenshot` (cheaper, faster, parseable) +- Refs break after navigation/scroll/DOM changes - re-snapshot before using stale refs +- **Refs recover automatically**: if a ref goes stale but the element still exists, the tool finds it via a robust selector + text/role scan (response carries `via: "fallback"`). If the element was scrolled away entirely, the response carries `freshRefs: [...]` — retry with one of those new refs in the same step, no separate snapshot needed. +- **`isNew: true`** on a snapshot ref means the element appeared since the last snapshot. After an action opens an overlay/dropdown, filter to `isNew` refs instead of re-reading the whole tree. +- After `browser_scroll`, expect `refsMayBeStale: true` on feeds that recycle DOM nodes (Facebook/Instagram/Twitter) — re-snapshot before interacting. +- `browser_evaluate` now runs in the page's MAIN world via chrome.scripting (no debugger banner, CSP-safe). It no longer needs CDP. + +## Tools + +Every page-interaction tool takes `tabId` (number, required). Listed below. + +### Navigation & Tabs +- `browser_navigate` - Go to URL in a tab. Params: `url`, `tabId` (optional, defaults to active), `waitUntil` +- `browser_tabs` - List/create/close/focus/lock tabs. Params: `action` (list|create|close|focus|lock|unlock), `tabId`, `url` + +### Interaction (all require `tabId`) +- `browser_click` - Click by ref OR selector. Params: `tabId`, `ref`, `selector`, `button`, `doubleClick` +- `browser_click_text` - Click by visible text. Params: `tabId`, `text`, `index`, `exact` +- `browser_type` - Type text. Params: `tabId`, `ref` OR `selector`, `text`, `clear` +- `browser_press_key` - Keyboard. Params: `tabId`, `key`, `modifiers`, `ref`, `selector` +- `browser_scroll` - Scroll. Params: `tabId`, `direction`, `amount`, `selector`, `toElement`, `position` +- `browser_hover` - Hover. Params: `tabId`, `ref` OR `selector` +- `browser_select` - Select from ``. Params: `ref` OR `selector`, `value` OR `label` OR `index` - -### Reading -- `browser_snapshot` - Accessibility tree with refs. Params: `selector` -- `browser_screenshot` - Capture visible tab as image. Params: `format` (png|jpeg), `quality` -- `browser_text` - Get text content. Params: `selector`, `maxLength` -- `browser_find` - Find elements by natural language. Params: `query`, `limit` - -### JavaScript & Dialogs -- `browser_evaluate` - Execute JS via chrome.debugger CDP. Bypasses CSP but shows debugger banner and may close popups. Use as LAST RESORT only. -- `browser_handle_dialog` - Handle alert/confirm/prompt. Call BEFORE triggering actions that cause dialogs. Params: `action` (accept|dismiss), `promptText` - -### Waiting -- `browser_wait` - Wait for element or delay. Params: `selector`, `state` (visible|hidden|attached), `timeout`, `delay` - -### Debug -- `browser_console` - Read console messages (read-only). Params: `clear` -- `browser_network` - Read network requests. Params: `filter` (regex), `clear` - -## React Dropdowns / Portals / Overlays (IMPORTANT) - -Snapshot refs DO NOT capture React portal overlays (rendered outside main DOM tree). DO NOT use `browser_evaluate` — it uses chrome.debugger which steals focus and closes the dropdown. - -Proven pattern that works on all sites including strict-CSP (GitHub, Google): - -1. `browser_click` on the trigger button (use ref from snapshot) -2. `browser_wait` with `delay: 400` for the overlay to render -3. `browser_text` with broad CSS selectors to confirm dropdown is open and read options: - `"selector": "[role=menu],[role=listbox],.ActionList,.SelectMenu-list,.Overlay,details-menu"` -4. `browser_click` with a CSS selector targeting the desired child: - `"selector": ".ActionList li:first-child"` or use `browser_click_text` with the option text - -If `browser_click_text` is available, prefer it — it finds elements by visible text, picks the deepest DOM match, and is CSP-safe. - -## Workflow - -1. `browser_snapshot` to see page structure and get element refs -2. Use refs with `browser_click`, `browser_type`, etc. -3. For dropdowns/menus: follow the pattern above (click trigger → wait → text → click child) -4. After actions, re-snapshot or `browser_find` to verify result -5. `browser_wait` before interacting with dynamically loaded content -6. `browser_press_key` for keyboard shortcuts (Enter to submit, Escape to close modals) diff --git a/agent-config/setup.mjs b/agent-config/setup.mjs index f38503f..4eb215c 100644 --- a/agent-config/setup.mjs +++ b/agent-config/setup.mjs @@ -8,7 +8,7 @@ const __dirname = dirname(fileURLToPath(import.meta.url)); const target = process.argv[2]; if (!target || !['cursor', 'claude', 'all'].includes(target)) { - console.log('Usage: real-browser-mcp --setup '); + console.log('Usage: browser-controller --setup '); console.log(''); console.log(' cursor - Install Cursor rule + command'); console.log(' claude - Copy AGENTS.md to your project'); @@ -39,8 +39,8 @@ function installCursor() { const rulesDir = join(home, '.cursor', 'rules'); ensureDir(rulesDir); copyIfNeeded( - join(__dirname, 'cursor', 'rules', 'real-browser-mcp.mdc'), - join(rulesDir, 'real-browser-mcp.mdc'), + join(__dirname, 'cursor', 'rules', 'browser-controller.mdc'), + join(rulesDir, 'browser-controller.mdc'), 'rule', ); @@ -64,8 +64,8 @@ function installClaude() { if (existsSync(dest)) { const existing = readFileSync(dest, 'utf-8'); - if (existing.includes('real-browser-mcp')) { - console.log(' AGENTS.md already contains real-browser-mcp config. Skipping.'); + if (existing.includes('browser-controller')) { + console.log(' AGENTS.md already contains browser-controller config. Skipping.'); return; } const addition = readFileSync(src, 'utf-8'); diff --git a/agent-config/skills/browser-automation/SKILL.md b/agent-config/skills/browser-automation/SKILL.md index c1e2c0d..8c5202a 100644 --- a/agent-config/skills/browser-automation/SKILL.md +++ b/agent-config/skills/browser-automation/SKILL.md @@ -1,49 +1,74 @@ --- name: browser-automation -description: Control the user's real browser via Real Browser MCP. Use when asked to interact with web pages, test UIs, fill forms, or read page content. +description: Control the user's real browser via Browser Controller. Use when asked to interact with web pages, test UIs, fill forms, or read page content. --- -# Browser Automation with Real Browser MCP +# Browser Automation with Browser Controller Use this skill when you need to interact with the user's actual browser - clicking, typing, reading pages, taking screenshots, or navigating. +## CRITICAL: Always target a tab explicitly (v2) + +This is a **multi-client** server — other agents may be using other tabs. You MUST pass `tabId` to every page-interaction tool; actions never hit "the active tab" implicitly. + +1. First call `browser_tabs` with `action: "list"` to discover tabs and get a `tabId` +2. Pass that same `tabId` to `browser_snapshot`, `browser_click`, `browser_type`, etc. +3. Refs from a snapshot are valid **only for the tabId that produced them** — never reuse a ref from tab 10 on tab 20 +4. If you need exclusive use of a tab, `browser_tabs { action: "lock", tabId }` first, then `unlock` when done + +If `tabId` is missing you'll get: *"tabId is required. Call browser_tabs list first."* + ## Before You Start -1. Verify the extension is connected: try `browser_tabs` with action "list" first +1. Verify the extension is connected: `browser_tabs { action: "list" }` (returns the tab list) 2. If disconnected, ask the user to check the extension icon (should show green "ON") 3. Never close tabs you didn't create ## Reading Pages -Start with `browser_snapshot` to get the accessibility tree. This gives you refs like "e12" that you use for interaction. +`browser_snapshot { tabId }` returns the accessibility tree with refs like "e12" that you use for interaction. + +For large pages, scope with a selector: `browser_snapshot { tabId, selector: "main" }`. -For large pages, scope with a CSS selector: `browser_snapshot` with `selector: "main"` or `selector: ".content"`. +Use `browser_text { tabId }` to extract raw text when you need full content. -Use `browser_text` to extract raw text when you need the full content. +### Reading efficiently (don't re-scan everything) + +- **`isNew: true`** on a ref means that element appeared since the last snapshot. When something just changed (an overlay opened after a click, a dropdown rendered), filter to `isNew` refs instead of re-reading the whole tree — big token saver on large pages. +- After `browser_scroll`, expect **`refsMayBeStale: true`** — the feed may have recycled DOM nodes. Re-snapshot before your next interaction. ## Interacting -Always snapshot first, then use refs: -- `browser_click` with `ref: "e12"` to click -- `browser_type` with `ref: "e5"` and `text: "hello"` to type -- `browser_press_key` with `key: "Enter"` to submit -- `browser_scroll` with `direction: "down"` to scroll +Always pass `tabId`, then use refs from that tab's snapshot: +- `browser_click { tabId, ref: "e12" }` +- `browser_type { tabId, ref: "e5", text: "hello" }` +- `browser_press_key { tabId, key: "Enter" }` +- `browser_scroll { tabId, direction: "down" }` + +### When a ref breaks (automatic recovery) + +On dynamic sites (React re-renders, virtualized feeds) refs can go stale. The extension handles this automatically — you usually don't need to do anything: + +- The response includes **`via: "fallback"`** if the element was found via a robust selector or text+role scan. Keep going, but know that ref may be stale for future calls — re-snapshot when convenient. +- If the response includes **`freshRefs: [...]`**, the element was scrolled away entirely and a fresh snapshot was captured inline. **Retry with one of the new refs in the same step** — no separate snapshot needed. ## Dynamic Content (SPAs, social media) -1. `browser_scroll` down to load more content -2. `browser_wait` with a selector for lazy-loaded elements -3. Snapshot again after scrolling - refs are regenerated -4. For virtual scroll containers (Twitter feeds, Reddit), pass the container's CSS selector to `browser_scroll` +1. `browser_scroll { tabId, direction: "down" }` to load more +2. `browser_wait { tabId, selector }` for lazy-loaded elements +3. After scrolling, expect `refsMayBeStale: true` → snapshot again before interacting +4. For virtual scroll containers (Twitter, Reddit), pass the container's CSS selector to `browser_scroll` -## Debugging +## Debugging (per-tab) -- `browser_console` reads console.log/warn/error output -- `browser_network` shows XHR/fetch requests with status codes -- `browser_screenshot` captures what the user sees +- `browser_console { tabId }` reads console output for that tab +- `browser_network { tabId }` shows XHR/fetch requests for that tab +- `browser_screenshot { tabId }` captures the tab (activates it first to capture) ## Common Mistakes -- Using stale refs after navigation or scroll (always re-snapshot) -- Trying to click elements in iframes (scope snapshot to the iframe) -- Not waiting for page load after navigation +- Forgetting `tabId` → "tabId is required" error +- Using a ref from one tabId against a different tabId +- Ignoring `freshRefs` and re-sending a stale ref that already failed +- Re-reading the whole tree when `isNew` refs would suffice +- Holding a tab lock too long (other agents queue behind you) diff --git a/assets/icon.svg b/assets/icon.svg new file mode 100644 index 0000000..fc32502 --- /dev/null +++ b/assets/icon.svg @@ -0,0 +1,53 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/assets/logo.png b/assets/logo.png index d06d36f..87b594c 100644 Binary files a/assets/logo.png and b/assets/logo.png differ diff --git a/extension/background.js b/extension/background.js index aeb43e1..369bafd 100644 --- a/extension/background.js +++ b/extension/background.js @@ -1,10 +1,33 @@ +/** + * Browser Controller — background service worker. + * + * v2 architecture (plan tasks 1.2–1.5, 2.1–2.5): + * - Every page-interaction tool takes an explicit `tabId`. There is NO global + * "active tab" concept on the dispatch path — the user can freely switch + * tabs/move the mouse while an agent works. (1.2) + * - Element refs are stored per-tab (refsByTab), so a ref from tab 10 can never + * resolve against tab 20's DOM. (1.3) + * - console/network buffers are per-tab (capped 200). NOTE: these live in + * service-worker memory and are LOST when Chrome recycles the worker + * (~30s idle). They are debug aids, not durable state — the agent can + * always re-read them after the page emits new messages. (1.4) + * - browser_evaluate runs via chrome.scripting MAIN world — no debugger banner. (1.5) + * - Per-tab mutex: only one tool runs against a given tab at a time; different + * tabs run in parallel. (2.1) + * - Per-agent sessionId + tab lock: browser_tabs lock claims a tab so other + * agents queue behind it instead of racing. (2.2) + * - Snapshot traverses shadow DOM + iframes. (2.4) + * - safeExec turns "can't touch chrome:// pages" into a clear error. (2.5) + */ const DEFAULT_WS_PORT = 7225; const RECONNECT_BASE_MS = 1000; const RECONNECT_MAX_MS = 30000; const KEEPALIVE_ALARM = 'keepalive'; const KEEPALIVE_INTERVAL_MIN = 0.4; // ~24s, under Chrome's 30s limit +const PER_TAB_CAP = 200; let wsPort = DEFAULT_WS_PORT; +let wsToken = ''; // auth token (3.1); appended as ?token= let ws = null; let isConnected = false; let reconnectAttempts = 0; @@ -12,31 +35,110 @@ let reconnectTimeout = null; let nextRetryMs = 0; let connectedSince = null; let lastError = null; -let consoleMessages = []; -let networkRequests = []; -let currentActivity = null; -let activeTabId = null; -let pendingDialog = null; -// --- Connection Management --- +import { TabMutexMap, TabLockMap, runOnTab as runOnTabLib } from './lib/tab-concurrency.js'; +import { PAGE_FALLBACK_FN, PAGE_RESOLVE_FALLBACK_FN } from './utils/smart-selector.js'; + +// --- Per-tab state (tasks 1.3, 1.4) --------------------------------------- +// Map> and Map>. +// In-memory only: Chrome recycles the service worker (~30s idle), which wipes +// these. They're debug aids; the agent re-reads after new messages arrive. +// (Audit C3: the header previously claimed chrome.storage.session persistence +// that was never implemented. Persisting on every message would be write- +// heavy; the honest fix is to document the in-memory behavior.) +const consoleByTab = new Map(); +const networkByTab = new Map(); +/** + * Smart-selector fallbacks (plan tasks 2–4). Map>. + * Populated at snapshot time; consumed by click/type when a ref no longer + * resolves. Never sent to the agent — it keeps payloads token-cheap. + */ +const fallbackByTab = new Map(); +/** + * isNew feature (borrowed from Page-Agent's *[index] idea): Map> + * of "fingerprints" (role|name) from the PREVIOUS snapshot. The next snapshot marks + * any ref whose fingerprint isn't in this set as `isNew: true`, so the agent can + * focus on what changed (e.g. an overlay that appeared after a click) instead of + * re-reading the whole tree — a big token saver on dynamic sites. + */ +const lastSnapshotFingerprints = new Map(); +let currentActivity = null; // tool currently running (single overlay label source) + +// --- Per-tab mutex (task 2.1) + per-agent tab locks (task 2.2) ------------ +// Pure, unit-tested primitives in lib/tab-concurrency.js. +const tabMutex = new TabMutexMap(); +const tabLocks = new TabLockMap(); + +// --- Connection Management ------------------------------------------------ + +/** + * Auto-pair: fetch the daemon's auth token over localhost. The background + * service worker can fetch() (it has host_permissions for 127.0.0.1:7225), and + * it MUST own this — the popup only exists while open, but the WS connection is + * driven by the background at startup and via the keepalive alarm. Returns the + * token, or '' if the daemon isn't up yet (caller retries via reconnect loop). + */ +async function autoPairToken() { + try { + const res = await fetch(`http://127.0.0.1:${wsPort}/pair`, { cache: 'no-store' }); + if (!res.ok) return ''; + const data = await res.json(); + if (data && typeof data.token === 'string' && data.token) { + if (data.token !== wsToken) { + wsToken = data.token; + chrome.storage.local.set({ wsToken }); + } + return data.token; + } + } catch { + // daemon not reachable yet + } + return ''; +} async function initConnection() { try { - const stored = await chrome.storage.local.get('wsPort'); + const stored = await chrome.storage.local.get(['wsPort', 'wsToken']); if (stored.wsPort) wsPort = stored.wsPort; + if (stored.wsToken) wsToken = stored.wsToken; } catch {} chrome.alarms.create(KEEPALIVE_ALARM, { periodInMinutes: KEEPALIVE_INTERVAL_MIN }); + // Always try to (re)pair at startup so the WS carries a fresh token even if + // the daemon rotated it or storage is empty. + await autoPairToken(); connect(); } -function connect() { +function wsUrl() { + // 127.0.0.1 (not 'localhost') so it matches the daemon's IPv4 bind. + // On macOS 'localhost' can resolve to IPv6 ::1 and refuse. + const base = `ws://127.0.0.1:${wsPort}`; + return wsToken ? `${base}?token=${encodeURIComponent(wsToken)}` : base; +} + +/** + * Track whether the socket closed BEFORE opening (handshake destroyed). That's + * the signature of a bad/missing token — so the next reconnect re-pairs first. + */ +let lastWasHandshakeClose = false; + +async function connect() { if (ws && ws.readyState === WebSocket.OPEN) return; + // If the last attempt died mid-handshake, the token is likely stale/empty — + // re-fetch it before reconnecting so we don't loop on a bad token. + if (lastWasHandshakeClose) { + await autoPairToken(); + lastWasHandshakeClose = false; + } + + let opened = false; try { - ws = new WebSocket(`ws://localhost:${wsPort}`); + ws = new WebSocket(wsUrl()); ws.onopen = () => { + opened = true; isConnected = true; reconnectAttempts = 0; connectedSince = Date.now(); @@ -46,6 +148,8 @@ function connect() { }; ws.onclose = () => { + // closed before open = the daemon destroyed the upgrade (bad/no token) + if (!opened) lastWasHandshakeClose = true; isConnected = false; connectedSince = null; ws = null; @@ -62,15 +166,25 @@ function connect() { }; ws.onmessage = async (event) => { + let msgId = null; try { const msg = JSON.parse(event.data); + msgId = msg.id ?? null; if (msg.type === 'ping') { ws.send(JSON.stringify({ type: 'pong' })); return; } await handleMessage(msg); } catch (err) { - console.error('[RealBrowser] Message error:', err); + console.error('[BrowserController] Message error:', err); + // Audit C4/6d: a throw inside handleMessage (e.g. bad params, a handler + // bug) used to leave the daemon hanging until its timeout. Reply with + // an explicit error so the agent gets an actionable message fast. + if (msgId && ws && ws.readyState === WebSocket.OPEN) { + try { + ws.send(JSON.stringify({ id: msgId, success: false, error: err?.message || String(err) })); + } catch { /* ws gone — close path will reject pending on the daemon */ } + } } }; } catch { @@ -92,7 +206,7 @@ chrome.alarms.onAlarm.addListener((alarm) => { if (alarm.name === KEEPALIVE_ALARM && !isConnected) connect(); }); -// --- Badge --- +// --- Badge & Overlay (unchanged behavior) --------------------------------- function updateBadge(status) { const config = { @@ -106,17 +220,15 @@ function updateBadge(status) { chrome.action.setBadgeText({ text: c.text }); } -// --- Activity Overlay --- - async function showOverlay(tabId, label) { try { await chrome.scripting.executeScript({ target: { tabId }, func: (lbl) => { - let el = document.getElementById('__rbmcp-overlay'); + let el = document.getElementById('__bc-overlay'); if (!el) { el = document.createElement('div'); - el.id = '__rbmcp-overlay'; + el.id = '__bc-overlay'; el.style.cssText = 'position:fixed;top:12px;right:12px;background:linear-gradient(135deg,#667eea,#764ba2);color:#fff;' + 'padding:6px 14px;border-radius:16px;font:500 13px system-ui,sans-serif;z-index:2147483647;' + @@ -138,7 +250,7 @@ async function hideOverlay(tabId) { try { await chrome.scripting.executeScript({ target: { tabId }, - func: () => document.getElementById('__rbmcp-overlay')?.remove(), + func: () => document.getElementById('__bc-overlay')?.remove(), }); } catch {} } @@ -149,7 +261,7 @@ function getConnectionState() { return 'disconnected'; } -function buildStatusPayload(message) { +function buildStatusPayload(message, tabs) { return { type: 'status', connectionState: getConnectionState(), @@ -159,40 +271,123 @@ function buildStatusPayload(message) { connectedSince, lastError, activity: currentActivity, + tabLocks: tabLocksToJSON(), + // Open-tab snapshot so the popup can render a Pin dropdown per tab without + // a second round-trip. undefined when the caller didn't fetch tabs (the + // popup treats missing the same as empty — see updateUI). + tabs: tabs || [], statusMessage: message || null, }; } -function broadcastStatus(message) { - chrome.runtime.sendMessage(buildStatusPayload(message)).catch(() => {}); +function tabLocksToJSON() { + return tabLocks.snapshot(); } -// --- Message Router --- +/** + * Open tabs in the current window, each annotated with its lock owner so the + * popup can show "Locked: {session}" and preselect it in the Pin dropdown. + * Mirrors the shape of the `browser_tabs list` tool result (handleTabs) so the + * popup and the agent-facing tool stay consistent. + */ +async function getOpenTabs() { + try { + const tabs = await chrome.tabs.query({ currentWindow: true }); + return tabs.map((t) => ({ + id: t.id, + url: t.url, + title: t.title, + active: t.active, + lockedBy: tabLocks.owner(t.id) || null, + })); + } catch { + return []; + } +} -async function handleMessage(msg) { - const { id, tool, params } = msg; +async function broadcastStatus(message) { + // Tabs are fetched first so the popup's "Open Tabs" panel stays in sync with + // lock changes (a lock op changes lockedBy, which the popup re-renders). + const tabs = await getOpenTabs(); + chrome.runtime.sendMessage(buildStatusPayload(message, tabs)).catch(() => {}); +} - let tabId = null; - try { - const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); - tabId = tab?.id; - } catch {} +// --- Message Router ------------------------------------------------------- +// +// sessionId arrives as a first-class top-level field on the WS message (audit +// M1) — the daemon no longer injects it into params. We read it here so the +// per-tab mutex + lock layer can attribute the call; it never reaches a tool +// handler. (Previously it was smuggled through params.__sessionId, which +// coupled the multiplexer to the extension's wire format.) - currentActivity = tool; - activeTabId = tabId; - updateBadge('active'); - if (tabId) await showOverlay(tabId, tool.replace('browser_', '')); +async function handleMessage(msg) { + // Control messages (non-tool) from the daemon. These carry a `type` and no + // `tool`; handle them here before the tool-dispatch path assumes a tool call. + if (msg.type === 'releaseSession') { + // An agent disconnected — release any tabs it had locked, so a crashed/quit + // agent doesn't leave orphaned locks blocking other agents forever. Locks + // are keyed by agentName, so release by that (the daemon only sends this + // when NO surviving connection shares the agentName — see daemon close handler). + const owner = msg.agentName || msg.sessionId; + if (owner) { + const released = tabLocks.releaseByOwner(owner); + if (released.length) { + broadcastStatus(`Released ${released.length} lock(s) from disconnected agent ${owner}`); + } + } + return; // control message — no response expected + } + if (msg.type === 'ping') { + // already handled in onmessage, but be defensive + return; + } - try { - const result = await dispatch(tool, params || {}); - sendResponse(id, { success: true, result }); - } catch (err) { - sendResponse(id, { success: false, error: err.message || String(err) }); - } finally { - currentActivity = null; - updateBadge(isConnected ? 'connected' : 'disconnected'); - if (tabId) await hideOverlay(tabId); + const { id, tool, params } = msg; + const p = params || {}; + const sessionId = msg.sessionId || null; + // agentName is the STABLE identity for tab locks (survives MCP-client + // reconnects where sessionId churns s3→s4). Locks are keyed by agentName; + // sessionId is kept only for logging/UI. + const agentName = msg.agentName || null; + + const tabId = extractTabId(tool, p); + + // Tools without a tabId (tabs list/create, console-less) run directly. + if (tabId == null) { + try { + const result = await dispatch(tool, p, sessionId, agentName); + sendResponse(id, { success: true, result }); + } catch (err) { + sendResponse(id, { success: false, error: err.message || String(err) }); + } + return; } + + // Acquire this tab's mutex (2.1) and honor per-agent locks (2.2). Locks are + // keyed by agentName (stable across reconnects), so waitFor checks agentName. + runOnTabLib( + tabLocks, + tabMutex, + tabId, + agentName, + async () => { + currentActivity = tool; + updateBadge('active'); + await showOverlay(tabId, tool.replace('browser_', '')); + try { + const result = await dispatch(tool, p, sessionId, agentName); + sendResponse(id, { success: true, result }); + } catch (err) { + sendResponse(id, { success: false, error: err.message || String(err) }); + } finally { + currentActivity = null; + updateBadge(isConnected ? 'connected' : 'disconnected'); + await hideOverlay(tabId); + } + }, + ).catch((err) => { + sendResponse(id, { success: false, error: err.message || String(err) }); + }); } function sendResponse(id, response) { @@ -201,9 +396,23 @@ function sendResponse(id, response) { } } -// --- Tool Dispatch --- +/** Which tabId does this call target? null = tab-agnostic (tabs list/create). */ +function extractTabId(tool, params) { + if (typeof params.tabId === 'number') return params.tabId; + // navigate allows omitting tabId → falls back to active tab inside its handler. + if (tool === 'browser_navigate') return null; // handled in-handler + if (tool === 'browser_tabs') return null; // create/list don't target; close/focus read their own tabId + return null; +} + +// --- Per-tab mutex + tab lock: see lib/tab-concurrency.js (tasks 2.1, 2.2) - +// runOnTabLib(locks, mutex, tabId, sessionId, fn) serializes per-tab work and +// makes non-owner sessions wait for the lock owner. Tested in +// tests/tab-concurrency.test.ts. -async function dispatch(tool, params) { +// --- Tool Dispatch -------------------------------------------------------- + +async function dispatch(tool, params, sessionId, agentName) { const handlers = { browser_navigate: handleNavigate, browser_click: handleClick, @@ -225,67 +434,217 @@ async function dispatch(tool, params) { browser_run_action: handleRunAction, browser_drag: handleDrag, browser_fill_form: handleFillForm, - find: handleFind, + // Legacy wire-name aliases (find / get_page_text) removed: the tool files + // now send their canonical .name (browser_find / browser_text), so the + // aliases would only mask future drift. (audit C1) browser_find: handleFind, - get_page_text: handleGetPageText, browser_text: handleGetPageText, }; const handler = handlers[tool]; if (!handler) throw new Error(`Unknown tool: ${tool}`); - return handler(params); + return handler(params, sessionId, agentName); } -// --- Helpers --- +// --- Tab resolution (task 1.2) ------------------------------------------- -async function getActiveTab() { - const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); - if (!tab) throw new Error('No active tab'); - return tab; +/** + * Resolve a tab by id, throwing a clear, actionable error if it's gone. + * Replaces the old getActiveTab() which silently grabbed whatever tab was + * focused — the root cause of agents acting on the wrong page. + */ +async function resolveTab(tabId) { + if (tabId == null || typeof tabId !== 'number') { + throw new Error('tabId is required. Call browser_tabs list first to get a tabId.'); + } + try { + const tab = await chrome.tabs.get(tabId); + if (!tab) throw new Error(`Tab ${tabId} not found, call browser_tabs list first.`); + return tab; + } catch (err) { + throw new Error(`Tab ${tabId} not found, call browser_tabs list first. (${err.message || err})`); + } } -async function execInTab(tabId, func, args = []) { - const sanitized = args.map(a => a === undefined ? null : a); - const results = await chrome.scripting.executeScript({ target: { tabId }, func, args: sanitized }); - return results[0]?.result; +/** + * Get the stored smart-selector fallback for a (tabId, ref). Returns null if the + * ref was never snapshotted or the snapshot pre-dates the fallback feature. + */ +function getFallback(tabId, ref) { + if (tabId == null || !ref) return null; + const map = fallbackByTab.get(tabId); + if (!map) return null; + return map.get(ref) || null; } -// --- Tool Handlers --- +/** + * Auto-re-snapshot helper (Facebook/Instagram virtualization recovery). + * + * On virtualized sites (FB/IG/Twitter feeds), scrolling can REMOVE a post from + * the DOM entirely — so a stale `ref` + every in-page fallback all return null. + * Rather than forcing the agent to do a full round-trip (error → snapshot → + * retry), we snapshot the tab HERE and embed the fresh refs in the error so the + * agent can retry in one step using the new refs. + * + * Returns a compact summary (refs + names) suitable for an error payload — NOT + * the full tree (keeps it token-cheap). null if the re-snapshot itself failed. + */ +async function autoReSnapshot(tabId) { + try { + const res = await handleSnapshot({ tabId, compact: true }); + if (!res || !res.success || !res.tree) return null; + // Flatten ref → {role, name} so the agent can pick the right new ref. + const refs = []; + const walk = (n) => { + if (!n) return; + if (Array.isArray(n)) { n.forEach(walk); return; } + if (n.ref) refs.push({ ref: n.ref, role: n.role || '', name: (n.name || '').slice(0, 60) }); + if (n.children) walk(n.children); + }; + walk(res.tree); + return { refs: refs.slice(0, 40), url: res.url, title: res.title }; + } catch { + return null; + } +} -async function handleNavigate(params) { - const { url, waitUntil = 'load' } = params; - const tab = await getActiveTab(); +/** + * safeExec (task 2.5): run chrome.scripting.executeScript against a tab, + * turning "can't access chrome:// / webstore / devtools pages" into a clear + * error instead of a silent timeout. + */ +async function safeExec(tabId, func, args = []) { + const tab = await resolveTab(tabId); + if (/^(chrome|chrome-extension|devtools|edge|about):/i.test(tab.url || '')) { + throw new Error(`Cannot access protected page (${tab.url}). Tab ${tabId} is a browser-internal page.`); + } + const sanitized = args.map((a) => (a === undefined ? null : a)); + try { + const results = await chrome.scripting.executeScript({ target: { tabId }, func, args: sanitized }); + return results[0]?.result; + } catch (err) { + const msg = err?.message || String(err); + if (/cannot access|Cannot access|not allowed|No tab with id/i.test(msg)) { + throw new Error(`Cannot execute on tab ${tabId}: ${msg}`); + } + throw err; + } +} + +// --- Per-tab console/network storage (task 1.4) -------------------------- + +function getTabBuffer(map, tabId) { + let arr = map.get(tabId); + if (!arr) { + arr = []; + map.set(tabId, arr); + } + return arr; +} + +function pushCapped(arr, item, cap = PER_TAB_CAP) { + arr.push(item); + if (arr.length > cap) arr.splice(0, arr.length - cap); +} + +// --- Tool Handlers ------------------------------------------------------- - return new Promise((resolve, reject) => { - const listener = (tabId, changeInfo) => { - if (tabId !== tab.id) return; +/** Active-tab fallback, used ONLY by navigate when the caller omits tabId. */ +async function getActiveTab() { + const [tab] = await chrome.tabs.query({ active: true, currentWindow: true }); + if (!tab) throw new Error('No active tab and no tabId given. Call browser_tabs list first.'); + return tab; +} + +async function handleNavigate(params) { + const { url, waitUntil = 'load', tabId } = params; + // navigate is the one page tool allowed to omit tabId → active tab fallback. + const tab = tabId != null ? await resolveTab(tabId) : await getActiveTab(); + + // Fix #1 (SPA-ready): Chrome fires `complete` as soon as the HTML loads, but + // SPAs (React/Vue/...) render content via JS AFTER that — so a snapshot taken + // immediately returns an empty body. We (a) wait for `complete`, (b) give the + // SPA ~500ms to render, then (c) return a snapshot inline so the agent doesn't + // waste a separate browser_snapshot call on a still-empty page. On a protected + // page (chrome://, 401 Basic auth) the snapshot will surface a clear error + // instead of silently returning empty. + await new Promise((resolve, reject) => { + const listener = (tId, changeInfo) => { + if (tId !== tab.id) return; if (changeInfo.status === 'complete' || (waitUntil === 'domcontentloaded' && changeInfo.status === 'complete')) { chrome.tabs.onUpdated.removeListener(listener); - resolve({ url, status: 'navigated' }); + resolve(); } }; - chrome.tabs.onUpdated.addListener(listener); chrome.tabs.update(tab.id, { url }).catch(reject); - setTimeout(() => { chrome.tabs.onUpdated.removeListener(listener); - resolve({ url, status: 'timeout' }); + resolve(); // timeout — still proceed to settle + snapshot }, 55000); }); + + // SPA settle window. 500ms is a cheap, robust default that covers most + // client-rendered apps without making navigation feel slow for static pages. + await new Promise((r) => setTimeout(r, 500)); + + // Return the snapshot inline — saves the agent a step (no separate call). + try { + const snap = await handleSnapshot({ tabId: tab.id, compact: true }); + const snapObj = typeof snap === 'string' ? JSON.parse(snap) : snap; + return { + url, + status: 'navigated', + tabId: tab.id, + // The tool returns textResult(JSON.stringify(...)); inline the snapshot + // under `snapshot` so it's visible to the agent alongside nav status. + snapshot: snapObj && snapObj.content ? snapObj.content : snapObj, + }; + } catch { + // snapshot failed (protected page / 401 / etc) — navigation still succeeded. + return { url, status: 'navigated', tabId: tab.id }; + } } async function handleClick(params) { - const { ref, selector, button = 'left', doubleClick = false } = params; - const tab = await getActiveTab(); + const { tabId, ref, selector, button = 'left', doubleClick = false } = params; + await resolveTab(tabId); + const fb = getFallback(tabId, ref); + const resolveFallbackSrc = PAGE_RESOLVE_FALLBACK_FN.toString(); - return execInTab(tab.id, (_ref, _sel, _btn, _dbl) => { + return safeExec(tabId, async (_ref, _sel, _btn, _dbl, _fb, resolveFallbackSrc) => { let el = _ref ? document.querySelector(`[data-mcp-ref="${_ref}"]`) : null; - if (!el && _sel) el = document.querySelector(_sel); - if (!el) return { success: false, error: 'Element not found' }; + let via = 'ref'; + if (!el && _sel) { el = document.querySelector(_sel); via = 'selector'; } + // Rebuild the resolver from its source (chrome.scripting can't serialize fns). + let resolveFallback = null; + try { resolveFallback = eval('(' + resolveFallbackSrc + ')'); } catch {} + // Smart-selector fallback (plan task 3): ref broke → try robust selector, + // then text+role+tag scan. The agent doesn't request this; it's automatic. + if (!el && _fb && resolveFallback) { el = resolveFallback(_fb); if (el) via = 'fallback'; } + if (!el) { + // Element is gone (likely virtualized away on scroll). Abort WITHOUT + // clicking — the background auto-re-snapshots and embeds fresh refs. + return { success: false, error: 'REF_GONE', _ref }; + } el.scrollIntoView({ behavior: 'instant', block: 'center' }); + // Fix #2 (visibility retry): after scrollIntoView, the element may still be + // off-screen or zero-size if layout hasn't reflowed yet. Give it one short + // settle (200ms) and re-read the element once. This kills the common "element + // present but click landed nowhere" failure on lazy-rendered lists. Bounded + // to a single retry so a truly-hidden element still surfaces honestly. + const rect0 = el.getBoundingClientRect(); + const visible0 = rect0.width > 0 && rect0.height > 0; + if (!visible0) { + await new Promise((r) => setTimeout(r, 200)); + // re-resolve the element (it may have been re-rendered with a new node) + el = _ref ? document.querySelector(`[data-mcp-ref="${_ref}"]`) : el; + if (el) el.scrollIntoView({ behavior: 'instant', block: 'center' }); + } + if (!el) return { success: false, error: 'REF_GONE', _ref }; + const rect = el.getBoundingClientRect(); const x = rect.left + rect.width / 2; const y = rect.top + rect.height / 2; @@ -305,18 +664,43 @@ async function handleClick(params) { el.dispatchEvent(new MouseEvent('dblclick', init)); } - return { success: true }; - }, [ref, selector, button, doubleClick]); + return { success: true, ...(via !== 'ref' ? { via } : {}) }; + }, [ref, selector, button, doubleClick, fb, resolveFallbackSrc]); + + // The page function returns REF_GONE when the element (and all fallbacks) + // can't be found — typical of virtualized feeds (FB/IG) after scrolling. + // Auto-re-snapshot and embed fresh refs so the agent retries in ONE step. + // We do NOT auto-retry the click: it's non-idempotent and the element that + // re-appears may be a different post after the scroll shifted the feed. + if (res && res.success === false && res.error === 'REF_GONE') { + const fresh = await autoReSnapshot(tabId); + return { + success: false, + error: `Element ${res._ref || ref} is gone from the DOM (feed scrolled/virtualized). Fresh refs captured — retry with a new ref.`, + freshRefs: fresh, + }; + } + return res; } async function handleType(params) { - const { ref, selector, text, clear = false } = params; - const tab = await getActiveTab(); + const { tabId, ref, selector, text, clear = false } = params; + await resolveTab(tabId); + const fb = getFallback(tabId, ref); + const resolveFallbackSrc = PAGE_RESOLVE_FALLBACK_FN.toString(); - return execInTab(tab.id, (_ref, _sel, _text, _clear) => { + return safeExec(tabId, (_ref, _sel, _text, _clear, _fb, resolveFallbackSrc) => { let el = _ref ? document.querySelector(`[data-mcp-ref="${_ref}"]`) : null; - if (!el && _sel) el = document.querySelector(_sel); - if (!el) return { success: false, error: 'Element not found' }; + let via = 'ref'; + if (!el && _sel) { el = document.querySelector(_sel); via = 'selector'; } + let resolveFallback = null; + try { resolveFallback = eval('(' + resolveFallbackSrc + ')'); } catch {} + if (!el && _fb && resolveFallback) { el = resolveFallback(_fb); if (el) via = 'fallback'; } + if (!el) { + // Element gone (virtualized feed) — abort WITHOUT typing; background + // auto-re-snapshots and embeds fresh refs for a one-step retry. + return { success: false, error: 'REF_GONE', _ref }; + } el.focus(); @@ -341,15 +725,27 @@ async function handleType(params) { } el.dispatchEvent(new Event('change', { bubbles: true })); - return { success: true, typed: _text }; - }, [ref, selector, text, clear]); + return { success: true, typed: _text, ...(via !== 'ref' ? { via } : {}) }; + }, [ref, selector, text, clear, fb, resolveFallbackSrc]); + + // Virtualization recovery (same as click): type target is gone, so + // auto-re-snapshot and embed fresh refs. No auto-retry (non-idempotent). + if (res && res.success === false && res.error === 'REF_GONE') { + const fresh = await autoReSnapshot(tabId); + return { + success: false, + error: `Element ${res._ref || ref} is gone from the DOM (feed scrolled/virtualized). Fresh refs captured — retry with a new ref.`, + freshRefs: fresh, + }; + } + return res; } async function handleScroll(params) { - const { direction = 'down', amount = 500, selector, toElement, position } = params; - const tab = await getActiveTab(); + const { tabId, direction = 'down', amount = 500, selector, toElement, position } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_dir, _amt, _sel, _toEl, _pos) => { + return safeExec(tabId, (_dir, _amt, _sel, _toEl, _pos) => { if (_toEl) { const el = document.querySelector(`[data-mcp-ref="${_toEl}"]`) || document.querySelector(_toEl); if (el) { @@ -383,14 +779,21 @@ async function handleScroll(params) { else target.scrollBy(scrollOpts); return { success: true, direction: _dir, amount: _amt }; - }, [direction, amount, selector, toElement, position]); + }, [direction, amount, selector, toElement, position]).then((res) => { + // Scrolling a virtualized feed (FB/IG/Twitter) recycles DOM nodes, so any + // refs the agent holds are now likely stale. Hint it to re-snapshot. We + // don't auto-snapshot here (every scroll would be expensive); the hint is + // enough for a well-behaved agent to snapshot before its next interaction. + if (res && res.success) res.refsMayBeStale = true; + return res; + }); } async function handlePressKey(params) { - const { key, modifiers = [], ref, selector } = params; - const tab = await getActiveTab(); + const { tabId, key, modifiers = [], ref, selector } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_key, _mods, _ref, _sel) => { + return safeExec(tabId, (_key, _mods, _ref, _sel) => { let target = document.activeElement || document.body; if (_ref) { const el = document.querySelector(`[data-mcp-ref="${_ref}"]`); @@ -420,20 +823,19 @@ async function handlePressKey(params) { } async function handleWait(params) { - const { selector, state = 'visible', timeout = 10000, delay } = params; + const { tabId, selector, state = 'visible', timeout = 10000, delay } = params; if (delay) { - await new Promise(r => setTimeout(r, Math.min(delay, 30000))); + await new Promise((r) => setTimeout(r, Math.min(delay, 30000))); return { success: true, waited: delay }; } if (!selector) return { success: false, error: 'Need selector or delay' }; - - const tab = await getActiveTab(); + await resolveTab(tabId); const start = Date.now(); while (Date.now() - start < timeout) { - const found = await execInTab(tab.id, (_sel, _state) => { + const found = await safeExec(tabId, (_sel, _state) => { const el = document.querySelector(_sel); if (_state === 'hidden') return !el || el.offsetParent === null; if (_state === 'attached') return !!el; @@ -441,17 +843,17 @@ async function handleWait(params) { }, [selector, state]); if (found) return { success: true, selector, state, elapsed: Date.now() - start }; - await new Promise(r => setTimeout(r, 200)); + await new Promise((r) => setTimeout(r, 200)); } return { success: false, error: `Timeout waiting for ${selector} to be ${state}` }; } async function handleHover(params) { - const { ref, selector } = params; - const tab = await getActiveTab(); + const { tabId, ref, selector } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_ref, _sel) => { + return safeExec(tabId, (_ref, _sel) => { let el = _ref ? document.querySelector(`[data-mcp-ref="${_ref}"]`) : null; if (!el && _sel) el = document.querySelector(_sel); if (!el) return { success: false, error: 'Element not found' }; @@ -471,10 +873,10 @@ async function handleHover(params) { } async function handleSelect(params) { - const { ref, selector, value, label, index } = params; - const tab = await getActiveTab(); + const { tabId, ref, selector, value, label, index } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_ref, _sel, _val, _lbl, _idx) => { + return safeExec(tabId, (_ref, _sel, _val, _lbl, _idx) => { let el = _ref ? document.querySelector(`[data-mcp-ref="${_ref}"]`) : null; if (!el && _sel) el = document.querySelector(_sel); if (!el) return { success: false, error: 'Element not found' }; @@ -482,7 +884,7 @@ async function handleSelect(params) { if (_val !== null) el.value = _val; else if (_lbl !== null) { - const opt = Array.from(el.options).find(o => o.textContent.trim() === _lbl); + const opt = Array.from(el.options).find((o) => o.textContent.trim() === _lbl); if (opt) el.value = opt.value; else return { success: false, error: `Option "${_lbl}" not found` }; } else if (_idx !== null) { @@ -496,12 +898,33 @@ async function handleSelect(params) { }, [ref, selector, value, label, index]); } +/** + * Snapshot (task 2.4): builds an accessibility tree INCLUDING shadow DOM and + * same-origin iframes. Refs are stamped via data-mcp-ref and are valid only for + * the tab that produced them (enforced by resolveTab in the consuming tools). + */ async function handleSnapshot(params) { - const { selector, compact = true } = params; - const tab = await getActiveTab(); - - return execInTab(tab.id, (_sel, _compact) => { + const { tabId, selector, compact = true } = params; + await resolveTab(tabId); + + // chrome.scripting cannot serialize functions across the service worker + // boundary, so pass the fallback generator as its SOURCE STRING and eval it + // in the page to rebuild the live function. + const genFallbackSrc = PAGE_FALLBACK_FN.toString(); + // isNew feature: pass the fingerprints seen in the PREVIOUS snapshot so the + // page function can mark newly-appeared elements. Array is serializable. + const prevFingerprints = lastSnapshotFingerprints.get(tabId) || []; + + return safeExec(tabId, (_sel, _compact, genFallbackSrc, _prevFingerprints) => { let refCount = 0; + /** @type {Record} ref -> fallback, returned to background */ + const fallbacks = {}; + /** @type {string[]} fingerprints of THIS snapshot (role|name), returned to background */ + const fingerprints = []; + const prevSet = new Set(_prevFingerprints); + // Rebuild the live function from its source string (see comment at call site). + let genFallback = null; + try { genFallback = eval('(' + genFallbackSrc + ')'); } catch {} const skipTags = new Set(['SCRIPT', 'STYLE', 'NOSCRIPT', 'TEMPLATE', 'SVG', 'PATH', 'BR', 'HR', 'WBR', 'META', 'LINK']); function vis(el) { @@ -515,10 +938,10 @@ async function handleSnapshot(params) { const r = el.getAttribute('role'); if (r) return r; const map = { - A:'link',BUTTON:'button',SELECT:'combobox',TEXTAREA:'textbox',IMG:'img', - H1:'heading',H2:'heading',H3:'heading',H4:'heading',H5:'heading',H6:'heading', - NAV:'navigation',MAIN:'main',HEADER:'banner',FOOTER:'contentinfo',FORM:'form', - TABLE:'table',UL:'list',OL:'list',LI:'listitem', + A: 'link', BUTTON: 'button', SELECT: 'combobox', TEXTAREA: 'textbox', IMG: 'img', + H1: 'heading', H2: 'heading', H3: 'heading', H4: 'heading', H5: 'heading', H6: 'heading', + NAV: 'navigation', MAIN: 'main', HEADER: 'banner', FOOTER: 'contentinfo', FORM: 'form', + TABLE: 'table', UL: 'list', OL: 'list', LI: 'listitem', }; if (el.tagName === 'INPUT') { const t = el.type?.toLowerCase(); @@ -543,7 +966,7 @@ async function handleSnapshot(params) { } function isInteractive(el) { - const tags = ['A','BUTTON','INPUT','SELECT','TEXTAREA']; + const tags = ['A', 'BUTTON', 'INPUT', 'SELECT', 'TEXTAREA']; return tags.includes(el.tagName) || el.onclick || el.getAttribute('tabindex') !== null || el.getAttribute('role') === 'button' || el.getAttribute('role') === 'link' || el.getAttribute('role') === 'tab' || el.getAttribute('role') === 'menuitem' || @@ -551,7 +974,24 @@ async function handleSnapshot(params) { el.getAttribute('contenteditable') === 'true'; } - const landmarkRoles = new Set(['navigation','main','banner','contentinfo','form','search','complementary','region']); + const landmarkRoles = new Set(['navigation', 'main', 'banner', 'contentinfo', 'form', 'search', 'complementary', 'region']); + + // Children including shadow DOM (open roots) and same-origin iframes. + function childrenOf(el) { + const out = []; + for (const c of el.children) out.push(c); + if (el.shadowRoot) { + for (const c of el.shadowRoot.children) out.push(c); + } + // same-origin iframes: expose their document body children too. + if (el.tagName === 'IFRAME') { + try { + const doc = el.contentDocument; + if (doc && doc.body) for (const c of doc.body.children) out.push(c); + } catch { /* cross-origin: skip */ } + } + return out; + } function buildCompact(el) { if (!el || el.nodeType !== 1) return null; @@ -563,7 +1003,7 @@ async function handleSnapshot(params) { const isLandmark = landmarkRoles.has(r); const kids = []; - for (const c of el.children) { + for (const c of childrenOf(el)) { const cn = buildCompact(c); if (cn) Array.isArray(cn) ? kids.push(...cn) : kids.push(cn); } @@ -575,9 +1015,16 @@ async function handleSnapshot(params) { const ref = `e${refCount++}`; el.setAttribute('data-mcp-ref', ref); const n = elName(el); + try { if (genFallback) fallbacks[ref] = genFallback(el); } catch {} + + // isNew: mark elements whose (role|name) wasn't in the previous snapshot. + const fp = `${r}|${n}`; + fingerprints.push(fp); + const isNew = !prevSet.has(fp); const node = { ref, role: r }; if (n) node.name = n; + if (isNew) node.isNew = true; if (el.value !== undefined && el.value !== '') node.value = String(el.value); if (el.checked !== undefined) node.checked = el.checked; if (el.disabled) node.disabled = true; @@ -598,7 +1045,7 @@ async function handleSnapshot(params) { if (r === 'generic' && !n && !ia && depth > 1) { const kids = []; - for (const c of el.children) { + for (const c of childrenOf(el)) { const cn = buildFull(c, depth + 1); if (cn) Array.isArray(cn) ? kids.push(...cn) : kids.push(cn); } @@ -607,17 +1054,24 @@ async function handleSnapshot(params) { const ref = `e${refCount++}`; el.setAttribute('data-mcp-ref', ref); + try { if (genFallback) fallbacks[ref] = genFallback(el); } catch {} + + // isNew: mark elements whose (role|name) wasn't in the previous snapshot. + const fp = `${r}|${n}`; + fingerprints.push(fp); + const isNew = !prevSet.has(fp); const node = { ref, role: r }; if (r === 'generic') node.tag = el.tagName.toLowerCase(); if (n) node.name = n; + if (isNew) node.isNew = true; if (el.value !== undefined && el.value !== '') node.value = String(el.value); if (el.checked !== undefined) node.checked = el.checked; if (el.disabled) node.disabled = true; if (el.href && el.tagName === 'A') node.href = el.href; const kids = []; - for (const c of el.children) { + for (const c of childrenOf(el)) { const cn = buildFull(c, depth + 1); if (cn) Array.isArray(cn) ? kids.push(...cn) : kids.push(cn); } @@ -636,13 +1090,35 @@ async function handleSnapshot(params) { title: document.title, compact: _compact, tree, + // internal: background stores these per-tab; never sent to the agent. + __fallbacks: fallbacks, + __fingerprints: fingerprints, }; - }, [selector, compact]); + }, [selector, compact, genFallbackSrc, prevFingerprints]).then((res) => { + // Store the fallbacks per-tab so click/type can resolve stale refs. + if (res && res.__fallbacks) { + const map = new Map(Object.entries(res.__fallbacks)); + fallbackByTab.set(tabId, map); + delete res.__fallbacks; // keep it out of the agent-visible payload + } + // Store THIS snapshot's fingerprints so the next snapshot can compute isNew. + if (res && res.__fingerprints) { + lastSnapshotFingerprints.set(tabId, res.__fingerprints); + delete res.__fingerprints; + } + return res; + }); } async function handleScreenshot(params) { - const { format = 'png', quality = 80 } = params; - const tab = await getActiveTab(); + const { tabId, format = 'png', quality = 80 } = params; + const tab = await resolveTab(tabId); + // captureVisibleTab is window-scoped: ensure the tab is the active one in its + // window first (a no-op if it already is). + if (!tab.active) { + await chrome.tabs.update(tabId, { active: true }).catch(() => {}); + await new Promise((r) => setTimeout(r, 150)); // let the paint settle + } const dataUrl = await chrome.tabs.captureVisibleTab(tab.windowId, { format, quality: format === 'jpeg' ? quality : undefined, @@ -651,29 +1127,39 @@ async function handleScreenshot(params) { } async function handleConsole(params) { - const { clear = false } = params; - const msgs = [...consoleMessages]; - if (clear) consoleMessages = []; + const { tabId, clear = false } = params; + const buf = getTabBuffer(consoleByTab, tabId); + const msgs = [...buf]; + if (clear) consoleByTab.set(tabId, []); return { success: true, messages: msgs }; } async function handleNetwork(params) { - const { filter, clear = false } = params; - let reqs = [...networkRequests]; + const { tabId, filter, clear = false } = params; + let reqs = [...getTabBuffer(networkByTab, tabId)]; if (filter) { const re = new RegExp(filter); - reqs = reqs.filter(r => re.test(r.url)); + reqs = reqs.filter((r) => re.test(r.url)); } - if (clear) networkRequests = []; + if (clear) networkByTab.set(tabId, []); return { success: true, requests: reqs }; } -async function handleTabs(params) { +async function handleTabs(params, sessionId, agentName) { const { action, tabId, url } = params; switch (action) { case 'list': { const tabs = await chrome.tabs.query({ currentWindow: true }); - return { success: true, tabs: tabs.map(t => ({ id: t.id, url: t.url, title: t.title, active: t.active })) }; + return { + success: true, + tabs: tabs.map((t) => ({ + id: t.id, + url: t.url, + title: t.title, + active: t.active, + lockedBy: tabLocks.owner(t.id) || null, + })), + }; } case 'create': { const t = await chrome.tabs.create({ url: url || 'about:blank' }); @@ -682,6 +1168,7 @@ async function handleTabs(params) { case 'close': { if (!tabId) throw new Error('tabId required'); await chrome.tabs.remove(tabId); + tabLocks.release(tabId); return { success: true, closed: tabId }; } case 'focus': { @@ -689,15 +1176,33 @@ async function handleTabs(params) { await chrome.tabs.update(tabId, { active: true }); return { success: true, focused: tabId }; } - default: throw new Error(`Unknown action: ${action}`); + case 'lock': { + if (!tabId) throw new Error('tabId required'); + // Lock by agentName (stable across reconnects) when available; fall back + // to sessionId for the popup path (which only knows sessionId from /status). + const owner = agentName || sessionId; + if (!owner) throw new Error('lock requires a session (called outside daemon?)'); + tabLocks.lock(tabId, owner); + broadcastStatus(`Tab ${tabId} locked by ${owner}`); + return { success: true, locked: tabId, owner }; + } + case 'unlock': { + if (!tabId) throw new Error('tabId required'); + const was = tabLocks.owner(tabId); + tabLocks.release(tabId); + broadcastStatus(`Tab ${tabId} unlocked (was ${was || '-'})`); + return { success: true, unlocked: tabId, previousSession: was || null }; + } + default: + throw new Error(`Unknown action: ${action}`); } } async function handleFind(params) { - const { query, limit = 10 } = params; - const tab = await getActiveTab(); + const { tabId, query, limit = 10 } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_q, _lim) => { + return safeExec(tabId, (_q, _lim) => { const qLow = _q.toLowerCase(); const matches = []; @@ -709,7 +1214,7 @@ async function handleFind(params) { function aRole(el) { const r = el.getAttribute('role'); if (r) return r; - const map = { A:'link', BUTTON:'button', INPUT:'input', SELECT:'combobox', TEXTAREA:'textbox', IMG:'image' }; + const map = { A: 'link', BUTTON: 'button', INPUT: 'input', SELECT: 'combobox', TEXTAREA: 'textbox', IMG: 'image' }; return map[el.tagName] || el.tagName.toLowerCase(); } @@ -732,8 +1237,9 @@ async function handleFind(params) { const ref = `f${rc++}`; node.setAttribute('data-mcp-ref', ref); - matches.push({ ref, role: r, name: n.slice(0, 100), tag: node.tagName.toLowerCase(), score, - bounds: { x: Math.round(rect.x), y: Math.round(rect.y), width: Math.round(rect.width), height: Math.round(rect.height) } + matches.push({ + ref, role: r, name: n.slice(0, 100), tag: node.tagName.toLowerCase(), score, + bounds: { x: Math.round(rect.x), y: Math.round(rect.y), width: Math.round(rect.width), height: Math.round(rect.height) }, }); } @@ -743,11 +1249,11 @@ async function handleFind(params) { } async function handleGetPageText(params) { - const { selector, maxLength = 50000 } = params; - const tab = await getActiveTab(); + const { tabId, selector, maxLength = 50000 } = params; + await resolveTab(tabId); const args = selector === undefined ? [null, maxLength] : [selector, maxLength]; - return execInTab(tab.id, (_sel, _max) => { + return safeExec(tabId, (_sel, _max) => { const root = _sel ? document.querySelector(_sel) : document.body; if (!root) return { success: false, error: 'Element not found' }; @@ -760,31 +1266,40 @@ async function handleGetPageText(params) { }, args); } +/** + * evaluate (task 1.5): runs in the page's MAIN world via chrome.scripting — no + * chrome.debugger, so no yellow "is being debugged" banner. Replaces the old + * CDP Runtime.evaluate path. + */ async function handleEvaluate(params) { - const { expression } = params; - const tab = await getActiveTab(); - - await chrome.debugger.attach({ tabId: tab.id }, '1.3'); - try { - const { result, exceptionDetails } = await chrome.debugger.sendCommand( - { tabId: tab.id }, - 'Runtime.evaluate', - { expression: `(async () => { ${expression} })()`, awaitPromise: true, returnByValue: true }, - ); - if (exceptionDetails) { - return { success: false, error: exceptionDetails.exception?.description || exceptionDetails.text }; - } - return { success: true, result: result.value }; - } finally { - try { await chrome.debugger.detach({ tabId: tab.id }); } catch {} + const { tabId, expression } = params; + await resolveTab(tabId); + const tab = await chrome.tabs.get(tabId); + if (/^(chrome|chrome-extension|devtools|edge|about):/i.test(tab.url || '')) { + throw new Error(`Cannot evaluate on protected page (${tab.url}).`); } + + // Wrap the user expression in an async IIFE so `await` works, then stringify + // the function and run it in the MAIN world (page's own JS context). + const wrapped = `(async () => { ${expression} })()`; + const results = await chrome.scripting.executeScript({ + target: { tabId }, + world: 'MAIN', + func: async (code) => { + // eslint-disable-next-line no-eval + return await eval(code); + }, + args: [wrapped], + }); + const value = results?.[0]?.result; + return { success: true, result: value }; } async function handleClickByText(params) { - const { text, index = 0, exact = false } = params; - const tab = await getActiveTab(); + const { tabId, text, index = 0, exact = false } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_text, _index, _exact) => { + return safeExec(tabId, (_text, _index, _exact) => { const textLower = _text.toLowerCase(); const candidates = []; const walker = document.createTreeWalker(document.body, NodeFilter.SHOW_ELEMENT); @@ -806,7 +1321,7 @@ async function handleClickByText(params) { } } - function getDepth(el) { let d = 0; let p = el; while (p = p.parentElement) d++; return d; } + function getDepth(el) { let d = 0; let p = el; while ((p = p.parentElement)) d++; return d; } candidates.sort((a, b) => b.depth - a.depth); @@ -831,10 +1346,10 @@ async function handleClickByText(params) { } async function handleDialog(params) { - const { action = 'accept', promptText } = params; - const tab = await getActiveTab(); + const { tabId, action = 'accept', promptText } = params; + await resolveTab(tabId); - return execInTab(tab.id, (_action, _promptText) => { + return safeExec(tabId, (_action, _promptText) => { window.__mcpDialogLog = window.__mcpDialogLog || []; window.__mcpDialogAction = _action; window.__mcpDialogPromptText = _promptText || ''; @@ -843,53 +1358,35 @@ async function handleDialog(params) { window.__mcpDialogOverrides = true; window.alert = function (msg) { - window.__mcpDialogLog.push({ - type: 'alert', - message: String(msg), - timestamp: Date.now(), - handled: window.__mcpDialogAction, - }); + window.__mcpDialogLog.push({ type: 'alert', message: String(msg), timestamp: Date.now(), handled: window.__mcpDialogAction }); }; window.confirm = function (msg) { const accepted = window.__mcpDialogAction === 'accept'; - window.__mcpDialogLog.push({ - type: 'confirm', - message: String(msg), - timestamp: Date.now(), - result: accepted, - }); + window.__mcpDialogLog.push({ type: 'confirm', message: String(msg), timestamp: Date.now(), result: accepted }); return accepted; }; window.prompt = function (msg, def) { const accepted = window.__mcpDialogAction === 'accept'; const text = accepted ? (window.__mcpDialogPromptText || def || '') : null; - window.__mcpDialogLog.push({ - type: 'prompt', - message: String(msg), - timestamp: Date.now(), - result: text, - }); + window.__mcpDialogLog.push({ type: 'prompt', message: String(msg), timestamp: Date.now(), result: text }); return accepted ? text : null; }; } const log = [...window.__mcpDialogLog]; window.__mcpDialogLog = []; - return { - success: true, - dialogs: log, - message: log.length ? 'Retrieved dialog history' : 'Overrides configured', - }; + return { success: true, dialogs: log, message: log.length ? 'Retrieved dialog history' : 'Overrides configured' }; }, [action, promptText]); } async function handleRunAction(params) { - const { code, actionParams = {} } = params; + const { tabId, code, actionParams = {} } = params; if (!code) throw new Error('code is required'); - const tab = await getActiveTab(); + const tab = await resolveTab(tabId); + // run_action stays on CDP (plan decision: CDP-only, can't be scripted). await chrome.debugger.attach({ tabId: tab.id }, '1.3'); try { const paramsJson = JSON.stringify(actionParams); @@ -911,11 +1408,12 @@ async function handleRunAction(params) { } async function handleUploadFile(params) { - const { ref, selector, filePath, files: fileList } = params; - const tab = await getActiveTab(); + const { tabId, ref, selector, filePath, files: fileList } = params; + const tab = await resolveTab(tabId); const filePaths = fileList || (filePath ? [filePath] : []); if (filePaths.length === 0) throw new Error('filePath or files required'); + // upload_file stays on CDP (DOM.setFileInputFiles is CDP-only). await chrome.debugger.attach({ tabId: tab.id }, '1.3'); try { await chrome.debugger.sendCommand({ tabId: tab.id }, 'DOM.enable', {}); @@ -944,13 +1442,13 @@ async function handleUploadFile(params) { } async function handleDrag(params) { - const { startRef, startSelector, endRef, endSelector, startX, startY, endX, endY, steps = 10 } = params; - const tab = await getActiveTab(); + const { tabId, startRef, startSelector, endRef, endSelector, startX, startY, endX, endY, steps = 10 } = params; + const tab = await resolveTab(tabId); let sx = startX, sy = startY, ex = endX, ey = endY; if (sx == null || sy == null || ex == null || ey == null) { - const coords = await execInTab(tab.id, (_sRef, _sSel, _eRef, _eSel) => { + const coords = await safeExec(tabId, (_sRef, _sSel, _eRef, _eSel) => { function find(ref, sel) { let el = ref ? document.querySelector(`[data-mcp-ref="${ref}"]`) : null; if (!el && sel) el = document.querySelector(sel); @@ -959,10 +1457,7 @@ async function handleDrag(params) { const r = el.getBoundingClientRect(); return { x: r.left + r.width / 2, y: r.top + r.height / 2 }; } - return { - start: find(_sRef, _sSel), - end: find(_eRef, _eSel), - }; + return { start: find(_sRef, _sSel), end: find(_eRef, _eSel) }; }, [startRef, startSelector, endRef, endSelector]); if (coords.start) { sx = coords.start.x; sy = coords.start.y; } @@ -973,6 +1468,7 @@ async function handleDrag(params) { throw new Error('Could not determine drag coordinates. Provide refs/selectors or explicit x,y coordinates.'); } + // drag stays on CDP (Input.dispatchMouseEvent is CDP-only). await chrome.debugger.attach({ tabId: tab.id }, '1.3'); try { await chrome.debugger.sendCommand({ tabId: tab.id }, 'Input.dispatchMouseEvent', { @@ -1000,13 +1496,13 @@ async function handleDrag(params) { } async function handleFillForm(params) { - const { fields, submit } = params; + const { tabId, fields, submit } = params; if (!fields || !Array.isArray(fields) || fields.length === 0) { throw new Error('fields array is required'); } - const tab = await getActiveTab(); + await resolveTab(tabId); - return execInTab(tab.id, (_fields, _submit) => { + return safeExec(tabId, (_fields, _submit) => { const results = []; for (const field of _fields) { const { ref, selector, value, clear } = field; @@ -1057,14 +1553,29 @@ async function handleFillForm(params) { }, [fields, submit]); } -// --- Events --- +// --- Events (per-tab console/network capture: task 1.4) ------------------ chrome.runtime.onMessage.addListener((msg, sender, respond) => { - if (msg.type === 'console') { - consoleMessages.push({ level: msg.level, text: msg.text, timestamp: Date.now(), url: sender.tab?.url }); - } else if (msg.type === 'getStatus') { - respond(buildStatusPayload()); - } else if (msg.type === 'setPort') { + // NOTE: do NOT `return true` unconditionally. Returning true tells Chrome the + // listener will call `respond()` ASYNCHRONOUSLY; if the sender (popup) closes + // before that, Chrome logs "message channel closed before a response was + // received". Every branch below responds synchronously (or is fire-and-forget), + // so we return false (or nothing) — Chrome handles it without the warning. + if (msg.type === 'console' && sender.tab?.id != null) { + const buf = getTabBuffer(consoleByTab, sender.tab.id); + pushCapped(buf, { level: msg.level, text: msg.text, timestamp: Date.now(), url: sender.tab.url }); + return false; // fire-and-forget; no response expected + } + if (msg.type === 'getStatus') { + // Async: fetch tabs before responding so the popup gets a full snapshot + // (connection + locks + open tabs) in one message. Returning true signals + // Chrome we'll call respond() asynchronously. + getOpenTabs().then((tabs) => { + respond(buildStatusPayload(undefined, tabs)); + }); + return true; // async response + } + if (msg.type === 'setPort') { const p = parseInt(msg.port, 10); if (p > 0 && p < 65536) { wsPort = p; @@ -1078,19 +1589,77 @@ chrome.runtime.onMessage.addListener((msg, sender, respond) => { } else { respond({ success: false, error: 'Invalid port' }); } + return false; + } + if (msg.type === 'setToken') { + // extension popup can store the auth token (3.1) once. + wsToken = (msg.token || '').trim(); + chrome.storage.local.set({ wsToken }); + ws?.close(); + ws = null; + isConnected = false; + reconnectAttempts = 0; + connect(); + respond({ success: true }); + return false; + } + if (msg.type === 'unlockAll') { + tabLocks.unlockAll(); + broadcastStatus('All tab locks cleared'); + respond({ success: true }); + return false; + } + if (msg.type === 'lockTab') { + // { tabId, agentName?, sessionId? } — pin a tab to an agent from the popup. + // Prefer agentName (stable across reconnects); fall back to sessionId for + // backward compat. Same primitive as the browser_tabs lock tool. + const owner = msg.agentName || msg.sessionId; + if (msg.tabId == null || !owner) { + respond({ success: false, error: 'tabId and agentName/sessionId required' }); + return false; + } + tabLocks.lock(msg.tabId, owner); + broadcastStatus(`Tab ${msg.tabId} pinned to ${owner}`); + respond({ success: true }); + return false; } - return true; + if (msg.type === 'unlockTab') { + // { tabId } — release one tab's lock (vs unlockAll which clears all). + if (msg.tabId == null) { + respond({ success: false, error: 'tabId required' }); + return false; + } + const was = tabLocks.owner(msg.tabId); + tabLocks.release(msg.tabId); + broadcastStatus(`Tab ${msg.tabId} unpinned (was ${was || '-'})`); + respond({ success: true, previousSession: was || null }); + return false; + } + return false; // unrecognized message — no async response promised }); chrome.webRequest.onCompleted.addListener( (details) => { - networkRequests.push({ - method: details.method, url: details.url, - status: details.statusCode, type: details.type, timestamp: details.timeStamp, + if (details.tabId == null || details.tabId < 0) return; // not a real tab + const buf = getTabBuffer(networkByTab, details.tabId); + pushCapped(buf, { + method: details.method, + url: details.url, + status: details.statusCode, + type: details.type, + timestamp: details.timeStamp, }); - if (networkRequests.length > 200) networkRequests = networkRequests.slice(-200); }, { urls: [''] }, ); +// A tab closing should release its lock and drop its buffers. +chrome.tabs.onRemoved.addListener((tabId) => { + tabLocks.release(tabId); + consoleByTab.delete(tabId); + networkByTab.delete(tabId); + fallbackByTab.delete(tabId); + lastSnapshotFingerprints.delete(tabId); +}); + initConnection(); diff --git a/extension/content.js b/extension/content.js index de17714..6412c32 100644 --- a/extension/content.js +++ b/extension/content.js @@ -1,7 +1,7 @@ (function () { 'use strict'; - if (window.__realBrowserMcpInjected) return; - window.__realBrowserMcpInjected = true; + if (window.__browserControllerInjected) return; + window.__browserControllerInjected = true; const orig = { log: console.log, @@ -17,6 +17,13 @@ if (typeof a === 'object') { try { return JSON.stringify(a); } catch { return String(a); } } return String(a); }).join(' '); + // Drop well-known benign warnings so they don't spam the captured log: + // - ResizeObserver loop (element resized during its own RO callback) + // - "message channel closed before a response was received" — happens on + // heavy SPA pages (Meta.ai etc.) when an async onMessage sender closes + // before the listener replies. Harmless; nothing our code can do about it. + if (text.indexOf('ResizeObserver loop') !== -1) return; + if (text.indexOf('message channel closed before a response was received') !== -1) return; try { chrome.runtime.sendMessage({ type: 'console', level, text }); } catch {} } @@ -27,6 +34,10 @@ console.debug = (...a) => capture('debug', ...a); window.addEventListener('error', (e) => { + // Silence the well-known ResizeObserver loop warning: it's a benign browser + // notice (element resized during its own observation callback), not a real + // error. Every RO-based UI triggers it; capturing it just spams the console. + if (e && typeof e.message === 'string' && e.message.indexOf('ResizeObserver loop') !== -1) return; capture('error', `Uncaught: ${e.message} at ${e.filename}:${e.lineno}`); }); diff --git a/extension/icons/icon128.png b/extension/icons/icon128.png index 9b5e7d9..e8ca8a7 100644 Binary files a/extension/icons/icon128.png and b/extension/icons/icon128.png differ diff --git a/extension/icons/icon16.png b/extension/icons/icon16.png index b16c484..6049cef 100644 Binary files a/extension/icons/icon16.png and b/extension/icons/icon16.png differ diff --git a/extension/icons/icon48.png b/extension/icons/icon48.png index 7b31949..d198142 100644 Binary files a/extension/icons/icon48.png and b/extension/icons/icon48.png differ diff --git a/extension/lib/tab-concurrency.js b/extension/lib/tab-concurrency.js new file mode 100644 index 0000000..d811235 --- /dev/null +++ b/extension/lib/tab-concurrency.js @@ -0,0 +1,152 @@ +/** + * Pure tab-concurrency primitives (plan tasks 2.1 + 2.2), framework-free so they + * can be unit-tested with vitest WITHOUT a Chrome instance. + * + * - {@link TabMutexMap}: one promise-chain per tabId. Calls to the SAME tab + * serialize; calls to DIFFERENT tabs run in parallel. (2.1) + * - {@link TabLockMap}: per-agent ownership of a tab. A call from a non-owner + * session waits (queues) until the owner releases. (2.2) + * + * background.js wires these together via runOnTab(); the vitest suite in + * tests/tab-concurrency.test.ts proves the ordering guarantees. + */ + +export class TabMutexMap { + constructor() { + /** @type {Map>} */ + this.queues = new Map(); + } + + /** + * Run `fn` serialized for `tabId`: at most one fn per tab runs at a time. + * Different tabIds run concurrently. Resolves/rejects with fn's result. + * @param {number} tabId + * @param {() => (Promise | unknown)} fn + * @returns {Promise} + */ + run(tabId, fn) { + const prev = this.queues.get(tabId) || Promise.resolve(); + const next = prev.then(fn, fn); + // Swallow rejection on the stored chain so a later call isn't poisoned, + // but the returned `next` still surfaces the real error to the caller. + const settled = next.then( + () => {}, + () => {}, + ); + this.queues.set(tabId, settled); + // Leak fix: once this chain settles, drop the entry IF it's still ours. + // A later run() may have chained onto `settled` and replaced the map value — + // in that case `settled !== current`, and deleting would break the newer + // chain's serialization. The identity check makes this safe. Without this, + // every tab's first run() left a permanent entry → memory leak + isIdle() + // returned false forever after the first call. + settled.then(() => { + if (this.queues.get(tabId) === settled) { + this.queues.delete(tabId); + } + }); + return next; + } + + /** For tests: whether anything is currently pending for a tab. */ + isIdle(tabId) { + return !this.queues.has(tabId); + } +} + +export class TabLockMap { + constructor() { + /** @type {Map} tabId -> sessionId */ + this.locks = new Map(); + /** lock-wait timeout so a forgotten lock can't hang a caller forever */ + this.acquireTimeoutMs = 30_000; + } + + /** Who owns tabId, or undefined if unlocked. */ + owner(tabId) { + return this.locks.get(tabId); + } + + /** Claim tabId for sessionId. */ + lock(tabId, sessionId) { + this.locks.set(tabId, sessionId); + } + + /** Release tabId (only if currently owned by sessionId). */ + unlock(tabId, sessionId) { + if (sessionId == null || this.locks.get(tabId) === sessionId) { + this.locks.delete(tabId); + } + } + + /** Release tabId unconditionally (used when the tab is closed). */ + release(tabId) { + this.locks.delete(tabId); + } + + /** + * Release every tab locked to `sessionId`. Called when an agent disconnects + * (daemon close handler → bridge → extension) so a crashed/quit agent doesn't + * leave orphaned locks that block other agents forever. Returns the tabIds + * that were released (empty if the session owned nothing). + * @param {string} sessionId + * @returns {number[]} + */ + releaseByOwner(sessionId) { + const released = []; + for (const [tabId, owner] of this.locks) { + if (owner === sessionId) { + this.locks.delete(tabId); + released.push(tabId); + } + } + return released; + } + + /** @returns {Array<{tabId:number,sessionId:string}>} serializable snapshot. */ + snapshot() { + return Array.from(this.locks.entries()).map(([tabId, sessionId]) => ({ tabId, sessionId })); + } + + /** Release all locks. */ + unlockAll() { + this.locks.clear(); + } + + /** + * Wait until tabId is unlocked OR owned by sessionId (i.e. this session may + * proceed). Resolves immediately if already eligible; otherwise polls. + * @returns {Promise} + */ + async waitFor(tabId, sessionId) { + if (sessionId == null) return; // no session → no locking semantics + const owner = this.locks.get(tabId); + if (owner === undefined || owner === sessionId) return; + + const start = Date.now(); + return new Promise((resolve) => { + const tick = () => { + const cur = this.locks.get(tabId); + if (cur === undefined || cur === sessionId || Date.now() - start > this.acquireTimeoutMs) { + resolve(); + } else { + setTimeout(tick, 50); + } + }; + tick(); + }); + } +} + +/** + * Convenience wrapper combining lock + mutex, matching runOnTab in background.js. + * @param {TabLockMap} locks + * @param {TabMutexMap} mutex + * @param {number} tabId + * @param {string|null} sessionId + * @param {() => (Promise | unknown)} fn + */ +export async function runOnTab(locks, mutex, tabId, sessionId, fn) { + await locks.waitFor(tabId, sessionId); + return mutex.run(tabId, fn); +} diff --git a/extension/manifest.json b/extension/manifest.json index 673d4eb..3dd04bf 100644 --- a/extension/manifest.json +++ b/extension/manifest.json @@ -1,7 +1,7 @@ { "manifest_version": 3, - "name": "Real Browser MCP", - "version": "1.3.0", + "name": "Browser Controller", + "version": "2.0.0", "description": "Let AI agents control your real browser - your tabs, your sessions, your logins", "icons": { "16": "icons/icon16.png", @@ -18,7 +18,9 @@ "alarms" ], "host_permissions": [ - "" + "", + "http://127.0.0.1:7225/*", + "http://localhost:7225/*" ], "background": { "service_worker": "background.js", @@ -34,7 +36,7 @@ ], "action": { "default_popup": "popup/popup.html", - "default_title": "Real Browser MCP", + "default_title": "Browser Controller", "default_icon": { "16": "icons/icon16.png", "48": "icons/icon48.png", diff --git a/extension/popup/popup.html b/extension/popup/popup.html index 17b4ea4..063abda 100644 --- a/extension/popup/popup.html +++ b/extension/popup/popup.html @@ -1,117 +1,376 @@ - +
-

Real Browser MCP

+

Browser Controller

-
Checking...
+
Checking…
- +
+
+ + +
+ +
Open Tabs
+
none
+ +
Connected Agents
+
none
+ +
Tab Locks
+
none
+ +
- Docs
+
+ diff --git a/extension/popup/popup.js b/extension/popup/popup.js index 5548421..c808782 100644 --- a/extension/popup/popup.js +++ b/extension/popup/popup.js @@ -2,19 +2,206 @@ const dot = document.getElementById('dot'); const statusEl = document.getElementById('status'); const detailEl = document.getElementById('detail'); const portInput = document.getElementById('port'); +const tokenInput = document.getElementById('token'); const logEl = document.getElementById('log'); const versionEl = document.getElementById('version'); +const locksEl = document.getElementById('locks'); +const unlockAllBtn = document.getElementById('unlockAll'); +const agentsEl = document.getElementById('agents'); +const openTabsEl = document.getElementById('openTabs'); const manifest = chrome.runtime.getManifest(); versionEl.textContent = `v${manifest.version}`; +// Shared state: agents come from the daemon /status, tabs+locks come from the +// background getStatus. Each panel needs the other's data to render its +// controls (the Pin dropdown lists agents; the Disconnect rows know nothing +// about tabs). Keep the latest snapshot of both so a re-render of either panel +// has the full picture without waiting for the other poll to land. +let lastAgents = []; // [{sessionId, name, connectedAt}] +let lastTabs = []; // [{id, url, title, active, lockedBy}] + +// Daemon HTTP base (same port as the WebSocket). The popup talks to it to +// auto-pair the token (no fs access in MV3) and to show connected agents. +function daemonHttpBase() { + return `http://127.0.0.1:${portInput.value || 7225}`; +} + +/** + * Auto-pair: fetch the daemon's token over localhost and hand it to the + * background worker so the WS connection carries ?token=. Runs once on popup + * open; safe to re-run. Silent if the daemon isn't up yet (it'll retry). + */ +async function autoPairToken() { + try { + const res = await fetch(`${daemonHttpBase()}/pair`, { cache: 'no-store' }); + if (!res.ok) return; + const data = await res.json(); + if (data && data.token && data.token !== tokenInput.value) { + tokenInput.value = data.token; + chrome.runtime.sendMessage({ type: 'setToken', token: data.token }); + } + } catch { + // daemon not reachable yet — the user will see "Disconnected"; no spam. + } +} + +/** Poll the daemon for connected agents and render them. */ +async function refreshAgents() { + try { + const res = await fetch(`${daemonHttpBase()}/status`, { cache: 'no-store' }); + if (!res.ok) { setAgents([]); return; } + const data = await res.json(); + setAgents(Array.isArray(data.agents) ? data.agents : []); + } catch { + setAgents(null); // null = daemon down + } +} + +function setAgents(agents) { + // null signals "daemon down" — keep an empty agent list (so the Pin dropdown + // has nothing to list) but render the unreachable banner in the agents panel. + lastAgentsDown = agents === null; + lastAgents = Array.isArray(agents) ? agents : []; + renderAgents(); + // tabs depend on agents (the Pin dropdown lists them) — re-render too. + renderOpenTabs(); +} + +let lastAgentsDown = false; + +function renderAgents() { + if (lastAgentsDown) { + agentsEl.innerHTML = '
daemon not reachable
'; + return; + } + const agents = lastAgents; + if (agents.length === 0) { + agentsEl.innerHTML = '
none
'; + return; + } + agentsEl.innerHTML = agents + .map((a) => { + const name = escapeHtml(a.name || 'agent'); + const sid = escapeHtml(a.sessionId || ''); + const ago = a.connectedAt ? Math.round((Date.now() - a.connectedAt) / 1000) : 0; + const dur = ago < 60 ? `${ago}s` : `${Math.round(ago / 60)}m`; + // data-sid carries the session id to the click handler (delegation). + return `
+ ${name} + ${sid} · ${dur} + +
`; + }) + .join(''); +} + +/** + * Kick a connected agent off the daemon immediately (don't wait ~45s for the + * heartbeat to evict it). Calls the daemon's GET /kill?sessionId=… endpoint. + */ +async function disconnectAgent(sessionId) { + try { + const res = await fetch(`${daemonHttpBase()}/kill?sessionId=${encodeURIComponent(sessionId)}`, { cache: 'no-store' }); + const data = await res.json(); + if (data && data.ok) { + addLog(`Disconnected ${sessionId}`, 'warn'); + refreshAgents(); // immediately reflect the eviction + } else { + addLog(`Disconnect failed: ${data?.error || 'unknown'}`, 'err'); + } + } catch (err) { + addLog(`Disconnect failed: daemon not reachable`, 'err'); + } +} + +/** Render the Open Tabs panel with a Pin dropdown per tab. */ +// Signature of the last-rendered tab set — skip the innerHTML rebuild when +// nothing material changed (avoids the 2s poll destroying an open open (mid-interaction), do NOT rebuild — + // replacing innerHTML closes the dropdown and loses the in-progress selection. + const openSelect = openTabsEl.querySelector('select[data-action="pickAgent"]'); + if (openSelect && openSelect === document.activeElement) { + return; // user is interacting with a dropdown; leave the DOM alone + } + // GUARD 2: only rebuild if the tab set + lock state + agent roster changed. + // The signature encodes tab ids, titles, lockedBy, and the agent list. Without + // this, every 2s poll re-renders identical rows, flickering the dropdown. + const agents = (lastAgents || []).filter(Boolean); + const sig = JSON.stringify({ + tabs: tabs.map(t => ({ id: t.id, title: t.title, lockedBy: t.lockedBy })), + agents: agents.map(a => a.sessionId), + }); + if (sig === lastTabsSignature) return; // nothing changed — keep the DOM stable + lastTabsSignature = sig; + + openTabsEl.innerHTML = tabs + .map((t) => { + const title = escapeHtml(t.title || t.url || `tab ${t.id}`); + const cls = t.active ? 'tab-title active' : 'tab-title'; + const lockedBy = t.lockedBy; + if (lockedBy) { + // Locked: show owner + an unpin (✕) button. No dropdown. + const ownerName = agentLabelFor(lockedBy); + return `
+ ${title} + + 🔒 ${escapeHtml(ownerName)} + + +
`; + } + // Unlocked: dropdown of agents + Pin button. The option VALUE is the + // agentName (the stable lock identity — survives reconnects where the + // sessionId churns s3→s4). The label shows both for clarity. + const opts = [''] + .concat(agents.map((a) => { + const name = a.name || 'agent'; + const label = escapeHtml(`${name} · ${a.sessionId}`); + const val = escapeHtml(name); + return ``; + })) + .join(''); + return `
+ ${title} + + + + +
`; + }) + .join(''); +} + +/** Human label for a sessionId, falling back to the bare id. */ +function agentLabelFor(sessionId) { + const a = (lastAgents || []).find((x) => x.sessionId === sessionId); + return a ? `${a.name || 'agent'} · ${sessionId}` : sessionId; +} + +function escapeHtml(s) { + return String(s).replace(/[&<>"']/g, (c) => ({ '&': '&', '<': '<', '>': '>', '"': '"', "'": ''' }[c])); +} + function updateUI(state) { if (!state) return; dot.className = `dot ${state.connectionState || 'disconnected'}`; const labels = { - connected: 'Connected to MCP server', + connected: 'Connected to daemon', reconnecting: 'Reconnecting...', disconnected: 'Disconnected', error: 'Connection error', @@ -23,7 +210,7 @@ function updateUI(state) { ? `Active: ${state.activity}` : labels[state.connectionState] || 'Unknown'; - let detail = `ws://localhost:${state.port || '7225'}`; + let detail = `ws://127.0.0.1:${state.port || '7225'}`; if (state.connectionState === 'reconnecting' && state.reconnectAttempts > 0) { detail += ` · attempt ${state.reconnectAttempts}`; if (state.nextRetryMs) detail += ` · retry in ${Math.round(state.nextRetryMs / 1000)}s`; @@ -36,6 +223,26 @@ function updateUI(state) { detailEl.textContent = detail; if (state.port) portInput.value = state.port; + + // Tab locks (task 3.3) — kept as a flat list with Unlock All. + const locks = Array.isArray(state.tabLocks) ? state.tabLocks : []; + if (locks.length === 0) { + locksEl.innerHTML = '
none
'; + unlockAllBtn.disabled = true; + } else { + locksEl.innerHTML = locks + .map((l) => { + const owner = agentLabelFor(l.sessionId); + return `
tab ${l.tabId}${escapeHtml(owner)}
`; + }) + .join(''); + unlockAllBtn.disabled = false; + } + + // Open Tabs panel (driven by the background getStatus payload). Cache so a + // later agents poll can re-render the Pin dropdowns with fresh agent names. + lastTabs = Array.isArray(state.tabs) ? state.tabs : []; + renderOpenTabs(); } function addLog(text, level) { @@ -48,10 +255,27 @@ function addLog(text, level) { while (logEl.children.length > 50) logEl.removeChild(logEl.firstChild); } -chrome.runtime.sendMessage({ type: 'getStatus' }, (response) => { - if (response) { +/** Fetch connection + tabs + locks state from the background worker. */ +function refreshStatus() { + chrome.runtime.sendMessage({ type: 'getStatus' }, (response) => { + if (chrome.runtime.lastError || !response) return; updateUI(response); - addLog(`Status: ${response.connectionState || 'unknown'}`, response.connectionState === 'connected' ? 'ok' : ''); + }); +} + +refreshStatus(); + +// On open: auto-pair the token (so the WS carries ?token=) and start polling +// the daemon for connected agents AND the background for tabs/locks. Both +// polls stop when the popup is hidden (closes) to avoid leaking timers. +autoPairToken(); +refreshAgents(); +let agentsTimer = setInterval(refreshAgents, 2000); +let statusTimer = setInterval(refreshStatus, 2000); +document.addEventListener('visibilitychange', () => { + if (document.visibilityState === 'hidden') { + clearInterval(agentsTimer); + clearInterval(statusTimer); } }); @@ -65,6 +289,7 @@ chrome.runtime.onMessage.addListener((msg) => { } }); +// Port (debounced) let debounce = null; portInput.addEventListener('input', () => { clearTimeout(debounce); @@ -77,3 +302,171 @@ portInput.addEventListener('input', () => { }); }, 600); }); + +// Token (task 3.1): the daemon generates token.json; the extension must present +// the same token on connect. Set it here once. +let tokenDebounce = null; +tokenInput.addEventListener('input', () => { + clearTimeout(tokenDebounce); + tokenDebounce = setTimeout(() => { + chrome.runtime.sendMessage({ type: 'setToken', token: tokenInput.value }, (resp) => { + if (resp?.success) addLog('Token updated, reconnecting', 'warn'); + }); + }, 600); +}); + +// Unlock All (task 3.3) +unlockAllBtn.addEventListener('click', () => { + chrome.runtime.sendMessage({ type: 'unlockAll' }, (resp) => { + if (resp?.success) addLog('All tab locks released', 'warn'); + }); +}); + +// Event delegation for the dynamic Open Tabs / Connected Agents panels. +// Rows are re-rendered on every poll (innerHTML), so attaching listeners to +// individual buttons would be lost — route all clicks through the document +// and dispatch on data-action. +document.addEventListener('click', (e) => { + const btn = e.target.closest('[data-action]'); + if (!btn) return; + const action = btn.dataset.action; + + if (action === 'disconnect') { + const sid = btn.dataset.sid; + if (sid) disconnectAgent(sid); + return; + } + + if (action === 'lockTab') { + const tabId = Number(btn.dataset.tab); + // find this tab row's