feat(browser): full-Chrome fallback integration + parallel sub-agent spawns

[raroque]

Summary

• New browser integration: 10 MCP tools wrapping the agent-browser CLI (open, snapshot, click, fill, press, get_text, get_url, wait, screenshot, close), exposed to sub-agents only.
• Real Chrome by default — auto-detects /Applications/Google Chrome.app/... on macOS and /usr/bin/google-chrome[-stable]/chromium on Linux. Falls back to agent-browser's Chrome for Testing if neither is present. Overridable via BOOP_BROWSER_EXECUTABLE.
• Headed by default — Chrome for Testing's --headless=new is fingerprinted instantly by Cloudflare/Reddit/etc., so we drop it via the AGENT_BROWSER_HEADED=1 env. Toggle is now a per-runtime setting (settings.browser_headed) with a switch inside the Settings tab's "Full browser use" card.
• Dispatcher prompt updated:
  • "Tool selection priority" block makes browser a fallback for sites without a native Composio toolkit. If a Gmail/Slack/etc. task lands, the agent uses that toolkit, not the browser.
  • "Parallel spawning" block tells the dispatcher it can emit multiple spawn_agent tool_use blocks in one turn for independent sub-tasks. The runtime already supports concurrent agents (server/execution-agent.ts:10); this just unlocks the model side.
  • One send_ack covers multiple parallel spawns now (no more triple-acks).
• Debug UI: BrowserSection card in Settings (moved out of Connections) — Chrome status indicator, "Show Chrome window" toggle, "Install Chrome for Testing" one-time button, and a "Log in to a site" helper that opens Chrome under the boop profile so cookies persist for future agent runs.
• New centralized server/browser/config.ts for profile dir / session / executable / env so tools, routes, and the login script all stay in sync.

Notable design choices

• Single shared --session boop across all sub-agents. agent-browser launches one Chrome per (--session, profile dir), and Chrome enforces one process per profile dir via SingletonLock — so per-agent sessions on a shared profile collide. v0 trades parallel-browser-isolation for reliability; commands queue through the daemon.
• Dedicated profile dir at ~/.boop/agent-browser-profile. Separate from the user's daily Chrome profile, so no lock contention with their everyday browsing.
• Browser is fallback-only. Reinforced in three places: integration registration description, dispatcher prompt, execution-agent prompt. The dispatcher is gated against passing browser for tasks a native toolkit covers.

Tradeoffs / known limits

• Some sites (X.com is the worst offender) detect any CDP-controlled Chrome regardless of binary or headed mode. For those, the Composio toolkit is the right answer; the fallback exists for the long tail of sites with no native integration.
• A real Chrome window pops up while sub-agents browse. This is intentional (much lower fingerprint), but visually noisy. If it bites, the headed toggle goes off in one click.

Test plan

• npm run typecheck clean
• npm run dev boots; server log shows [browser] using real Chrome at ... and [browser] registered ...
• Settings tab shows "Full browser use" card with status, in-card headed toggle, install button, login input
• Click "Open & sign in" with a URL — Chrome window opens under the boop profile
• Send the dispatcher: "open hacker news /newest and tell me the top story" — sub-agent uses browser_open + browser_snapshot, returns real content (verified pre-PR)
• Send a Gmail task — dispatcher passes ["gmail"], NOT ["browser"] (verify in agent log)
• Send: "check gmail unreads AND summarize today's calendar" — dispatcher fans out two spawn_agent calls in one turn (parallel verified by overlapping agent_spawned events)
• Toggle "Show Chrome window" off in Settings → next agent run launches Chrome headlessly within ~30s

[greptile-apps]

Greptile Summary

This PR adds a real-Chrome browser integration (10 MCP tools via agent-browser CLI with CDP stealth patching), a pause-and-resume flow for login walls, parallel sub-agent dispatch in the dispatcher, and a cookie-import UI for lifting session cookies from the user's daily Chrome profile.

Two P1 issues in the new server/browser/ layer:

• spawnChrome hardcodes /Applications/Google Chrome.app/... as the fallback when no system Chrome is detected. On Linux with only Chrome for Testing installed, Chrome never starts, every browser_* call times out, and the agent sees a misleading "install agent-browser" hint.
• DAILY_CHROME_DIR is macOS-only (~/Library/Application Support/Google/Chrome). On Linux, listDailyProfiles silently returns [], making cookie import entirely non-functional with no user-facing error.

Confidence Score: 3/5

Safe to merge for macOS-only deployments; two P1 regressions affect Linux users.

Two P1 findings both in the new browser subsystem: the stealth-launcher Chrome fallback path and the macOS-only DAILY_CHROME_DIR break the integration on Linux. The rest of the PR is well-structured and correct.

server/browser/stealth-launcher.ts (fallback binary), server/browser/cookies.ts (DAILY_CHROME_DIR)

Important Files Changed

Filename	Overview
server/browser/stealth-launcher.ts	New file — launches a real Chrome via CDP, injects stealth patches on every new document, and manages lifecycle. P1: hardcoded macOS fallback binary in spawnChrome breaks the integration on Linux when only Chrome for Testing is installed.
server/browser/cookies.ts	New file — snapshots and queries the user's daily Chrome profile Cookies SQLite DB, then copies selected rows into the boop profile. P1: DAILY_CHROME_DIR is hardcoded to the macOS path; cookie import silently returns no profiles on Linux.
server/browser/tools.ts	New file — exposes 9 browser_* MCP tools wrapping the agent-browser CLI. No browser_close tool is registered (correct — lifecycle is server-managed).
server/browser/config.ts	New file — centralises profile dir, session name, CDP port, and getBrowserEnv(). Clean; detects real Chrome on macOS and Linux, falls back to null (issue surfaces in stealth-launcher).
server/browser-routes.ts	New file — Express routes for browser status, install, cookie profile/scan/import, and the login helper. Properly guards cookie import against concurrent agents and validates URLs for the login endpoint.
server/pause-tools.ts	New file — pause_for_user MCP tool that sends a user message, persists a continuation, and sets pausedFlag.paused. Clean implementation.
server/interaction-agent.ts	Updated dispatcher: adds parallel spawn instructions, pending-continuation injection, and clear_pending_continuation tool.
server/execution-agent.ts	Adds pause_for_user MCP server and promotes status to include "paused". Clean signal path via pausedFlag.
convex/schema.ts	Adds pendingContinuations and cookieImports tables plus "paused" status literal. Consistent with new mutation/query files.
server/index.ts	Mounts browser router and registers signal handlers to stop Chrome on server shutdown.

Sequence Diagram

sequenceDiagram
    participant D as Dispatcher
    participant E as Execution Agent
    participant P as pause-tools MCP
    participant S as stealth-launcher
    participant AB as agent-browser CLI
    participant C as Chrome CDP :9222

    D->>E: spawnExecutionAgent
    E->>S: ensureStealthChrome()
    S->>C: spawn binary (CHROME_PATH ?? macOS fallback)
    S->>C: attachWebSocket + STEALTH_SCRIPT on every new doc
    E->>AB: agent-browser --session boop --cdp 9222 open url
    AB->>C: drives tab via CDP
    AB-->>E: stdout result
    E-->>D: SpawnResult completed

    Note over E,P: If login wall hit
    E->>P: pause_for_user(message, resume_task)
    P->>D: sendImessage + persist pendingContinuation
    P-->>E: pausedFlag.paused = true
    E-->>D: SpawnResult paused
    D-->>D: dispatcherSilent = true

    Note over D: Next user message
    D->>D: read pendingContinuation from Convex
    D->>E: spawnExecutionAgent resume_task

Loading

Comments Outside Diff (2)

1. server/browser/stealth-launcher.ts, line 2856-2858 (link)

   Hardcoded macOS fallback breaks Chrome for Testing on Linux

   When CHROME_PATH is null (no system Chrome found), spawnChrome falls back to the macOS binary path regardless of platform. On a Linux system where only Chrome for Testing is installed via agent-browser install, spawn() receives a non-existent path, Chrome never starts, and waitForCdpEndpoint times out after 15 s. Every browser_* tool call then returns exitCode: null, which triggers the "Is agent-browser installed?" hint — pointing users at a completely wrong fix.

   The log in tools.ts says "falling back to Chrome for Testing" but the stealth launcher has no logic to actually locate Chrome for Testing's binary.

   You'll need to either resolve Chrome for Testing's installed binary path (e.g., by running agent-browser doctor and parsing the output) or skip the stealth-launcher entirely on the fallback path and let agent-browser manage its own Chrome lifecycle.

   

2. server/browser/cookies.ts, line 2305-2312 (link)

   DAILY_CHROME_DIR is macOS-only — cookie import silently returns nothing on Linux

   The constant is hardcoded to ~/Library/Application Support/Google/Chrome, which is the macOS path. On Linux, Chrome stores its profiles under ~/.config/google-chrome (or ~/.config/chromium). Since listDailyProfiles starts with if (!existsSync(DAILY_CHROME_DIR)) return [], the feature silently produces zero profiles on Linux with no feedback to the user.

   The PR description mentions Linux support for the browser integration, so users installing on Linux who try to import cookies will see an empty profile list with no explanation.

   

_{Reviews (4): Last reviewed commit: "feat(browser): stealth Chrome + cookie i..." | Re-trigger Greptile}

[greptile-apps]

Misleading install hint fires on timeout, not just missing binary

exitCode === null is true in two distinct cases: (a) agent-browser is not found (ENOENT), and (b) execa kills the process because the 30 s TIMEOUT_MS expired. Case (b) is a routine occurrence for browser_wait whenever the agent passes a duration ≥ 30 000 ms (e.g. "45000" for a slow load), or for any browser command on a sluggish page. In that scenario the agent sees:

[browser error] exit=null
Command timed out after 30000ms

Is agent-browser installed? Run `npx agent-browser install` once on this machine.

The hint is flat-out wrong; agent-browser is installed. The agent will either waste a turn trying to install it or give up on the task with a confusing diagnosis. Reserve the install hint for the ENOENT case only:

Suggested change

	const hint =
	r.stderr.includes("ENOENT") || r.exitCode === null
	? "\n\nIs agent-browser installed? Run `npx agent-browser install` once on this machine."
	: "";
	const hint =
	r.stderr.includes("ENOENT")
	? "\n\nIs agent-browser installed? Run `npx agent-browser install` once on this machine."
	: "";

The underlying cause is also worth noting: TIMEOUT_MS = 30_000 is shared by browser_wait, but browser_wait accepts arbitrary millisecond durations with no stated upper bound. Any target ≥ 30 000 will always timeout silently. Consider either documenting a <30 s cap in the tool description, or giving browser_wait a higher per-call timeout (e.g., 90 s).

Suggested change

	const hint =
	r.stderr.includes("ENOENT") || r.exitCode === null
	? "\n\nIs agent-browser installed? Run `npx agent-browser install` once on this machine."
	: "";
	const hint =
	r.stderr.includes("ENOENT")
	? "\n\nIs agent-browser installed? Run `npx agent-browser install` once on this machine."
	: "";