[Feature]: t3 browser CLI — drive the collaborative browser without MCP

[YoruSamaOG]

Before submitting

• I searched existing issues and did not find a duplicate.
• I am describing a concrete problem or use case, not just a vague idea.

Area

apps/server

Problem or use case

The collaborative browser is only reachable through the MCP preview_* tools, so an agent without MCP cannot drive it at all. That covers a plain shell, a script, a provider whose harness has no MCP, and a second agent that wants to look at what the first one is building. Those callers currently reach for Playwright, which opens a browser nobody is watching — losing the one property that makes the preview worth having, that a human can see and take over the same tab.

There is a second, quieter problem for callers that do have MCP. preview_snapshot is budgeted and viewport-shaped, which is right for orientation and wrong for reading data. Getting a 240-row table means scrolling and re-snapshotting, paying for nav and chrome on every pass, with no way to tell a truncated read from a complete one.

Proposed solution

A t3 browser subcommand that goes through the same PreviewAutomationBroker the MCP tools use, so a CLI-driven tab is the same collaborative tab a human sees. It reads the running server's origin from the persisted runtime state and mints a short-lived scoped session, so there is nothing to configure.

Two smaller pieces that turned out to matter more than the transport:

• Element refs. Snapshots return @e1-style handles and actions accept a ref instead of a selector. This removes the guessing game of writing a locator that matches, and a stale ref fails as a distinct "stale ref, take a new snapshot" error rather than silently clicking the wrong element.
• extract. Structured row reading with cursor pagination, so a whole table is one call and the response states the true total rather than however many rows happened to fit.

Why this matters

It makes the collaborative browser usable from anything that can run a command, instead of only from a provider with MCP wired up. And it turns "what does this dashboard actually say" into one call rather than a scroll-and-hope loop.

Smallest useful scope

The CLI alone, over the existing broker, with open, snapshot, click, type, and status. Refs and extract are separable and could land later, or never.

Alternatives considered

Playwright or raw CDP. Both drive a browser the human cannot see, which gives up the collaborative tab.

Expanding the MCP tool surface instead. This seems like the wrong direction given #6792 and #6863, where people are asking for fewer preview tools, not more. A CLI adds no MCP tools at all — it is reachable only by callers that deliberately run it.

Doing nothing. Reasonable if MCP-less agents are not a case you care about. That is the main thing I would like a read on.

Risks or tradeoffs

• A second entry point into the broker is a second thing to keep in step with the MCP tools. Sharing one operation registry and decoding CLI input against the MCP tool's own schema keeps them from drifting, at the cost of the CLI never being able to accept something the MCP tool would reject.
• Minting an administrative session per command is real auth surface, even though it is short-lived and revoked when the command exits.
• extract is a genuinely new capability rather than a transport change, and is the piece most reasonable to say no to.

Examples or references

Related, and pulling the opposite way on tool surface: #6792 (disable built-in browser/preview MCP tools), #6863 (skip injecting preview tools when no host is connected).

Contribution

• I would be open to helping implement this.

---

Being upfront: I have this working on a fork and have been using it daily, which is how the refs and extract parts surfaced. I am raising it here rather than opening a PR because CONTRIBUTING.md asks for an issue first on non-trivial work, and because a large unsolicited feature PR is explicitly what you have asked people not to do. Happy to break it into small pieces if it is a direction you want, and equally happy to hear it is out of scope.

[blas0]

Arrived at nearly this design from the token-cost side, so rather than open a competing thread: strong +1 on routing through PreviewAutomationBroker instead of standalone Playwright. Two things I'd change, and some measurements.

Measurements (at e145c5f2, building each toolkit with the repo's effect@4.0.0-rc.112 and serializing as McpServer.registerToolkit does):

Toolkit	Tools	Definition bytes
preview_*	14	17,545
device_*	4	2,141
pull requests	3	1,811
Total	21	21,497 (~5–7k tokens, estimated from bytes, not tokenized)

Harnesses that load MCP definitions up front pay that in every thread, browser or not. Claude's SDK defers them by default, so the idle cost there is just the 21 names (~650 B), but each first use still pulls whole definitions in (preview_navigate alone is 2.9 KB).

Change 1: reuse the thread credential instead of minting an admin session. The McpSessionRegistry token already carries exactly the right authority: threadId, providerSessionId, and the capability set from enableAgentBrowserAccess/enableAgentDeviceAccess. A small HttpApiMiddleware mirroring McpAuthMiddlewareLive can resolve that same bearer and provide McpInvocationContext, so requireMcpCapability behaves identically on both transports and no new authority appears. Keeping the same providerSessionId also preserves the broker's host/tab stickiness, which is the property that makes this the collaborative tab and not just another browser.

Delivering it can also fix #10020 on the way: prepareMcpSession writes a 0600 session file the way AgentDeviceTarget already writes host configs, and only the path goes into the env (T3_AGENT_SESSION_FILE) via the existing withAgentDeviceEnvironment plumbing, which already reaches all six adapters. The token then never sits in argv or the environment.

Change 2: the same treatment applies to the pull-request tools, and they're the cheaper place to start. They're 3 tools plus an always-on <pull_request_linking> block appended for every provider, with no binary payloads and no broker involved. Shipping t3 agent pr link|unlink|list first proves the contract, the middleware, the session file, and the PATH shim on the smallest possible surface, and browser can follow with the risky parts already settled.

On drift, which you flagged as the main cost: it may not need a separate operation registry. The handlers already read scope from McpInvocationContext and toolkits run outside MCP via Toolkit.handle(name, payload) — the existing handler tests do this. So a typed AgentHttpApi group in packages/contracts can reuse the current tool schemas and have each endpoint call toolkit.handle(...) over the existing handler layers. One definition, two transports, no second implementation to keep in step.

One more piece worth folding in: browser snapshot writing the PNG and bounded metadata to disk and printing a short summary with paths, reusing boundSnapshotMetadata/saveScreenshot out of McpHttpServer.ts. That's the same direction as #11295 and #10501, and it's most of the per-call win: file-first output is a few hundred bytes per command instead of an in-band PNG plus up to 60 KB of text.

The precedent for all of this is the maintainers' own, in docs/internals/devices.md: four device_* tools, driving through the agent-device CLI on a PATH shim, because usage instructions returned on demand "cost nothing in threads that never open a device."

Full write-up with per-adapter notes and a test plan: blas0#1