Repository navigation
Let agents drive hosted macOS apps through a host-owned driver #2477
Description
Activity
Claimed: feat/2477-hosted-agent-driver
Merged #2479: the host driver boundary (
HostedAgentHost, refcount, restart, grant scoping), the agent-device adapter (explicit binary path, claim with the daemon as child, verified stop) and the/device-host/agent/<session>/route. The adapter stays gated: agent-device has no remote lease limited to one macOS app, sostart()andissue()refuse and hosted apps get{ driver: 'none' }with a notice. Lease binding inforward()must be added when that lease exists. Still to do after #2473 merges: callappRunning/appStoppedfromDeviceHost(replacing the fixednonein #2473), and add the doctor line for a hosted macOS app running with no driver. The upstream proposal is drafted and has not been filed.Claimed: feat/2477-agent-driver-wiring (wiring of appRunning/appStopped and the no-driver doctor/guide line; the upstream-lease part stays open)
Upstream proposal filed: callstack/agent-device#3229 (a
macos-applease backend scoped to one app). The driver stays gated off ({driver:'none'}) until it lands.Merged #2484 (55b77d7):
DeviceHostnow callsappRunning/appStoppedon hosted macOS install, stop, revocation and close, andapp.attach/app.launchreturn the manager's grant ornonewith its notice. The install worker returns the app pid.stim doctoron the hosting Mac notes a ready hosted macOS app whilehosting.agentDriverisnone; the guide, website and server README say so. Still open here: the upstream macOS app lease (proposal drafted, not filed), lease binding inforward()and theagent-deviceadapter'sissue(), the client's 0600 remote-config (#2475), and the real two-Mac run. Known gap: grants are in memory, so an app still running after a stim-server restart reports `none"; nothing re-attaches it today.Claimed: feat/2477-macos-app-lease (implement the macos-app lease in agent-device per callstack/agent-device#3229, use it from Stim's agent-device adapter behind feature detection, and write the client remote-config lease fields)
- added a commit that references this issue
on Oct 5, 2026 Upstream draft PR for the macos-app lease: callstack/agent-device#3236 (implements #3229). Stim side: #2516 (driver enables only when the daemon's /health lists macos-app; setting default stays none). The two-Mac run is pending: redeploying the mini's stim-server with STIM_AGENT_DEVICE_BIN was blocked by the session's permission classifier.
End-to-end result for the host-owned agent driver on the real tailnet path (MacBook client, Mac mini host, current main 787f051, signed Stim Host 0.1.0, agent-device from callstack/agent-device#3236 draft at 5c958e109,
AGENT_DEVICE_MACOS_APP_BACKEND=native).Setup on the host:
hosting.agentDriver = agent-deviceon the mini; the LaunchAgent carriesSTIM_AGENT_DEVICE_BINandAGENT_DEVICE_MACOS_HELPER_BINand runs under Stim Host (Screen & System Audio Recording and Device Control and Data Access allowed). The setup stays in place for dogfooding.Results
stim macos --host janics-mac-miniwith a fixture SwiftPM app: built here, delivered, launched as<id>.hosted1;status --jsonreportedagent.driver = "agent-device"with the remote config.- agent-device
open,snapshot -i,click,fill,type,find,get text,screenshot(1800x900, only the app window),closeall worked over https 7443. - Refused as intended:
openof another app (UNAUTHORIZED, "only opens "),open --surface desktop,screenshot --fullscreen,apps/devices("Unsupported request"),--platform ioson the leased session (CONNECTION_PLATFORM_CONFLICT),clickoutside any accessibility element. - The same flow with
apps/desktopfrom a fresh worktree (hosted Stim Desktop, 200 files delivered, ~63 s build): snapshot of 49 nodes, clickingGet Started, dismissing the onboarding sheet, opening Machines, window-only screenshot. - The mini's agent-device proxy starts with the first hosted app and its
/healthlistsmacos-appinleaseBackends; it exits afterstim stop.stim stopremoved the hosted apps and the remote config.
Observations
- After
stim stop, the client's local agent-device session stays bound to the old remote config, so the next workspace's--remote-configfails with "A different remote connection is already active" untilagent-device disconnectruns.stim guide macoscould say to runagent-device closethenagent-device disconnectbeforestim stop. - Once, the first
screenshotafter aclickreturned a 132x40 image (a transient window-sharing indicator,WindowSharingSessionButton, appeared in the snapshot). It did not recur in the next 7 captures. - In the onboarding sheet of hosted Desktop,
click @reffailed several times with helper reasonno-accessible-target(point-based hit test), then worked after the sheet changed; typing Escape dismissed the sheet. ~/.stim/server/agent-device/sessions/stim.<session>_defaultdirectories remain on the host after each hosted session.
Backlog triage: Stim's driver lifecycle/scoping, wiring and macos-app lease adapter landed in #2479, #2484 and 38da60b (#2516). The recorded two-Mac run satisfies the driver acceptance, including other-app and broad-surface refusals plus daemon teardown: #2477 (comment) . The connection/session cleanup observations were fixed by acbf044 (#2527).
On 43395b6,
pnpm test packages/stim-cli/src/__tests__/hosted-agent-driver.test.ts packages/stim-cli/src/__tests__/hosted-macos-client.test.tspassed 50 tests. Closing completed Stim implementation/acceptance. The runtime evidence used the upstream callstack/agent-device#3236 draft build; an installed agent-device must advertisemacos-appor Stim still refuses the grant as designed. This closure does not claim that the upstream capability has shipped.
Track E of #2403. Contracts: contracts comment, sections 1 (agent route), 5 and 6. Shared types and the
hosting.agentDriversetting: #2472.Problem
A coding agent on the client Mac cannot drive a hosted macOS app (snapshot, click, type, screenshot) the way it drives local simulators with agent-device. The maintainer decided:
hosting.agentDriver(defaultnone, opt-inagent-device);Evidence
Local experiments with isolated state directories and a fixture app:
agent-device(0.21.12) open, snapshot -i, click and screenshot work on astim macosapp. The screenshot is the window only.agent-device proxyplusconnect proxy,devices --platform macoslists the host, butopenrefuses: remote leases exist only forios-simulator,ios-instance,android-instanceandharmonyos-instance. This holds in 0.21.12 and 0.21.20 and on upstream main, whereLEASE_BACKEND_BY_PLATFORMmaps the macOS desktop host to no backend. fix(remote): speak one platform axis between a device and its bound connection (#2962) callstack/agent-device#2989 (the platform-axis fix, 0.21.16) does not change this.daemonBaseUrl,daemonAuthToken,leaseId,leaseBackendandplatform.PATHis only system directories. agent-device lives at~/.local/bin/agent-device(0.21.20 on the mini).Fix idea / scope
desktop/frontmost-app/menubarsurfaces, install and launch of other apps, and non-window screenshots. A host administrator allocates the lease. Search upstream first, and link the result here./device-host/agent/<session>/*is forwarded with the session token, only from the client's node.--remote-configand fillHostedAgentAccess(handled with Run macOS apps on a hosting Mac with stim macos --host #2475).doctorandguidenamehosting.agentDriverwhen a hosted macOS app runs with no driver.Until the upstream lease exists,
issuerefuses, and deliveries carry{ driver: 'none' }with a notice. No unscoped desktop access is ever handed out.Acceptance
Out of scope
Other drivers (one adapter each, later).
The upstream proposal and the adapter skeleton can start now. End-to-end use depends on #2473 and #2475.