feat(computer-use): create platform abstraction layer for cross-platform backends

[0xMudit]

Summary

The Computer Use backend (\packages/computer-use/src/maka-cu-backend.ts) is deeply coupled to macOS-specific APIs (AXRaise, AXMenu, ScreenCaptureKit, macOS seatbelt). A platform abstraction layer is needed to support future Linux/Windows backends.

Current State

• **\packages/computer-use/src/maka-cu-backend.ts**: 1300+ lines of macOS-specific code
• **\packages/computer-use/src/select-backend.ts**: Hardcoded platform check
• **\�pps/desktop/src/main/computer-use/**: 7 macOS-only Electron components

Proposed Solution

1. Define a \ComputerUseBackend\ interface abstracting platform operations
2. Move macOS-specific code into a \DarwinComputerUseBackend\ implementation
3. Create stub implementations for Linux (\LinuxComputerUseBackend) and Windows (\WindowsComputerUseBackend)
4. Update \select-backend.ts\ to use the abstraction layer
5. Add platform guards to desktop computer-use components

Acceptance Criteria

• \ComputerUseBackend\ interface is defined and documented
• macOS backend implements the interface
• Linux/Windows stubs return appropriate errors or no-ops
• Desktop components have platform guards
• Tests pass on all platforms

Labels

enhancement, help wanted

[1625567290]

Proposal: make the shared executor-host lifecycle the missing platform abstraction

The direction of this issue is still useful, but current main has already moved part of the way toward the requested abstraction:

• the Runtime exposes a platform-neutral CuDispatchBackend;
• maka-cu-backend.ts implements that dispatch contract;
• select-backend.ts owns platform selection.

The Windows work in #4318 and #4409 reveals the remaining architectural gap more clearly: platform selection is abstracted, but native-executor supervision is not. A new backend currently has to reimplement process spawning, handshake, pending-request ownership, cancellation settlement, timeouts, restart, generation invalidation, and disposal.

I propose narrowing this issue from “introduce another backend interface and platform stubs” to “extract one shared supervised-executor core, then keep platform backends thin.”

Proposed boundary

Runtime / model-facing Computer Use actions
  -> CuDispatchBackend
  -> thin platform backend (Darwin, Windows, future Linux)
  -> shared SupervisedChildRpc lifecycle core
  -> isolated native executor

CuDispatchBackend remains the model-facing action boundary. The new shared core is an internal transport/lifecycle primitive, not another model-facing backend interface and not a new public maka.cu protocol revision.

Shared core responsibilities

The shared core should own only behavior that is independent of AX, UIA, AT-SPI, capture APIs, or executor language:

• validated launch specification and executable-integrity hook;
• child spawn, bounded handshake, and confirmed child exit;
• framed request/response transport with request IDs and output limits;
• pending-request stages such as queued, writing, delivered, and settled;
• abort and timeout settlement, preserving outcome_unknown when a mutation may have been delivered;
• single execution-lane ownership where required;
• bounded restart/backoff and executor-generation invalidation;
• shutdown, parent-death behavior, disposal, and lifecycle telemetry;
• deterministic hooks for fake-child and failure-injection tests.

Platform backends should continue to own:

• permission and approval policy;
• application/window/element identity and snapshot-token rules;
• capability discovery and action routing;
• capture and semantic/input implementation;
• platform-specific error translation and effect verification;
• private executor codec and protocol details.

The abstraction should not choose Rust versus .NET, repository ownership, or the final Windows executor. Those remain evidence-driven decisions under #4318/#3785.

Migration plan

1. Extract the generic lifecycle behavior from MakaCuService behind characterization tests, with no macOS behavior or protocol change.
2. Make the Darwin backend use the extracted core and keep its current protocol, release events, recovery semantics, and tests byte-for-byte compatible at the public boundary.
3. Require every comparative Windows executor to integrate through the same core and a thin Windows codec/backend adapter.
4. Remove any parallel Windows-specific supervisor before a Windows backend can become distribution-ready.
5. Add future Linux support by implementing a platform backend and private executor adapter, not by copying lifecycle authority again.

Acceptance criteria

• There is exactly one implementation of spawn/handshake/pending-request/cancel/timeout/restart/dispose authority in packages/computer-use.
• Existing macOS Computer Use behavior and focused tests remain unchanged at the public boundary.
• Fake-child tests cover cancellation before write, after delivery, blocked requests, malformed/oversized responses, crash during startup, crash after mutation delivery, restart exhaustion, confirmed old-child exit, and disposal races.
• Executor generation invalidates every platform session/snapshot that depends on the old child.
• Platform backends advertise capabilities dynamically and return typed unsupported/refused results; stubs must never silently succeed or no-op.
• The Windows comparison under feat(windows): feasibility spike for a supervised C#/.NET maka-cu Windows helper #4318 can swap executors without introducing another process-lifecycle implementation.
• No new public wire protocol, Windows product enablement, or executor-language decision is bundled into this refactor.

Why separate this from the Windows implementation

This extraction is useful even if the Windows architecture decision changes. It lowers the cost of evaluating .NET, Rust/direct-COM, or another isolated executor against the same host guarantees, while preventing a feasibility prototype from becoming a second permanent lifecycle authority.

If maintainers agree with this revised scope, #3896 can become the platform-neutral prerequisite for production Windows/Linux backends, while #4318 remains a disposable feasibility and comparative-evidence track.

中文说明

当前主干已经具备平台中立的 CuDispatchBackend 和 backend 选择层，因此本 issue 不必再新增一套相似的模型侧接口。#4318/#4409 暴露出的真实缺口，是不同平台仍会重复实现 native executor 的启动、握手、pending request、取消结束、超时、重启、代次失效和销毁。

建议把 #3896 收敛为一套共享的 SupervisedChildRpc 生命周期核心：Runtime 继续面向 CuDispatchBackend，Darwin/Windows/Linux 只保留薄的平台 backend 与私有协议适配器，底层共同使用唯一的子进程监管实现。

共享核心只负责与平台无关的 transport/lifecycle；权限、窗口与元素身份、snapshot token、能力路由、截图、动作实现和效果验证仍由平台 backend 负责。该重构不决定 Windows 使用 Rust 还是 .NET，也不引入新的公开协议或直接启用 Windows 产品能力。

实施顺序应先通过特征测试从 MakaCuService 无行为变化地抽取核心，再迁移现有 Darwin backend，随后要求 Windows 对照执行器都通过同一核心接入；Windows backend 分发前必须删除并行的专用 supervisor。这样 #4318 可以继续做可抛弃的选型实验，而不会把实验代码固化成第二套生命周期权威。