Host: remote verification through Simlock-backed fresh devices (ADR 0021) #3264

Purpose

A remote verification worker connects to agent-device Host on a self-hosted Mac, installs a PR's CI build on a fresh device, drives it, collects artifacts, and releases the lease. Host gets one fresh device per lease from a local Simlock daemon.

ADR 0021, as amended in #3095, is the source of truth. callstackincubator/simlock#70 tracks the Simlock side.

The work has two halves. The Host front-end is everything a remote caller touches: the agent-device host command, the service credential, trusted identity, the public route policy, device-by-type requests, and installing verification builds. The lease side covers the allocator contract trim, the Simlock adapter, the coordinator and journal, renewal, and restart recovery.

Children

• Host: agent-device host front-end command and persistent service credential #3265 agent-device host front-end command and persistent credential (PR feat(host): add the agent-device host front-end #3271)
• Host: strip client identity, hand the principal to the daemon over loopback, enforce the public route policy #3266 Identity stripping, principal handoff over loopback, public route policy (PR feat(host): strip client identity and enforce the public route policy #3272)
• Host: request devices by shape (--device "iPhone 16" / "Pixel 7") #3267 Shape requests (--device "iPhone 16" / "Pixel 7") (PR feat(host): request devices by shape #3273)
• Host: install verification builds through Host, including private GitHub Actions artifacts #3268 install-from-source through Host, including private GitHub Actions artifacts (PR feat(host): install verification builds through Host #3274)
• Host lease side: allocator contract trim, Simlock adapter, coordinator/journal, renewal, recovery #3269 Lease side: allocator contract trim, Simlock adapter, coordinator/journal, renewal, recovery
• Host acceptance: end-to-end verification run on iOS and Android from a second machine (ADR 0021 §10) #3270 Acceptance end-to-end run (ADR 0021 §10), iOS and Android, from a second machine

Design

I worked these out from the ADRs and the code before writing anything, and updated them to match the PRs. Section numbers refer to ADR 0021 unless noted.

D1. Where agent-device host lives (§3)

agent-device host is a CLI command built on @agent-device/proxy, not a new package. The command is src/cli/commands/host.ts and its credential and policy modules are in src/cli/host/.

It runs as its own process. Like proxy, it starts or reuses the local daemon in HTTP mode and forwards to it over loopback. It wraps createDaemonProxy(...).handle and serves it through the proxy's Node request listener, so transport, uploads, artifacts, instance checks and the token rewrite stay exactly as they are. The proxy command does not change.

Host has no second consumer yet, and a package would bring its own exports map, publish entry and layering rules. If one shows up, extracting it is a mechanical refactor(move).

D2. Principal handoff over loopback (§6)

This is the contract with the lease side. It stays a proposal until the lease side agrees; the comments below describe it.

The front-end authenticates the caller with the service credential and drops every identity the caller claims: the x-agent-device-tenant and x-agent-device-principal headers, params.meta.tenantId, params.flags.tenant, and params.tenant / params.tenantId on lease methods. After authentication it adds one header to the outbound loopback request, x-agent-device-principal: <principal>.

The daemon reads that header only on HTTP requests that already passed the daemon-token check. It validates the value with the tenant format, keeps it as daemon-internal state (internal.hostPrincipal), and pins meta.tenantId to it with sessionIsolation: 'tenant'. That is the same path a tenant attested by the auth hook takes today. The daemon never reads a principal from the body, and daemon-internal state never comes from the wire. If an auth hook is configured, the header is refused.

The trust comes from three facts. Holding the daemon token already grants local-operator authority, the same as /admin/*. The daemon listens on 127.0.0.1 only. Plain proxy forwards a header allowlist that leaves this header out, and a test pins that.

The principal is host-svc-<credentialId>. It stays the same across restarts because the credential id is persisted.

A forwarded request looks like this:

POST http://127.0.0.1:<daemon http port>/rpc
authorization: Bearer <daemon token>
x-agent-device-network-access: public-only
x-agent-device-principal: host-svc-3f9c2a1b
content-type: application/json

{"jsonrpc":"2.0","id":"1","method":"agent_device.lease.allocate",
 "params":{"token":"<daemon token>","runId":"verify-812","clientId":"ab12cd34",
           "platform":"ios","device":"iPhone 16"}}

D3. Service credential (§6, §8)

Host stores one credential at <state dir>/host/service-credential.json, written atomically, with the directory at 0700 and the file at 0600. It holds { version: 1, credentialId, token, principal, createdAt }, and the token is 32 random bytes in hex. Host creates it on the first start and reuses it after every restart. The state dir survives restarts and the process manager restarts Host.

Host refuses to start when the file or directory is open to group or others or owned by another user (host-credential-insecure), and when the file is malformed (host-credential-invalid). It never regenerates a bad file, because that would lock out every worker. These checks, and reading the TLS files (host-tls-unreadable), all happen before any daemon starts.

The token is printed only on the start that creates it. Later starts print the file path and principal, so process-manager logs never collect the secret. Rotation means deleting the file and restarting; live rotation and revocation are deferred (§9).

With --tls-cert and --tls-key, Host serves HTTPS; the key must match the certificate. Without TLS, Host serves plain HTTP and only on loopback (host-tls-required otherwise), for use behind a TLS tunnel. A wildcard bind such as --host 0.0.0.0 advertises the machine's hostname, since a worker cannot dial 0.0.0.0. A new credential reaches disk only once Host is serving, so a failed first start never hides the token from the next one.

D4. Public route policy (§5, §6, §10 item 6)

Host serves the same routes as proxy: /health, /rpc, uploads, /artifacts and request diagnostics. Unserved routes get 404, and a missing or wrong credential gets 401.

/admin/* is not served at all (404). Host applies its policy through an admitRpc hook the proxy calls after it has authenticated the request, so Host never repeats the proxy's auth or body handling. It refuses four kinds of request with HTTP 403, code UNSUPPORTED_OPERATION and a typed details.reason. Callers and tests check the reason, never the message text.

• host-admin-refused: lease.allocate for the host-allocated macos-app backend, matched the way the daemon normalizes backend names.
• host-path-refused: any input naming a path on the Host machine, batch steps included. Input keys come from HOST_PATH_INPUT_KEYS in src/daemon/host-path-inputs.ts, which the macos-app lease reads too. Positionals come from each command's own schema (path, appOrPath, payloadOrJson), so a new command is covered without editing Host. Only the exact temp locations the remote client writes for screenshots and recordings are accepted, and an upload id exempts only install and reinstall.
• host-component-download-refused: allowDownload anywhere in the request.
• host-script-refused: replay and test, whose nested actions Host cannot check before the daemon runs them.

Anonymous /health returns only { ok, service: "agent-device-host", rpcProtocolVersion }. Authenticated /health adds version, instanceId, upstream and features. A raw UDID or serial outside a Host lease is the lease binding's job, on the lease side.

D5. Device-by-type requests: a capability, not an rpcProtocolVersion bump (§5, §8, ADR 0006)

On a Host, every lease-allocating command needs --device "<type>" and --platform (or a platform already on the connection), and no UDID or serial; the client refuses anything else before a lease request. After allocation the command addresses the device the lease bound, by its UDID or serial, and a session that holds one type refuses another (host-shape-mismatch). A missing token shows up as host-unauthenticated, because only authenticated health carries the instance id.

Everything here is additive under the ADR 0006 bump rules: a new command and a new optional features field on /health. lease.allocate does not change shape. The device type, platform and --os-version travel in the device-selection fields it already carries (device, platform, providerOsVersion), and the daemon builds the strict { platform, deviceType, osVersion? } shape from them.

An old client talking to Host resolves --device against public inventory. Managed devices are not in it (§5), so the old client fails clearly before any mutation. A new client treats --device as a type only when authenticated health says service: "agent-device-host" and advertises features: ["device-shape"]. A Host without the feature fails with host-shape-unsupported before any lease request, which is §8's "protocol mismatch fails before mutation". Plain proxy reports service: "agent-device-proxy" and resolves inventory as before.

Clients authenticate with the same bearer token as proxy. The principal header only travels between the Host front-end and its own daemon. The wire-compat ledger records the additive health changes as compatibleChanges acknowledgements.

ADR 0006 prefers capability metadata to a protocol bump when the shared transport contract is unaffected, so §8's "RPC protocol change" is a capability here.

D6. install-from-source through Host (§6)

url sources (public hosts only, through the existing download approval) and uploaded artifacts already worked through proxy, so they work through Host. path sources are refused at the edge.

github-actions-artifact was the gap. The HTTP server parsed it, but the local daemon refused it and nothing held a GitHub credential. The daemon now resolves it through the GitHub REST API by id, by run and name, or by the newest live artifact with that name from the repository's own runs, never a fork pull request. It reads the token only from its own environment (AGENT_DEVICE_GITHUB_TOKEN), and AGENT_DEVICE_GITHUB_REPOSITORIES can limit which repositories it reads. Requests never carry the token, and daemon.json holds only its fingerprint, so a client holding a token refuses to reuse a daemon started without it or with another. The token goes only to an archive URL under the named repository on the GitHub API, and the downloader drops it at the cross-origin storage redirect. The lookup runs after the install's local checks and has its own timeout. Refusals are typed: github-token-missing, github-token-rejected, github-repository-not-allowed, github-artifact-not-found, github-artifact-expired, github-artifact-url-unexpected, github-api-refused, github-api-timeout.

Related

• docs: simplify Host to fresh Simlock devices and lease recovery #3095: ADR 0021 amendment (fresh devices, lease recovery)
• Host integration: use Simlock as the local device allocator callstackincubator/simlock#70: Simlock-side Host integration
• refactor(proxy): export the Node adapter from the main entry point #3262: proxy single entry point. If it lands first, Host imports from @agent-device/proxy only.