Skip to content

Targeted fleet spawn reports spawn_failed/spawn_unconfirmed for agents that launched and are ready #1841

Description

@khaliqgant

Problem

A targeted agent-relay fleet spawn … --node <name> reports failure even when the agent launched fine. Users who then retry can start a duplicate (see the duplicate-spawn / retire hazard in #1593).

Seen live on 2026-09-22, agent-relay 12.4.1 on both nodes (MacBook + Mac mini, same workspace, both started with node up --background --broker-name …):

  1. fleet spawn claude --node xmach-mini --name Probe … exited 1 with:
    node 'xmach-mini' reported completed for spawn without explicit spawned:true and ready:true proof
    {"error":{"code":"spawn_failed","state":"failed",...,"dispatchState":"dispatched","receipt":{"status":"completed",...}}}
    
    On the mini, node agent list showed Probe with "ready": true, current_state: working, and a live claude pid.
  2. fleet spawn claude --node xmach-laptop --name Receiver … and fleet spawn claude --node xmach-mini --name Sender … each exited with spawn_unconfirmed ("never reported a result within 120000ms"). Both agents were running with "ready": true on their nodes (the laptop one within seconds), and Receiver later received and acted on a cross-machine DM.

Likely cause (to confirm)

The SDK's confirmation check (packages/sdk/src/messaging/relaycast.ts ~L779 and ~L999, hasExplicitSpawnReadinessProof) requires spawned:true and ready:true in the invocation output. The broker has two result builders:

  • crates/broker/src/runtime/fleet.rs:157 verified_spawn_ready_result{spawned:true, ready:true}
  • crates/broker/src/runtime/fleet.rs:1884 fleet_spawn_action_result{spawned:true} only, no ready

Case 1 fits the second path: the result reached the cloud as completed without ready, and the SDK treated it as a failure. For case 2, the verified-ready result seems never to be posted, or not before the 120s budget, even though the worker was ready. Check whether the PendingVerifiedSpawn deadline/generation path fires for PTY claude workers and whether the result reaches the invocation record getInvocation reads.

Expected

When the node reports a spawn completed and the worker becomes ready, the CLI exits 0. spawn_failed / spawn_unconfirmed appear only when the agent really didn't start.

Acceptance

  • Broker and SDK agree on one readiness contract. Either every success path emits {spawned:true, ready:true} once ready, or the SDK accepts the shape the broker actually sends. Cover both the verified and unverified broker paths with tests.
  • A targeted spawn onto a healthy remote node of a logged-in claude returns success within the default confirm budget. Prove it with a test that fails on the current code (mutate-to-red, per the repo's proof rules).
  • A changelog entry under [Unreleased].

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions