What happened
Every session (27 of them) flipped to error simultaneously, and new sessions failed with failed to start session: tmux new-session failed: server exited unexpectedly.
Root cause: tmux did not crash. A background agent working in this repo ran, as part of probing tmux's copy-command defaults:
tmux kill-server 2>/dev/null; tmux new-session -d -s __probe__ ... ; tmux kill-server 2>/dev/null
Bare tmux kill-server — no -L — so it hit the default socket, which is where fleet keeps every session. Timeline from debug.log:
14:39:29 — kill-server runs
14:39:31 — every session's SessionEnd hook fires status=dead
14:39:32 — all 27 sessions → error, reason=tmux_gone
Two seconds from cause to total loss. The agent's other probes were correctly socket-scoped (-L probe205b, -L fleettest_env); one command forgot the flag.
Why it's bad
fleet deliberately shares the user's default tmux server (no dedicated socket) because copy-command is a server option — see the copy-to-clipboard note in CLAUDE.md. That decision means any process doing routine tmux hygiene on the default socket wipes the entire fleet. This is not hypothetical: agents that fleet itself runs work on tmux code and probe tmux behavior.
Two things worth fixing
1. Blast radius — dedicated socket. Running fleet's sessions on -L fleet would make a stray default-socket kill-server harmless. Cost: interacts with the copy-command server-option decision (a dedicated server needs its own copy-command set, which is arguably cleaner — no more "only set when unset" tiptoeing around the user's config). Worth evaluating.
2. Recovery UX. Today this presents as a cryptic server exited unexpectedly on every row, and recovery needed manual ps/kill surgery at the shell. When every session goes tmux_gone in the same tick, fleet knows the server died — it could say so plainly ("tmux server is gone — all sessions stopped") and offer one-key recovery instead of 27 identical error rows.
Related but distinct: a separate incident days earlier had the same symptom from a genuinely wedged server (alive, holding the socket, spinning at 147 CPU-hours, not serving clients). tmux ls failed while a tmux process existed. Recovery detection should handle both the gone case and the wedged case.
What happened
Every session (27 of them) flipped to error simultaneously, and new sessions failed with
failed to start session: tmux new-session failed: server exited unexpectedly.Root cause: tmux did not crash. A background agent working in this repo ran, as part of probing tmux's
copy-commanddefaults:Bare
tmux kill-server— no-L— so it hit the default socket, which is where fleet keeps every session. Timeline fromdebug.log:14:39:29—kill-serverruns14:39:31— every session'sSessionEndhook firesstatus=dead14:39:32— all 27 sessions → error,reason=tmux_goneTwo seconds from cause to total loss. The agent's other probes were correctly socket-scoped (
-L probe205b,-L fleettest_env); one command forgot the flag.Why it's bad
fleet deliberately shares the user's default tmux server (no dedicated socket) because
copy-commandis a server option — see the copy-to-clipboard note inCLAUDE.md. That decision means any process doing routine tmux hygiene on the default socket wipes the entire fleet. This is not hypothetical: agents that fleet itself runs work on tmux code and probe tmux behavior.Two things worth fixing
1. Blast radius — dedicated socket. Running fleet's sessions on
-L fleetwould make a stray default-socketkill-serverharmless. Cost: interacts with thecopy-commandserver-option decision (a dedicated server needs its owncopy-commandset, which is arguably cleaner — no more "only set when unset" tiptoeing around the user's config). Worth evaluating.2. Recovery UX. Today this presents as a cryptic
server exited unexpectedlyon every row, and recovery needed manualps/killsurgery at the shell. When every session goestmux_gonein the same tick, fleet knows the server died — it could say so plainly ("tmux server is gone — all sessions stopped") and offer one-key recovery instead of 27 identical error rows.Related but distinct: a separate incident days earlier had the same symptom from a genuinely wedged server (alive, holding the socket, spinning at 147 CPU-hours, not serving clients).
tmux lsfailed while a tmux process existed. Recovery detection should handle both the gone case and the wedged case.