Skip to content

rpc: shared-host session worker open deadline is a hardcoded 30s wall clock, so attach silently falls back on a loaded machine #1719

Description

@code-yeongyu

Summary

On a busy machine, attaching an interactive session to the shared RPC host always fails: the session worker cannot finish open_session inside the hardcoded 30s budget, the host quarantines the session, and the TUI silently falls back to its own local runtime. The budget is the only lifecycle window in the shared-host stack with no override, so an operator has no way to make shared-host attach usable.

The failure is not caused by session size — a brand-new, empty session file fails the same way.

Reproduction

  1. Start a shared multi-session host: senpi --internal-rpc-host-supervisor --socket <agent-dir>/rpc-tui/rpc.sock with SENPI_RPC_CLIENT_CAPABILITIES=extension_events,custom_unsupported and SENPI_RPC_HOST_COLD_START=persistent.
  2. Put the machine under sustained load (here: 14 cores, 1-minute load average ~46, from other agent processes).
  3. Send {"id":"probe-open","type":"open_session","sessionPath":"/tmp/probe.jsonl","cwd":"/tmp"} over the public socket. The path does not exist, so there is nothing to load.

Expected

Either the open completes (a worker that is alive and progressing is given the time it needs), or the budget is operator-tunable the same way every other shared-host window already is, and the client reports the actual reason.

Actual

open_session took 30006ms
{"success": false, "error": "session_closing"}
list_sessions -> [{"sessionId":"rpc-10","status":"closing","sessionPath":"/private/tmp/probe.jsonl"}]

Host stderr:

senpi rpc host ready on unix://<agent-dir>/rpc-tui/rpc.sock (coldStart=persistent, idleExitMs=never)
senpi rpc session rpc-1 quarantined: session_worker_request_timeout

Interactive client (ENABLE_SHARED_HOST=1), after the same timeout:

Warning: shared interactive host unavailable; continuing locally

Every interactive pane that tried to attach ended up on its own local runtime, so the shared host stayed at zero sessions.

Root cause

packages/coding-agent/src/modes/rpc/session-worker-protocol.ts:13

export const SESSION_WORKER_LIMITS = {
	...
	openMs: 30_000,
	controlMs: 5_000,
} as const;

packages/coding-agent/src/modes/rpc/session-worker-requests.ts:25 turns it into one wall-clock deadline for the whole open:

private readonly openingDeadline = Date.now() + SESSION_WORKER_LIMITS.openMs;

Every non-control request during the open shares that single deadline, and expiry calls fail("session_worker_request_timeout") (session-worker-client.ts:48), which quarantines the session.

Two properties make this hard to live with:

  • No override. The neighbouring shared-host windows are all operator-tunable and documented in docs/rpc.md (SENPI_RPC_HOST_COLD_START, SENPI_RPC_HOST_IDLE_EXIT_MS, SENPI_RPC_HOST_EMPTY_EXIT_MS, SENPI_RPC_SESSION_IDLE_EVICTION_MS). The open deadline is the exception.
  • Wall clock, not liveness. A worker that is merely slow (CPU-starved host, cold extension/skill load) is indistinguishable from a wedged one, so a healthy worker is killed for being scheduled late.

Introduced with the bounded-worker isolation in #1499.

Scope and acceptance criteria

  • The session-worker open budget is configurable (e.g. SENPI_RPC_SESSION_OPEN_MS), defaulting to today's 30s, validated like the other windows (positive integer, invalid falls through to the default).
  • The deadline is liveness-aware: a worker that is still alive and making progress is not failed purely on elapsed wall clock; a silent worker still fails.
  • The interactive fallback warning names the cause (worker open timed out) instead of the generic "shared interactive host unavailable", so the operator can act on it.
  • docs/rpc.md documents the new knob beside the existing shared-host windows.
  • Regression coverage: an open that exceeds the default budget succeeds under a raised budget, and a worker that never answers still fails and quarantines.

Related

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions