Repository navigation
fix: detect harness death via parent PID watchdog - #20
Merged
Merged
Conversation
Existing shutdown handlers (signals, stdin events, transport-close) all rely on the harness gracefully tearing down. When the harness dies abruptly (SIGKILL'd, crashed, OS reap) none of them fire — the child is reparented to PID 1 (launchd on macOS) and heartbeats continue indefinitely as a zombie. Capture ppid at startup, check it every 30s in the heartbeat, exit cleanly when reparented. Logs a "reparented" event for triage. Also includes ppid in the alive heartbeat so we can correlate orphan processes with their original parent next time it happens. Verified against ~14 silent-disconnect entries in ~/.mailbox-mcp/debug.log where heartbeats stop with no termination event recorded.
The stdout/stdin error handlers called process.exit(0) on any stream error. Transient backpressure errors (EAGAIN/EWOULDBLOCK) — which Node surfaces on a busy pipe while flushing a large search_emails/read_email response — were treated as fatal. The server exited cleanly (exit code=0, so the lifecycle log showed nothing abnormal) and the client saw a dropped transport, forcing a manual /mcp reconnect mid-session. Classify stream errors: only a genuinely broken pipe (EPIPE, ECONNRESET, ERR_STREAM_DESTROYED, ERR_STREAM_WRITE_AFTER_END) exits. Transient errors are logged (stdin/stdout-error-transient) and the server keeps serving — the SDK transport recovers on the next read/write. shutdownClean is now idempotent so overlapping end/close/error events can't double-fire the drain timer. Diagnosis: across the full ~/.mailbox-mcp/debug.log history there are zero uncaughtException/unhandledRejection/fatal/non-zero-exit events — every prior resilience handler (process-level guards, provider-cache eviction on auth/connection errors) is already working. The only remaining mid-session drop vector was the stream error handlers exiting on non-fatal errors, which leave a clean exit code=0 and so never showed up as a crash. This is the targeted fix for that path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR closes the silent-disconnect gap when the Claude Code harness dies abruptly (SIGKILL, crash, OS reap) without firing any of our existing shutdown hooks.
Background
~/.mailbox-mcp/debug.logshows ~14 entries where heartbeats stop cold with no termination event recorded — no signal, notransport-close, no stdin event, no exception. The existing handlers all require the harness to tear down gracefully. They can't catch the case where the parent process is killed without warning and we get reparented to PID 1 (launchd on macOS).Change
Capture
process.ppidat startup. Every 30s in the heartbeat, check if it changed. If yes, the harness is gone — log areparentedevent and exit cleanly via the existingshutdownCleanpath.Also includes
ppid=in everyaliveheartbeat so we can correlate orphan processes with their original parent in future triage.Why this works
process.ppidis updated by the kernel when the parent dies. Reparenting to PID 1 is observable, doesn't require a signal, and doesn't depend on the SDK. Catches the gap left by SIGKILL-uncatchable signal handlers.Test plan
startlog line includesppid=ppid=valuekill -9) from another terminal, confirm mailbox-mcp exits within 30s and writes areparentedevent to debug.logFollow-up commit: don't exit on transient stdio backpressure errors
This is the fix for the reported symptom (server dropping intermittently mid-session, needing a manual
/mcpreconnect).Root cause: the
process.stdout/process.stdinerror handlers calledprocess.exit(0)on any stream error. Transient backpressure errors (EAGAIN/EWOULDBLOCK), which Node surfaces on a busy pipe while flushing a largesearch_emails/read_emailresponse, were treated as fatal. The server exited cleanly (exit code=0, so the lifecycle log showed nothing abnormal) and the client saw a dropped transport.Diagnosis note: across the full
~/.mailbox-mcp/debug.loghistory there are zerouncaughtException/unhandledRejection/fatal/non-zero-exit events. The existing process-level guards and provider-cache eviction are already working — no crash path remains. The only mid-session drop vector left was these stream handlers exiting on non-fatal errors, which leaves a cleanexit code=0and never registers as a crash.Change: classify stream errors. Only a genuinely broken pipe (
EPIPE,ECONNRESET,ERR_STREAM_DESTROYED,ERR_STREAM_WRITE_AFTER_END) exits. Transient errors are logged (stdin-error-transient/stdout-error-transient) and the server keeps serving — the SDK transport recovers on the next read/write.shutdownCleanis now idempotent so overlappingend/close/errorevents can't double-fire the drain timer.Verification:
npm run buildclean,vitest220/220 passing.