Skip to content

infra: reap CDP/Chrome browsers — bound launches + force-kill on agent exit (host OOM root cause) #10

Description

@jg-noncelogic

HIGH — root cause of a full host crash. Agent/tool CDP launches (chrome --remote-debugging-port=9222) leaked: 12 Chrome processes survived their launcher (reparented to systemd --user), lived ~12h, and consumed ~1GB until the host OOM'd and the delivery-train stalled.

Fix:

  • Every CDP/Chrome launch must be tracked and force-killed when the launching agent/tool finishes or errors (try/finally around the browser handle; kill the process tree, not just the client).
  • A reaper/guard: cap concurrent browser instances, set a per-browser memory ceiling, and a sweep that kills orphaned remote-debugging-port chrome older than N minutes whose parent is dead (reparented to pid 1 / systemd-user).
  • Consider --single-process/--disable-dev-shm-usage and explicit browser.close() + SIGKILL fallback.

Filed 2026-06-16 after the delivery-train host OOM crash mid-run (CDP/Chrome memory leak + no agent resource cleanup). The run had ~12 orphaned --remote-debugging-port=9222 Chrome procs (~1GB, reparented to systemd-user, 12h old) and ~7 leftover git worktrees that were never reaped — host ran out of memory and stalled, killing the workflow ~16/18 tickets in.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions