macOS: `ocx service repair` unconditionally evicts the running LaunchAgent, accepts a no-op `launchctl load` as success, and leaves the hub down with no log line

[lidge-jun]

Severity: high — a single ocx service repair can take a hub fully offline (public proxy + management ingress + loopback client listener) with no error shown, no rollback, and no automatic recovery. Recovery needs a launchctl command the CLI never prints.

Environment

• opencodex 2.51.0 (source checkout run via bundled Bun 1.4.2)
• macOS 26 / Darwin 27.0.0, arm64
• runtimeRole: "hub", hostname: "100.76.170.81" (Tailscale-only public bind), port: 10100
• hub.managementIngress: { enabled: true, port: 10102 }
• unauthenticatedLoopbackListener: { enabled: true, port: 10104 }
• LaunchAgent com.opencodex.proxy at ~/Library/LaunchAgents/com.opencodex.proxy.plist, KeepAlive: true

All line references are to the source checkout at commit 6d3ad12e3 (main, after PR #4196).

---

Summary

Four defects, all on the macOS launchd path. (1) is the outage. (2) can send an operator into (1) on a false premise. (3) and (4) turn permanently-fatal config states into silent KeepAlive crash loops / false warnings.

1. installLaunchd() evicts the live job with a domain-explicit bootout, re-registers with the domain-implicit legacy launchctl load -w, and accepts exit-0-with-empty-stderr as success — then writes install state. No pre-check that the job is already healthy, no settle delay, no bootstrap fallback, no plist backup, no rollback. macOS is the only backend with "evict first, verify never"; the Windows branch of the same function has a full preserve/restore protocol.
2. diagnoseService() derives "loaded" from sh("launchctl list | grep <label> || true") — session-relative domain, and || true plus a catch swallow every exit code, so every failure mode collapses into "not loaded". That single bit also drives enabled, running, viable and therefore isServiceViable(), which the update path uses to decide whether to start a competing proxy.
3. An occupied unauthenticatedLoopbackListener.port / hub.managementIngress.port is a fatal startup crash that is then misdiagnosed as a public port conflict, so the retry waits on the wrong port and the job KeepAlive-loops with stdio: "ignore" upstream.
4. ocx status compares client fence ports against the public listen port only, ignoring unauthenticatedLoopbackListener.port, so it warns about the very port opencodex's own startup just wrote — and recommends ocx ensure, which on a non-loopback-hostname hub would repoint local clients at an address they cannot reach.

---

Repro

Hub with a non-loopback hostname, management ingress enabled, unauthenticatedLoopbackListener enabled, service installed and healthy and serving traffic.

# before: 100.76.170.81:10100, 127.0.0.1:10102, 127.0.0.1:10104 all LISTEN
ocx service repair
ocx start --port 10100     # also attempted; see note

Expected: repair rewrites the plist and leaves the job bootstrapped in gui/$uid and serving. On any failure it restores the previously running registration, or exits non-zero naming the teardown explicitly and the command needed to recover.

Actual: all three listeners gone.

• ~/Library/LaunchAgents/com.opencodex.proxy.plist present and plutil -lint-clean.
• ~/.opencodex/service-api-token rewritten with the same mtime as the plist — i.e. installLaunchd() definitely ran to at least :2370.
• launchctl list shows no com.opencodex.proxy.
• Nothing appended to ~/.opencodex/service.log after the eviction, because the job never started again. The outage leaves no log trace at all, which is what made it undiagnosable.
• ocx status: ❌ Proxy: not running / Service: installed, not loaded (launchd) / re-run 'ocx service repair' — i.e. it recommends the command that caused this.
• Every endpoint refuses connections, including remote clients arriving over Tailscale/cloudflared.

Recovery required exactly one command, which ocx never issues or suggests anywhere:

launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.opencodex.proxy.plist

That restored all three listeners from the unmodified plist and the unmodified config, first try, runs = 1, no crash loop. Nothing was wrong with the artifacts repair wrote — only the (re)load verb and the absent verification.

Note on ocx start --port: it is not the tear-down agent. handleStart never touches launchd, and with an explicit --port it sets killOcxHolders: false (src/cli/index.ts:210-221) and refuses to hop. It is implicated only as the command that then fails to recover the evicted state. The kill-then-repair-then-maybe-nothing sequence lives in the update paths — see "Related: update fallback" below.

---

Defect 1 — installLaunchd(): unconditional teardown, unverified reload, no rollback

ocx service repair on darwin is installLaunchd():

src/service.ts:3214-3217

if (platform === "darwin") {
  (deps.repairLaunchd ?? installLaunchd)();
  return;
}

src/service.ts:2356-2412

2360  recordOwnedConfigPath(getConfigDir(), serviceStatePath());
2362  writeServiceApiTokenFile();
2369  const launcher = stableLauncherEntry();
2370  writeServiceDefinitionFile(p, buildPlist(resolvedProxyEnv(), { launcher }), "utf8");
2385  const bootoutTarget = `${launchdGuiDomain()}/${LABEL}`;
2386  run(["bootout", bootoutTarget]);          // EVICTS the running hub
2387  let loaded = run(["load", "-w", p]);      // legacy, domain-implicit
2388  if (launchctlLoadFailed(loaded.stderr)) { run(["bootout", bootoutTarget]); loaded = run(["load", "-w", p]); }
2396  if (!loaded.ok || launchctlLoadFailed(loaded.stderr)) { throw new Error(...) }
2411  writeServiceInstallState("scheduler", launcher);

1a. Asymmetric domain targeting

bootout is addressed to gui/$uid via launchdGuiDomain() (src/service.ts:968-970). load -w is not addressed at all — legacy load acts on the caller's bootstrap domain. Any invocation from outside the Aqua session (ssh, a user/$uid-bootstrapped parent, a cron/daemon context, a tray helper in another bootstrap context) deletes the gui-domain job and registers nothing there. launchctl bootstrap gui/$uid <plist> is the verb that pairs with the bootout already in use.

The codebase knows these domains are disjoint — src/service-manager-probe.ts:334-352:

"Measured on macOS 27.0: the shipped agent answers 0 under gui/<uid> and 113 under user/<uid>. Asking only one leaves the other free to hold a job"

Confirmed on the affected machine: launchctl print gui/501/com.opencodex.proxy → exit 0; launchctl print user/501/com.opencodex.proxy → exit 113.

1b. Success is inferred from a stderr regex, never verified

launchctlLoadFailed() (src/service.ts:964-966) is /\b(?:Load|Bootstrap) failed\b/i over stderr. Deprecated load can exit 0 with empty stderr and do nothing — which satisfies the guard at :2396 — and it can also emit a diagnostic on a load that did work, which converts a success into a throw-and-leave-it-down.

The correct probe already exists and is already used by the sibling function:

• launchdJobMatchesPlist() (src/service.ts:979-991) runs launchctl print gui/$uid/<label> and compares the live arguments.
• startLaunchd() cross-checks with it before throwing (src/service.ts:2427-2447, call at :2434-2439).
• installLaunchd() never calls it — despite being the only path that evicts first.

Its own header comment at src/service.ts:973-978 says list "only proves domain membership" and print is "the only way to catch a load that silently no-op'd". The install path skips precisely that check.

1c. The teardown is unconditional

repairService() validates supported / conflict / installed (src/service.ts:3015-3026) and then never consults diag.running or diag.viable before delegating to a function whose own comment (:2373-2383) says it "EVICTS the running job". A repair of an already-healthy service is therefore guaranteed to interrupt service, with recovery depending entirely on 1a/1b.

1d. No settle delay, so the retry is useless

bootout is asynchronous. Both load attempts fire back-to-back (:2386-2389), so a job still exiting yields Load failed: 5: Input/output error twice and the "bounded retry" adds nothing. The Windows path has SCHEDULER_SETTLE_DELAYS_MS / settleDelay (src/service.ts:1477, :3162-3169); launchd has neither that nor a kickstart -k fallback.

1e. No plist backup and no rollback — unlike every other backend

writeServiceDefinitionFile (src/service.ts:2470-2496) overwrites in place with no backup, at :2370, before the eviction. If both loads fail, the function throws at :2396: the evicted job is not re-bootstrapped, the previous plist is gone, and writeServiceInstallState at :2411 is skipped. Terminal state = plist on disk, nothing in launchd, nothing listening.

Compare Windows, same file: repairService preserves and restarts the still-registered definition on failure (src/service.ts:3101-3186), with installFreshWindowsSchedulerSafely (:3878-3960) and rollbackWindowsSchedulerTaskOwnedByAttempt (:1301). The comment there states that leaving a previously runnable proxy stopped would make the user "worse off than before the repair". darwin has none of this.

1f. The failure never reaches the verification step

serviceCommand's repair branch (src/service.ts:4581-4589) has no try/catch, so a throw from installLaunchd escapes through src/cli/dispatch.ts:564-570 to the top level and reportServiceServing("repaired") is never reached. Even when it is reached, it only prints and sets process.exitCode = 1 (:790-819) — it never retries or reloads.

1g. The plist is silently rewritten from the caller's PATH

stableLauncherEntry() (src/service.ts:102-124) resolves ocx from the invoking shell's PATH. A repair run from a context without ocx on PATH rewrites a working launcher-form plist into the bun+CLI pair (expectedLaunchdCommand / cliEntry, :614-636, :73-100) — and then boots out the healthy job to load it. Nothing checks the new command is runnable, and there is no rollback if it isn't.

Related latent trap: expectedLaunchdCommand's fallback at :614-636 means that if ~/.opencodex/service-state.json is ever lost, status and start compare a healthy launcher-form job against the bun+CLI pair and report "launchd is running an OLDER plist" (:2444, :4409-4412), with startLaunchd throwing instead of no-op'ing.

1h. The plain install path has the same shape, with a verb that cannot work

serviceCommand install (:4608-4627) → installServiceSafely (:3816) → prepareServiceInstall (:3783-3814) → darwin ServiceInstallCleanupOps (src/service.ts:3628-3637):

status: () => { const listing = sh("launchctl list"); return listing.split("\n").some(line => line.includes(LABEL)) ? listing : null; },
stop:   () => { sh(`launchctl unload "${plistPath()}"`); },

That stop uses legacy unload, which installLaunchd's own comment at :2371-2375 says does not evict a gui-domain job. Teardown first, then installLaunchd with its own bootout; any throw surfaces as ❌ Service install cleanup failed + exitCode = 1 (:4625-4628) with nothing restored.

Suggested fix

1. Skip the bootout entirely when launchdJobMatchesPlist() already reports { loaded: true, matchesPlist: true } and the newly rendered plist is byte-identical — a repair of a healthy service should be a no-op, not an outage.
2. Replace run(["load", "-w", p]) with run(["bootstrap", launchdGuiDomain(), p]) — the verb that matches the bootout target — and add a settle delay between bootout and bootstrap, plus a kickstart -k gui/$uid/<label> fallback.
3. Require launchdJobMatchesPlist(expectedLaunchdCommand(installedServiceListenPort())) to confirm { loaded: true, matchesPlist: true } before writeServiceInstallState(...); treat anything else as failure regardless of stderr.
4. Back up the previous plist before overwriting at :2370; on terminal failure restore it, re-bootstrap, and only then throw — with an error that says the job was evicted and is currently down and names launchctl bootstrap gui/$uid <plist> as the manual remedy.
5. Wrap the repair branch at :4581 so the serving verification runs even on a thrown repair.

---

Defect 2 — the "loaded" bit is a swallowed, session-relative string grep

src/service.ts:2449

function statusLaunchd(): string { try { return sh(`launchctl list | grep ${LABEL} || true`); } catch { return ""; } }

src/service.ts:4321-4330 (diagnoseService(), darwin branch) — consumed by ocx status at src/cli/status.ts:216-222

const installed = existsSync(plistPath());
const running = installed && Boolean(statusLaunchd());
...
: running ? `installed and loaded (launchd; ${diagnostics})`
          : `installed, not loaded (launchd; ${diagnostics})`;

• Session-relative domain. Bare launchctl list enumerates the caller's own domain, not gui/$uid, while every mutating/inspecting call in the file targets gui/$uid explicitly. From a non-Aqua session a healthy gui-domain job is invisible → installed, not loaded + "re-run ocx service repair" for a hub that is serving traffic. Following that advice triggers Defect 1.
• Every exit code is discarded, twice. || true makes the shell exit 0 whatever launchctl or grep did, and catch { return "" } eats the rest. A bootstrap-server/EPERM error, a missing grep, or an execSync maxBuffer overflow on a host with thousands of agents all collapse into the same empty string as genuine absence. There is no tri-state: diagnoseService has no "unknown", unlike the Windows probe (probeWindowsSchedulerTask → "unknown", src/service.ts:1171) and unlike runLaunchctl, which deliberately preserves the numeric status specifically to separate 112 (no such domain) from 113 (no such service) (src/service.ts:944-956).
• Its own twin fails the opposite way. The install-cleanup probe for the identical question omits || true and therefore throws (src/service.ts:3630-3633). Two probes of one fact with opposite failure semantics.
• Unanchored regex — the inverse false positive. grep ${LABEL} is an unquoted pattern with unescaped dots matched anywhere on the line, so com.opencodex.proxy.helper, or a foreign label matching com?opencodex?proxy, reads as ours → "installed and loaded" for a job that is not the proxy. Domain membership is also not serving: a job bootstrapped from an older plist, or listed but never bound, also reads as "loaded".
• The right probes exist and are not wired in. launchdJobMatchesPlist (:979) is used only for the stale-plist hint line in serviceStatusReport (:4399-4412); inspectLaunchd (src/service-manager-probe.ts:331, queries both domains and is 112/113-aware) is used only for ownership. Neither feeds diagnoseService.
• Not a locale problem. The locale-sensitive decoding in this file is Windows-only (decodeSchtasksOutput :999, windowsSchedulerCsvIncludesTask :1149). Domain scope and swallowed exit status are the causes.

Blast radius beyond the cosmetic string. enabled, running and viable all come from that one bit (:4330), so isServiceViable() (:4175) returns false for a healthy loaded job — and that is what drives the update fallback at src/update/index.ts:372-380, which then treats a successful repair as non-viable and starts a competing proxy.

Why it reads as self-contradictory. src/cli/status.ts:216-222 cross-checks the summary against the live health probe and appends "— registered but NOT serving" only when service.installed && !live. So when the probe succeeds and the grep fails, status prints ✅ Proxy: running immediately above a bare Service: installed, not loaded (launchd) with no reconciling suffix — which is exactly the originally reported symptom.

Honest scoping of this report: in the incident above the job genuinely was not loaded, so that particular message was accurate. The defect is the earlier, reproducible complaint — installed, not loaded printed while launchctl list showed a live pid — which 2a/2b explain, and which is dangerous precisely because the remedy it prints is Defect 1. This is also already recorded as known-but-deferred in the repo: devlog/_plan/260910_post249_round2/_research/4141.md ("statusLaunchd → print gui/<uid>/<label> … changes running / isServiceViable", filed under POLICY (not mechanical)) and devlog/_plan/260910_post249_round2/040_4141_launchctl_bootout.md:58 ("statusLaunchd stays launchctl list | grep").

Suggested fix

Drive the darwin branch from runLaunchctl(["print", ${launchdGuiDomain()}/${LABEL}]) (or inspectLaunchd), keep the 112/113 distinction, query both gui/$uid and user/$uid, and report four states: not installed / not loaded / loaded from a stale plist / loaded from the current plist — plus an explicit unknown that does not recommend ocx service repair and does not make isServiceViable() false.

---

Defect 3 — an occupied secondary listener port is fatal and misdiagnosed

Both secondary listeners bind inside one startup transaction, after the public listener, with no prior availability probe:

src/server/index.ts:2405-2448

server = Bun.serve({ ...serveOptions, port: listenPort, hostname: bindHost });
if (loopbackListenerPort !== null) { try { loopbackServer = Bun.serve({ ..., port: loopbackListenerPort, hostname: "127.0.0.1" }); } catch (error) { void server.stop(true); throw error; } }
if (managementIngressPort !== null) { try { managementIngressServer = Bun.serve({ ... }); } catch ...

The pre-bind guard only stops the public listener from stealing the loopback port (src/cli/index.ts:193-203, reservedPort in src/server/ports.ts:70-100). Nothing checks that 10104 or 10102 is free. When one is held by a foreign process the transaction rolls back and rethrows, and handleStart cannot tell which listener failed:

src/cli/index.ts:367-380

if (!isAddrInUse(err) || attempt >= 2) throw err;
if (requestedPort !== undefined) { /* waits for the PUBLIC port to free */ }

So with --port (the launchd case) it waits on the wrong port, retries twice, then prints ❌ Port <public> stayed busy; refusing to hop and exits 1 — naming a port that was never the problem. Under KeepAlive: true that is a loaded job that never serves, relaunched forever.

This machine hit exactly that, historically, when the management ingress was configured on 10101 (held by an unrelated local service):

error: Failed to start server. Is port 10101 in use?
 code: "EADDRINUSE"
      at startServer (src/server/index.ts:2433)
      at handleStart (src/cli/index.ts:360)

repeated until the port was changed by hand. ocx status reported only installed, not loaded throughout.

Related: the schema at src/config.ts:1195-1198 is a discriminated union with .optional().catch(undefined), so a malformed hand edit silently disables the loopback listener instead of erroring. The relationship checks (loopbackListenerPortError, src/config.ts:2772-2797; ingress cross-check :2838-2841) are write-time only and cover internal collisions only — a foreign holder is never considered, at write time or at startup.

Suggested fix

Probe unauthenticatedLoopbackListener.port and hub.managementIngress.port for availability in the install / repair / start pre-flight and fail with the holding process named; and tag the rethrown bind error with which listener failed so handleStart stops blaming (and waiting on) the public port. A config error that can never succeed on retry should not be handed to KeepAlive.

---

Defect 4 — false "fence drift" warning against the loopback listener, with a destructive remedy

With the hub healthy, ocx status prints:

Grok Build config points at port 10104, but the proxy is on 10100;
grok turns will retry against a closed port. Run 'ocx ensure' to repoint it.

10104 is the unauthenticatedLoopbackListener port and it answers GET /v1/models with HTTP 200. opencodex's own startup wrote that value: the same service.log startup block contains both ⚠️ Unauthenticated loopback listener active on http://127.0.0.1:10104 and + Grok Build config updated (~/.grok/config.toml).

• src/cli/status.ts:283-294 calls grokFenceEndpointDrift(readGrokStatus(), health.ok ? listen.port : undefined) — public listen port only.
• src/grok/status.ts:119-133 then flags any mismatch: if (!Number.isFinite(fencePort) || fencePort === livePort) return null;

Neither side knows about config.unauthenticatedLoopbackListener.port. On a hub whose public bind is a Tailscale address, 10100 is unreachable from loopback by design — that is the entire reason the 10104 listener exists — so running the suggested ocx ensure would repoint ~/.grok/config.toml, and likely ~/.codex/config.toml's openai_base_url, at an address local clients cannot connect to.

Suggested fix

Treat the locally reachable set as { listen.port } ∪ { unauthenticatedLoopbackListener.port when enabled } and warn only when the fence port is in neither. Apply the same rule wherever ocx ensure / ocx sync choose a client base_url.

---

Secondary: assertNotAdminToken under KeepAlive is invisible

Earlier in the same log, the installed plist exported a service token that collided with the admin token, and startup threw ~20 times in a row:

error: OPENCODEX_API_AUTH_TOKEN is a management (admin) token. The service exports it as the
data-plane secret, which fences the whole management API closed and makes every ocx management
command fail with 503. Unset OPENCODEX_API_AUTH_TOKEN, or set it to a distinct data-plane key,
then rerun the install.
      at assertNotAdminToken (...)
      at handleStart (src/cli/index.ts:231)
      at dispatchCommand (src/cli/dispatch.ts)

(token values redacted)

To be precise about what is and is not wrong here:

• The ordering relative to teardown is correct, and this was not the cause of the outage above. serviceCommand runs assertServiceAuthEnvironment() before repairService() (src/service.ts:4582-4583) and before install (:4610), and inside installLaunchd the token write (:2362) precedes the bootout (:2386). A collision aborts before anything is evicted.
• Nothing in the product can mint a colliding data-plane token. writeServiceApiTokenFile (:479-500) only copies process.env.OPENCODEX_API_AUTH_TOKEN; the sole minter is the management token (src/server/management-auth.ts:154), and detection is prefix/byte-equality (src/lib/admin-secrets.ts:28-50). The plist never embeds the token — it cats the file at launch (src/service.ts:591-606).
• Where it does bite is src/cli/index.ts:277-283: a legacy ~/.opencodex/service-api-token holding an admin token makes every supervised ocx start throw at boot. Under KeepAlive that is a crash loop in which the job is loaded, nothing ever listens, ocx status says only installed, not loaded, and — because the update fallback spawns with stdio: "ignore" (src/update/index.ts:408-417) — the message is invisible. src/cli/doctor.ts:188-207 already documents this legacy state; validating the on-disk token during install/repair would surface it in the command the operator is actually running.

---

Related: the update fallback amplifies all of the above

The "killed the proxy, then failed to bring it back" shape also exists outside repair, and it depends on the broken viability bit from Defect 2:

• src/update/index.ts:349-357 — reclaim with killOcxHolders: capturedListen.oldPid != null, then service repair; on failure serviceViable = isServiceViable() (:376) → either :386-396 "refusing to hop" (nothing started) or :408-417 a fire-and-forget spawn(["start","--port",…], { detached, stdio: "ignore" }) with no health wait, so a child that dies leaves nothing listening and no diagnostics anywhere.
• src/update/job.ts:1111-1122 — same downstream with killOcxHolders: true, killAllOcxOnPort: true.
• src/lib/process-control.ts:205-215 — post-stop reclaim with killOcxHolders: !!(stoppedPid …).

---

Not a factor: .opencodex-uninstall.json

Worth stating because the file's name and mtime make it look like an uninstall ran. It is the owned-path manifest (CONFIG_UNINSTALL_MANIFEST, src/lib/config-ownership.ts:18; writer recordOwnedConfigPath, :267-289), refreshed by installLaunchd at :2360 — i.e. before the bootout, so a manifest failure aborts before any teardown. The manifest on the affected machine is intact, with ownerId present and 38+ paths. No uninstall path ran.

---

Impact

On a hub topology (runtimeRole: "hub", non-loopback hostname), ocx service repair — the command ocx status itself recommends, sometimes on the false premise of Defect 2 — can take the proxy, the management ingress and the loopback client listener down simultaneously, report no failure, write no log line, and require a launchctl bootstrap the CLI never prints. Remote clients arriving over Tailscale/cloudflared go down with it, and because ~/.opencodex/service.log gains nothing after the eviction, there is no artifact to diagnose from.

[lidge-jun]

리뷰 · 우선순위 74 / 80

이 이슈는 macOS에서 ocx service repair 한 번이 살아 있던 LaunchAgent 허브를 통째로 내리는데도, CLI는 성공처럼 보이거나 같은 repair를 다시 권하는 상태를 남긴다는 고심각도 버그 리포트다. 제목만 읽으면 “서비스 관리 문구”처럼 보이지만, 본문이 말하는 실제 피해는 공개 프록시·management ingress·루프백 리스너가 한꺼번에 사라지는 운영 중단이다. 지금 dev HEAD df65a2587(패키지 2.51.0, 최근에는 L2/L3 레인 #4230/#4231이 들어간 상태)에서도 인용한 코드 경로는 그대로다. src/service.ts의 installLaunchd()는 여전히 bootout gui/$uid/<label>으로 기존 잡을 먼저 비운 뒤, 도메인을 명시하지 않는 레거시 launchctl load -w만 다시 치고, stderr에 “Load/Bootstrap failed” 문자열이 없으면 성공으로 본다. 같은 파일의 launchdJobMatchesPlist()와 startLaunchd()는 이미 launchctl print로 “진짜로 현재 plist가 올라갔는지”를 검사하는데, 퇴거를 하는 쪽인 install/repair만 그 검사를 건너뛴다. 이슈가 재현에서 확인한 복구 명령 launchctl bootstrap gui/$(id -u) ~/Library/LaunchAgents/com.opencodex.proxy.plist도 HEAD 어디에도 자동 제안·자동 실행되지 않는다.

왜 지금 dev에서 점수가 높은가. 현재 메인은 카탈로그/계정 풀 레인이지만, hub 런타임의 “repair가 서비스를 죽인다”는 플랫폼 결함은 L2/L3보다 먼저 터지면 그 레인 검증 자체를 못 한다. 특히 Tailscale-only hostname + managementIngress + unauthenticatedLoopbackListener를 켠 허브(이슈 환경 그대로)는 remote client까지 한 번에 끊긴다. 본문이 정리한 네 갈래는 서로 연쇄한다. (1) installLaunchd의 무조건 퇴거+검증 없는 reload가 실제 장애이고, (2) diagnoseService가 launchctl list | grep으로 “loaded”를 만들면 건강한 gui-domain 잡도 “not loaded”로 보여 repair로 유도하고, (3) 루프백/management 포트 충돌을 public 포트 충돌처럼 오진하면 KeepAlive 크래시 루프가 되고, (4) ocx status의 fence 비교가 public listen만 보면 자기 설정이 쓴 루프백 포트를 경고한다. Windows 쪽 repair는 실패 시 이전 정의를 보존·재기동하는 프로토콜이 있는데 darwin만 “먼저 내리고 검증은 안 하는” 비대칭이다. 이슈 본문의 라인 인용(대략 src/service.ts:2356-2412, :2449, :3012-3027, :4318+)은 HEAD에서 줄 번호가 크게 어긋나지 않고, src/service-manager-probe.ts의 양쪽 도메인 조사가 이미 있다는 점도 맞다. 즉 “설계 문서”가 아니라 지금 트리에서 고칠 수 있는 구체 결함이다.

다만 이슈는 메인(6d3ad12e3) 기준으로 적혀 있고, 아직 대응 PR은 없다. 우선순위 74는 “당장 머지할 PR이 있다”가 아니라 “허브 운영자가 repair/update 경로로 재현 가능한 완전 중단이고, 수정 방향이 이미 코드 안에 있는 헬퍼로 명확하다”는 뜻이다. types.ts/config.ts 스플릿과는 무관하고, 닫을 중복도 지금 검색상 없다. 고칠 때는 installLaunchd 한 함수만 손보면 끝나는 것처럼 보여도, diagnose/status/update fallback(isServiceViable)까지 같이 안 고치면 “죽은 뒤 status가 다시 repair를 권하는” 순환이 남는다.

라인 2386 - bootout은 gui/$uid를 명시하는데 바로 다음 load -w는 도메인 비명시라, Aqua 밖(ssh·cron·다른 bootstrap)에서 호출하면 잡을 지우고 다시 못 올린다.
라인 2387-2395 - 레거시 load가 exit 0 + 빈 stderr로 no-op해도 성공으로 통과하고, bootout 직후 settle 없이 재시도해서 “Load failed: 5” 레이스를 못 잡는다.
라인 2352-2354 / 2411 - 주석이 스스로 launchdJobMatchesPlist를 안 쓴다고 적어 두고, 성공 판정 뒤 writeServiceInstallState만 쓴다. 퇴거 경로에서 검증을 빼면 이슈의 “로그도 안 남는 다운”이 된다.
라인 2449 - statusLaunchd가 launchctl list | grep … || true라서 실패·다른 도메인·오탐이 전부 “없음/있음” 한 비트로 뭉개진다.
경로 src/service.ts diagnoseService(darwin) - 그 한 비트가 running/enabled/viable로 퍼져 isServiceViable()와 update fallback이 건강한 서비스에도 경쟁 프록시를 띄울 수 있다.
경로 repairService → installLaunchd - repair는 installed만 보고 running/viable를 보지 않은 채 퇴거형 install로 넘어가, 이미 건강한 허브도 중단시킨다.
경로 Windows 대비 - 같은 파일의 스케줄러 repair는 실패 시 보존/롤백이 있는데 darwin installLaunchd에는 plist 백업·재bootstrap·에러에 bootstrap 복구 안내가 없다.

메인테이너의 판단이 필요한 지점

• 이 이슈를 현재 7레인(L2/L3 등) 사이 긴급 핫픽스로 끼울지, 아니면 platform 스택 전용 작은 PR로 레인 사이에 독립 랜딩할지.
• repair가 “이미 현재 plist와 일치하면 no-op”이어야 하는지, 아니면 의도적으로 항상 reload인지(지금 동작은 후자인데 운영 비용이 너무 큼).
• load -w를 버리고 bootstrap gui/$uid로 통일할지, 구형 macOS 호환을 위해 둘 다 시도할지.
• Defect 3·4(포트 오진, status fence)를 같은 PR에 묶을지, Defect 1+2만 먼저 막을지.

너의 추천

• 닫지 말고 연 채로 두고, Defect 1+2만 먼저 고치는 작은 darwin PR을 연다. installLaunchd에서 (a) 새 plist와 launchdJobMatchesPlist가 이미 일치하면 bootout 생략, (b) reload는 bootstrap+settle(+필요 시 kickstart), (c) install state 쓰기 전에 print 검증 필수, (d) 실패 시 이전 plist 복구 후 bootstrap 재시도하고 에러에 복구 명령을 명시. diagnose의 loaded 판정은 inspectLaunchd/launchdJobMatchesPlist로 교체해 status가 다시 repair로 몰아가지 않게 한다. Defect 3·4는 follow-up 이슈/PR로 분리해도 된다. 현재 open PR 중 이 이슈를 닫는 후보는 없다.

이 댓글은 grok-bot이 작성했습니다

[lidge-jun]

Follow-up: on a non-loopback hub, most local clients ignore unauthenticatedLoopbackListener and dial 127.0.0.1:<public port>

Found while recovering the same hub. With hostname: "100.76.170.81" (Tailscale-only bind) and unauthenticatedLoopbackListener: { enabled: true, port: 10104 }, only the ocx sync writers honor the loopback listener (Codex, Grok, aside, opencode via opencodeProxyBaseUrl). Every other local integration hardcodes the public port on loopback, which does not exist on such a hub:

Site	What it writes / dials
src/cli/claude.ts:150, :194 (buildClaudeEnv)	ANTHROPIC_BASE_URL=http://127.0.0.1:${port} — route = port at :621 is the public port
src/cli/claude.ts:349 (fetchClaudeCodeState)	http://127.0.0.1:${port}/api/claude-code — fails, so ocx claude falls back to native launch ("Codex integration OFF / native" in the log)
src/claude/desktop-3p.ts:338	inferenceGatewayBaseUrl: http://127.0.0.1:${port} (Claude Desktop)
src/claude/gateway-cache.ts:105	same
src/server/system-env.ts:88 (ownedBaseUrl), system-env-shell.ts:84	launchctl setenv / shell export of ANTHROPIC_BASE_URL=http://127.0.0.1:${port}
src/server/management/cursor-integration-routes.ts:110	http://127.0.0.1:${port}/v1
src/server/management/api-access.ts:107	http://127.0.0.1:${port}/v1
src/vision/routed-describe.ts:79	http://127.0.0.1:${port}

Net effect: ocx sync reports success and Codex works, while ocx claude, Claude Desktop, Cursor, the system-env injection and the vision helper all point at a closed port — the "Codex works but nothing else does" symptom. grep -rn 'unauthenticatedLoopbackListener' src/cli/claude.ts src/claude src/server/system-env.ts src/server/management/cursor-integration-routes.ts returns nothing.

Suggested fix: one resolver (localClientBaseUrl(config, port)) that returns 127.0.0.1:<loopbackListener.port> when the listener is enabled, else 127.0.0.1:<port> only if isLoopbackHostname(config.hostname), else the public hostname:port plus the admission header — and route all eight sites through it.

Follow-up 2: hub-role gate reports "Codex integration is OFF" when the toggle is ON

src/codex/desired-state.ts:79-92 — shouldSyncCodexOnStart = localClientSyncAllowed(config) && codexIntegrationEnabled(config), where localClientSyncAllowed is false for runtimeRole: "hub" unless the loopback listener is enabled. The gate itself is a reasonable decision, but every caller reports the skipped sync with the toggle's message:

• ocx sync → Codex integration is OFF; catalog and models cache refreshed, Codex config untouched.
• ocx ensure → Codex integration OFF; startup left Codex native. and then removes the managed Grok block (↩️ Removed the opencodex managed block from Grok config.)
• ocx restore back → setIntegrationEnabled("codex", true) succeeds (src/cli/dispatch.ts:120), then syncModelsToCodex returns skipped, so it prints Codex integration is OFF; restore back did not change Codex. Retry after the competing integration change finishes. (:126) — there is no competing change; clientIntegrations is null.

Repro: runtimeRole: "hub", hostname: "127.0.0.1", loopback listener disabled, clientIntegrations absent → all three commands claim the toggle is off and restore back tells the operator to wait for a phantom writer. Nothing in the output mentions the hub gate or the loopback listener.

Suggested fix: give the skip a distinct reason ("hub-gated") and message: "This is a hub; local Codex/Grok/Claude sync is disabled unless unauthenticatedLoopbackListener is enabled." ocx restore back should surface that instead of the conflict text, and ocx ensure should not strip the Grok block on a hub-gated skip when the operator did not turn Grok off.

Recovery notes for this host (for the record)

• Reverted hostname to 127.0.0.1 (the value it had before the hub migration) and kept unauthenticatedLoopbackListener on 10104 so the hub gate still allows local sync. Hardcoded clients now hit 127.0.0.1:10100, sync-managed clients hit 10104; remote GUI is unaffected because it already goes through Tailscale Serve → 127.0.0.1:10102.
• The usage log showed zero non-loopback admissions, so the Tailscale-only public bind was serving no remote client.
• Restarted via launchctl kickstart -k gui/$uid/com.opencodex.proxy, not ocx service repair.

[Ingwannu]

The new local-client followup needs two destination contracts, not one base URL substituted everywhere. At dev babb764, loopbackRouteAllowed is an explicit data-plane allowlist; it does not admit /api/claude-code or /v1/messages. Moving fetchClaudeCodeState or every Claude caller to that port alone would therefore still fail even though Codex Responses works.

Separate (1) authenticated local management discovery/state from (2) the inference endpoint supported by each client wire. Resolve management through its intended authenticated ingress, and route inference only to an endpoint that admits that exact wire. Do not add /api/* to the unauthenticated listener or copy an admin credential into exported client configuration to hide the mismatch.

The launchd repair outage and the hub-gated-sync/managed-block removal followups should retain separate regression cases. Mock domain-qualified service commands and both client destinations in isolated homes; a host restart is not needed to validate those decisions. I have not run repair/ensure/sync or changed any local configuration.