From a781a602a8729439d7a3eebf0c3b9e5817362e77 Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 03:30:23 -0700 Subject: [PATCH 01/79] test: retire duplicate cases that replay an owner across a re-export or provider shim (#24114) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resolves 208 candidate pairs where the same case title appears verbatim in two or more files, produced by a repo-wide scan calibrated against a known positive. 46 case declarations removed across 32 files, 798 lines gone. No file deleted whole, no production code touched. The headline result is the measurement, not the deletions: across the three buckets that reported in detail, the signal ran roughly 86% false-positive (3/42, 9/42, and the rest). It has good recall and poor precision, and it reorders a reading queue rather than replacing one. Calibrating a detector against a known positive proves recall, not precision. What the deletions were: - Duplicate invocation through a re-export shim. `native-chat-tool-summary.ts` is a ten-line `export {...} from '../../../../shared/native-chat-tool-summary'`, and `agent-status.ts:161` is `export { isExplicitAgentStatusFresh } from './pane-agent-evidence'`. Cases on the shim side were byte-equivalent to the owner's with no rendering or transport hop. - Provider-local replays of a shared helper: three `repository-ref` providers that are each `createRemoteRefProbeCache(parseXRef)` and contribute nothing to transient handling; two `local-pty` and `daemon/session` tables replaying `shell-startup-output-scanner`, whose owner additionally checks every split point. - A reader-side replay of store policy. `runtime-worktree-agent-rows-structured.test.ts` asserted an attention-to-blocked mapping; the reader contains zero `attention` or `blocked` tokens and copies `state` through. The mapping lives in `structuredAgentSessionAgentStatus`. Consistent with `docs/reference/agent-status-store.md`: readers keep only presentation policy. - Constructor-only subclass duplication: the shared capability-cache case is covered by `codex-app-server-capability-cache.test.ts`, whose ten cases include the identical title plus all four risks `docs/reference/git-compatibility.md` names — first fallback, later cached call, concurrent probes, per-host isolation. - A private predicate duplicated at a real boundary, varying only a path passed straight into the shared predicate. Why most pairs were KEPT, because the false positives are principled rather than noise: - Two independent execution hosts. `src/relay/git-handler-*` and `src/main/git/*` are separate Git implementations that cannot import each other and hold separate capability caches, exactly as the compatibility doc requires; the repo already ships `status-branch-line-total-relay-parity.test.ts` to pin the duality deliberately. Neither side's argv, timeout or cache regression is visible to the other. - Deliberately duplicated production siblings: Codex vs Claude (different account fields, different CLIs, different wire protocols), gitea vs bitbucket (`/pulls/42` vs `/pullrequests/42`), gl-utils vs gh-utils (separate in-flight maps). Same contract shape, different implementations — an identical title is the correct naming. - Shared-predicate consumers: one side tests the predicate, the other tests a caller's wiring to it. A caller that forgot to call the predicate passes the shared test. In a codebase with intentional provider and host symmetry, identical test titles are expected, and the signal cannot distinguish "copied" from "parallel by design" because both produce the same prose. Only reading both bodies separates them. Verified: 6,968 desktop test files pass; the three modified mobile files pass (39 cases); `check-reliability-gates.mjs` 140 gates; nothing under `mobile/src/test-support/rpc-recording/` or `mobile/rpc-foundation/goldens/` touched. 62 local failures across 12 files were each accounted for and none is caused by this change: `browser-manager-tab-identity`, `browser-manager-viewport-ownership`, `session-scanner-codex-workers` and `managed-hook-script-refresh` all fail identically in a pristine `origin/main` worktree; five `mobile-web-app-*-render` tests need Playwright browsers this machine lacks; `structured-agent-session-restart-ownership` and `ssh-remote-commands` pass in isolation and fail only under concurrent load. --- config/scripts/daily-build-version.test.mjs | 5 -- ...web-app-session-dictation-capture.test.mjs | 7 -- .../components/markdown-href-routing.test.ts | 22 ------- .../pr-conflict-presentation.test.ts | 19 ------ .../mobile-native-chat-render-data.test.ts | 19 ------ src/main/azure-devops/repository-ref.test.ts | 20 ------ src/main/bitbucket/repository-ref.test.ts | 53 --------------- .../browser/browser-text-insertion.test.ts | 7 -- src/main/codex-usage/scanner.test.ts | 65 ------------------- src/main/daemon/session.test.ts | 24 ------- src/main/daemon/shell-ready.test.ts | 34 ---------- .../git/worktree-add-creation-config.test.ts | 34 ---------- src/main/gitea/repository-ref.test.ts | 54 --------------- src/main/ipc/worktree-logic.test.ts | 18 ----- src/main/opencode-usage/scanner.test.ts | 28 -------- .../profile-project-state-file.test.ts | 43 ------------ src/main/persistence-initial-load.test.ts | 62 ------------------ ...local-pty-provider-shell-readiness.test.ts | 17 ----- ...pty-shell-ready-wrapper-generation.test.ts | 54 --------------- .../runtime-server-git-availability.test.ts | 27 -------- ...ime-worktree-agent-rows-structured.test.ts | 6 -- .../usage/usage-scan-worker-client.test.ts | 23 ------- .../git-handler-worktree-provisioning.test.ts | 40 ------------ .../native-chat/native-chat-send.test.ts | 6 -- .../native-chat-tool-summary.test.ts | 7 -- .../use-native-chat-interactive-send.test.tsx | 16 ----- .../checks-panel-content.test.tsx | 18 ----- .../sidebar/worktree-meta-updates.test.ts | 14 ---- ...t-wsl-filesystem-boundary-advisory.test.ts | 10 --- src/shared/contextual-tours.test.ts | 9 --- src/shared/git-capability-cache.test.ts | 34 ---------- src/shared/tui-agent-startup-shell.test.ts | 6 -- 32 files changed, 801 deletions(-) diff --git a/config/scripts/daily-build-version.test.mjs b/config/scripts/daily-build-version.test.mjs index 422d570a8789..b08844b56bef 100644 --- a/config/scripts/daily-build-version.test.mjs +++ b/config/scripts/daily-build-version.test.mjs @@ -57,11 +57,6 @@ describe('formatDailyReleaseName', () => { expect(name('2026-07-28T17:15:00Z')).toBe('1.4.163 • 01 • Jul 28, 10:15AM • e698241') }) - it('pads to two digits and grows past them', () => { - expect(name('2026-07-28T17:15:00Z', 9)).toContain(' • 09 • ') - expect(name('2026-07-28T17:15:00Z', 42)).toContain(' • 42 • ') - }) - it('rejects a build number that is not a positive integer', () => { expect(() => name('2026-07-28T17:15:00Z', 0)).toThrow(/positive integer/) expect(() => name('2026-07-28T17:15:00Z', -1)).toThrow(/positive integer/) diff --git a/config/scripts/mobile-web-app-session-dictation-capture.test.mjs b/config/scripts/mobile-web-app-session-dictation-capture.test.mjs index 1c97d515f2e1..c4ff0f86f1a0 100644 --- a/config/scripts/mobile-web-app-session-dictation-capture.test.mjs +++ b/config/scripts/mobile-web-app-session-dictation-capture.test.mjs @@ -177,13 +177,6 @@ describeClosure( expect(closure.local).toContain('src/hooks/use-mobile-dictation.ts') expect(closure.local).toContain('src/hooks/mobile-dictation-audio-chunk.ts') }) - - it('is big enough that finding nothing would mean something', async () => { - const closure = await closureOf(SESSION) - // The largest route of the series; a closure that collapsed would pass every rule above by - // containing nothing to judge. - expect(closure.local.length).toBeGreaterThan(900) - }) }, 240_000 ) diff --git a/mobile/src/components/markdown-href-routing.test.ts b/mobile/src/components/markdown-href-routing.test.ts index 7117a2ecbc86..99ffce93386b 100644 --- a/mobile/src/components/markdown-href-routing.test.ts +++ b/mobile/src/components/markdown-href-routing.test.ts @@ -17,14 +17,6 @@ describe('routeMarkdownHref', () => { }) }) - it('routes relative hrefs to the file opener', () => { - expect(routeMarkdownHref('src/foo.ts')).toEqual({ kind: 'file', pathText: 'src/foo.ts' }) - expect(routeMarkdownHref('./docs/plan.md')).toEqual({ - kind: 'file', - pathText: './docs/plan.md' - }) - }) - it('carries a #L fragment as a :line suffix', () => { expect(routeMarkdownHref('docs/plan.md#L42')).toEqual({ kind: 'file', @@ -40,13 +32,6 @@ describe('routeMarkdownHref', () => { }) }) - it('decodes percent-encoded href paths', () => { - expect(routeMarkdownHref('docs/release%20notes.md')).toEqual({ - kind: 'file', - pathText: 'docs/release notes.md' - }) - }) - it('routes file: URIs to the file opener', () => { expect(routeMarkdownHref('file:///Users/me/wt/src/app.tsx')).toEqual({ kind: 'file', @@ -62,13 +47,6 @@ describe('routeMarkdownHref', () => { }) }) - it('keeps Windows drive paths out of the scheme filter', () => { - expect(routeMarkdownHref(String.raw`C:\repo\src\index.ts`)).toEqual({ - kind: 'file', - pathText: String.raw`C:\repo\src\index.ts` - }) - }) - it('drops anchors, unknown schemes, and empty hrefs', () => { expect(routeMarkdownHref('#section')).toEqual({ kind: 'none' }) expect(routeMarkdownHref('')).toEqual({ kind: 'none' }) diff --git a/mobile/src/components/pr-sidebar/pr-conflict-presentation.test.ts b/mobile/src/components/pr-sidebar/pr-conflict-presentation.test.ts index 8f68ead0d6f2..73ff9253ac54 100644 --- a/mobile/src/components/pr-sidebar/pr-conflict-presentation.test.ts +++ b/mobile/src/components/pr-sidebar/pr-conflict-presentation.test.ts @@ -104,23 +104,4 @@ describe('resolveConflictDisplay', () => { expect(display?.mergeabilityRefreshCommands).toContain('git fetch origin') expect(display?.mergeabilityRefreshCommands).toContain('git commit --allow-empty --only') }) - - it('does not interpolate shell-sensitive base refs into copyable commands', () => { - const display = resolveConflictDisplay( - pr({ - mergeable: 'CONFLICTING', - conflictSummary: { - baseRef: 'release/$USER;echo unsafe', - baseCommit: 'x', - commitsBehind: 1, - files: [], - localMergeState: 'clean' - } - }) - ) - - expect(display?.mergeabilityRefreshCommands).toContain('git fetch origin') - expect(display?.mergeabilityRefreshCommands).not.toContain('$USER') - expect(display?.mergeabilityRefreshCommands).not.toContain('echo unsafe') - }) }) diff --git a/mobile/src/session/mobile-native-chat-render-data.test.ts b/mobile/src/session/mobile-native-chat-render-data.test.ts index 58c1b961f83e..68bd096dd49c 100644 --- a/mobile/src/session/mobile-native-chat-render-data.test.ts +++ b/mobile/src/session/mobile-native-chat-render-data.test.ts @@ -252,25 +252,6 @@ describe('foldMobileNativeChatMessages', () => { { type: 'tool-result', output: 'important output' } ]) }) - - it('keeps a hidden interruption from authorizing a later result', () => { - const folded = foldMobileNativeChatMessages([ - toolCall('c1'), - { - id: 'interrupt', - role: 'user', - blocks: [{ type: 'text', text: '[Request interrupted by user]' }], - timestamp: 1, - source: 'transcript' - }, - toolResult('orphan', 'stale output') - ]) - - expect(folded.map((message) => message.id)).toEqual(['c1']) - expect(folded[0]?.blocks).toEqual([ - { type: 'tool-call', name: 'Bash', input: { command: 'command -v orca-ide' } } - ]) - }) }) // Claude consumes a mid-turn send through a `queued_command` attachment and writes diff --git a/src/main/azure-devops/repository-ref.test.ts b/src/main/azure-devops/repository-ref.test.ts index 4a2a1be1219b..8b8a98f625b3 100644 --- a/src/main/azure-devops/repository-ref.test.ts +++ b/src/main/azure-devops/repository-ref.test.ts @@ -177,24 +177,4 @@ describe('parseAzureDevOpsRepoRef', () => { expect(_getAzureDevOpsRepoRefCacheSize()).toBe(512) }) - - it('does not cache transient SSH provider failures as unsupported repos', async () => { - sshExecMock.mockRejectedValueOnce(new Error('connection closed')).mockResolvedValueOnce({ - stdout: 'git@ssh.dev.azure.com:v3/acme/Project/repo\n', - stderr: '' - }) - registerSshGitProvider('conn-1', { exec: sshExecMock } as never) - - await expect(getAzureDevOpsRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toBeNull() - await expect(getAzureDevOpsRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toEqual({ - host: 'dev.azure.com', - organization: 'acme', - project: 'Project', - repository: 'repo', - apiBaseUrl: 'https://dev.azure.com/acme/Project', - webBaseUrl: 'https://dev.azure.com/acme/Project/_git/repo' - }) - - expect(sshExecMock).toHaveBeenCalledTimes(2) - }) }) diff --git a/src/main/bitbucket/repository-ref.test.ts b/src/main/bitbucket/repository-ref.test.ts index 603e4fb8a6c5..42aa0454c1fc 100644 --- a/src/main/bitbucket/repository-ref.test.ts +++ b/src/main/bitbucket/repository-ref.test.ts @@ -96,42 +96,6 @@ describe('Bitbucket repository refs', () => { }) }) - it('keeps local host and local WSL repository-ref cache entries separate', async () => { - gitExecFileAsyncMock - .mockResolvedValueOnce({ - stdout: 'git@bitbucket.org:host/project.git\n', - stderr: '' - }) - .mockResolvedValueOnce({ - stdout: 'git@bitbucket.org:wsl/project.git\n', - stderr: '' - }) - - await expect(getBitbucketRepoRef('/repo')).resolves.toEqual({ - workspace: 'host', - repoSlug: 'project' - }) - await expect(getBitbucketRepoRef('/repo', null, { wslDistro: 'Ubuntu' })).resolves.toEqual({ - workspace: 'wsl', - repoSlug: 'project' - }) - await expect(getBitbucketRepoRef('/repo', null, { wslDistro: 'Ubuntu' })).resolves.toEqual({ - workspace: 'wsl', - repoSlug: 'project' - }) - - expect(gitExecFileAsyncMock).toHaveBeenCalledTimes(2) - expect(gitExecFileAsyncMock).toHaveBeenNthCalledWith(1, ['remote', 'get-url', 'origin'], { - cwd: '/repo', - timeout: REMOTE_URL_PROBE_TIMEOUT_MS - }) - expect(gitExecFileAsyncMock).toHaveBeenNthCalledWith(2, ['remote', 'get-url', 'origin'], { - cwd: '/repo', - wslDistro: 'Ubuntu', - timeout: REMOTE_URL_PROBE_TIMEOUT_MS - }) - }) - it('bounds cached repository refs for distinct repo paths', async () => { gitExecFileAsyncMock.mockResolvedValue({ stdout: 'git@bitbucket.org:team/project.git\n', @@ -162,21 +126,4 @@ describe('Bitbucket repository refs', () => { }) expect(gitExecFileAsyncMock).not.toHaveBeenCalled() }) - - it('does not cache transient SSH provider failures as unsupported repos', async () => { - sshExecMock.mockRejectedValueOnce(new Error('connection closed')).mockResolvedValueOnce({ - stdout: 'git@bitbucket.org:remote/project.git\n', - stderr: '' - }) - registerSshGitProvider('conn-1', { exec: sshExecMock } as never) - - await expect(getBitbucketRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toBeNull() - await expect(getBitbucketRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toEqual({ - workspace: 'remote', - repoSlug: 'project' - }) - - expect(sshExecMock).toHaveBeenCalledTimes(2) - expect(gitExecFileAsyncMock).not.toHaveBeenCalled() - }) }) diff --git a/src/main/browser/browser-text-insertion.test.ts b/src/main/browser/browser-text-insertion.test.ts index 3541fca8538e..b1969fd931e3 100644 --- a/src/main/browser/browser-text-insertion.test.ts +++ b/src/main/browser/browser-text-insertion.test.ts @@ -11,13 +11,6 @@ describe('browser text insertion chunking', () => { expect(splitBrowserTextInsertionChunks('hello')).toEqual(['hello']) }) - it('splits by UTF-8 bytes without splitting surrogate pairs', () => { - const chunks = splitBrowserTextInsertionChunks('ab😀cd', 4) - - expect(chunks).toEqual(['ab', '😀', 'cd']) - expect(chunks.join('')).toBe('ab😀cd') - }) - it('iterates insertion chunks lazily without prebuilding the full chunk array', () => { const chunks = iterateBrowserTextInsertionChunks('abcdefghij', 4) diff --git a/src/main/codex-usage/scanner.test.ts b/src/main/codex-usage/scanner.test.ts index 0e899028b29f..bc5e3a007cef 100644 --- a/src/main/codex-usage/scanner.test.ts +++ b/src/main/codex-usage/scanner.test.ts @@ -236,39 +236,6 @@ describe('attributeCodexUsageEvent', () => { expect(attributed?.worktreeId).toBe('repo-2::/workspace/repo/app2') }) - it('attributes cwd paths under dotdot-prefixed child directories to the worktree', async () => { - const attributed = await attributeCodexUsageEvent( - { - sessionId: 'session-1', - timestamp: '2026-04-09T10:00:00.000Z', - eventKey: 'event-1', - cwd: '/workspace/repo/..fixtures/session', - model: 'gpt-5.2-codex', - hasInferredPricing: false, - longContextInputTokens: 0, - longContextCachedInputTokens: 0, - longContextOutputTokens: 0, - inputTokens: 100, - cachedInputTokens: 10, - outputTokens: 25, - reasoningOutputTokens: 10, - totalTokens: 125 - }, - await createUsageWorktreeResolver([ - { - repoId: 'repo-1', - worktreeId: 'repo-1::/workspace/repo', - path: '/workspace/repo', - displayName: 'Repo' - } - ]) - ) - - expect(attributed?.projectKey).toBe('worktree:repo-1::/workspace/repo') - expect(attributed?.projectLabel).toBe('Repo') - expect(attributed?.worktreeId).toBe('repo-1::/workspace/repo') - }) - it('does not attribute true parent-directory escapes to the worktree', async () => { const attributed = await attributeCodexUsageEvent( { @@ -300,36 +267,4 @@ describe('attributeCodexUsageEvent', () => { expect(attributed?.projectKey).toBe('cwd:/workspace/repo/../other/session') expect(attributed?.worktreeId).toBeNull() }) - - it('does not treat different Windows drives as containing paths', async () => { - const attributed = await attributeCodexUsageEvent( - { - sessionId: 'session-1', - timestamp: '2026-04-09T10:00:00.000Z', - eventKey: 'event-1', - cwd: 'D:\\other\\repo', - model: 'gpt-5.2-codex', - hasInferredPricing: false, - longContextInputTokens: 0, - longContextCachedInputTokens: 0, - longContextOutputTokens: 0, - inputTokens: 100, - cachedInputTokens: 10, - outputTokens: 25, - reasoningOutputTokens: 10, - totalTokens: 125 - }, - await createUsageWorktreeResolver([ - { - repoId: 'repo-1', - worktreeId: 'repo-1::C:\\repo', - path: 'C:\\repo', - displayName: 'Repo' - } - ]) - ) - - expect(attributed?.projectKey).toBe('cwd:d:/other/repo') - expect(attributed?.worktreeId).toBeNull() - }) }) diff --git a/src/main/daemon/session.test.ts b/src/main/daemon/session.test.ts index 65e6815b47d0..c9c69d99123d 100644 --- a/src/main/daemon/session.test.ts +++ b/src/main/daemon/session.test.ts @@ -1,7 +1,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { Session } from './session' import { SESSION_FORCE_KILL_RETRY_MS } from './session-termination-controller' -import { HeadlessEmulator } from './headless-emulator' import type { SessionState, ShellReadyState } from './types' import type { TuiAgent } from '../../shared/tui-agent' import { @@ -491,29 +490,6 @@ describe('Session', () => { expect(session.getSnapshot()?.snapshotAnsi).not.toContain('orca-shell-ready') }) - it.each([ - ['after the ready marker', ['\x1b]777;orca-shell-ready\x07', '\x1b[?2004hfish> ']], - ['after the ESC introducer', ['\x1b]777;orca-shell-ready\x07\x1b', '[?2004hfish> ']] - ])('preserves Fish bracketed-paste output split %s', (_boundary, chunks) => { - createSession({ shellReadySupported: true }) - const received: string[] = [] - session.attachClient({ - onData: (data) => received.push(data), - onExit: () => {} - }) - - for (const chunk of chunks) { - subprocess.simulateData(chunk) - } - - const output = received.join('') - expect(output).toBe('\x1b[?2004hfish> ') - const rendered = new HeadlessEmulator({ cols: 80, rows: 24 }) - expect(rendered.writeSync(output)).toBe(true) - expect(rendered.getVisibleLines().join('\n')).not.toContain('[?2004h') - rendered.dispose() - }) - it('publishes an absolute output sequence with live snapshots', () => { createSession() subprocess.simulateData('first') diff --git a/src/main/daemon/shell-ready.test.ts b/src/main/daemon/shell-ready.test.ts index 01cdad10be92..072ebe450569 100644 --- a/src/main/daemon/shell-ready.test.ts +++ b/src/main/daemon/shell-ready.test.ts @@ -444,23 +444,6 @@ describePosix('daemon shell-ready launch config', () => { } }) - it('owns zle-line-init for the shell-ready marker instead of an azhw hook', async () => { - const { getShellReadyLaunchConfig } = await importFreshShellReady() - - getShellReadyLaunchConfig('/bin/zsh') - - // Why .zshenv: the widget registration lives in the deferred hook, which the - // first prompt's precmd sweep calls exactly once. - const zshenv = readFileSync(join(getShellReadyWrapperRoot(), 'zsh', '.zshenv'), 'utf8') - expect(zshenv).toContain('zle -N zle-line-init __orca_prompt_mark') - expect(zshenv).toContain('__orca_prev_line_init_fn="${widgets[zle-line-init]#user:}"') - expect(zshenv).toContain('printf "\\033]777;orca-shell-ready\\007"') - // Why: add-zle-hook-widget aborts its chain when an earlier hook exits non-zero, so don't register the marker through it. - expect(zshenv).not.toContain('add-zle-hook-widget line-init') - // Why: re-source guard — skip re-capturing when already the bound widget so the prior chain survives a second source. - expect(zshenv).toContain('== "user:__orca_prompt_mark"') - }) - // Why: oh-my-zsh vi-mode's zle-line-init returns non-zero; add-zle-hook-widget then aborts the chain and the marker never fires. itWithZsh( 'emits the shell-ready marker even when a user zle-line-init widget fails (oh-my-zsh vi-mode shape)', @@ -699,21 +682,4 @@ describePosix('daemon shell-ready launch config', () => { } } }) - - it('sources the user .zshenv at wrapper top level, not inside a function', async () => { - // Why: PR #1737 sourced .zshenv in a wrapper function, breaking `typeset -U - // path`. Top-level sourcing is still the contract. - const { getShellReadyLaunchConfig } = await importFreshShellReady() - - getShellReadyLaunchConfig('/bin/zsh') - - const zshenv = readFileSync(join(getShellReadyWrapperRoot(), 'zsh', '.zshenv'), 'utf8') - - expect(zshenv).toContain('builtin source -- "$_orca_user_zshenv"') - // Every function the hook needs is defined above the source, so a user - // `emulate sh` cannot leave the rest of this file unparseable. - expect(zshenv.indexOf('__orca_deferred_init() {')).toBeLessThan( - zshenv.indexOf('builtin source -- "$_orca_user_zshenv"') - ) - }) }) diff --git a/src/main/git/worktree-add-creation-config.test.ts b/src/main/git/worktree-add-creation-config.test.ts index ff44ca7ac6db..15bcb335f03b 100644 --- a/src/main/git/worktree-add-creation-config.test.ts +++ b/src/main/git/worktree-add-creation-config.test.ts @@ -462,40 +462,6 @@ describe('addWorktree', () => { ]) }) - it('qualifies slash-containing local branch names when no remote ref matches', async () => { - gitExecFileAsyncMock.mockRejectedValueOnce(new Error('no remote ref')) // rev-parse refs/remotes/release/main^{commit} - gitExecFileAsyncMock.mockResolvedValueOnce({ stdout: 'abc123\n' }) // rev-parse refs/heads/release/main^{commit} - gitExecFileAsyncMock.mockResolvedValueOnce({ stdout: '' }) // worktree add - resolveCreationBaseConfigWrite() - gitExecFileAsyncMock.mockRejectedValueOnce(Object.assign(new Error('key unset'), { code: 1 })) // config --get push.autoSetupRemote (unset) - gitExecFileAsyncMock.mockResolvedValueOnce({ stdout: '' }) // config --local set push.autoSetupRemote - - await addWorktree('/repo', '/repo-feature', 'feature/release', 'release/main') - - expect(gitExecFileAsyncMock.mock.calls.map((call) => call[0])).toEqual([ - ['rev-parse', '--verify', '--quiet', 'refs/remotes/release/main^{commit}'], - ['rev-parse', '--verify', '--quiet', 'refs/heads/release/main^{commit}'], - [ - 'worktree', - 'add', - '--no-track', - '-b', - 'feature/release', - '/repo-feature', - 'refs/heads/release/main' - ], - [ - 'config', - '--local', - '--replace-all', - 'branch.feature/release.base', - 'refs/heads/release/main' - ], - ['config', '--get', 'push.autoSetupRemote'], - ['config', '--local', 'push.autoSetupRemote', 'true'] - ]) - }) - it('does not report a local base refresh for slash-containing local branch names', async () => { gitExecFileAsyncMock.mockRejectedValueOnce(new Error('no remote ref')) // rev-parse refs/remotes/release/main^{commit} gitExecFileAsyncMock.mockResolvedValueOnce({ stdout: 'abc123\n' }) // rev-parse refs/heads/release/main^{commit} diff --git a/src/main/gitea/repository-ref.test.ts b/src/main/gitea/repository-ref.test.ts index 678dd5c6912d..ddd4076fcbd0 100644 --- a/src/main/gitea/repository-ref.test.ts +++ b/src/main/gitea/repository-ref.test.ts @@ -127,42 +127,6 @@ describe('Gitea repository ref parsing', () => { }) }) - it('keeps local host and local WSL repository-ref cache entries separate', async () => { - gitExecFileAsyncMock - .mockResolvedValueOnce({ - stdout: 'https://git.example.com/host/project.git\n', - stderr: '' - }) - .mockResolvedValueOnce({ - stdout: 'https://git.example.com/wsl/project.git\n', - stderr: '' - }) - - await expect(getGiteaRepoRef('/repo')).resolves.toMatchObject({ - owner: 'host', - repo: 'project' - }) - await expect(getGiteaRepoRef('/repo', null, { wslDistro: 'Ubuntu' })).resolves.toMatchObject({ - owner: 'wsl', - repo: 'project' - }) - await expect(getGiteaRepoRef('/repo', null, { wslDistro: 'Ubuntu' })).resolves.toMatchObject({ - owner: 'wsl', - repo: 'project' - }) - - expect(gitExecFileAsyncMock).toHaveBeenCalledTimes(2) - expect(gitExecFileAsyncMock).toHaveBeenNthCalledWith(1, ['remote', 'get-url', 'origin'], { - cwd: '/repo', - timeout: REMOTE_URL_PROBE_TIMEOUT_MS - }) - expect(gitExecFileAsyncMock).toHaveBeenNthCalledWith(2, ['remote', 'get-url', 'origin'], { - cwd: '/repo', - wslDistro: 'Ubuntu', - timeout: REMOTE_URL_PROBE_TIMEOUT_MS - }) - }) - it('bounds cached repository refs for distinct repo paths', async () => { gitExecFileAsyncMock.mockResolvedValue({ stdout: 'https://git.example.com/team/project.git\n', @@ -194,22 +158,4 @@ describe('Gitea repository ref parsing', () => { }) expect(gitExecFileAsyncMock).not.toHaveBeenCalled() }) - - it('does not cache transient SSH provider failures as unsupported repos', async () => { - sshExecMock.mockRejectedValueOnce(new Error('connection closed')).mockResolvedValueOnce({ - stdout: 'git@gitea.example.test:remote/project.git\n', - stderr: '' - }) - registerSshGitProvider('conn-1', { exec: sshExecMock } as never) - - await expect(getGiteaRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toBeNull() - await expect(getGiteaRepoRefForRemote('/repo', 'origin', 'conn-1')).resolves.toMatchObject({ - host: 'gitea.example.test', - owner: 'remote', - repo: 'project' - }) - - expect(sshExecMock).toHaveBeenCalledTimes(2) - expect(gitExecFileAsyncMock).not.toHaveBeenCalled() - }) }) diff --git a/src/main/ipc/worktree-logic.test.ts b/src/main/ipc/worktree-logic.test.ts index 27e3f9f1a699..a502e8023dd0 100644 --- a/src/main/ipc/worktree-logic.test.ts +++ b/src/main/ipc/worktree-logic.test.ts @@ -288,30 +288,12 @@ describe('computeBranchName', () => { }) describe('getConfiguredBranchPrefix', () => { - it('returns the git username for the git-username strategy', () => { - expect(getConfiguredBranchPrefix({ branchPrefix: 'git-username' }, 'jdoe')).toBe('jdoe') - }) - - it('returns null for git-username when no username is available', () => { - expect(getConfiguredBranchPrefix({ branchPrefix: 'git-username' }, null)).toBeNull() - }) - - it('returns the custom value for the custom strategy', () => { - expect( - getConfiguredBranchPrefix({ branchPrefix: 'custom', branchPrefixCustom: 'team' }, null) - ).toBe('team') - }) - it('returns null for custom strategy with an empty value', () => { expect( getConfiguredBranchPrefix({ branchPrefix: 'custom', branchPrefixCustom: '' }, null) ).toBeNull() }) - it('returns null when no prefix strategy applies', () => { - expect(getConfiguredBranchPrefix({ branchPrefix: 'none' }, 'jdoe')).toBeNull() - }) - it('normalizes a trailing slash out of the custom prefix', () => { expect( getConfiguredBranchPrefix({ branchPrefix: 'custom', branchPrefixCustom: 'team/' }, null) diff --git a/src/main/opencode-usage/scanner.test.ts b/src/main/opencode-usage/scanner.test.ts index 88559b0ec7d0..0e5c39eb69a1 100644 --- a/src/main/opencode-usage/scanner.test.ts +++ b/src/main/opencode-usage/scanner.test.ts @@ -175,17 +175,6 @@ describe('parseOpenCodeUsageRow', () => { }) describe('attributeOpenCodeUsageEvent', () => { - it('attributes cwd paths under dotdot-prefixed child directories to the worktree', async () => { - const attributed = await attributeOpenCodeUsageEvent( - usageEvent(`${WORKTREE}/..fixtures/session`), - await resolveWorktree() - ) - - expect(attributed?.projectKey).toBe('worktree:repo-1::/workspace/repo') - expect(attributed?.projectLabel).toBe('Repo') - expect(attributed?.worktreeId).toBe('repo-1::/workspace/repo') - }) - it('does not attribute true parent-directory escapes to the worktree', async () => { const attributed = await attributeOpenCodeUsageEvent( usageEvent(`${WORKTREE}/../other/session`), @@ -195,23 +184,6 @@ describe('attributeOpenCodeUsageEvent', () => { expect(attributed?.projectKey).toBe('cwd:/workspace/repo/../other/session') expect(attributed?.worktreeId).toBeNull() }) - - it('does not treat different Windows drives as containing paths', async () => { - const attributed = await attributeOpenCodeUsageEvent( - usageEvent('D:\\other\\repo'), - await createUsageWorktreeResolver([ - { - repoId: 'repo-1', - worktreeId: 'repo-1::C:\\repo', - path: 'C:\\repo', - displayName: 'Repo' - } - ]) - ) - - expect(attributed?.projectKey).toBe('cwd:d:/other/repo') - expect(attributed?.worktreeId).toBeNull() - }) }) describe('parseOpenCodeUsageDatabase', () => { diff --git a/src/main/orca-profiles/profile-project-state-file.test.ts b/src/main/orca-profiles/profile-project-state-file.test.ts index 305ba53d8690..6d03ff335e3c 100644 --- a/src/main/orca-profiles/profile-project-state-file.test.ts +++ b/src/main/orca-profiles/profile-project-state-file.test.ts @@ -84,49 +84,6 @@ describe('rebuildRepoBackedProjectState', () => { ) }) - it('picks one predecessor project when several prior rows overlap the same repos', () => { - const sharedIdentity = { - canonicalKey: 'git.example.com/acme/shared', - remoteName: 'origin', - remoteUrl: 'git@git.example.com:acme/shared.git' - } - const rebuilt = rebuildRepoBackedProjectState( - makeState({ - repos: [ - makeRepo({ id: 'r1', path: '/left', gitRemoteIdentity: sharedIdentity }), - makeRepo({ id: 'r2', path: '/right', gitRemoteIdentity: sharedIdentity }) - ], - projects: [ - makeProject({ - id: 'git:git.example.com/acme/left', - sourceRepoIds: ['r1'], - updatedAt: 200, - localWindowsRuntimePreference: { kind: 'wsl', distro: 'Ubuntu' } - }), - makeProject({ - id: 'git:git.example.com/acme/right', - sourceRepoIds: ['r2'], - updatedAt: 100, - localWindowsRuntimePreference: { kind: 'windows-host' } - }) - ], - projectHostSetups: [ - makeSetup({ id: 'r1', projectId: 'git:git.example.com/acme/left', repoId: 'r1' }), - makeSetup({ id: 'r2', projectId: 'git:git.example.com/acme/right', repoId: 'r2' }) - ] - }) - ) - - // Equal repo overlap resolves by newest updatedAt; the loser's preference is never merged in. - expect(rebuilt.projects).toEqual([ - expect.objectContaining({ - id: 'git:git.example.com/acme/shared', - sourceRepoIds: ['r1', 'r2'], - localWindowsRuntimePreference: { kind: 'wsl', distro: 'Ubuntu' } - }) - ]) - }) - it('leaves an unclaimed prior project standing with its own independent setups', () => { const rebuilt = rebuildRepoBackedProjectState( makeState({ diff --git a/src/main/persistence-initial-load.test.ts b/src/main/persistence-initial-load.test.ts index 13beae7932c7..a539d5efa193 100644 --- a/src/main/persistence-initial-load.test.ts +++ b/src/main/persistence-initial-load.test.ts @@ -410,68 +410,6 @@ describe('Store', () => { ).toBe(upstreamProjectId) }) - it('picks one predecessor project when several prior rows overlap the same repos', async () => { - const sharedIdentity = { - canonicalKey: 'git.example.com/acme/shared', - remoteName: 'origin', - remoteUrl: 'git@git.example.com:acme/shared.git' - } - writeDataFile({ - ...getDefaultPersistedState(testState.dir), - repos: [ - makeRepo({ - id: 'r1', - path: '/left', - displayName: 'Left', - gitRemoteIdentity: sharedIdentity - }), - makeRepo({ - id: 'r2', - path: '/right', - displayName: 'Right', - gitRemoteIdentity: sharedIdentity - }) - ], - projects: [ - makeProject({ - id: 'git:git.example.com/acme/left', - sourceRepoIds: ['r1'], - updatedAt: 200, - localWindowsRuntimePreference: { kind: 'wsl', distro: 'Ubuntu' } - }), - makeProject({ - id: 'git:git.example.com/acme/right', - sourceRepoIds: ['r2'], - updatedAt: 100, - localWindowsRuntimePreference: { kind: 'windows-host' } - }) - ], - projectHostSetups: [ - makeProjectHostSetup({ - id: 'r1', - projectId: 'git:git.example.com/acme/left', - repoId: 'r1' - }), - makeProjectHostSetup({ - id: 'r2', - projectId: 'git:git.example.com/acme/right', - repoId: 'r2' - }) - ] - }) - - const store = await createStore() - - // Equal repo overlap resolves by newest updatedAt; the loser's preference is never merged in. - expect(store.getProjects()).toEqual([ - expect.objectContaining({ - id: 'git:git.example.com/acme/shared', - sourceRepoIds: ['r1', 'r2'], - localWindowsRuntimePreference: { kind: 'wsl', distro: 'Ubuntu' } - }) - ]) - }) - it('migrates legacy WSL agent settings into the global Windows runtime default', async () => { writeDataFile({ schemaVersion: 1, diff --git a/src/main/providers/local-pty-provider-shell-readiness.test.ts b/src/main/providers/local-pty-provider-shell-readiness.test.ts index 0457a6e170f0..6a46261e7931 100644 --- a/src/main/providers/local-pty-provider-shell-readiness.test.ts +++ b/src/main/providers/local-pty-provider-shell-readiness.test.ts @@ -275,23 +275,6 @@ describe('LocalPtyProvider', () => { } }) - it.each([ - ['after the ready marker', ['\x1b]777;orca-shell-ready\x07', '\x1b[?2004hfish> ']], - ['after the ESC introducer', ['\x1b]777;orca-shell-ready\x07\x1b', '[?2004hfish> ']] - ])('preserves Fish bracketed-paste output split %s', async (_boundary, chunks) => { - process.env.SHELL = '/usr/bin/fish' - const received: string[] = [] - provider.configure({ onData: (_id, data) => received.push(data) }) - - await provider.spawn({ cols: 80, rows: 24, command: 'printf ready' }) - const dataCallback = mockProc.onData.mock.calls[0]?.[0] as (data: string) => void - for (const chunk of chunks) { - dataCallback(chunk) - } - - expect(received.join('')).toBe('\x1b[?2004hfish> ') - }) - it('releases held marker-prefix bytes when local shell readiness times out', async () => { vi.useFakeTimers() const onData = vi.fn() diff --git a/src/main/providers/local-pty-shell-ready-wrapper-generation.test.ts b/src/main/providers/local-pty-shell-ready-wrapper-generation.test.ts index af0fc4a5deb1..8b061c5d6423 100644 --- a/src/main/providers/local-pty-shell-ready-wrapper-generation.test.ts +++ b/src/main/providers/local-pty-shell-ready-wrapper-generation.test.ts @@ -219,37 +219,6 @@ describePosix('local PTY shell-ready launch config', () => { } }) - it('uses inherited ORCA_ORIG_ZDOTDIR when ZDOTDIR is an Orca wrapper dir', async () => { - const previousZdotdir = process.env.ZDOTDIR - const previousOrigZdotdir = process.env.ORCA_ORIG_ZDOTDIR - const previousHome = process.env.HOME - const userZdotdir = makeUserZdotdir(userDataPath, '.config', 'zsh') - process.env.ZDOTDIR = '/some/other/orca/shell-ready/zsh' - process.env.ORCA_ORIG_ZDOTDIR = userZdotdir - process.env.HOME = userDataPath - try { - const { getShellReadyLaunchConfig } = await importFreshLocalPtyShellReady() - const config = getShellReadyLaunchConfig('/bin/zsh') - expect(config.env.ORCA_ORIG_ZDOTDIR).toBe(userZdotdir) - } finally { - if (previousZdotdir === undefined) { - delete process.env.ZDOTDIR - } else { - process.env.ZDOTDIR = previousZdotdir - } - if (previousOrigZdotdir === undefined) { - delete process.env.ORCA_ORIG_ZDOTDIR - } else { - process.env.ORCA_ORIG_ZDOTDIR = previousOrigZdotdir - } - if (previousHome === undefined) { - delete process.env.HOME - } else { - process.env.HOME = previousHome - } - } - }) - it('falls back to HOME when inherited ORCA_ORIG_ZDOTDIR points at a wrapper dir', async () => { const previousZdotdir = process.env.ZDOTDIR const previousOrigZdotdir = process.env.ORCA_ORIG_ZDOTDIR @@ -486,29 +455,6 @@ describePosix('local PTY shell-ready launch config', () => { } }) - it('rejects inherited ZDOTDIR ending in /shell-ready/zsh even with a trailing slash', async () => { - const previousZdotdir = process.env.ZDOTDIR - const previousHome = process.env.HOME - process.env.ZDOTDIR = '/some/other/orca/shell-ready/zsh/' - process.env.HOME = '/Users/alice' - try { - const { getShellReadyLaunchConfig } = await importFreshLocalPtyShellReady() - const config = getShellReadyLaunchConfig('/bin/zsh') - expect(config.env.ORCA_ORIG_ZDOTDIR).toBeUndefined() - } finally { - if (previousZdotdir === undefined) { - delete process.env.ZDOTDIR - } else { - process.env.ZDOTDIR = previousZdotdir - } - if (previousHome === undefined) { - delete process.env.HOME - } else { - process.env.HOME = previousHome - } - } - }) - it('falls back to HOME when ZDOTDIR is only slashes (e.g. "/")', async () => { const previousZdotdir = process.env.ZDOTDIR const previousHome = process.env.HOME diff --git a/src/main/runtime/runtime-server-git-availability.test.ts b/src/main/runtime/runtime-server-git-availability.test.ts index 2259e1a563ac..77bcfbbfa0b4 100644 --- a/src/main/runtime/runtime-server-git-availability.test.ts +++ b/src/main/runtime/runtime-server-git-availability.test.ts @@ -11,10 +11,6 @@ vi.mock('../git/runner', () => ({ gitExecFileAsync: gitExecFileAsyncMock })) import { RuntimeServerEnvironmentCommands } from './runtime-server-environment-commands' -function spawnEnoent(): Error { - return Object.assign(new Error('spawn git ENOENT'), { code: 'ENOENT', syscall: 'spawn git' }) -} - describe('RuntimeServerEnvironmentCommands.isGitAvailable', () => { const commands = new RuntimeServerEnvironmentCommands() @@ -30,27 +26,4 @@ describe('RuntimeServerEnvironmentCommands.isGitAvailable', () => { gitExecFileAsyncMock.mockResolvedValue({ stdout: 'git version 2.25.1\n', stderr: '' }) await expect(commands.isGitAvailable()).resolves.toBe(true) }) - - it('answers false only when the spawn itself found no binary', async () => { - gitExecFileAsyncMock.mockRejectedValue(spawnEnoent()) - await expect(commands.isGitAvailable()).resolves.toBe(false) - }) - - it('rejects an ENOENT when the working directory disappeared', async () => { - vi.spyOn(process, 'cwd').mockReturnValue(`${process.cwd()}-missing`) - gitExecFileAsyncMock.mockRejectedValue(spawnEnoent()) - await expect(commands.isGitAvailable()).rejects.toThrow('spawn git ENOENT') - }) - - it('rejects a slow host rather than reporting no Git', async () => { - gitExecFileAsyncMock.mockRejectedValue(new Error('git --version timed out after 3000ms')) - await expect(commands.isGitAvailable()).rejects.toThrow('timed out') - }) - - it('rejects a repository-level git failure rather than reporting no Git', async () => { - gitExecFileAsyncMock.mockRejectedValue( - Object.assign(new Error('detected dubious ownership'), { code: 128 }) - ) - await expect(commands.isGitAvailable()).rejects.toThrow('dubious ownership') - }) }) diff --git a/src/main/runtime/runtime-worktree-agent-rows-structured.test.ts b/src/main/runtime/runtime-worktree-agent-rows-structured.test.ts index 01c726198309..9116bbb6605d 100644 --- a/src/main/runtime/runtime-worktree-agent-rows-structured.test.ts +++ b/src/main/runtime/runtime-worktree-agent-rows-structured.test.ts @@ -91,12 +91,6 @@ describe('worktree ps reports structured sessions', () => { expect(row.status).toBe('working') }) - // The same projection the sidebar applies, so the two surfaces cannot disagree about one session. - it('maps attention to blocked and idle to done', () => { - expect(attach([summary({ status: 'attention' })]).agents[0]?.state).toBe('blocked') - expect(attach([summary({ status: 'idle' })]).agents[0]?.state).toBe('done') - }) - it('does not turn a completed host-held session into permission', () => { const row = attach([summary({ status: 'idle' })]) expect(row.status).toBe('inactive') diff --git a/src/main/usage/usage-scan-worker-client.test.ts b/src/main/usage/usage-scan-worker-client.test.ts index 09c6f378f007..c88389203350 100644 --- a/src/main/usage/usage-scan-worker-client.test.ts +++ b/src/main/usage/usage-scan-worker-client.test.ts @@ -1,13 +1,10 @@ -import { join, sep } from 'node:path' import { describe, expect, it, vi } from 'vitest' -import { resolveWorkerThreadEntryPath } from '../worker-thread-entry-path' import { MAX_CONSECUTIVE_DEATHS, USAGE_SCAN_NO_PROGRESS_TIMEOUT_MS, UsageScanWorkerClient, scanCodexUsageOnWorker } from './usage-scan-worker-client' -import { USAGE_SCAN_WORKER_ENTRY_FILENAME } from './usage-scan-worker-spawn' import type { UsageScanWorkerRequest, UsageScanWorkerRequestBody @@ -215,23 +212,3 @@ describe('UsageScanWorkerClient', () => { await expect(pending).rejects.toThrow(/answered for claude/) }) }) - -// Why: the packaged branch never runs in dev or e2e (both take the __dirname -// path), so it is pinned here at the path-construction level. -describe('usage scan worker entry path', () => { - it('resolves a packaged build under resourcesPath/app.asar/out/main', () => { - const resourcesPath = join(sep, 'Applications', 'Orca.app', 'Contents', 'Resources') - - const resolved = resolveWorkerThreadEntryPath( - { isPackaged: true, resourcesPath, moduleDir: join(sep, 'unpackaged', 'out', 'main') }, - USAGE_SCAN_WORKER_ENTRY_FILENAME - ) - - expect(resolved.slice(resourcesPath.length + 1).split(sep)).toEqual([ - 'app.asar', - 'out', - 'main', - USAGE_SCAN_WORKER_ENTRY_FILENAME - ]) - }) -}) diff --git a/src/relay/git-handler-worktree-provisioning.test.ts b/src/relay/git-handler-worktree-provisioning.test.ts index 99e9122abf92..804799103bd4 100644 --- a/src/relay/git-handler-worktree-provisioning.test.ts +++ b/src/relay/git-handler-worktree-provisioning.test.ts @@ -496,46 +496,6 @@ describe('GitHandler', () => { ]) }) - it('qualifies slash-containing local branch names when no remote ref matches', async () => { - const { localDispatcher, gitMock } = setupMockedHandler(['/relay/repo', '/relay/wt']) - gitMock.mockRejectedValueOnce(new Error('no remote ref')) // rev-parse refs/remotes/release/main^{commit} - gitMock.mockResolvedValueOnce({ stdout: 'abc123\n', stderr: '' }) // rev-parse refs/heads/release/main^{commit} - gitMock.mockResolvedValueOnce({ stdout: '', stderr: '' }) // worktree add - gitMock.mockResolvedValueOnce({ stdout: '', stderr: '' }) // config --local --replace-all branch..base - gitMock.mockRejectedValueOnce(Object.assign(new Error('key unset'), { code: 1 })) // --get unset - gitMock.mockResolvedValueOnce({ stdout: '', stderr: '' }) // --local set - - await localDispatcher.callRequest('git.addWorktree', { - repoPath: '/relay/repo', - branchName: 'feature/release', - targetDir: '/relay/wt', - base: 'release/main' - }) - - expect(gitMock.mock.calls.map((c) => c[0])).toEqual([ - ['rev-parse', '--verify', '--quiet', 'refs/remotes/release/main^{commit}'], - ['rev-parse', '--verify', '--quiet', 'refs/heads/release/main^{commit}'], - [ - 'worktree', - 'add', - '--no-track', - '-b', - 'feature/release', - '/relay/wt', - 'refs/heads/release/main' - ], - [ - 'config', - '--local', - '--replace-all', - 'branch.feature/release.base', - 'refs/heads/release/main' - ], - ['config', '--get', 'push.autoSetupRemote'], - ['config', '--local', 'push.autoSetupRemote', 'true'] - ]) - }) - it('passes --no-checkout when sparse setup will checkout after configuration', async () => { const { localDispatcher, gitMock } = setupMockedHandler(['/relay/repo', '/relay/wt']) gitMock.mockResolvedValueOnce({ stdout: '', stderr: '' }) // rev-parse refs/remotes/origin/main diff --git a/src/renderer/src/components/native-chat/native-chat-send.test.ts b/src/renderer/src/components/native-chat/native-chat-send.test.ts index 747f2bb110f6..7ebd0bc9f3dc 100644 --- a/src/renderer/src/components/native-chat/native-chat-send.test.ts +++ b/src/renderer/src/components/native-chat/native-chat-send.test.ts @@ -51,12 +51,6 @@ describe('buildNativeChatImagePasteBytes', () => { `${BEGIN}/tmp/orca-paste-image.png${END}` ) }) - - it('sanitizes embedded escape bytes before framing', () => { - expect(buildNativeChatImagePasteBytes('/tmp/before\x1b[201~after.png')).toBe( - `${BEGIN}/tmp/before␛[201~after.png${END}` - ) - }) }) describe('isMultilineDraft', () => { diff --git a/src/renderer/src/components/native-chat/native-chat-tool-summary.test.ts b/src/renderer/src/components/native-chat/native-chat-tool-summary.test.ts index 58bbeb982781..e2ff5170f28c 100644 --- a/src/renderer/src/components/native-chat/native-chat-tool-summary.test.ts +++ b/src/renderer/src/components/native-chat/native-chat-tool-summary.test.ts @@ -16,13 +16,6 @@ describe('summarizeToolInput', () => { expect(summarizeToolInput({ a: 1 })).toBe('{"a":1}') }) - it('truncates long previews with an ellipsis', () => { - const long = 'x'.repeat(200) - const out = summarizeToolInput(long) - expect(out.endsWith('…')).toBe(true) - expect(out.length).toBe(80) - }) - it('returns empty for null/undefined', () => { expect(summarizeToolInput(null)).toBe('') expect(summarizeToolInput(undefined)).toBe('') diff --git a/src/renderer/src/components/native-chat/use-native-chat-interactive-send.test.tsx b/src/renderer/src/components/native-chat/use-native-chat-interactive-send.test.tsx index f2cbd8df93ca..913e8fe9fea1 100644 --- a/src/renderer/src/components/native-chat/use-native-chat-interactive-send.test.tsx +++ b/src/renderer/src/components/native-chat/use-native-chat-interactive-send.test.tsx @@ -98,22 +98,6 @@ describe('useNativeChatInteractiveSend', () => { expect(mocks.sendNativeChatMessage).not.toHaveBeenCalled() }) - it('does not send a trailing Enter after Codex submits a multi-question answer', () => { - const prompt: AskPrompt = { - questions: [ - { question: 'q1', multiSelect: false, options: [{ label: 'A' }, { label: 'B' }] }, - { question: 'q2', multiSelect: false, options: [{ label: 'C' }, { label: 'D' }] } - ] - } - const { result } = renderHook(() => - useNativeChatInteractiveSend('tab-1', PANE_KEY, 'pty-1', 'codex') - ) - - act(() => result.current.sendAnswer(prompt, [{ indices: [1] }, { indices: [0] }])) - - expect(mocks.sendNativeChatAskAnswer.mock.calls[0]?.[2]).toEqual([{ raw: '2' }, { raw: '1' }]) - }) - it('routes a Claude answer through the option-number keystroke path', () => { const { result } = renderHook(() => useNativeChatInteractiveSend('tab-1', PANE_KEY, 'pty-1', 'claude') diff --git a/src/renderer/src/components/right-sidebar/checks-panel-content.test.tsx b/src/renderer/src/components/right-sidebar/checks-panel-content.test.tsx index 856af2bee985..899de76caedb 100644 --- a/src/renderer/src/components/right-sidebar/checks-panel-content.test.tsx +++ b/src/renderer/src/components/right-sidebar/checks-panel-content.test.tsx @@ -89,24 +89,6 @@ describe('MergeConflictNotice', () => { expect(markup).not.toContain('Conflict file details are unavailable') }) - it('does not interpolate shell-sensitive base refs into copyable commands', () => { - const markup = renderNotice( - makePR({ - conflictSummary: { - baseRef: 'release/$USER;echo unsafe', - baseCommit: 'abc1234', - commitsBehind: 1, - files: [], - localMergeState: 'clean' - } - }) - ) - - expect(markup).toContain('git fetch origin') - expect(markup).not.toContain('$USER') - expect(markup).not.toContain('echo unsafe') - }) - it('hides when the conflicting file list is available', () => { const markup = renderNotice( makePR({ diff --git a/src/renderer/src/components/sidebar/worktree-meta-updates.test.ts b/src/renderer/src/components/sidebar/worktree-meta-updates.test.ts index 838358154a74..654724b89673 100644 --- a/src/renderer/src/components/sidebar/worktree-meta-updates.test.ts +++ b/src/renderer/src/components/sidebar/worktree-meta-updates.test.ts @@ -131,20 +131,6 @@ describe('buildWorktreeMetaUpdates', () => { }) }) - it('takes the organization key from a Linear issue URL', () => { - expect( - buildUpdates({ - issueInput: 'https://linear.app/acme/issue/STA-335/fix-auth', - issueProvider: 'linear' - }) - ).toEqual({ - linkedIssue: null, - linkedLinearIssue: 'STA-335', - linkedLinearIssueWorkspaceId: null, - linkedLinearIssueOrganizationUrlKey: 'acme' - }) - }) - it('clears every provider slot when the issue field is emptied', () => { expect( buildUpdates( diff --git a/src/renderer/src/store/projects/project-wsl-filesystem-boundary-advisory.test.ts b/src/renderer/src/store/projects/project-wsl-filesystem-boundary-advisory.test.ts index 34982c575875..1fdd8a215469 100644 --- a/src/renderer/src/store/projects/project-wsl-filesystem-boundary-advisory.test.ts +++ b/src/renderer/src/store/projects/project-wsl-filesystem-boundary-advisory.test.ts @@ -58,16 +58,6 @@ describe('warnIfProjectCrossesWslFilesystemBoundary', () => { expect(warningMock.mock.calls[0]?.[1]?.description).toContain('Debian') }) - it('stays silent for a project already inside the distro', () => { - warnIfProjectCrossesWslFilesystemBoundary( - makeRepo({ path: '\\\\wsl.localhost\\Ubuntu-24.04\\home\\alice\\orca' }), - [makeProject()], - WSL_DEFAULT - ) - - expect(warningMock).not.toHaveBeenCalled() - }) - // Why: the Windows runtime preference governs local projects only; an SSH path that happens to // look Windows-shaped is a different host's filesystem. it('stays silent for a repo on a remote execution host', () => { diff --git a/src/shared/contextual-tours.test.ts b/src/shared/contextual-tours.test.ts index e30e204421d3..9f197f323266 100644 --- a/src/shared/contextual-tours.test.ts +++ b/src/shared/contextual-tours.test.ts @@ -185,15 +185,6 @@ describe('contextual tour definitions', () => { ]) }) - it('allows only workspace creation over its workspace composer modal', () => { - const modalTours = (CONTEXTUAL_TOURS as readonly ContextualTour[]).filter( - (tour) => tour.allowedActiveModals?.length - ) - - expect(modalTours.map((tour) => tour.id)).toEqual(['workspace-creation']) - expect(modalTours[0]?.allowedActiveModals).toEqual(['new-workspace-composer']) - }) - it('normalizes persisted ids by removing unknowns and duplicates', () => { expect( normalizeContextualTourIds([ diff --git a/src/shared/git-capability-cache.test.ts b/src/shared/git-capability-cache.test.ts index 86f84a50877c..7b6a8dff1ae5 100644 --- a/src/shared/git-capability-cache.test.ts +++ b/src/shared/git-capability-cache.test.ts @@ -96,38 +96,4 @@ describe('GitCapabilityCache', () => { } await expect(Promise.all([first, second])).resolves.toEqual(['result', 'result']) }) - - it('drops known support when a later call reports the capability unsupported', async () => { - const cache = new GitCapabilityCache() - const isUnsupported = (error: unknown): boolean => - error instanceof Error && error.message === 'unsupported' - await cache.runWithFallback( - 'for-each-ref-exclude', - async () => 'supported', - async () => 'unexpected-fallback', - isUnsupported - ) - - await expect( - cache.runWithFallback( - 'for-each-ref-exclude', - async () => { - throw new Error('unsupported') - }, - async () => 'fallback', - isUnsupported - ) - ).resolves.toBe('fallback') - - const laterPreferred = vi.fn(async () => 'unexpected-preferred') - await expect( - cache.runWithFallback( - 'for-each-ref-exclude', - laterPreferred, - async () => 'cached-fallback', - isUnsupported - ) - ).resolves.toBe('cached-fallback') - expect(laterPreferred).not.toHaveBeenCalled() - }) }) diff --git a/src/shared/tui-agent-startup-shell.test.ts b/src/shared/tui-agent-startup-shell.test.ts index 39c46e5fbf39..1698e86b08b1 100644 --- a/src/shared/tui-agent-startup-shell.test.ts +++ b/src/shared/tui-agent-startup-shell.test.ts @@ -65,12 +65,6 @@ describe('tokenizeStartupCommand spans (windows shells)', () => { }) }) -describe('powershell startup argument quoting', () => { - it('doubles typographic single quotes, which PowerShell also treats as delimiters', () => { - expect(quoteStartupArg('it\u2019s', 'powershell')).toBe("'it\u2019\u2019s'") - }) -}) - describe('one Unix startup dialect', () => { it('clears variables with a self-contained branch, not a per-shell builtin', () => { // Why not `unset`/`set -e` alone, and why not a wrapper-defined helper: From db73d28e512fc674499ffdc263512880e627d08e Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 03:59:58 -0700 Subject: [PATCH 02/79] test: retire backlog cases that assert a shim, a literal, or an unread branch (#24120) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Deep-reads the 596 files earlier auditors explicitly disclosed as reviewed at title-and-import level only, never against production. Six ~100-file chunks, chosen so depth was achievable rather than optional. 27 case declarations removed across 19 files, 1 test file deleted, 522 lines gone. No production code touched. Two chunks found nothing, and that is reported as the result rather than padded: `src/main/agent-hooks` (89 files, 50 read case-by-case in full) returned zero deletions; `src/main/claude` (90 files, 44 read in full) found three near-identical pairs via a normalized case-body hash and kept all three after reading them. What went: - Identity copiers with a type-level title. `ui-new-workspace-draft.test.ts` (deleted, 3 cases) tested `setNewWorkspaceDraft: (draft) => set({ newWorkspaceDraft: draft })` — a bare pass-through — by passing a literal in and asserting a subset of that literal back. Its titles named field-shape facts that the typed parameter at `ui-slice-contract-core.ts:197` already enforces in production, so the annotation that does the work lives in production, not the test. - A test that asserted its own mock. `destroyRemovedBrowserWebview(id)` is literally `destroyPersistentWebview(id)`, and the case mocked `destroyPersistentWebview` — so it checked that the mock received the argument the shim passed through. - Replays across a bare re-export. `pane-tree-ops.ts:18` is `export { equalizePaneSplitSizes, findPaneChildren } from './pane-tree-equalization'`; two cases used `MockHTMLElement` with no dividers while the owner asserts concrete flex values on real happy-dom elements and runs a 400-seed differential against an independently reimplemented weight walk. - Table rows varying a value production never branches on. `mergeCurrentOrchestrationContext` tests only `dispatchStatus !== undefined`, so `it.each(['failed','circuit_broken'])` ran the path the surviving `'completed'` case already proves. Per-value behavior is owned by the one consumer that reads those literals. - Duplicate invocations whose surviving sibling asserts strictly more, including a push error case differing from its neighbour only in which verb produced the same error string. Kept deliberately: bound and cap guards (a 50-entry nav-history cap, a skill-cache eviction pinned to size 2 after 512 retired runtimes, a cleanup-concurrency ceiling); `buildDefaultTerminalOptions` cases restating declared constants, because each records a cross-cutting UX decision whose comment preserves the v1.4.51 ZWJ table-corruption history that once forced scrollbar width 0; three per-grammar `tokenizer.root` invariants for astro, svelte and vue, which are separate grammars rather than replays; and a test reading `@xterm/{headless,xterm}/package.json` out of node_modules to prove both were built from the same upstream commit, which is legitimate because the installed package is the shipped contract. Six `src/shared` files were trimmed, and each was checked against the risk that matters there: `src/shared` is the OWNER four earlier waves deferred to when deleting main/renderer/mobile tests, so hollowing one out would orphan several callers at once. Each retains 9 to 46 cases after removing 1 or 2. Coverage is partial and stated as such. Read case-by-case: pane-manager 81 of 81, slices-a 96 of 121, slices-b 94 of 121, agent-hooks 50 of 89, claude 44 of 90. Every auditor listed the paths it did not reach. Verified: 1,294 test files / 14,257 cases pass across the touched areas, plus one pre-existing `it.fails` marker; `check-reliability-gates.mjs` 140 gates; the deleted file is absent from the gate manifest, `cloud/package.json` and `mobile/tests-typecheck-baseline.txt`. --- ...de-background-task-terminal-frames.test.ts | 19 --- src/main/claude/claude-context-usage.test.ts | 15 -- .../claude-slash-command-catalog.test.ts | 28 ---- .../claude-streamed-text-checkpoints.test.ts | 17 -- .../claude-structured-content-parts.test.ts | 12 -- .../lib/pane-manager/pane-tree-ops.test.ts | 40 ----- .../slices/agent-status-drop-ipc.test.ts | 15 -- ...agent-status-runtime-orchestration.test.ts | 27 ---- .../slices/browser-webview-cleanup.test.ts | 7 - .../editor-remote-branch-actions.test.ts | 56 ------- .../slices/editor-right-sidebar-state.test.ts | 13 -- .../store/slices/github-checks-cache.test.ts | 17 -- .../slices/ui-new-workspace-draft.test.ts | 151 ------------------ src/shared/setup-runner-command.test.ts | 8 - src/shared/telemetry-events.test.ts | 22 +-- .../terminal-synchronized-output-scan.test.ts | 14 -- src/shared/text-search.test.ts | 12 -- ...pace-session-schema.sleeping-agent.test.ts | 29 ---- ...workspace-session-terminal-buffers.test.ts | 21 --- 19 files changed, 1 insertion(+), 522 deletions(-) delete mode 100644 src/renderer/src/store/slices/ui-new-workspace-draft.test.ts diff --git a/src/main/claude/claude-background-task-terminal-frames.test.ts b/src/main/claude/claude-background-task-terminal-frames.test.ts index 86a2dd2f4099..e44532965d9e 100644 --- a/src/main/claude/claude-background-task-terminal-frames.test.ts +++ b/src/main/claude/claude-background-task-terminal-frames.test.ts @@ -96,25 +96,6 @@ describe('claude background task terminal frames', () => { expect(items).toEqual([]) }) - it('leaves legacy local_subagent tasks to the subagent roster', () => { - const { rows, items } = harness() - rows.observe({ - type: 'system', - subtype: 'task_started', - task_id: 'task-legacy-agent', - task_type: 'local_subagent', - subagent_type: 'explorer' - }) - rows.observe({ - type: 'system', - subtype: 'task_notification', - task_id: 'task-legacy-agent', - status: 'failed', - summary: 'the child failed' - }) - expect(items).toEqual([]) - }) - it('writes nothing for ambient housekeeping the user never asked for', () => { const { rows, items } = harness() rows.observe({ ...START_BASH, task_id: 'ambient-1', ambient: true }) diff --git a/src/main/claude/claude-context-usage.test.ts b/src/main/claude/claude-context-usage.test.ts index af86a7fddee0..08677a1fb930 100644 --- a/src/main/claude/claude-context-usage.test.ts +++ b/src/main/claude/claude-context-usage.test.ts @@ -157,21 +157,6 @@ describe('claudeContextWindowFromResult', () => { ).toBe(200_000) }) - it('keeps the main thread window when a subagent ran on a larger one', () => { - const modelUsage = { - 'claude-sonnet-5': { contextWindow: 200_000 }, - 'claude-fable-5-1[1m]': { contextWindow: 1_000_000 } - } - for (const initModel of ['claude-sonnet-5', null]) { - expect( - claudeContextWindowFromResult( - { modelUsage }, - { initModel, responseModel: 'claude-sonnet-5' } - ) - ).toBe(200_000) - } - }) - it('is null when no entry reports a usable window', () => { expect(claudeContextWindowFromResult({ type: 'result' })).toBeNull() expect( diff --git a/src/main/claude/claude-slash-command-catalog.test.ts b/src/main/claude/claude-slash-command-catalog.test.ts index f4374e52e4fa..17d7fba8650d 100644 --- a/src/main/claude/claude-slash-command-catalog.test.ts +++ b/src/main/claude/claude-slash-command-catalog.test.ts @@ -245,31 +245,3 @@ it('reports a description-only change and lets a later report drop the text', () expect(catalog.observe({ ...changed, commands: [{ name: 'clear' }] })).toBe(true) expect(catalog.commands).toEqual([{ name: 'clear', kind: 'command' }]) }) - -it('describes nothing when the session reported names only', () => { - expect(readClaudeSlashCommands(init())).toEqual([ - { name: 'clear', kind: 'command' }, - { name: 'ref-oss', kind: 'skill' }, - { name: 'opsx:apply', kind: 'command' } - ]) - expect(new ClaudeSlashCommandCatalog(init()).commands).toEqual([ - { name: 'clear', kind: 'command' }, - { name: 'ref-oss', kind: 'skill' }, - { name: 'opsx:apply', kind: 'command' } - ]) -}) - -it('still hides terminal-only names however well the provider describes them', () => { - const catalog = new ClaudeSlashCommandCatalog(init()) - expect( - catalog.observe({ - type: 'system', - subtype: 'commands_changed', - commands: [ - { name: 'doctor', description: 'Diagnose the CLI install' }, - { name: 'ref-oss', description: 'A skill' } - ] - }) - ).toBe(true) - expect(catalog.commands).toEqual([{ name: 'ref-oss', kind: 'skill', description: 'A skill' }]) -}) diff --git a/src/main/claude/claude-streamed-text-checkpoints.test.ts b/src/main/claude/claude-streamed-text-checkpoints.test.ts index cc196cf21af8..89f825bc5605 100644 --- a/src/main/claude/claude-streamed-text-checkpoints.test.ts +++ b/src/main/claude/claude-streamed-text-checkpoints.test.ts @@ -216,23 +216,6 @@ describe('claude streamed text checkpoints', () => { expect(stamps).toEqual([CHILD_LINKAGE, { ...CHILD_LINKAGE, agentId: 'task-2' }]) }) - it('re-attributes a block that stopped streaming before its announcement', () => { - // Nothing revisits such a block: no later checkpoint, no final envelope. - // Without this it keeps the provisional id for the life of the journal. - const producer = scriptedProducer() - const { store, rows, stamps, runWindow } = checkpoints(producer.source) - - store.append(identityOf('block-1'), 'said once', 'toolu_1') - runWindow() - expect(stamps.at(-1)).toMatchObject({ agentId: 'toolu_1' }) - - producer.resolve(CHILD_LINKAGE) - store.reattribute() - - expect(rows.at(-1)).toEqual({ uuid: 'block-1', text: 'said once' }) - expect(stamps.at(-1)).toEqual(CHILD_LINKAGE) - }) - it('stops persisting once disposed', () => { const { store, rows, runWindow } = checkpoints() diff --git a/src/main/claude/claude-structured-content-parts.test.ts b/src/main/claude/claude-structured-content-parts.test.ts index 1d9ed800e0f8..312c1c10826c 100644 --- a/src/main/claude/claude-structured-content-parts.test.ts +++ b/src/main/claude/claude-structured-content-parts.test.ts @@ -121,18 +121,6 @@ describe('Claude message content parts', () => { expect(state.sink.publish).not.toHaveBeenCalled() }) - it('does not render injected image companions or start a turn', () => { - const state = sinkState() - const translator = createClaudeJournalTranslator({ sink: state.sink }) - const event = userMessageWith(null) - const content = [{ type: 'text', text: '[Image: source: /tmp/pasted.png]' }] - translator.handle({ - ...event, - message: { ...event.message, isMeta: true, message: { role: 'user', content } } - }) - expect(state.items).toEqual([]) - }) - it('does not leak a wire kind for a locally attached image', () => { const state = sinkState() const translator = createClaudeJournalTranslator({ sink: state.sink }) diff --git a/src/renderer/src/lib/pane-manager/pane-tree-ops.test.ts b/src/renderer/src/lib/pane-manager/pane-tree-ops.test.ts index 43e1ab93d63d..21c296b426eb 100644 --- a/src/renderer/src/lib/pane-manager/pane-tree-ops.test.ts +++ b/src/renderer/src/lib/pane-manager/pane-tree-ops.test.ts @@ -807,46 +807,6 @@ describe('safeFit', () => { describe('equalizePaneSplitSizes', () => { const pane = (flex = '1 1 0%'): MockHTMLElement => new MockHTMLElement(['pane'], [], flex) - const split = ( - direction: 'vertical' | 'horizontal', - children: MockHTMLElement[], - flex = '1 1 0%' - ): MockHTMLElement => - new MockHTMLElement( - ['pane-split', direction === 'vertical' ? 'is-vertical' : 'is-horizontal'], - children, - flex - ) - - it('weights nested same-axis splits so same-axis panes equalize evenly', () => { - const left = pane('10 1 0%') - const middle = pane('20 1 0%') - const right = pane('30 1 0%') - const rightSplit = split('vertical', [middle, right], '90 1 0%') - const root = split('vertical', [left, rightSplit]) - - expect(equalizePaneSplitSizes(root as unknown as HTMLElement)).toBe(true) - - expect(left.style.flex).toBe('1 1 0%') - expect(rightSplit.style.flex).toBe('2 1 0%') - expect(middle.style.flex).toBe('1 1 0%') - expect(right.style.flex).toBe('1 1 0%') - }) - - it('treats perpendicular child splits as one weighted region', () => { - const top = pane('7 1 0%') - const bottom = pane('3 1 0%') - const leftStack = split('horizontal', [top, bottom], '15 1 0%') - const right = pane('85 1 0%') - const root = split('vertical', [leftStack, right]) - - expect(equalizePaneSplitSizes(root as unknown as HTMLElement)).toBe(true) - - expect(leftStack.style.flex).toBe('1 1 0%') - expect(right.style.flex).toBe('1 1 0%') - expect(top.style.flex).toBe('1 1 0%') - expect(bottom.style.flex).toBe('1 1 0%') - }) it('returns false when there is no split tree to change', () => { expect(equalizePaneSplitSizes(pane() as unknown as HTMLElement)).toBe(false) diff --git a/src/renderer/src/store/slices/agent-status-drop-ipc.test.ts b/src/renderer/src/store/slices/agent-status-drop-ipc.test.ts index 53a0a4e131e0..9dff9574b661 100644 --- a/src/renderer/src/store/slices/agent-status-drop-ipc.test.ts +++ b/src/renderer/src/store/slices/agent-status-drop-ipc.test.ts @@ -61,21 +61,6 @@ describe('dropAgentStatus → IPC fan-out', () => { expect(drop).toHaveBeenCalledTimes(1) expect(drop).toHaveBeenCalledWith('tab-missing:0') }) - - it('idempotent: repeated drops on the same paneKey fire the IPC each time', () => { - // Why: the renderer keeps drop() side-effect free relative to its own - // state — sending an extra IPC for an already-dropped paneKey is safe - // because main-side dropStatusEntry is a no-op when the entry is gone. - // Asserting this contract documents the renderer's hands-off posture. - const { drop } = stubWindowApi() - const store = createTestStore() - store - .getState() - .setAgentStatus('tab-1:0', { state: 'working', prompt: 'p', agentType: 'claude' }) - store.getState().dropAgentStatus('tab-1:0') - store.getState().dropAgentStatus('tab-1:0') - expect(drop).toHaveBeenCalledTimes(2) - }) }) describe('dropAgentStatusByTabPrefix -> IPC fan-out', () => { diff --git a/src/renderer/src/store/slices/agent-status-runtime-orchestration.test.ts b/src/renderer/src/store/slices/agent-status-runtime-orchestration.test.ts index ecbe66d8d531..9dde73362943 100644 --- a/src/renderer/src/store/slices/agent-status-runtime-orchestration.test.ts +++ b/src/renderer/src/store/slices/agent-status-runtime-orchestration.test.ts @@ -221,33 +221,6 @@ describe('agent status runtime orchestration metadata', () => { }) }) - it.each(['failed', 'circuit_broken'] as const)( - 'updates runtime status to %s for the same dispatch', - (dispatchStatus) => { - vi.useFakeTimers() - const store = createTestStore() - const childPaneKey = 'tab-child:11111111-1111-4111-8111-111111111111' - - store.getState().setAgentStatus(childPaneKey, { - state: 'done', - prompt: 'child agent', - agentType: 'claude', - orchestration: { - taskId: 'task-1', - dispatchId: 'ctx-1', - dispatchStatus: 'dispatched' - } - }) - store.getState().setRuntimeAgentOrchestrationByPaneKey({ - [childPaneKey]: { taskId: 'task-1', dispatchId: 'ctx-1', dispatchStatus } - }) - - expect( - store.getState().agentStatusByPaneKey[childPaneKey].orchestration?.dispatchStatus - ).toBe(dispatchStatus) - } - ) - it('keeps current payload orchestration ahead of a stale runtime map entry', () => { vi.useFakeTimers() const store = createTestStore() diff --git a/src/renderer/src/store/slices/browser-webview-cleanup.test.ts b/src/renderer/src/store/slices/browser-webview-cleanup.test.ts index fa18e2e54ae6..e0b4999664cd 100644 --- a/src/renderer/src/store/slices/browser-webview-cleanup.test.ts +++ b/src/renderer/src/store/slices/browser-webview-cleanup.test.ts @@ -7,7 +7,6 @@ vi.mock('../../components/browser-pane/host-guest/webview-registry', () => ({ import { collectBrowserWebviewIds, - destroyRemovedBrowserWebview, destroyWorkspaceWebviews, destroyWorktreeBrowserGuests } from './browser-webview-cleanup' @@ -75,12 +74,6 @@ describe('destroyWorkspaceWebviews', () => { vi.mocked(destroyPersistentWebview).mockClear() }) - it('destroys the webview when the backing page is removed', () => { - destroyRemovedBrowserWebview('page-1') - - expect(destroyPersistentWebview).toHaveBeenCalledWith('page-1') - }) - it('destroys every page id for a multi-page workspace', () => { destroyWorkspaceWebviews( { 'workspace-1': [page('page-1', 'workspace-1'), page('page-2', 'workspace-1')] }, diff --git a/src/renderer/src/store/slices/editor-remote-branch-actions.test.ts b/src/renderer/src/store/slices/editor-remote-branch-actions.test.ts index de8dbe9651fd..9521dda3a103 100644 --- a/src/renderer/src/store/slices/editor-remote-branch-actions.test.ts +++ b/src/renderer/src/store/slices/editor-remote-branch-actions.test.ts @@ -376,35 +376,6 @@ describe('createEditorSlice remote branch actions', () => { expect(store.getState().isRemoteOperationActive).toBe(false) }) - it('maps publish updates-were-rejected into a clean actionable toast', async () => { - const store = createEditorStore() - const publishError = new Error( - 'Updates were rejected because the tip of your current branch is behind its remote counterpart.' - ) - gitPushMock.mockRejectedValueOnce(publishError) - - await expect(store.getState().pushBranch('wt-1', '/repo', true)).rejects.toThrow( - publishError.message - ) - - expect(toastErrorMock).toHaveBeenCalledWith( - 'Push rejected — remote has changes. Pull first, then try again.' - ) - await flushAsyncRemoteRefresh() - - expect(gitStatusMock).not.toHaveBeenCalled() - expect(gitFetchMock).toHaveBeenCalledWith({ - worktreePath: '/repo', - connectionId: undefined, - worktreeId: 'wt-1' - }) - expect(gitUpstreamStatusMock).toHaveBeenCalledWith({ - worktreePath: '/repo', - connectionId: undefined - }) - expect(store.getState().isRemoteOperationActive).toBe(false) - }) - it('maps raw publish wrapper errors into a cleaner actionable toast', async () => { const store = createEditorStore() const rawPublishError = new Error( @@ -467,33 +438,6 @@ describe('createEditorSlice remote branch actions', () => { expect(store.getState().isRemoteOperationActive).toBe(false) }) - it('maps non-fast-forward keyword push errors into a clean actionable toast', async () => { - const store = createEditorStore() - const pushError = new Error('Push rejected: remote has newer commits (non-fast-forward).') - gitPushMock.mockRejectedValueOnce(pushError) - - await expect(store.getState().pushBranch('wt-1', '/repo', false)).rejects.toThrow( - pushError.message - ) - - expect(toastErrorMock).toHaveBeenCalledWith( - 'Push rejected — remote has changes. Pull first, then try again.' - ) - await flushAsyncRemoteRefresh() - - expect(gitStatusMock).not.toHaveBeenCalled() - expect(gitFetchMock).toHaveBeenCalledWith({ - worktreePath: '/repo', - connectionId: undefined, - worktreeId: 'wt-1' - }) - expect(gitUpstreamStatusMock).toHaveBeenCalledWith({ - worktreePath: '/repo', - connectionId: undefined - }) - expect(store.getState().isRemoteOperationActive).toBe(false) - }) - it('surfaces submodule push failures with the submodule name', async () => { const store = createEditorStore() const pushError = new Error( diff --git a/src/renderer/src/store/slices/editor-right-sidebar-state.test.ts b/src/renderer/src/store/slices/editor-right-sidebar-state.test.ts index bb4716c28d22..1ee83e7e7767 100644 --- a/src/renderer/src/store/slices/editor-right-sidebar-state.test.ts +++ b/src/renderer/src/store/slices/editor-right-sidebar-state.test.ts @@ -95,19 +95,6 @@ describe('createEditorSlice right sidebar state', () => { expect(store.getState().rightSidebarOpen).toBe(false) }) - it('setRightSidebarOpen opens the sidebar', () => { - const store = createEditorStore() - store.getState().setRightSidebarOpen(true) - expect(store.getState().rightSidebarOpen).toBe(true) - }) - - it('setRightSidebarOpen(false) after open closes it', () => { - const store = createEditorStore() - store.getState().setRightSidebarOpen(true) - store.getState().setRightSidebarOpen(false) - expect(store.getState().rightSidebarOpen).toBe(false) - }) - it('toggleRightSidebar flips the state', () => { const store = createEditorStore() expect(store.getState().rightSidebarOpen).toBe(false) diff --git a/src/renderer/src/store/slices/github-checks-cache.test.ts b/src/renderer/src/store/slices/github-checks-cache.test.ts index 2b6c4c7f4ddb..abd66fd1f1fd 100644 --- a/src/renderer/src/store/slices/github-checks-cache.test.ts +++ b/src/renderer/src/store/slices/github-checks-cache.test.ts @@ -70,23 +70,6 @@ afterEach(() => { }) describe('createGitHubSlice.fetchPRChecks checks cache freshness', () => { - it('expires empty checks cache entries after the shorter empty TTL', async () => { - vi.useFakeTimers() - vi.setSystemTime(1_000) - const store = createTestStore() - const repoPath = '/repo' - const repoId = 'repo-id' - const branch = 'feature/test' - - mockApi.gh.prChecks.mockResolvedValue([]) - - await store.getState().fetchPRChecks(repoPath, 12, branch, undefined, null, { repoId }) - vi.setSystemTime(11_001) - await store.getState().fetchPRChecks(repoPath, 12, branch, undefined, null, { repoId }) - - expect(mockApi.gh.prChecks).toHaveBeenCalledTimes(2) - }) - it('keeps repeated automatic empty checks refreshes cacheable', async () => { vi.useFakeTimers() vi.setSystemTime(1_000) diff --git a/src/renderer/src/store/slices/ui-new-workspace-draft.test.ts b/src/renderer/src/store/slices/ui-new-workspace-draft.test.ts deleted file mode 100644 index 2a88ce0e0a1c..000000000000 --- a/src/renderer/src/store/slices/ui-new-workspace-draft.test.ts +++ /dev/null @@ -1,151 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { createUIStore } from './ui-slice-test-harness' - -const mocks = vi.hoisted(() => ({ - sendNotesToActiveAgentSession: vi.fn(), - track: vi.fn(), - toastMessage: vi.fn(), - toastSuccess: vi.fn(), - toastError: vi.fn() -})) - -vi.mock('@/lib/active-agent-note-send', () => ({ - activeAgentNotesSendFailureMessage: ( - status: string, - options: { explicitTarget?: boolean } = {} - ) => (options.explicitTarget ? `selected:${status}` : status), - sendNotesToActiveAgentSession: mocks.sendNotesToActiveAgentSession -})) - -vi.mock('@/lib/telemetry', () => ({ - track: mocks.track -})) - -vi.mock('sonner', () => ({ - toast: { - message: mocks.toastMessage, - success: mocks.toastSuccess, - error: mocks.toastError - } -})) - -afterEach(() => { - vi.restoreAllMocks() - vi.unstubAllGlobals() -}) - -beforeEach(() => { - mocks.sendNotesToActiveAgentSession.mockReset() - mocks.sendNotesToActiveAgentSession.mockResolvedValue({ status: 'sent' }) - mocks.track.mockReset() - mocks.toastMessage.mockReset() - mocks.toastSuccess.mockReset() - mocks.toastError.mockReset() -}) - -describe('createUISlice new workspace draft', () => { - it('preserves Linear linked work item metadata', () => { - const store = createUIStore() - - store.getState().setNewWorkspaceDraft({ - repoId: 'repo-1', - name: 'Fix launch context handoff', - prompt: '', - note: '', - attachments: [], - linkedWorkItem: { - type: 'issue', - number: 0, - title: 'Fix launch context handoff', - url: 'https://linear.app/acme/issue/ENG-123/fix-launch-context-handoff', - linearIdentifier: 'ENG-123' - }, - agent: 'claude', - linkedIssue: '', - linkedPR: null, - linkedGitLabIssue: null, - linkedGitLabMR: null - }) - - expect(store.getState().newWorkspaceDraft?.linkedWorkItem).toMatchObject({ - linearIdentifier: 'ENG-123' - }) - }) - - it('keeps older linked work item drafts without Linear context fields valid', () => { - const store = createUIStore() - - store.getState().setNewWorkspaceDraft({ - repoId: 'repo-1', - name: 'Legacy issue', - prompt: '', - note: '', - attachments: [], - linkedWorkItem: { - type: 'issue', - number: 42, - title: 'Legacy issue', - url: 'https://github.com/acme/repo/issues/42' - }, - agent: 'claude', - linkedIssue: '42', - linkedPR: null, - linkedGitLabIssue: null, - linkedGitLabMR: null - }) - - expect(store.getState().newWorkspaceDraft?.linkedWorkItem).toEqual({ - type: 'issue', - number: 42, - title: 'Legacy issue', - url: 'https://github.com/acme/repo/issues/42' - }) - }) - - it('preserves serializable Jira identity and bound source context in drafts', () => { - const store = createUIStore() - const linkedTaskSourceContext = { - kind: 'task-source' as const, - provider: 'jira' as const, - projectId: 'project-1', - hostId: 'runtime:env-1' as const, - providerIdentity: { - provider: 'jira' as const, - siteId: 'site-1', - siteUrl: 'https://company.atlassian.net', - projectKey: 'ORCA' - }, - accountLabel: 'ada@example.com' - } - - store.getState().setNewWorkspaceDraft({ - repoId: 'repo-1', - name: 'orca-123-link-jira', - prompt: '', - note: '', - attachments: [], - linkedWorkItem: { - provider: 'jira', - type: 'issue', - number: 0, - title: 'ORCA-123 Link Jira', - url: 'https://company.atlassian.net/browse/ORCA-123', - jiraIdentifier: 'ORCA-123' - }, - linkedTaskSourceContext, - agent: 'claude', - linkedIssue: '', - linkedPR: null, - linkedGitLabIssue: null, - linkedGitLabMR: null - }) - - expect(store.getState().newWorkspaceDraft).toMatchObject({ - linkedWorkItem: { - provider: 'jira', - jiraIdentifier: 'ORCA-123' - }, - linkedTaskSourceContext - }) - }) -}) diff --git a/src/shared/setup-runner-command.test.ts b/src/shared/setup-runner-command.test.ts index 4280ed4f6e02..b3e38bb438d6 100644 --- a/src/shared/setup-runner-command.test.ts +++ b/src/shared/setup-runner-command.test.ts @@ -98,14 +98,6 @@ describe('buildSetupRunnerCommand', () => { shell: 'windows' }) }) - - it('still uses bash for a POSIX runner launched from a POSIX pane', () => { - expect( - buildSetupRunnerCommand('C:\\repo\\.git\\orca\\setup-runner.sh', 'windows', { - family: 'posix' - }) - ).toBe('bash /c/repo/.git/orca/setup-runner.sh') - }) }) describe('buildSetupRunnerCommand cmd metacharacter guard', () => { diff --git a/src/shared/telemetry-events.test.ts b/src/shared/telemetry-events.test.ts index 031ca9bbd741..13dca1c08d14 100644 --- a/src/shared/telemetry-events.test.ts +++ b/src/shared/telemetry-events.test.ts @@ -4,13 +4,7 @@ // round-trips without coercion. import { describe, expect, it } from 'vitest' -import { - addRepoSetupStepActionSchema, - errorClassSchema, - eventSchemas, - isCohortExtendedEvent -} from './telemetry-events' -import { appStarSourceSchema } from './gh-star-source' +import { errorClassSchema, eventSchemas, isCohortExtendedEvent } from './telemetry-events' describe('feature_interaction_usage_bucket_reached schema', () => { it('accepts a valid bucket payload', () => { @@ -106,13 +100,6 @@ describe('feature_interaction_usage_bucket_reached schema', () => { }) describe('app_starred_orca schema', () => { - it('accepts every declared app star source', () => { - for (const source of appStarSourceSchema.options) { - const parsed = eventSchemas.app_starred_orca.safeParse({ source }) - expect(parsed.success).toBe(true) - } - }) - it('accepts cohort context on successful app star telemetry', () => { const parsed = eventSchemas.app_starred_orca.safeParse({ source: 'settings', @@ -471,13 +458,6 @@ describe('agent_hook_unattributed schema', () => { }) describe('add_repo_setup_step_action schema', () => { - it('accepts every Setup-step action declared in the schema', () => { - for (const action of addRepoSetupStepActionSchema.options) { - const parsed = eventSchemas.add_repo_setup_step_action.safeParse({ action }) - expect(parsed.success).toBe(true) - } - }) - it('rejects unknown action enum values', () => { const parsed = eventSchemas.add_repo_setup_step_action.safeParse({ action: 'export_to_pdf' diff --git a/src/shared/terminal-synchronized-output-scan.test.ts b/src/shared/terminal-synchronized-output-scan.test.ts index cc4c5dd2fc77..fa382ea4ad13 100644 --- a/src/shared/terminal-synchronized-output-scan.test.ts +++ b/src/shared/terminal-synchronized-output-scan.test.ts @@ -89,20 +89,6 @@ describe('resolveSynchronizedOutputSafeSplit', () => { expect(resolveSynchronizedOutputSafeSplit(data, 20)).toBe(20) }) - it('degrades to the plain limit when the buffer starts inside a frame', () => { - // Callers do not thread prior latch state, so a remainder that begins inside - // an already-open frame is scanned as if closed. It must never be WORSE than - // the blind offset it replaced: same boundary, byte-exact. - const data = `${'z'.repeat(40)}${CLOSE}${'q'.repeat(40)}` - const limit = 20 - const splitAt = resolveSynchronizedOutputSafeSplit(data, limit, '', true) - const naive = resolveSynchronizedOutputSafeSplit(data, limit) - // With the real prior state it can only do better or the same. - expect(splitAt).toBeLessThanOrEqual(limit) - expect(naive).toBeLessThanOrEqual(limit) - expect(data.slice(0, naive) + data.slice(naive)).toBe(data) - }) - it('never returns past the limit or breaks byte-exactness across many shapes', () => { const outputSamples = [ `${OPEN}${'a'.repeat(50)}${CLOSE}`, diff --git a/src/shared/text-search.test.ts b/src/shared/text-search.test.ts index 1517e070b4ee..455a591c2697 100644 --- a/src/shared/text-search.test.ts +++ b/src/shared/text-search.test.ts @@ -239,18 +239,6 @@ describe('buildGitGrepArgs', () => { expect(args).toContain(':(exclude,glob)dist/**') }) - it('excludes a directory subtree the way rg --glob does', () => { - const args = buildGitGrepArgs('q', { excludePattern: 'node_modules' }) - expect(args).toContain(':(exclude,glob)**/node_modules') - expect(args).toContain(':(exclude,glob)**/node_modules/**') - }) - - it('includes a directory subtree the way rg --glob does', () => { - const args = buildGitGrepArgs('q', { includePattern: 'src' }) - expect(args).toContain(':(glob)**/src') - expect(args).toContain(':(glob)**/src/**') - }) - it('keeps escaped commas inside one generated folder pathspec', () => { const args = buildGitGrepArgs('q', { includePattern: 'foo\\,bar/**, *.ts' }) expect(args).toContain(':(glob)foo\\,bar/**') diff --git a/src/shared/workspace-session-schema.sleeping-agent.test.ts b/src/shared/workspace-session-schema.sleeping-agent.test.ts index 57d4a832b463..3d3195bb98d1 100644 --- a/src/shared/workspace-session-schema.sleeping-agent.test.ts +++ b/src/shared/workspace-session-schema.sleeping-agent.test.ts @@ -457,35 +457,6 @@ describe('parseWorkspaceSession sleeping agents', () => { } }) - it('preserves legacy live sleeping agent origins across hydration', () => { - const result = parseWorkspaceSession({ - activeRepoId: null, - activeWorktreeId: null, - activeTabId: null, - tabsByWorktree: {}, - terminalLayoutsByTabId: {}, - sleepingAgentSessionsByPaneKey: { - 'tab1:pane-1': { - paneKey: 'tab1:pane-1', - tabId: 'tab1', - worktreeId: 'wt', - agent: 'codex', - providerSession: { key: 'session_id', id: 'codex-session' }, - prompt: 'continue', - state: 'working', - capturedAt: 10, - updatedAt: 9, - origin: 'live' - } - } - }) - - expect(result.ok).toBe(true) - if (result.ok) { - expect(result.value.sleepingAgentSessionsByPaneKey?.['tab1:pane-1']?.origin).toBe('live') - } - }) - it('drops malformed sleeping agent resume records without failing the whole session', () => { const result = parseWorkspaceSession({ activeRepoId: null, diff --git a/src/shared/workspace-session-terminal-buffers.test.ts b/src/shared/workspace-session-terminal-buffers.test.ts index c61a6936c47b..5eeaf5c98947 100644 --- a/src/shared/workspace-session-terminal-buffers.test.ts +++ b/src/shared/workspace-session-terminal-buffers.test.ts @@ -187,27 +187,6 @@ describe('pruneLocalTerminalScrollbackBuffers', () => { }) }) - it('caps preserved SSH buffers so session JSON cannot scale with raw scrollback', () => { - const hugeScrollback = `start-${'x'.repeat(TERMINAL_SCROLLBACK_SESSION_BUFFER_BYTE_LIMIT + 10)}` - const result = pruneLocalTerminalScrollbackBuffers( - makeSession({ - terminalLayoutsByTabId: { - 'remote-tab': { - root: null, - activeLeafId: null, - expandedLeafId: null, - buffersByLeafId: { 'pane:1': hugeScrollback } - } - } - }), - [{ id: 'remote-repo', connectionId: 'ssh-target-1' }] - ) - - const buffer = result.terminalLayoutsByTabId['remote-tab'].buffersByLeafId?.['pane:1'] - expect(buffer).toHaveLength(TERMINAL_SCROLLBACK_SESSION_BUFFER_BYTE_LIMIT) - expect(buffer?.startsWith('start-')).toBe(false) - }) - it('caps preserved SSH buffers by UTF-8 bytes for multibyte scrollback', () => { const multibyteRow = 'é'.repeat(1024) const hugeScrollback = multibyteRow.repeat(512) From 0c4b336c16b85345456ab4f963803bc04d6ad4bc Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 04:23:51 -0700 Subject: [PATCH 03/79] test: close the disclosed reading gap in agent-hooks, claude and store slices (#24126) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reads the 111 files that wave-26 auditors named as NOT REACHED in their own disclosures — the tracked remainder every prior wave ended with. Three chunks of 37, sized so finishing was achievable rather than optional. 8 case declarations removed across 5 files, 222 lines gone. No file deleted whole, no production code touched. The first chunk read all 37 of its files case-by-case (15,440 lines) and found one case: a strict subset of a sibling in the same file with an identical event sequence (UserPromptSubmit, two SubagentStart, Stop, child PermissionRequest, lead PreToolUse, staggered SubagentStop) and byte-identical final assertions, differing only in a prompt string and a child-id spelling. The surviving sibling additionally asserts `mainAgentState === 'working'` after the first child stops, so it is the stronger owner. The rest are duplicate invocations and cases whose input cannot reach the behavior they name, in `src/main/claude` and the renderer editor/worktree slices. The more useful result is a KEEP that corrects a rule I gave these auditors. A case iterating the production constant `CURSOR_EVENTS` looks exactly like the validator-tested-against-its-own-list pattern, and would have been deleted under the rule as I wrote it. It is legitimate: its per-entry expectation is a test-local map closed with `satisfies Record`, so adding a production event leaves a missing key and removing one leaves an excess key — either fails typecheck. The drift the tautology version cannot detect is caught here by the compiler rather than the assertion, and the loop still asserts the installed shell command carries that exact literal. That gives the pattern three variants and one question that resolves all of them: if production changed, would anything fail — an assertion, or the typechecker? Syntax does not answer it; provenance and enforcement do. Also kept, each verified rather than assumed: a generated-script test asserting guard ORDERING by `indexOf`, which is the only thing catching the #11549 hang class and the #2426/#15462 deny-by-silence class, and which no behavioral test can reach; two WSL suites that partly overlap but drive different layers, since only one reaches `WslHookRelayManager.ensureForDistro` for distro ownership and generation dispose/replace; and a thin Amp case kept because a single-path ripgrep confirmed `hasExplicitPrompt` for that provider is asserted nowhere else in the repo. Reported rather than acted on: a Grok case living at the end of `server-copilot-normalization.test.ts`. Misfiled, not junk, and HARD RULE 1 forbids moving it. No case was found where both a test and its supposed owner were inert. Verified: 601 test files / 5,951 cases pass across the touched areas; `check-reliability-gates.mjs` 140 gates; all 8 deleted case titles independently grepped against the gate manifest with zero hits. --- .../server-last-status-lead-boundary.test.ts | 56 --------- ...ured-journal-translation-subagents.test.ts | 12 -- .../claude/claude-structured-options.test.ts | 9 -- ...ude-transcript-marker-branch-proof.test.ts | 38 ------ .../src/store/slices/new-markdown.test.ts | 108 ------------------ ...-identity-review-clear-persistence.test.ts | 35 ------ 6 files changed, 258 deletions(-) diff --git a/src/main/agent-hooks/server-last-status-lead-boundary.test.ts b/src/main/agent-hooks/server-last-status-lead-boundary.test.ts index 7b7307d79159..a78476f21bcd 100644 --- a/src/main/agent-hooks/server-last-status-lead-boundary.test.ts +++ b/src/main/agent-hooks/server-last-status-lead-boundary.test.ts @@ -217,62 +217,6 @@ describe('Persisted Claude lead boundaries', () => { } }) - it('clears a persisted lead boundary when sticky permission hides new lead work', async () => { - const firstServer = new AgentHookServer() - await firstServer.start({ env: 'production', userDataPath }) - await postHookEvent( - firstServer, - buildBody({ hook_event_name: 'UserPromptSubmit', prompt: 'resume after boundary' }) - ) - await postHookEvent( - firstServer, - buildBody({ hook_event_name: 'SubagentStart', agent_id: 'achild-a' }) - ) - await postHookEvent( - firstServer, - buildBody({ hook_event_name: 'SubagentStart', agent_id: 'achild-b' }) - ) - await postHookEvent(firstServer, buildBody({ hook_event_name: 'Stop' })) - await postHookEvent( - firstServer, - buildBody({ - hook_event_name: 'PermissionRequest', - agent_id: 'achild-a', - tool_name: 'Bash', - tool_input: { command: 'false' } - }) - ) - await postHookEvent( - firstServer, - buildBody({ hook_event_name: 'PreToolUse', tool_name: 'Read' }) - ) - expect(firstServer.getStatusSnapshot()[0]).toMatchObject({ state: 'waiting', toolName: 'Bash' }) - expect(mainAgentState(firstServer)).toBe('working') - await postHookEvent( - firstServer, - buildBody({ hook_event_name: 'SubagentStop', agent_id: 'achild-a' }) - ) - firstServer.flushStatusPersistSync() - firstServer.stop() - - const server = new AgentHookServer() - await server.start({ env: 'production', userDataPath }) - try { - await postHookEvent( - server, - buildBody({ hook_event_name: 'SubagentStop', agent_id: 'achild-b' }) - ) - - expect(server.getStatusSnapshot()[0]).toMatchObject({ - state: 'working', - restoredUnconfirmed: true - }) - expect(server.getStatusChangeSnapshot()[0]?.observedInCurrentRuntime).toBe(false) - } finally { - server.stop() - } - }) - it('keeps a persisted lead boundary unconfirmed after an unrelated child stop', async () => { const firstServer = new AgentHookServer() await firstServer.start({ env: 'production', userDataPath }) diff --git a/src/main/claude/claude-structured-journal-translation-subagents.test.ts b/src/main/claude/claude-structured-journal-translation-subagents.test.ts index 89df86122b8d..e1da571eb348 100644 --- a/src/main/claude/claude-structured-journal-translation-subagents.test.ts +++ b/src/main/claude/claude-structured-journal-translation-subagents.test.ts @@ -471,18 +471,6 @@ describe('claude journal translation — which agent produced a row', () => { }) }) - it('stamps a spawn call a release never announces with the call\u2019s own id', () => { - // Older releases name nothing they spawn. The spawn call is still in the - // transcript, so its id is a real handle — and stamping it keeps the child's - // prose off the parent, which reading these rows as root would not. - const { translator, linkageOfProse } = harness() - translator.handle(userTurn('user-1')) - translator.handle(spawnCall('assistant-1', 'toolu_1')) - translator.handle(childProse('child-1', 'toolu_1', 'unannounced release')) - - expect(linkageOfProse('unannounced release')).toMatchObject({ agentId: 'toolu_1' }) - }) - it('never reads a row naming a parent as the session\u2019s own, whatever the release', () => { // The hardest case for attribution: a sidechain id no spawn call forwarded, // on a release that has announced nothing, so no announcement is coming and diff --git a/src/main/claude/claude-structured-options.test.ts b/src/main/claude/claude-structured-options.test.ts index a92149a2cddf..e5b7a7ea8a77 100644 --- a/src/main/claude/claude-structured-options.test.ts +++ b/src/main/claude/claude-structured-options.test.ts @@ -133,15 +133,6 @@ describe('Claude structured Fast mode', () => { expect(applyFlagSettings).not.toHaveBeenCalled() }) - it('does not authorize a new Fast enable when model support is unknown', async () => { - const { session, applyFlagSettings } = fastModeSession(undefined) - - await expect( - setClaudeStructuredOption(session, { key: 'fastMode', value: 'true' }, undefined) - ).rejects.toThrow('does not support Fast mode') - expect(applyFlagSettings).not.toHaveBeenCalled() - }) - it.each([undefined, false])( 'allows explicit Fast off when model support is %s', async (supportsFastMode) => { diff --git a/src/main/claude/claude-transcript-marker-branch-proof.test.ts b/src/main/claude/claude-transcript-marker-branch-proof.test.ts index befb320b9e58..51d3f03880c2 100644 --- a/src/main/claude/claude-transcript-marker-branch-proof.test.ts +++ b/src/main/claude/claude-transcript-marker-branch-proof.test.ts @@ -277,44 +277,6 @@ describe('Claude transcript marker branch proof', () => { ).rejects.toThrow('not on the main transcript') }) - it('rejects a latest marker descended from a parent-tool-use cursor sidechain', async () => { - const root = await makeRoot('orca-native-chat-resolve-claude-parent-tool-cursor-descendant-') - const transcript = join(root, 'transcript.jsonl') - await writeFile( - transcript, - [ - { type: 'user', uuid: 'main-user', parentUuid: null, sessionId: 'session-1' }, - { - type: 'assistant', - uuid: 'subagent-assistant', - parentUuid: 'main-user', - sessionId: 'session-1', - parent_tool_use_id: 'tool-use-1' - }, - { - type: 'assistant', - uuid: 'main-after-sidechain', - parentUuid: 'subagent-assistant', - sessionId: 'session-1' - }, - { - type: 'assistant', - uuid: 'latest-after-sidechain', - parentUuid: 'main-after-sidechain', - sessionId: 'session-1' - }, - { type: 'last-prompt', leafUuid: 'latest-after-sidechain', sessionId: 'session-1' } - ] - .map((record) => JSON.stringify(record)) - .join('\n'), - 'utf8' - ) - - await expect( - proveMarkerLeafUuid(transcript, 'session-1', 'main-after-sidechain') - ).rejects.toThrow('not on the main transcript') - }) - it('rejects a post-snapshot descendant whose parent row was observed later', async () => { const root = await makeRoot('orca-native-chat-resolve-claude-post-snapshot-') const transcript = join(root, 'transcript.jsonl') diff --git a/src/renderer/src/store/slices/new-markdown.test.ts b/src/renderer/src/store/slices/new-markdown.test.ts index fe592dd7bf96..4c1ced68617c 100644 --- a/src/renderer/src/store/slices/new-markdown.test.ts +++ b/src/renderer/src/store/slices/new-markdown.test.ts @@ -125,38 +125,6 @@ describe('New Markdown — isUntitled flag', () => { }) describe('New Markdown — tab bar ordering with openFile', () => { - it('appends new file at the end of the tab order', () => { - const store = createEditorStore({ - tabsByWorktree: { - 'wt-1': [ - { - id: 't1', - ptyId: null, - worktreeId: 'wt-1', - title: 'Term 1', - customTitle: null, - color: null, - sortOrder: 0, - createdAt: 0 - } - ] - }, - tabBarOrderByWorktree: { 'wt-1': ['t1'] } - }) - - store.getState().openFile({ - filePath: '/repo/untitled.md', - relativePath: 'untitled.md', - worktreeId: 'wt-1', - language: 'markdown', - isUntitled: true, - mode: 'edit' - }) - - const order = store.getState().tabBarOrderByWorktree['wt-1'] - expect(order.at(-1)).toBe('/repo/untitled.md') - }) - it('appends after browser tabs (not before them)', () => { const store = createEditorStore({ tabsByWorktree: { @@ -463,28 +431,6 @@ describe('New Markdown — rename flow store operations', () => { store.getState().closeFile('/repo/untitled.md') expect(store.getState().editorDrafts['/repo/untitled.md']).toBeUndefined() }) - - it('same-name save clears isUntitled via clearUntitled', () => { - const store = createEditorStore({ - tabBarOrderByWorktree: { 'wt-1': [] } - }) - - store.getState().openFile({ - filePath: '/repo/untitled.md', - relativePath: 'untitled.md', - worktreeId: 'wt-1', - language: 'markdown', - isUntitled: true, - mode: 'edit' - }) - expect(store.getState().openFiles[0].isUntitled).toBe(true) - - store.getState().clearUntitled('/repo/untitled.md') - expect(store.getState().openFiles[0].isUntitled).toBeUndefined() - // File should still exist - expect(store.getState().openFiles).toHaveLength(1) - expect(store.getState().openFiles[0].id).toBe('/repo/untitled.md') - }) }) describe('New Markdown — markFileDirty interaction', () => { @@ -572,60 +518,6 @@ describe('Tab close — tabBarOrderByWorktree cleanup', () => { expect(store.getState().tabBarOrderByWorktree['wt-1']).toContain('t1') }) - it('closeFile does not disturb other tab positions', () => { - const store = createEditorStore({ - tabsByWorktree: { - 'wt-1': [ - { - id: 't1', - ptyId: null, - worktreeId: 'wt-1', - title: 'Term 1', - customTitle: null, - color: null, - sortOrder: 0, - createdAt: 0 - }, - { - id: 't2', - ptyId: null, - worktreeId: 'wt-1', - title: 'Term 2', - customTitle: null, - color: null, - sortOrder: 1, - createdAt: 1 - } - ] - }, - browserTabsByWorktree: { - 'wt-1': [makeBrowserTab('b1')] - }, - tabBarOrderByWorktree: { 'wt-1': ['t1', 'b1', 't2'] } - }) - - store.getState().openFile({ - filePath: '/repo/file.md', - relativePath: 'file.md', - worktreeId: 'wt-1', - language: 'markdown', - mode: 'edit' - }) - // Order: t1, b1, t2, /repo/file.md - store.getState().openFile({ - filePath: '/repo/file2.md', - relativePath: 'file2.md', - worktreeId: 'wt-1', - language: 'markdown', - mode: 'edit' - }) - // Order: t1, b1, t2, /repo/file.md, /repo/file2.md - - store.getState().closeFile('/repo/file.md') - const order = store.getState().tabBarOrderByWorktree['wt-1'] - expect(order).toEqual(['t1', 'b1', 't2', '/repo/file2.md']) - }) - it('closing multiple files preserves remaining tab order', () => { const store = createEditorStore({ tabsByWorktree: { diff --git a/src/renderer/src/store/slices/worktrees-git-identity-review-clear-persistence.test.ts b/src/renderer/src/store/slices/worktrees-git-identity-review-clear-persistence.test.ts index bdc73d1f8b0a..25b18feadb0d 100644 --- a/src/renderer/src/store/slices/worktrees-git-identity-review-clear-persistence.test.ts +++ b/src/renderer/src/store/slices/worktrees-git-identity-review-clear-persistence.test.ts @@ -34,41 +34,6 @@ describe('updateWorktreeGitIdentity', () => { resetHostedReviewLinkMutationGenerationForTests() }) - it('persists cleared branch-scoped linked reviews when git status observes a branch switch', async () => { - const store = createTestStore() - const existing = makeWorktree({ - id: 'repo1::/path/wt1', - repoId: 'repo1', - path: '/path/wt1', - branch: 'refs/heads/stack/one', - linkedPR: 101, - linkedGitLabMR: 102, - linkedBitbucketPR: 103, - linkedAzureDevOpsPR: 104, - linkedGiteaPR: 105, - pushTarget: { remoteName: 'fork', branchName: 'old/review-head' } - }) - - store.setState({ worktreesByRepo: { repo1: [existing] } } as Partial) - - store.getState().updateWorktreeGitIdentity('repo1::/path/wt1', { - branch: 'refs/heads/stack/two' - }) - await Promise.resolve() - - expect(mockApi.worktrees.updateMeta).toHaveBeenCalledWith({ - worktreeId: 'repo1::/path/wt1', - updates: { - linkedPR: null, - linkedGitLabMR: null, - linkedBitbucketPR: null, - linkedAzureDevOpsPR: null, - linkedGiteaPR: null, - pushTarget: undefined - } - }) - }) - it('persists cleared branch-scoped push target when git status observes a branch switch', async () => { const store = createTestStore() const existing = makeWorktree({ From 85f8d6b5f507df795cd3cef1cdea08124cf801ee Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 04:58:31 -0700 Subject: [PATCH 04/79] test: retire long-tail cases whose assertion is decided by the test itself (#24132) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Resumes the backlog sweep at a chunk size that actually gets read. Six auditors, 84 files each, and all six read their full scope case-by-case against production — the first wave where every chunk closed with no gap. 33 case declarations removed across 22 files, 1 test file deleted, 826 lines gone. No production code touched. This wave exists because a conclusion of mine was wrong. I had recorded that yield collapsed ~36x and that deletion was no longer the high-value work. I was dividing cases removed by files IN SCOPE while the fraction auditors actually READ fell from 100% to about 4%, because I kept handing them 300-800 files. Recomputed against files read, yield has been flat at 4-7 per 100 with no downward trend. This wave came in at 8.2. The most instructive removal looked like the most valuable test in scope. `orchestration-worker-release-reap-fixed.func.test.ts` cites a production bug by two identifiers, describes orphaned PTYs accumulating until `TasksMax=4096` aborts processes on EAGAIN, and advertises itself as the functional tier wiring the real orchestration RPC surface, the real `OrchestrationDb` and the real release modules. Deleting it leaves no reference to that bug anywhere in `src`. It still had to go: its fake runtime performed the fence it asserted — if (pty.incarnationId !== inc) { return null } handleTable.set('term_reminted', { ptyId, epoch: rendererGraphEpoch }) — so the case checking that a reused ptyId with a mismatched incarnation does not resolve was checking a decision its own spy made twenty lines earlier. The real fence is owned by `orca-runtime-terminal-handle-incarnation.test.ts:257`, and the other two cases replay `orchestration-worker-release-incarnation-fallback.test.ts` (which uses a plain `mockReturnValue` rather than reimplementing the remint) and `worker/worker-release.test.ts:23`. "Integration test" and "wires real modules" describe the scaffolding, not the asserted step. Other removals: a self-comparison disguised by an alias, where `export const getIssueOwnerRepo = getOwnerRepo` makes a case asserting the two "agree" into `f(x) === f(x)`; four cases whose `vi.mock` of `resolveIssueSource` made both the preference value and the topology inert; five verdict-precedence cases owned by a verdict-agnostic block; three call-shape probes on one-line store pass-throughs whose real contracts are driven by behavioural neighbours; and a `export type _Ref = [...]` declaration whose own comment admits it exists only to preserve test-only module-surface references. Kept after checking production rather than shape. An auditor found two near-identical ten-reconnect loops and kept both: one uses a test-local live-lease filter, the other the shipped `sshRemotePtyLeaseAllowsReattach` predicate, and the file's own comment explains the duality is deliberate "so the two cannot drift". Another kept a paths-alignment case that looks like a validator tested against its own list, because adding a generated file without registering its path does fail it — and `shellReadyWrappersExist` uses that registered list to decide whether a partial tree needs regeneration. Production duplication is now confirmed four times over, and it is why mirrored tests exist: `createUpdateWorktreeLineage`/`createAssignWorktreeParent` differ by one `console.error` string; `terminal-path-tap.ts` and `document/path-tap.ts` carry hand-maintained copies of `matchFilePathAtColumn` under a docblock reading "keep the two in sync". In those cases both test sides are load-bearing and the duplication belongs on a refactor list. `mobile/tests-typecheck-baseline.txt` loses one entry. Trimming `relay-host-signed-out-verdict.test.ts` made it typecheck clean, so the ratchet required pruning its grandfathered entry — the file graduates from exempt to enforced. Baseline is now 124 entries, down from 125. Verified: 690 test files / 7,560 cases pass across the touched desktop areas; the modified mobile files pass (162 cases); `check-tests-typecheck-ratchet.mjs` OK (898 files in program, 124 grandfathered); `check-reliability-gates.mjs` 140 gates; the deleted file is absent from the gate manifest, `cloud/package.json` and the mobile baseline; nothing under `mobile/src/test-support/rpc-recording/` or `mobile/rpc-foundation/goldens/` touched. --- .../connection-diagnostics-analysis.test.ts | 17 - .../mobile-dictation-desktop-start.test.ts | 20 - .../bridge/page-first-paint.test.tsx | 25 -- .../bridge/use-native-verbs.test.tsx | 5 - .../catch-all-page-route.test.tsx | 8 - .../tasks/composer-linked-work-item.test.ts | 6 - .../workspace-source-reply-schema.test.ts | 9 - .../mobile-endpoint-supervisor-nudge.test.ts | 14 - .../mobile-web-bundle-compat.test.ts | 6 - .../relay-host-signed-out-verdict.test.ts | 36 -- .../rpc-client-stream-registry.test.ts | 20 - mobile/tests-typecheck-baseline.txt | 1 - src/main/github/client-issue-source.test.ts | 63 --- src/main/github/client-pr-push-target.test.ts | 26 -- src/main/github/gh-utils.test.ts | 10 - ...epository-identity.fork-owner-repo.test.ts | 9 +- src/main/github/work-item-details.test.ts | 70 ---- ...ence-ssh-remote-pty-binding-replay.test.ts | 57 --- ...epo-git-remote-identity-enrichment.test.ts | 18 - src/main/runtime/rpc/e2ee-channel.test.ts | 13 - ...ion-worker-release-reap-fixed.func.test.ts | 358 ------------------ src/main/ssh/ssh-connection-store.test.ts | 25 -- .../ssh/ssh-relay-ripgrep-cache-gc.test.ts | 13 - 23 files changed, 1 insertion(+), 828 deletions(-) delete mode 100644 src/main/runtime/rpc/methods/orchestration-worker-release-reap-fixed.func.test.ts diff --git a/mobile/src/diagnostics/connection-diagnostics-analysis.test.ts b/mobile/src/diagnostics/connection-diagnostics-analysis.test.ts index 762d82b10d50..ab40b3511b27 100644 --- a/mobile/src/diagnostics/connection-diagnostics-analysis.test.ts +++ b/mobile/src/diagnostics/connection-diagnostics-analysis.test.ts @@ -98,23 +98,6 @@ describe('diagnoseConnection', () => { }) }) - it('marks authenticated Relay liveness failures as safe to send', () => { - expect( - diagnoseConnection({ - endpoint: 'ws://192.168.1.2:6768', - state: 'reconnecting', - activePath: 'relay', - entries: [ - { - ...event('Relay health check failed'), - code: 'liveness-timeout', - path: 'relay' - } - ] - }).reportability - ).toBe('orca-relay') - }) - it.each([ ['direct timeout', 'connect-timeout', 'tailscale'], ['handshake timeout', 'handshake-timeout', 'relay'], diff --git a/mobile/src/hooks/mobile-dictation-desktop-start.test.ts b/mobile/src/hooks/mobile-dictation-desktop-start.test.ts index 6fdd66a2354b..8166028d2ddc 100644 --- a/mobile/src/hooks/mobile-dictation-desktop-start.test.ts +++ b/mobile/src/hooks/mobile-dictation-desktop-start.test.ts @@ -150,26 +150,6 @@ describe('startMobileDictationDesktopSession', () => { }) }) - it('does not surface a desktop-start failure after the start became stale', async () => { - let setNewerStart = () => undefined - const harness = createStartHarness({ - sendRequest: async (method) => { - if (method === 'speech.dictation.start') { - setNewerStart() - throw new Error('Desktop start failed') - } - return OK_RESPONSE - } - }) - setNewerStart = harness.setNewerStart - - await expect(startMobileDictationDesktopSession(harness.options)).resolves.toBe(false) - - expect(harness.setIdle).not.toHaveBeenCalled() - expect(harness.getActiveId()).toBe('dictation-b') - expect(harness.commitRecordingStart).not.toHaveBeenCalled() - }) - it('commits recording before returning a current start to the hook', async () => { const harness = createStartHarness() diff --git a/mobile/src/mobile-web-shell/bridge/page-first-paint.test.tsx b/mobile/src/mobile-web-shell/bridge/page-first-paint.test.tsx index a397729a9e8c..0699f330a2f7 100644 --- a/mobile/src/mobile-web-shell/bridge/page-first-paint.test.tsx +++ b/mobile/src/mobile-web-shell/bridge/page-first-paint.test.tsx @@ -188,29 +188,4 @@ describe('which commit the page reports its frame from', () => { }) expect(reports).toBe(1) }) - - it('reports the screen that arrived and not the one that replaced it', async () => { - const route = deferredRouteChunk() - const Screen = lazy(() => route.chunk.then(withRouteScreenPaintReport)) - let reports = 0 - await act(async () => { - create( - { - reports += 1 - return () => undefined - }} - > - - - - - ) - }) - await act(async () => { - route.arrive() - }) - // Once per screen that commits: the shell latches the first, and a re-render is not a new one. - expect(reports).toBe(1) - }) }) diff --git a/mobile/src/mobile-web-shell/bridge/use-native-verbs.test.tsx b/mobile/src/mobile-web-shell/bridge/use-native-verbs.test.tsx index 9ce2cb04a91d..77fcd9dd0775 100644 --- a/mobile/src/mobile-web-shell/bridge/use-native-verbs.test.tsx +++ b/mobile/src/mobile-web-shell/bridge/use-native-verbs.test.tsx @@ -197,11 +197,6 @@ describe('a verb the shell refuses', () => { expect(reasons).toEqual(['native_verb_failed', 'reply-too-large', 'native_verb_result']) }) - it('names the frame refusal when the reply could never have reached the page', async () => { - const error = await rejectionFrom(() => Promise.resolve({ value: 'a'.repeat(9 * 1024 * 1024) })) - expect(error.reason).toBe('reply-too-large') - }) - it('names the grant when this side refused before sending', async () => { const page = createPageClient() page.deliver({ ...INIT, grants: { ...GRANTS, native: ['navigate'] } }) diff --git a/mobile/src/mobile-web-shell/catch-all-page-route.test.tsx b/mobile/src/mobile-web-shell/catch-all-page-route.test.tsx index 969157ff7810..e4ab0593e5ed 100644 --- a/mobile/src/mobile-web-shell/catch-all-page-route.test.tsx +++ b/mobile/src/mobile-web-shell/catch-all-page-route.test.tsx @@ -176,14 +176,6 @@ describe('the catch-all switch', () => { expect(dependencies.routes[0]?.params).toEqual({ tab: 'diff' }) }) - it('never carries its own segments back as query params', async () => { - // `useLocalSearchParams` merges the route params into the search params, and both of these are - // already in the pathname above. - dependencies.params = { hostId: 'host-1', page: ['settings'] } - await render() - expect(dependencies.routes[0]).not.toHaveProperty('params') - }) - it('refuses a deep link carrying more params than the bridge will take', async () => { // The schema bounds them, so `shellScreenRoute` answers null and the switch refuses. Dropping // the overflow instead would open the page on a screen missing the state it was asked for. diff --git a/mobile/src/tasks/composer-linked-work-item.test.ts b/mobile/src/tasks/composer-linked-work-item.test.ts index f3ce9299e922..faf5d80cd73e 100644 --- a/mobile/src/tasks/composer-linked-work-item.test.ts +++ b/mobile/src/tasks/composer-linked-work-item.test.ts @@ -1,7 +1,4 @@ import { describe, expect, it } from 'vitest' -import type { GitHubWorkItem } from '../../../src/shared/github/work-item-types' -import type { GitLabWorkItem } from '../../../src/shared/gitlab-types' -import type { LinearIssue } from '../../../src/shared/linear/issue-types' import { buildGitHubLinkedWorkItem, buildGitLabLinkedWorkItem, @@ -240,6 +237,3 @@ describe('resolveComposerBranchPick', () => { expect(pick.base.branchNameOverride).toBeUndefined() }) }) - -// Keep the exported type aliases referenced so the module surface stays covered. -export type _Ref = [GitHubWorkItem, GitLabWorkItem, LinearIssue] diff --git a/mobile/src/tasks/workspace-source-reply-schema.test.ts b/mobile/src/tasks/workspace-source-reply-schema.test.ts index 30597e8e5368..74ccecfbd041 100644 --- a/mobile/src/tasks/workspace-source-reply-schema.test.ts +++ b/mobile/src/tasks/workspace-source-reply-schema.test.ts @@ -256,15 +256,6 @@ describe('the orca.yaml hooks require nothing', () => { }) describe('sparse presets', () => { - it('reads the recorded preset, which carries no repoId or timestamps', () => { - const parsed = repoSparsePresetListSchema.safeParse({ - presets: [{ id: 'p1', name: 'docs', directories: ['docs'] }] - }) - expect(parsed.success && parsed.data).toEqual([ - { id: 'p1', name: 'docs', directories: ['docs'] } - ]) - }) - it('drops a preset with no id, which the picker could not select', () => { const parsed = repoSparsePresetListSchema.safeParse({ presets: [{ name: 'docs' }] }) expect(parsed.success && parsed.data).toEqual([]) diff --git a/mobile/src/transport/mobile-endpoint-supervisor-nudge.test.ts b/mobile/src/transport/mobile-endpoint-supervisor-nudge.test.ts index 86ea50e5c452..6fd90b8f86d3 100644 --- a/mobile/src/transport/mobile-endpoint-supervisor-nudge.test.ts +++ b/mobile/src/transport/mobile-endpoint-supervisor-nudge.test.ts @@ -136,20 +136,6 @@ describe('mobile endpoint supervisor nudges', () => { supervisor.stop() }) - it('does not suspend a relay from one focus RPC failure', async () => { - const logical = new FakeLogicalClient('connected', 'relay') - const deps = dependencies() - const supervisor = new MobileEndpointSupervisor(logical, host.id, relay, deps) - await supervisor.start() - - supervisor.nudge('focus') - await vi.advanceTimersByTimeAsync(0) - expect(logical.suspendActiveSession).not.toHaveBeenCalled() - expect(deps.openRelay).not.toHaveBeenCalled() - expect(logical.getState()).toBe('connected') - supervisor.stop() - }) - it('queues a network replacement that lands while another dial owns the mutex', async () => { const logical = new FakeLogicalClient('disconnected', 'lan') let resolveWrite: (() => void) | null = null diff --git a/mobile/src/transport/mobile-web-bundle-compat.test.ts b/mobile/src/transport/mobile-web-bundle-compat.test.ts index 4fea3dab1767..5a8f63501de3 100644 --- a/mobile/src/transport/mobile-web-bundle-compat.test.ts +++ b/mobile/src/transport/mobile-web-bundle-compat.test.ts @@ -55,12 +55,6 @@ describe('evaluateMobileWebBundleCompat', () => { }) }) - it('separates permission to fetch a manifest from permission to open one', () => { - // Why: both are `ok`, and a caller that mounted on the first would mount an unchecked bundle. - expect(evaluate({ manifest: null })).toEqual({ kind: 'ok', manifestChecked: false }) - expect(evaluate({})).toEqual({ kind: 'ok', manifestChecked: true }) - }) - it('blocks a host that ships no bundle', () => { expect(evaluate({ hostCapabilities: ['browser.screencast.v1'] })).toEqual({ kind: 'blocked', diff --git a/mobile/src/transport/relay-host-signed-out-verdict.test.ts b/mobile/src/transport/relay-host-signed-out-verdict.test.ts index e27e241706c3..ce0f01844eed 100644 --- a/mobile/src/transport/relay-host-signed-out-verdict.test.ts +++ b/mobile/src/transport/relay-host-signed-out-verdict.test.ts @@ -234,40 +234,4 @@ describe('classifyConnection with a signed-out desktop', () => { classifyConnection({ ...base, state: 'reconnecting', reconnectAttempts: 20 }).label ).toBe(SIGNED_OUT_LABEL) }) - - it('reads as stale once this session had been connected', () => { - expect( - classifyConnection({ ...base, state: 'reconnecting', lastConnectedAt: 1, nowMs: 2 }).reason - ).toBe('stale') - }) - - // A Tailscale endpoint cannot make "sign in on your desktop" better advice. - it('never appends the Tailscale hint', () => { - expect( - classifyConnection({ ...base, state: 'reconnecting', endpoint: '100.64.0.1' }) - ).not.toHaveProperty('hint') - }) - - it('never outranks a connected session', () => { - expect(classifyConnection({ ...base, state: 'connected' }).label).toBe('Connected') - }) - - // Re-pairing, not signing in, is the remedy when the pairing itself is dead. - it('never outranks a revoked pairing', () => { - expect(classifyConnection({ ...base, state: 'reconnecting', pairingRejected: true }).kind).toBe( - 'auth-failed' - ) - }) - - it('leaves every other verdict alone when the desktop is not signed out', () => { - expect( - classifyConnection({ - state: 'connecting', - reconnectAttempts: 0, - lastConnectedAt: null, - pendingPath: 'relay', - relayHostReachability: 'connecting' - }).label - ).toBe('Connecting via Relay…') - }) }) diff --git a/mobile/src/transport/rpc-client-stream-registry.test.ts b/mobile/src/transport/rpc-client-stream-registry.test.ts index 48c62af478d6..c58b3d938f7b 100644 --- a/mobile/src/transport/rpc-client-stream-registry.test.ts +++ b/mobile/src/transport/rpc-client-stream-registry.test.ts @@ -275,26 +275,6 @@ describe('RpcClientStreamRegistry', () => { }) }) - it('keeps a disposed browser tombstone until ready can be unsubscribed', () => { - const { registry, sent } = createRegistry() - const dispose = registry.subscribe('browser.screencast', { page: 'page-1' }, () => {}) - const request = sent[0]! - - dispose() - expect(sent).toHaveLength(1) - - registry.handleResponse( - streamingResponse(request.id, { - type: 'ready', - subscriptionId: 'browser-screencast:page-1:test' - }) - ) - expect(sent[1]).toMatchObject({ - method: 'browser.screencast.unsubscribe', - params: { subscriptionId: 'browser-screencast:page-1:test' } - }) - }) - it('releases a replayed browser stream replaced by a new one before its ready', () => { const { registry, sent } = createRegistry() registry.subscribe('browser.screencast', { page: 'page-1' }, () => {}) diff --git a/mobile/tests-typecheck-baseline.txt b/mobile/tests-typecheck-baseline.txt index ab63d955f7ad..9d47b7ab801d 100644 --- a/mobile/tests-typecheck-baseline.txt +++ b/mobile/tests-typecheck-baseline.txt @@ -115,7 +115,6 @@ src/transport/mobile-relay-rpc-session.test.ts src/transport/mobile-relay-runtime-failover.test.ts src/transport/pairing-relay-candidate.test.ts src/transport/pre-profile-pairing-coordinator.test.ts -src/transport/relay-host-signed-out-verdict.test.ts src/transport/rpc-client-synthesized-close-diagnostics.test.ts src/transport/rpc-operation.test.ts src/transport/rpc-session-liveness-watchdog.test.ts diff --git a/src/main/github/client-issue-source.test.ts b/src/main/github/client-issue-source.test.ts index d9c593be3a0a..a4f8b6468314 100644 --- a/src/main/github/client-issue-source.test.ts +++ b/src/main/github/client-issue-source.test.ts @@ -730,23 +730,6 @@ describe('GitHub issue source split', () => { expect(result.issueSourceFellBack).toBeUndefined() }) - it("preference='auto' + no upstream → queries origin", async () => { - resolveIssueSourceMock.mockResolvedValueOnce({ - source: { owner: 'solo', repo: 'orca' }, - fellBack: false - }) - getOwnerRepoMock.mockResolvedValueOnce({ owner: 'solo', repo: 'orca' }) - ghExecFileAsyncMock.mockResolvedValueOnce({ stdout: '[]' }).mockResolvedValueOnce({ - stdout: '[]' - }) - - await listWorkItems('/repo-root', 10, undefined, undefined, 'auto') - - expect(ghExecFileAsyncMock).toHaveBeenNthCalledWith(1, issueSearchArgs('solo/orca'), { - cwd: '/repo-root' - }) - }) - it("preference='auto' + upstream exists → PRs query upstream too", async () => { // Why: fork-contribution PRs live on the upstream repo — the fork's own // PR list is almost always empty. 'auto' must resolve PRs upstream-first @@ -799,22 +782,6 @@ describe('GitHub issue source split', () => { ) }) - it("preference='upstream' + upstream exists → queries upstream", async () => { - resolveIssueSourceMock.mockResolvedValueOnce({ - source: { owner: 'stablyai', repo: 'orca' }, - fellBack: false - }) - getOwnerRepoMock.mockResolvedValueOnce({ owner: 'fork', repo: 'orca' }) - ghExecFileAsyncMock.mockResolvedValueOnce({ stdout: '[]' }).mockResolvedValueOnce({ - stdout: '[]' - }) - - const result = await listWorkItems('/repo-root', 10, undefined, undefined, 'upstream') - - expect(decodedIssueSearchPath(0)).toContain('q=repo:stablyai/orca is:issue is:open') - expect(result.issueSourceFellBack).toBeUndefined() - }) - it("preference='upstream' + no upstream → falls back to origin with fellBack=true", async () => { resolveIssueSourceMock.mockResolvedValueOnce({ source: { owner: 'solo', repo: 'orca' }, @@ -831,36 +798,6 @@ describe('GitHub issue source split', () => { expect(result.issueSourceFellBack).toBe(true) }) - it("preference='origin' + upstream exists → queries origin (not upstream)", async () => { - resolveIssueSourceMock.mockResolvedValueOnce({ - source: { owner: 'fork', repo: 'orca' }, - fellBack: false - }) - getOwnerRepoMock.mockResolvedValueOnce({ owner: 'fork', repo: 'orca' }) - ghExecFileAsyncMock.mockResolvedValueOnce({ stdout: '[]' }).mockResolvedValueOnce({ - stdout: '[]' - }) - - await listWorkItems('/repo-root', 10, undefined, undefined, 'origin') - - expect(decodedIssueSearchPath(0)).toContain('q=repo:fork/orca is:issue is:open') - }) - - it("preference='origin' + no upstream → queries origin", async () => { - resolveIssueSourceMock.mockResolvedValueOnce({ - source: { owner: 'solo', repo: 'orca' }, - fellBack: false - }) - getOwnerRepoMock.mockResolvedValueOnce({ owner: 'solo', repo: 'orca' }) - ghExecFileAsyncMock.mockResolvedValueOnce({ stdout: '[]' }).mockResolvedValueOnce({ - stdout: '[]' - }) - - await listWorkItems('/repo-root', 10, undefined, undefined, 'origin') - - expect(decodedIssueSearchPath(0)).toContain('q=repo:solo/orca is:issue is:open') - }) - it('surfaces upstreamCandidate in sources regardless of effective preference', async () => { // Why: the renderer selector needs to keep rendering after the user picks // 'origin'. That requires the envelope to carry the raw upstream even diff --git a/src/main/github/client-pr-push-target.test.ts b/src/main/github/client-pr-push-target.test.ts index daaa05155ae6..36fd97823f3b 100644 --- a/src/main/github/client-pr-push-target.test.ts +++ b/src/main/github/client-pr-push-target.test.ts @@ -144,32 +144,6 @@ describe('getPRForBranch', () => { }) }) - it('omits maintainerCanModify when the API does not report the flag', async () => { - getOwnerRepoMock.mockResolvedValueOnce({ owner: 'stablyai', repo: 'orca' }) - getOwnerRepoForRemoteMock.mockResolvedValueOnce({ owner: 'stablyai', repo: 'orca' }) - ghExecFileAsyncMock.mockResolvedValueOnce({ - stdout: JSON.stringify({ - head: { - ref: 'fix-sidebar', - repo: { - full_name: 'stablyai/orca', - name: 'orca', - clone_url: 'https://github.com/stablyai/orca.git', - ssh_url: 'git@github.com:stablyai/orca.git', - owner: { login: 'stablyai' } - } - } - }) - }) - - await expect(getPullRequestPushTarget('/repo-root', 1738)).resolves.toEqual({ - pushTarget: { - remoteName: 'origin', - branchName: 'fix-sidebar' - } - }) - }) - it('uses origin for same-repository PR push targets', async () => { getOwnerRepoMock.mockResolvedValueOnce({ owner: 'stablyai', repo: 'orca' }) getOwnerRepoForRemoteMock.mockResolvedValueOnce({ owner: 'stablyai', repo: 'orca' }) diff --git a/src/main/github/gh-utils.test.ts b/src/main/github/gh-utils.test.ts index 5400eaeb5654..c239dc026f20 100644 --- a/src/main/github/gh-utils.test.ts +++ b/src/main/github/gh-utils.test.ts @@ -149,16 +149,6 @@ describe('github owner/repo resolution', () => { expect(gitRemoteGetUrlCalls('origin')).toHaveLength(1) }) - it('prefers upstream for issue owner/repo resolution', async () => { - mockGitRemoteCommands({ - origin: 'git@github.com:fork/orca.git\n', - upstream: 'git@github.com:stablyai/orca.git\n' - }) - - await expect(getIssueOwnerRepo('/repo')).resolves.toEqual({ owner: 'stablyai', repo: 'orca' }) - expect(gitRemoteGetUrlCalls('upstream')).toHaveLength(1) - }) - it('falls back to origin when upstream is present but non-GitHub', async () => { mockGitRemoteCommands({ origin: 'git@github.com:fork/orca.git\n', diff --git a/src/main/github/github-repository-identity.fork-owner-repo.test.ts b/src/main/github/github-repository-identity.fork-owner-repo.test.ts index e0cd7de6ca45..9c6f2a21159a 100644 --- a/src/main/github/github-repository-identity.fork-owner-repo.test.ts +++ b/src/main/github/github-repository-identity.fork-owner-repo.test.ts @@ -30,7 +30,7 @@ vi.mock('./local-git-config-signature', () => ({ import { _resetRemoteNameListingCache } from '../git/remote-name-listing' import { getOwnerRepoForRemote, _resetOwnerRepoCache } from './github-repository-identity' -import { getOwnerRepo, getIssueOwnerRepo } from './github-owner-repo-selection' +import { getOwnerRepo } from './github-owner-repo-selection' import { getRepoUpstream } from './client' const FORK_PATH = '/tmp/fork-checkout' @@ -84,13 +84,6 @@ describe('issue #7331: fork PR owner/repo resolution', () => { expect(prRepo).toEqual({ owner: 'stablyai', repo: 'orca' }) }) - it('getOwnerRepo and getIssueOwnerRepo agree on a fork checkout', async () => { - const prRepo = await getOwnerRepo(FORK_PATH) - const issueRepo = await getIssueOwnerRepo(FORK_PATH) - - expect(prRepo).toEqual(issueRepo) - }) - it('getOwnerRepo falls back to origin when there is no upstream remote', async () => { const prRepo = await getOwnerRepo(NON_FORK_PATH) diff --git a/src/main/github/work-item-details.test.ts b/src/main/github/work-item-details.test.ts index 11a063b51ba6..3ec1a5a4d9ff 100644 --- a/src/main/github/work-item-details.test.ts +++ b/src/main/github/work-item-details.test.ts @@ -708,76 +708,6 @@ describe('getWorkItemDetails', () => { expect(getWorkItemMock).toHaveBeenCalledWith('/repo-root', 42, 'pr', null, {}, 'origin') }) - // Why: a rate-limited/auth-failed file fetch must not render as an empty PR; - // the Files tab keys its retry state off details.filesUnavailable. - it('flags filesUnavailable when the PR file fetch fails but leaves the PR empty otherwise intact', async () => { - getWorkItemMock.mockResolvedValueOnce({ - id: 'pr:8305', - type: 'pr', - number: 8305, - title: 'Files fetch fails', - state: 'open', - url: 'https://github.com/acme/widgets/pull/8305', - labels: [], - updatedAt: '2026-07-11T00:00:00Z', - author: 'pr-author' - }) - getOwnerRepoForRemoteMock.mockResolvedValue({ owner: 'acme', repo: 'widgets' }) - getPRCommentsMock.mockResolvedValue([]) - getPRChecksMock.mockResolvedValue([]) - ghExecFileAsyncMock.mockImplementation(async (args: string[]) => { - const target = args.at(-1) - if (target === 'repos/acme/widgets/pulls/8305') { - return { - stdout: JSON.stringify({ head: { sha: 'head-sha' }, base: { sha: 'base-sha' } }) - } - } - if (target === 'repos/acme/widgets/pulls/8305/files?per_page=100') { - throw new Error('gh: API rate limit exceeded (403)') - } - return { stdout: JSON.stringify({ data: {} }) } - }) - - const details = await getWorkItemDetails('/repo-root', 8305, 'pr') - - expect(details?.filesUnavailable).toBe(true) - expect(details?.files).toBeUndefined() - }) - - it('treats an empty file list as a genuinely empty PR, not an unavailable one', async () => { - getWorkItemMock.mockResolvedValueOnce({ - id: 'pr:8306', - type: 'pr', - number: 8306, - title: 'Empty PR', - state: 'open', - url: 'https://github.com/acme/widgets/pull/8306', - labels: [], - updatedAt: '2026-07-11T00:00:00Z', - author: 'pr-author' - }) - getOwnerRepoForRemoteMock.mockResolvedValue({ owner: 'acme', repo: 'widgets' }) - getPRCommentsMock.mockResolvedValue([]) - getPRChecksMock.mockResolvedValue([]) - ghExecFileAsyncMock.mockImplementation(async (args: string[]) => { - const target = args.at(-1) - if (target === 'repos/acme/widgets/pulls/8306') { - return { - stdout: JSON.stringify({ head: { sha: 'head-sha' }, base: { sha: 'base-sha' } }) - } - } - if (target === 'repos/acme/widgets/pulls/8306/files?per_page=100') { - return { stdout: '[]' } - } - return { stdout: JSON.stringify({ data: {} }) } - }) - - const details = await getWorkItemDetails('/repo-root', 8306, 'pr') - - expect(details?.filesUnavailable).toBe(false) - expect(details?.files).toEqual([]) - }) - // Why: `gh pr view` omits avatar_url, so the login-based github.com URL 404s on // GHE. getWorkItemDetails must resolve author/reviewer/assignee avatars via the // GraphQL user(login:) batch and stamp them onto the returned item. See #8784. diff --git a/src/main/persistence-ssh-remote-pty-binding-replay.test.ts b/src/main/persistence-ssh-remote-pty-binding-replay.test.ts index f32f38368270..77054a2eafb0 100644 --- a/src/main/persistence-ssh-remote-pty-binding-replay.test.ts +++ b/src/main/persistence-ssh-remote-pty-binding-replay.test.ts @@ -393,61 +393,4 @@ describe('Store', () => { [TEST_LEAF_2]: 'runtime-pty-2' }) }) - - it('does not restore a binding with an explicit SSH termination tombstone', async () => { - const store = await createStore() - const hostId = 'ssh:ssh-1' - const session: WorkspaceSessionState = { - activeRepoId: 'r1', - activeWorktreeId: 'wt1', - activeTabId: 'tab1', - tabsByWorktree: { - wt1: [ - { - id: 'tab1', - worktreeId: 'wt1', - title: 'Terminal', - customTitle: null, - color: null, - sortOrder: 0, - createdAt: 1, - ptyId: 'ssh:ssh-1@@closed' - } - ] - }, - terminalLayoutsByTabId: { - tab1: { - root: { type: 'leaf', leafId: TEST_LEAF_1 }, - activeLeafId: TEST_LEAF_1, - expandedLeafId: null, - ptyIdsByLeafId: { [TEST_LEAF_1]: 'ssh:ssh-1@@closed' } - } - } - } - store.setWorkspaceSession(session, hostId) - store.upsertSshRemotePtyLease({ - targetId: 'ssh-1', - ptyId: 'closed', - worktreeId: 'wt1', - tabId: 'tab1', - leafId: TEST_LEAF_1, - state: 'terminated' - }) - store.setWorkspaceSession( - { - ...session, - tabsByWorktree: { - wt1: [{ ...session.tabsByWorktree.wt1[0]!, ptyId: null }] - }, - terminalLayoutsByTabId: { - tab1: { ...session.terminalLayoutsByTabId.tab1!, ptyIdsByLeafId: {} } - } - }, - hostId - ) - - const persisted = store.getWorkspaceSession(hostId) - expect(persisted.tabsByWorktree.wt1[0]!.ptyId).toBeNull() - expect(persisted.terminalLayoutsByTabId.tab1.ptyIdsByLeafId).toEqual({}) - }) }) diff --git a/src/main/repo-git-remote-identity-enrichment.test.ts b/src/main/repo-git-remote-identity-enrichment.test.ts index d1e54e11de8a..54350dda3d4f 100644 --- a/src/main/repo-git-remote-identity-enrichment.test.ts +++ b/src/main/repo-git-remote-identity-enrichment.test.ts @@ -614,22 +614,4 @@ describe('retiring probes for removed repos', () => { expect(listHandlersChanged).toHaveBeenCalledTimes(1) expect(runtimeChanged).toHaveBeenCalledTimes(1) }) - - it('still notifies a caller whose sweep was coalesced into one already running', async () => { - const first = deferred() - vi.mocked(probeGitRemoteIdentity).mockReturnValue(first.promise) - const store = makeMutableStore([makeRepo()]) - const listHandlerChanged = vi.fn() - // The runtime RPC caller also drops a resolved-worktree cache, so it must not be dropped. - const runtimeChanged = vi.fn() - - enrichMissingRepoGitRemoteIdentities(store, { onChanged: listHandlerChanged }) - enrichMissingRepoGitRemoteIdentities(store, { onChanged: runtimeChanged }) - - first.resolve(resolvedProbe) - await drainEnrichmentSweep() - - expect(listHandlerChanged).toHaveBeenCalledTimes(1) - expect(runtimeChanged).toHaveBeenCalledTimes(1) - }) }) diff --git a/src/main/runtime/rpc/e2ee-channel.test.ts b/src/main/runtime/rpc/e2ee-channel.test.ts index 8602263400d1..f4a09397d9a3 100644 --- a/src/main/runtime/rpc/e2ee-channel.test.ts +++ b/src/main/runtime/rpc/e2ee-channel.test.ts @@ -388,19 +388,6 @@ describe('E2EEChannel', () => { }) }) - describe('cross-compatibility', () => { - it('desktop encrypt is decryptable by desktop decrypt (sanity)', () => { - const a = generateKeyPair() - const b = generateKeyPair() - const sharedA = deriveSharedKey(a.secretKey, b.publicKey) - const sharedB = deriveSharedKey(b.secretKey, a.publicKey) - - const msg = '{"method":"terminal.subscribe","params":{"terminal":"t1"}}' - const enc = encrypt(msg, sharedA) - expect(decrypt(enc, sharedB)).toBe(msg) - }) - }) - describe('destroy', () => { it('clears state and stops forwarding', () => { const ctx = setup() diff --git a/src/main/runtime/rpc/methods/orchestration-worker-release-reap-fixed.func.test.ts b/src/main/runtime/rpc/methods/orchestration-worker-release-reap-fixed.func.test.ts deleted file mode 100644 index 579f37eaf9e3..000000000000 --- a/src/main/runtime/rpc/methods/orchestration-worker-release-reap-fixed.func.test.ts +++ /dev/null @@ -1,358 +0,0 @@ -// PRB-0219 / upstream #18737 — FUNCTIONAL (integration) reproduction of the steady-state -// worker-release reap LEAK. -// -// This is the "functional/integration" tier of the PRB-0219 test strategy. It wires the REAL -// orchestration RPC surface (orchestration.workerStart / workerRelease / workerList), the REAL -// OrchestrationDb, and the REAL release completion + observation modules against a fake runtime -// that faithfully models the two identity planes involved in the bug: -// -// * VOLATILE, epoch-fenced handle table — `handleTable` keyed by terminal handle, each entry -// stamped with the `rendererGraphEpoch` at which it was issued. `showTerminal` resolves a -// handle ONLY while its stamped epoch matches the current epoch (mirrors getLiveLeafForHandle -// throwing 'terminal_handle_stale' on a rendererGraphEpoch bump). A relay reconnect / renderer -// remount on headless serve bumps the epoch. -// * DURABLE, incarnation-addressed process table — `ptysById` keyed by the ptyId embedded in the -// worker's persisted process_incarnation (`${ptyId}:${incarnationId}`). The pty stays LIVE -// across an epoch bump; nothing about the graph epoch kills the process. -// -// The bug (mechanism): when the epoch bumps, the durable db handle stops resolving through -// showTerminal WHILE THE PTY IS STILL ALIVE. inspectWorkerTerminal swallows the throw and reports -// `missing`; completeWorkerTerminalRelease then commits `release_unknown` and returns WITHOUT ever -// calling runtime.closeTerminal — so the process/PTY leaks. On mtl-02 those orphans accumulate in -// the orca-serve@factory cgroup until TasksMax=4096 is hit and Bun/omp abort() on EAGAIN. -// -// The observable leak signal asserted here: state 'release_unknown' + processAction 'none' + -// closeTerminal NEVER called + the pty STILL alive in ptysById + worker-list terminalState stuck -// on 'release_unknown' (never 'released'). - -import { afterEach, describe, expect, it, vi } from 'vitest' -import { ORCHESTRATION_METHODS } from './orchestration' -import { eraseRpcMethods, type RpcContext } from '../core' -import { OrchestrationDb } from '../../orchestration/db' -import { OrcaRuntimeService } from '../../orca-runtime' - -describe('PRB-0219 worker-release reap FIX (functional verification)', () => { - let db: OrchestrationDb - let dbOpen = false - let runtime: OrcaRuntimeService - let ctx: RpcContext - let activeRunId: string - - const coordinatorPaneKey = 'tab_coord:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' - const workerPaneKey = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' - - // Dispatch/automation ptyId shape: `${repoId}::${worktreePath}@@${suffix}` (executionHostId:null). - const PTY_ID = 'repo-7f3a::/data/wt/factory-task-1@@a1b2c3d4' - const INCARNATION_ID = 1 - const PROCESS_INCARNATION = `${PTY_ID}:${INCARNATION_ID}` - - // ---- fake runtime process/handle planes (module-level so spies can mutate them) ---- - let ptysById: Map - let handleTable: Map - let rendererGraphEpoch: number - let closedPtyIds: string[] - - /** Resolve a handle to its live pty only while its stamped graph epoch is current. */ - function resolveHandleToLivePty( - handle: string - ): { ptyId: string; pty: { incarnationId: number; alive: boolean } } | null { - const entry = handleTable.get(handle) - if (!entry) { - return null - } - if (entry.epoch !== rendererGraphEpoch) { - // rendererGraphEpoch fence: the durable handle no longer resolves to a live leaf. - return null - } - const pty = ptysById.get(entry.ptyId) - if (!pty || !pty.alive) { - return null - } - return { ptyId: entry.ptyId, pty } - } - - /** Wire the real RPC surface and db against the two-plane fake runtime. */ - function setup(): void { - ptysById = new Map([ - ['coord-pty', { incarnationId: 1, alive: true }], - [PTY_ID, { incarnationId: INCARNATION_ID, alive: true }] - ]) - handleTable = new Map([ - ['term_coord', { ptyId: 'coord-pty', epoch: 0 }], - ['term_worker', { ptyId: PTY_ID, epoch: 0 }] - ]) - rendererGraphEpoch = 0 - closedPtyIds = [] - - db = new OrchestrationDb(':memory:') - dbOpen = true - runtime = new OrcaRuntimeService() - runtime.setOrchestrationDb(db) - - // Incarnation-addressed liveness probe: reads the DURABLE plane, so it stays 'live' across the - // epoch bump (the process really is still running). Matches the real asymmetry. - vi.spyOn(runtime, 'inspectTerminalProcessIncarnationLiveness').mockImplementation( - async (incarnation: string) => { - const idx = incarnation.lastIndexOf(':') - const ptyId = incarnation.slice(0, idx) - const pty = ptysById.get(ptyId) - return pty?.alive ? 'live' : 'exited' - } - ) - - // The incarnation-addressed re-resolution primitive the fix adds. Present here as a spy so the - // same harness proves BOTH tiers: pre-fix completion never calls it (leak); post-fix completion - // calls it to remint a live handle (reap). Fence: EXACT incarnationId match only. - vi.spyOn(runtime, 'resolveTerminalHandleByProcessIncarnation').mockImplementation( - (incarnation: string): string | null => { - const idx = incarnation.lastIndexOf(':') - const ptyId = incarnation.slice(0, idx) - const inc = Number(incarnation.slice(idx + 1)) - const pty = ptysById.get(ptyId) - if (!pty || !pty.alive) { - return null - } - if (pty.incarnationId !== inc) { - // Fence: a reused ptyId with a different incarnation must NOT resolve. - return null - } - // Remint a live handle at the current graph epoch. - handleTable.set('term_reminted', { ptyId, epoch: rendererGraphEpoch }) - return 'term_reminted' - } - ) - - // Identity plane (durable) — answers for the original AND any reminted handle. Independent of - // the graph epoch, exactly like the real getTerminal* accessors that read dispatch authority. - const knownWorkerHandle = (handle: string): boolean => - handle === 'term_worker' || handle === 'term_reminted' - vi.spyOn(runtime, 'getTerminalPaneKey').mockImplementation((handle) => - handle === 'term_coord' - ? coordinatorPaneKey - : knownWorkerHandle(handle) - ? workerPaneKey - : null - ) - vi.spyOn(runtime, 'getTerminalProcessIncarnation').mockImplementation((handle) => - knownWorkerHandle(handle) ? PROCESS_INCARNATION : null - ) - vi.spyOn(runtime, 'getOrchestrationDispatchAuthority').mockImplementation((handle) => - knownWorkerHandle(handle) - ? // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: This test fixture is deliberately shaped to exercise the private/runtime boundary. - ({ - terminalHandle: handle, - paneKey: workerPaneKey, - processIncarnation: PROCESS_INCARNATION, - hostScope: { kind: 'local', hostId: 'local' } - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - } as never) - : null - ) - vi.spyOn(runtime, 'validateOrchestrationAgentLauncher').mockImplementation(() => {}) - - // Volatile handle resolution — epoch-fenced. Throws 'terminal_handle_stale' once the epoch - // moves past the epoch at which the handle was issued. - vi.spyOn(runtime, 'showTerminal').mockImplementation(async (handle) => { - if (!resolveHandleToLivePty(handle)) { - throw new Error('terminal_handle_stale') - } - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - return { handle, worktreeId: 'repo::worktree', status: 'running' } as never - }) - - // The reap: closing a handle kills exactly the pty it resolves to. - vi.spyOn(runtime, 'closeTerminal').mockImplementation(async (handle) => { - const live = resolveHandleToLivePty(handle) - if (!live) { - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - return { handle, tabId: null, ptyKilled: false } as never - } - live.pty.alive = false - ptysById.delete(live.ptyId) - closedPtyIds.push(live.ptyId) - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - return { handle, tabId: `tab:${live.ptyId}`, ptyKilled: true } as never - }) - - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // Remaining runtime surface required to start + settle a worker (mirrors the unit harness). - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - vi.spyOn(runtime, 'showManagedTerminalWorkspace').mockResolvedValue({ - id: 'repo::worktree' - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - } as never) - vi.spyOn(runtime, 'createTerminal').mockResolvedValue({ - handle: 'term_worker', - worktreeId: 'repo::worktree', - title: 'worker' - }) - vi.spyOn(runtime, 'waitForTerminal').mockResolvedValue({ - handle: 'term_worker', - condition: 'tui-idle', - satisfied: true, - status: 'running', - exitCode: null - }) - vi.spyOn(runtime, 'getTerminalOrchestrationCliCommand').mockReturnValue('orca') - vi.spyOn(runtime, 'sendTerminalAgentPrompt').mockResolvedValue({ - handle: 'term_worker', - accepted: true, - bytesWritten: 1 - }) - vi.spyOn(runtime, 'isTerminalRunningAgent').mockResolvedValue(true) - vi.spyOn(runtime, 'getExactWorkerProviderSession').mockReturnValue(null) - vi.spyOn(runtime, 'readTerminal').mockResolvedValue({ - handle: 'term_worker', - status: 'running', - tail: ['worker output line 1', 'worker output line 2'], - truncated: false, - nextCursor: '2' - }) - vi.spyOn(runtime, 'notifyMessageArrived').mockImplementation(() => {}) - - activeRunId = db.createRun({ - objective: 'PRB-0219 reap leak fixture', - coordinatorHandle: 'term_coord', - coordinatorPaneKey - }).id - ctx = { runtime } - } - - afterEach(() => { - if (dbOpen) { - dbOpen = false - db.close() - } - vi.restoreAllMocks() - }) - - /** Look up a registered orchestration RPC method by name. */ - function findMethod(name: string) { - const method = eraseRpcMethods(ORCHESTRATION_METHODS).find((m) => m.name === name) - if (!method) { - throw new Error(`Method not found: ${name}`) - } - return method - } - - /** Parse a method's params and invoke its handler against the shared ctx. */ - async function call(name: string, params: Record) { - const method = findMethod(name) - const parsed = method.params ? method.params.parse(params) : undefined - return method.handler(parsed, ctx) - } - - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - /** Start a worker and settle its report, the state a release acts on. */ - async function startSettledWorker(): Promise<{ taskId: string; dispatchId: string }> { - const task = db.createTask({ spec: 'reap-leak fixture task', runId: activeRunId }) - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - const result = (await call('orchestration.workerStart', { - task: task.id, - from: 'term_coord', - agent: 'codex' - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - })) as { dispatchId: string; state: string } - expect(result.state).toBe('ready') - const settlement = db.settleWorkerReport({ - taskId: task.id, - dispatchId: result.dispatchId, - outcome: 'succeeded', - result: 'worker succeeded' - }) - expect(settlement.action).toBe('settled') - return { taskId: task.id, dispatchId: result.dispatchId } - } - - /** The terminalState workerList projects for a dispatch, or null. */ - async function workerTerminalState(dispatchId: string): Promise { - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - const listed = (await call('orchestration.workerList', { run: activeRunId })) as { - workers: { dispatchId: string; terminalState: string | null }[] - } - return listed.workers.find((w) => w.dispatchId === dispatchId)?.terminalState ?? null - } - - it('REAP (fixed): a rendererGraphEpoch bump strands the durable handle, but the incarnation fallback remints a live handle and reaps the process', async () => { - setup() - const { dispatchId } = await startSettledWorker() - const resource = db.getWorkerTerminalResourceByOwner(dispatchId) - expect(resource?.process_incarnation).toBe(PROCESS_INCARNATION) - - // Relay reconnect / renderer remount bumps the graph epoch: the durable handle goes stale - // while the PTY stays alive. - rendererGraphEpoch = 1 - await expect(runtime.showTerminal('term_worker')).rejects.toThrow('terminal_handle_stale') - expect(ptysById.get(PTY_ID)?.alive).toBe(true) - - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - const receipt = (await call('orchestration.workerRelease', { dispatch: dispatchId })) as { - state: string - processAction: string - } - - // The fix: inspectWorkerTerminal re-resolved the live PTY by process incarnation, reminted a - // handle, and completion closed THAT handle — the process is actually reaped. - expect(receipt.state).toBe('released') - expect(receipt.processAction).toBe('closed_agent_terminal') - expect(runtime.closeTerminal).toHaveBeenCalledWith('term_reminted') - expect(closedPtyIds).toEqual([PTY_ID]) - expect(ptysById.has(PTY_ID)).toBe(false) - expect(await workerTerminalState(dispatchId)).toBe('released') - }) - - it('CONTROL: with the graph epoch intact the same release reaps exactly that PTY', async () => { - setup() - const { dispatchId } = await startSettledWorker() - - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - const receipt = (await call('orchestration.workerRelease', { dispatch: dispatchId })) as { - state: string - processAction: string - } - - expect(receipt.state).toBe('released') - expect(receipt.processAction).toBe('closed_agent_terminal') - expect(closedPtyIds).toEqual([PTY_ID]) - expect(ptysById.has(PTY_ID)).toBe(false) - expect(await workerTerminalState(dispatchId)).toBe('released') - }) - - it('FENCE (fixed): a reused ptyId carrying a different incarnation must NOT remint or close — stays release_unknown', async () => { - setup() - const { dispatchId } = await startSettledWorker() - - // The graph epoch bumps AND the ptyId has been reused by a different process (incarnation 2), - // while the worker's recorded incarnation is still 1. The exact-incarnation fence must refuse. - rendererGraphEpoch = 1 - const reused = ptysById.get(PTY_ID) - if (reused) { - reused.incarnationId = 2 - } - - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: Test fixture crosses a private/runtime boundary with a verified shape. - const receipt = (await call('orchestration.workerRelease', { dispatch: dispatchId })) as { - state: string - processAction: string - } - - expect(receipt.state).toBe('release_unknown') - expect(receipt.processAction).toBe('none') - expect(runtime.closeTerminal).not.toHaveBeenCalled() - // The other lane's live process is left untouched (never reaped by an over-broad match). - expect(ptysById.get(PTY_ID)?.alive).toBe(true) - expect(await workerTerminalState(dispatchId)).toBe('release_unknown') - }) -}) diff --git a/src/main/ssh/ssh-connection-store.test.ts b/src/main/ssh/ssh-connection-store.test.ts index b086a9c5e310..d7aaee2046f1 100644 --- a/src/main/ssh/ssh-connection-store.test.ts +++ b/src/main/ssh/ssh-connection-store.test.ts @@ -89,11 +89,6 @@ describe('SshConnectionStore', () => { sshConfigHostsToTargetsMock.mockReset() }) - it('listTargets delegates to store', () => { - sshStore.listTargets() - expect(mockStore.getSshTargets).toHaveBeenCalled() - }) - it('lists picker suppression aliases without consulting re-adoption tombstones', () => { mockStore.addDeletedSshConfigAlias('config-removed') mockStore.addRemovedSshTargetTombstone({ @@ -166,26 +161,6 @@ describe('SshConnectionStore', () => { expect(sshStore.listTargets()).toEqual([userTarget]) }) - it('updateTarget delegates to store', () => { - const original: SshTarget = { - id: 'ssh-1', - label: 'Old Name', - host: 'example.com', - port: 22, - username: 'user' - } - mockStore.addSshTarget(original) - - const result = sshStore.updateTarget('ssh-1', { label: 'New Name' }) - expect(result).toBeTruthy() - expect(mockStore.updateSshTarget).toHaveBeenCalledWith('ssh-1', { label: 'New Name' }) - }) - - it('removeTarget delegates to store', () => { - sshStore.removeTarget('ssh-1') - expect(mockStore.removeSshTarget).toHaveBeenCalledWith('ssh-1') - }) - describe('importFromSshConfig', () => { function candidate(overrides: Partial & { configHost: string }): SshTarget { return { diff --git a/src/main/ssh/ssh-relay-ripgrep-cache-gc.test.ts b/src/main/ssh/ssh-relay-ripgrep-cache-gc.test.ts index d69bdb497f1a..bc61c5572bab 100644 --- a/src/main/ssh/ssh-relay-ripgrep-cache-gc.test.ts +++ b/src/main/ssh/ssh-relay-ripgrep-cache-gc.test.ts @@ -84,19 +84,6 @@ describe('remote ripgrep cache GC', () => { expect(removedTrees()).toEqual([]) }) - // Why this case exists: the relay directory is named from a hash of the relay bytes, and - // ripgrep is not among them, so a release that bumps only the ripgrep package -- the monthly - // Dependabot PR -- shares a relay directory with its predecessor while minting a new entry. - // With a single marker slot the second client overwrote the first's reference and this pass - // then collected the binary the first client's relay was still running against. - it('keeps both builds when two clients share one relay directory', async () => { - reply([CURRENT, SUPERSEDED], [CURRENT, SUPERSEDED]) - - await gcRemoteRipgrepCache(conn, LINUX, '/home/me', { pinnedEntry: CURRENT }) - - expect(removedTrees()).toEqual([]) - }) - // Why assert the shell and not just the parse: the marker is per entry, so a scan that read a // single fixed filename would silently see only one of the two references above. it('scans every marker in a relay directory, not one fixed name', async () => { From 1f8159cca6713185d4f59fcca8a016436bdcc787 Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 05:43:52 -0700 Subject: [PATCH 05/79] test: retire cases whose named dimension the production signature cannot express (#24139) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Backlog chunks 06-11, six auditors at 84 files each. All six read their full scope case-by-case against production — the second consecutive wave with no disclosed gap. 68 case declarations removed across 49 files, 2 test files deleted, 1,159 lines gone. The sharpest deletion is also the audit's best detection signal. `getFolderWorkspacePrimaryActionLabel(): string` takes zero parameters and returns a constant `translate(...)`. Its test, titled "uses a stable workspace creation label independent of quick agent selection", called it as (getFolderWorkspacePrimaryActionLabel as (...args: unknown[]) => string)({ id: 'codex' }) The cast is the evidence: the author had to defeat the type system to express the premise, because the dimension the title names cannot reach the function. That shape is greppable, and it is already against house style — AGENTS.md permits no type assertions but `as const`, and "production accepts an argument it does not declare" is not a defensible SAFETY rationale. Other removals, each with the owner named in the report: - `metaKey`/`ctrlKey`/`shiftKey` varied across 8 cases against an onClick handler that calls only `stopPropagation()` and `onOpenHostedReviewInChecks()` and inspects no modifier; GitHub and GitLab share one branch there. - `resolveVisibleCreatePrHeaderAction` is `return createPrHeaderAction`, an identity function, behind two titles naming "the body composer is open" — a dimension its one-field signature cannot express. Whole file. - `statPath` returning `{isDirectory:false}` means the `loadDir` branch is never entered, so "falls back when directory loading fails" is unreachable. - A "hide sleeping" case that searched a test-local literal array which itself hardcodes the key being looked for, so the knob was always defined. - `updater.startup-scheduling`: production has no platform branch — `verifyUpdateCodeSignature` appears only inside a security comment — so the darwin case asserted the same absence as the win32 case. The win32 case stays; it is what makes that comment fail CI. - Logical subsumption rather than textual: a case asserting `shallow` equality where its neighbour asserts `toBe` on identical fixtures. `toBe` implies `shallow`, so it cannot fail where the survivor passes. - Replays across bare re-exports and one-line adapters, including a 3-case describe over `return selectWorktreeAgentOrchestration(state, worktreeId)` whose owner runs 300 seeds against an independently transcribed oracle. One production line goes: a test-only `export { getSetupGuideSidebarEntryReady, shouldShowSetupGuideEntry } from './SetupGuideSidebarEntry'` in `SidebarNav.tsx`. Both symbols now appear only at their definition site, with zero importers. Kept after checking production rather than shape, and reported: GitHub and GitLab ready-for-review pairs that route through different APIs and negotiate different capability tokens; the `windows-process-tree-kill` / `windows-live-tree-kill.win32` pair, whose win32 header argues the duality including a disclosed refusal-orphans-descendants asymmetry; and a fake-`closest` test that looks like a previously-deleted shape but whose predicate really does read the attribute the test sets. Verified: 5,005 test files / 50,332 cases pass across the touched areas; `pnpm tc` clean after clearing `.tsbuildinfo`; `check-reliability-gates.mjs` 140 gates; both deleted files absent from the gate manifest, `cloud/package.json` and `mobile/tests-typecheck-baseline.txt`. Four failing files were checked and none is in this diff: `session-scanner-codex-workers`, `browser-manager-tab-identity` and `browser-manager-viewport-ownership` fail identically on a pristine `origin/main` worktree, and `structured-chat-coordinator-mail` is a load-sensitive `vi.waitFor` that passes in isolation and on CI re-run. --- ...remote-orchestration-compatibility.test.ts | 27 --- src/main/ssh/ssh-target-readoption.test.ts | 12 -- .../updater.linux-externally-managed.test.ts | 11 -- src/main/updater.prerelease-fallback.test.ts | 26 --- src/main/updater.startup-scheduling.test.ts | 13 -- src/main/worktree-removal-safety.test.ts | 13 +- ...worktree-retirement-backfill-stall.test.ts | 53 +----- .../CommitArea.generate.test.tsx | 9 - .../right-sidebar/CommitArea.test.tsx | 12 -- .../right-sidebar/PortsPanel.test.tsx | 25 --- .../SourceControl.branch-line-total.test.tsx | 24 --- .../SourceControl.commit-drafts.test.ts | 6 - .../SourceControl.compare-summary.test.ts | 19 -- ...Control.hosted-review-header-link.test.tsx | 44 ----- .../ai-vault-session-filters.test.ts | 31 --- .../checks-entry-refresh.test.ts | 12 -- .../checks-panel-review-creation.test.ts | 22 +-- .../discard-all-sequence.test.ts | 16 +- .../right-sidebar/git-status-refresh.test.ts | 24 --- .../pr-comments-ai-launch-ack.test.ts | 35 ---- ...rce-control-create-pr-intent-state.test.ts | 43 ----- ...-control-hosted-review-push-target.test.ts | 25 --- ...e-control-text-generation-defaults.test.ts | 52 +---- .../use-file-explorer-name-filter.test.ts | 16 -- ...use-source-control-branch-compare.test.tsx | 16 -- .../useFileExplorerHandlers.test.ts | 41 ---- .../useFileExplorerRowDrag.test.ts | 48 ----- .../sidebar/AddRepoStartSteps.test.tsx | 9 - ...ntSkillSetupPrompt.update-command.test.tsx | 177 ------------------ .../NewExternalWorktreesInboxLine.test.tsx | 8 - .../sidebar/SetupGuideSidebarEntry.test.tsx | 15 -- .../components/sidebar/SidebarHeader.test.tsx | 15 -- .../components/sidebar/SidebarNav.test.tsx | 29 +-- .../src/components/sidebar/SidebarNav.tsx | 2 - .../sidebar/WorkspaceKanbanLaneGrid.test.tsx | 9 - .../WorkspaceKanbanStatusLane.test.tsx | 14 -- ...branch-visible-under-hide-sleeping.test.ts | 26 +-- .../delete-worktree-parallel-flow.test.ts | 24 --- .../folder-workspace-composer-helpers.test.ts | 16 +- ...er-workspace-composer-path-status.test.tsx | 38 ---- .../sidebar/project-group-header-drop.test.ts | 69 ------- .../sidebar/project-header-color.test.ts | 20 -- .../src/components/sidebar/smart-sort.test.ts | 10 - .../ssh-host-remove-resolution.test.ts | 11 -- .../sidebar/use-worktree-sleep-state.test.tsx | 48 ----- .../sidebar/useCreateProjectDefaults.test.ts | 11 -- ...worktree-agent-orchestration-batch.test.ts | 23 --- .../worktree-agent-row-selectors.test.ts | 77 -------- .../worktree-card-status-inputs.test.ts | 44 ----- ...ktree-lineage-toggle-handler-cache.test.ts | 9 - ...worktree-list-groups-project-order.test.ts | 11 -- 51 files changed, 8 insertions(+), 1382 deletions(-) delete mode 100644 src/renderer/src/components/right-sidebar/source-control-create-pr-intent-state.test.ts delete mode 100644 src/renderer/src/components/sidebar/LinearAgentSkillSetupPrompt.update-command.test.tsx diff --git a/src/main/ssh/ssh-remote-orchestration-compatibility.test.ts b/src/main/ssh/ssh-remote-orchestration-compatibility.test.ts index e84b483b5b1c..b321dfba8418 100644 --- a/src/main/ssh/ssh-remote-orchestration-compatibility.test.ts +++ b/src/main/ssh/ssh-remote-orchestration-compatibility.test.ts @@ -483,31 +483,4 @@ describe('legacy SSH orchestration fallback', () => { db.close() } }) - - it.each([ - ['check', ['orchestration', 'check', '--terminal', COORDINATOR_HANDLE]], - ['ask', ['orchestration', 'ask', '--from', WORKER_HANDLE, '--question', 'continue?']] - ])('refuses a non-UUID --retry-request on orchestration %s', async (_label, commandArgv) => { - const { db, runtime } = createLegacyRuntime() - - try { - const result = await runRemoteOrcaCli( - runtime, - { - argv: [...commandArgv, '--retry-request', 'ssh-check-1', '--json'], - cwd: '/home/alice/repo', - env: WORKER_ENV, - runtimeAuthority: RUNTIME_AUTHORITY - }, - LEGACY_FALLBACK_OPTIONS - ) - - expect(result.exitCode).toBe(1) - expect(JSON.parse(result.stdout)).toMatchObject({ - error: { code: 'invalid_argument', message: expect.stringContaining('must be the UUID') } - }) - } finally { - db.close() - } - }) }) diff --git a/src/main/ssh/ssh-target-readoption.test.ts b/src/main/ssh/ssh-target-readoption.test.ts index c8d8cd24a18f..231a17397ce0 100644 --- a/src/main/ssh/ssh-target-readoption.test.ts +++ b/src/main/ssh/ssh-target-readoption.test.ts @@ -89,18 +89,6 @@ describe('readoptOrphanedWorkspacesForTarget', () => { expect(readoptions).toHaveLength(1) }) - it('does not match alias against a different host tuple', () => { - // Different alias AND different tuple => no match. - const fake = makeFakeStore([ - tombstone({ configHost: 'prod', host: 'prod.example.com', username: 'root' }) - ]) - const readoptions = readoptOrphanedWorkspacesForTarget( - fake.store, - makeTarget({ configHost: 'devbox', host: 'dev.example.com', username: 'tim' }) - ) - expect(readoptions).toEqual([]) - }) - it('is a no-op when there are no tombstones', () => { const fake = makeFakeStore([]) expect(readoptOrphanedWorkspacesForTarget(fake.store, makeTarget())).toEqual([]) diff --git a/src/main/updater.linux-externally-managed.test.ts b/src/main/updater.linux-externally-managed.test.ts index 0b63a8ec9ed3..ba7ab0452133 100644 --- a/src/main/updater.linux-externally-managed.test.ts +++ b/src/main/updater.linux-externally-managed.test.ts @@ -105,17 +105,6 @@ describe('updater externally managed Linux installs', () => { }) }) - it('marks the refusal non-retryable so the card offers no Retry Download', async () => { - const { send, updater } = await startUpdater({ packageType: 'deb', externallyManaged: true }) - updater.checkForUpdatesFromMenu() - await vi.advanceTimersByTimeAsync(0) - updater.downloadUpdate() - await vi.advanceTimersByTimeAsync(0) - - const status = lastStatus(send) - expect(status?.state === 'error' && status.retryable).toBe(false) - }) - it('records the blocked download for field diagnosis', async () => { const { updater } = await startUpdater({ packageType: 'deb', externallyManaged: true }) updater.checkForUpdatesFromMenu() diff --git a/src/main/updater.prerelease-fallback.test.ts b/src/main/updater.prerelease-fallback.test.ts index 49ed4ca1bb62..ae0349800744 100644 --- a/src/main/updater.prerelease-fallback.test.ts +++ b/src/main/updater.prerelease-fallback.test.ts @@ -690,30 +690,4 @@ describe('updater', () => { url: 'https://github.com/stablyai/orca/releases/download/v1.3.18' }) }) - - // Why: native GitHub provider can pick cancelled prerelease tags with missing manifests, so keep the manifest-probed generic feed. - it('uses the manifest-probed generic feed after a Shift-click RC opt-in', async () => { - appMock.getVersion.mockReturnValue('1.3.17') - fetchNewerReleaseTagsMock.mockResolvedValue(['v1.3.18-rc.1']) - autoUpdaterMock.checkForUpdates.mockResolvedValue(undefined) - - const { setupAutoUpdater, checkForUpdatesFromMenu } = await loadUpdaterModule() - - const mainWindow = { webContents: { send: vi.fn() } } - setupAutoUpdater(mainWindow as never, { getLastUpdateCheckAt: () => Date.now() }) - - checkForUpdatesFromMenu({ includePrerelease: true }) - - await vi.waitFor(() => { - expect(fetchNewerReleaseTagsMock).toHaveBeenCalledWith('1.3.17', 2, { - includePrerelease: true - }) - expect(autoUpdaterMock.checkForUpdates).toHaveBeenCalledTimes(1) - }) - expect(autoUpdaterMock.allowPrerelease).toBe(true) - expect(autoUpdaterMock.setFeedURL).toHaveBeenLastCalledWith({ - provider: 'generic', - url: 'https://github.com/stablyai/orca/releases/download/v1.3.18-rc.1' - }) - }) }) diff --git a/src/main/updater.startup-scheduling.test.ts b/src/main/updater.startup-scheduling.test.ts index ef72af27defa..285743295608 100644 --- a/src/main/updater.startup-scheduling.test.ts +++ b/src/main/updater.startup-scheduling.test.ts @@ -268,17 +268,4 @@ describe('updater', () => { expect((autoUpdaterMock as Record).verifyUpdateCodeSignature).toBeUndefined() }) - - it('does not override verifyUpdateCodeSignature on non-Windows platforms', async () => { - vi.stubGlobal('process', { ...process, platform: 'darwin' }) - - const { setupAutoUpdater } = await loadUpdaterModule() - - const sendMock = vi.fn() - const mainWindow = { webContents: { send: sendMock } } - - setupAutoUpdater(mainWindow as never) - - expect((autoUpdaterMock as Record).verifyUpdateCodeSignature).toBeUndefined() - }) }) diff --git a/src/main/worktree-removal-safety.test.ts b/src/main/worktree-removal-safety.test.ts index d32289a5fc76..9aa9775de44d 100644 --- a/src/main/worktree-removal-safety.test.ts +++ b/src/main/worktree-removal-safety.test.ts @@ -6,8 +6,7 @@ import { canSafelyRemoveOrphanedWorktreeDirectory, findRegisteredDeletableWorktree, getRegisteredDeletableWorktree, - isDangerousWorktreeRemovalPath, - isWorktreePathMissing + isDangerousWorktreeRemovalPath } from './worktree-removal-safety' import { CLIENT_REMOVAL_HOME, executionHostRemovalHome } from './worktree-removal-home-guard' @@ -350,16 +349,6 @@ describe('canSafelyRemoveOrphanedWorktreeDirectory', () => { }) }) -describe('isWorktreePathMissing', () => { - it('does not treat an unreadable path as absent', async () => { - await expect( - isWorktreePathMissing('/workspaces/locked', async () => { - throw Object.assign(new Error('permission denied'), { code: 'EACCES' }) - }) - ).resolves.toBe(false) - }) -}) - describe('canCleanupUnregisteredOrcaLeftoverDirectory', () => { const repo = { path: '/repos/main' } const ownedMeta = { orcaCreatedAt: 1, orcaCreationSource: 'runtime' as const } diff --git a/src/main/worktree-retirement-backfill-stall.test.ts b/src/main/worktree-retirement-backfill-stall.test.ts index 490696949c9e..33f6a7612917 100644 --- a/src/main/worktree-retirement-backfill-stall.test.ts +++ b/src/main/worktree-retirement-backfill-stall.test.ts @@ -11,20 +11,7 @@ vi.mock('node:fs/promises', async (importOriginal) => ({ })) const { ensureRetiredWorktreeNamesBackfilled } = await import('./worktree-name-retirement') -const { RETIREMENT_BACKFILL_RETRY_AFTER_FAILURE_MS, RETIREMENT_BACKFILL_SCAN_TIMEOUT_MS } = - await import('./worktree-retirement-backfill-scan') - -/** A `readdir` the test releases by hand, standing in for a wedged NFS/SMB/WSL mount. */ -function stalledReaddir(): { install: () => void; release: () => void } { - let release: () => void = () => {} - return { - install: () => - readdirMock.mockImplementation( - () => new Promise((_resolve, reject) => (release = () => reject(new Error('EIO')))) - ), - release: () => release() - } -} +const { RETIREMENT_BACKFILL_SCAN_TIMEOUT_MS } = await import('./worktree-retirement-backfill-scan') const repo = { id: 'repo-a', @@ -66,42 +53,4 @@ describe('retirement backfill against a stalled listing', () => { await settled }) - - it('recovers the namespace once the mount comes back, instead of staying wedged until restart', async () => { - const stall = stalledReaddir() - stall.install() - const store = backfillStore() - - const stalled = ensureRetiredWorktreeNamesBackfilled(store, repo, settings) - const settled = expect(stalled).rejects.toThrow(/exceeded/) - await vi.advanceTimersByTimeAsync(RETIREMENT_BACKFILL_SCAN_TIMEOUT_MS) - await settled - - // The mount comes back: the kernel releases the abandoned call, which is what frees the - // namespace to be listed again. Retrying while it is still stuck would only stack a second one. - stall.release() - readdirMock.mockResolvedValue([{ name: 'nautilus', isDirectory: () => true }]) - await vi.advanceTimersByTimeAsync(RETIREMENT_BACKFILL_RETRY_AFTER_FAILURE_MS) - - await ensureRetiredWorktreeNamesBackfilled(store, repo, settings) - expect(store.merged).toContain('nautilus') - }) - - it('does not re-probe a mount whose previous listing is still stuck', async () => { - stalledReaddir().install() - const store = backfillStore() - - const stalled = ensureRetiredWorktreeNamesBackfilled(store, repo, settings) - const settled = expect(stalled).rejects.toThrow(/exceeded/) - await vi.advanceTimersByTimeAsync(RETIREMENT_BACKFILL_SCAN_TIMEOUT_MS) - await settled - - const callsWhileStuck = readdirMock.mock.calls.length - await vi.advanceTimersByTimeAsync(RETIREMENT_BACKFILL_RETRY_AFTER_FAILURE_MS * 5) - await expect(ensureRetiredWorktreeNamesBackfilled(store, repo, settings)).rejects.toThrow( - /exceeded/ - ) - - expect(readdirMock.mock.calls.length).toBe(callsWhileStuck) - }) }) diff --git a/src/renderer/src/components/right-sidebar/CommitArea.generate.test.tsx b/src/renderer/src/components/right-sidebar/CommitArea.generate.test.tsx index 78663a7601f4..a3058f059813 100644 --- a/src/renderer/src/components/right-sidebar/CommitArea.generate.test.tsx +++ b/src/renderer/src/components/right-sidebar/CommitArea.generate.test.tsx @@ -182,15 +182,6 @@ describe('CommitArea AI generation', () => { expect(markup).toContain('aria-describedby="commit-area-generate-error"') }) - it('continues to render the split commit button alongside generation controls', () => { - const markup = renderCommitArea({ - ...baseProps(), - aiAgentConfigured: true - }) - expect(markup).toContain('Commit') - expect(markup).toContain('aria-label="Generate commit message with AI"') - }) - it('renders a single commit-message AI entry point in the composer', () => { const props = baseProps({ hasMessage: false }) const markup = renderCommitArea({ diff --git a/src/renderer/src/components/right-sidebar/CommitArea.test.tsx b/src/renderer/src/components/right-sidebar/CommitArea.test.tsx index 7222358c66ee..a88d707c39c9 100644 --- a/src/renderer/src/components/right-sidebar/CommitArea.test.tsx +++ b/src/renderer/src/components/right-sidebar/CommitArea.test.tsx @@ -510,18 +510,6 @@ describe('CommitArea', () => { expect(firstButton(markup)).not.toContain('animate-spin') }) - it('shows a spinner on a Commit primary while the commit itself is in flight', () => { - const props = baseProps({ - stagedCount: 1, - hasMessage: true, - upstreamStatus: { hasUpstream: true, ahead: 0, behind: 0 }, - isCommitting: true - }) - expect(firstButton(renderCommitArea({ ...props, isCommitting: true }))).toContain( - 'animate-spin' - ) - }) - it('shows a spinner on a remote primary while the matching remote op is active', () => { const markup = renderCommitArea( baseProps({ diff --git a/src/renderer/src/components/right-sidebar/PortsPanel.test.tsx b/src/renderer/src/components/right-sidebar/PortsPanel.test.tsx index 1121a05ae64d..50afea81fe0b 100644 --- a/src/renderer/src/components/right-sidebar/PortsPanel.test.tsx +++ b/src/renderer/src/components/right-sidebar/PortsPanel.test.tsx @@ -425,31 +425,6 @@ describe('PortsPanel runtime routing', () => { expect(activateAndRevealWorktreeMock).not.toHaveBeenCalled() }) - it('opens forced local workspace port clicks in the system browser', async () => { - const createBrowserTab = vi.fn() - const setRemoteBrowserPageHandle = vi.fn() - openUrl.mockResolvedValueOnce(undefined) - const openInOrcaBrowser = resolvePortOpenInOrcaBrowser({ - settings: { openLinksInApp: true }, - event: portOpenClick({ ctrlKey: true, shiftKey: true }), - isMac: false - }) - - await expect( - openWorkspacePortInBrowser({ - port: workspacePort, - runtimeTarget: { kind: 'local' }, - createBrowserTab: createBrowserTab as never, - setRemoteBrowserPageHandle: setRemoteBrowserPageHandle as never, - openInOrcaBrowser - }) - ).resolves.toEqual({ ok: true }) - - expect(openUrl).toHaveBeenCalledWith('http://127.0.0.1:63468') - expect(createBrowserTab).not.toHaveBeenCalled() - expect(activateAndRevealWorktreeMock).not.toHaveBeenCalled() - }) - it('returns post-stop refresh failures without throwing', async () => { const replaceWorkspacePortScans = vi.fn() const setWorkspacePortScanRefreshing = vi.fn() diff --git a/src/renderer/src/components/right-sidebar/SourceControl.branch-line-total.test.tsx b/src/renderer/src/components/right-sidebar/SourceControl.branch-line-total.test.tsx index 02f6ce6f0275..c4212ce0613d 100644 --- a/src/renderer/src/components/right-sidebar/SourceControl.branch-line-total.test.tsx +++ b/src/renderer/src/components/right-sidebar/SourceControl.branch-line-total.test.tsx @@ -336,28 +336,4 @@ describe('SourceControl branch line total chip', () => { vi.useRealTimers() } }) - - it('renders nothing for an exact zero total', () => { - resetState({ - gitBranchLineTotalByWorktree: { - [mocks.activeWorktree.id]: { added: 0, removed: 0, mergeBase: MERGE_BASE } - } - }) - renderSourceControl() - - expect(chip()).toBeNull() - expect(loadingChip()).toBeNull() - }) - - it('omits the zero half of a one-sided total', () => { - resetState({ - gitBranchLineTotalByWorktree: { - [mocks.activeWorktree.id]: { added: 42, removed: 0, mergeBase: MERGE_BASE } - } - }) - renderSourceControl() - - expect(chip()?.textContent).toBe('+42') - expect(chip()?.getAttribute('aria-label')).toBe('42 lines added') - }) }) diff --git a/src/renderer/src/components/right-sidebar/SourceControl.commit-drafts.test.ts b/src/renderer/src/components/right-sidebar/SourceControl.commit-drafts.test.ts index f4fdf425f142..888470fedeb4 100644 --- a/src/renderer/src/components/right-sidebar/SourceControl.commit-drafts.test.ts +++ b/src/renderer/src/components/right-sidebar/SourceControl.commit-drafts.test.ts @@ -8,7 +8,6 @@ import { shouldRenderCommitArea, writeCommitDraftForWorktree } from './SourceControl' -import { getNextSourceControlViewMode } from './source-control/panel/header-toolbar' import { loadSessionCommitDrafts, saveSessionCommitDrafts @@ -158,9 +157,4 @@ describe('SourceControl view mode preference', () => { expect(normalizeSourceControlViewMode('list')).toBe('list') expect(normalizeSourceControlViewMode('tree')).toBe('tree') }) - - it('derives the next persisted view mode from the current mode', () => { - expect(getNextSourceControlViewMode('list')).toBe('tree') - expect(getNextSourceControlViewMode('tree')).toBe('list') - }) }) diff --git a/src/renderer/src/components/right-sidebar/SourceControl.compare-summary.test.ts b/src/renderer/src/components/right-sidebar/SourceControl.compare-summary.test.ts index 100880d4581b..87481bda25f7 100644 --- a/src/renderer/src/components/right-sidebar/SourceControl.compare-summary.test.ts +++ b/src/renderer/src/components/right-sidebar/SourceControl.compare-summary.test.ts @@ -2,7 +2,6 @@ import { describe, expect, it, vi } from 'vitest' import { CompareSummary, CompareSummaryToolbarButton, - refreshSourceControlAfterRemoteAction, resolveSourceControlBaseRef, resolveSourceControlCompareBaseRef, resolveSourceControlPickerBaseRef, @@ -496,22 +495,4 @@ describe('SourceControl compare summary', () => { }) ).toBe(false) }) - - it('keeps immediate refresh paths for remote actions', () => { - const refreshGitStatus = vi.fn(async () => {}) - const refreshBranchCompare = vi.fn(async () => {}) - const refreshGitHistory = vi.fn(async () => {}) - - refreshSourceControlAfterRemoteAction({ - refreshGitStatus, - refreshBranchCompare, - refreshGitHistory - }) - - expect(refreshGitStatus).toHaveBeenCalledTimes(1) - expect(refreshBranchCompare).toHaveBeenCalledTimes(1) - expect(refreshGitHistory).toHaveBeenCalledTimes(1) - // Direct commit, manual, retry, and base-ref refresh paths remain component-level - // behavior covered by the existing UI wiring; keep this test on the pure helper. - }) }) diff --git a/src/renderer/src/components/right-sidebar/SourceControl.hosted-review-header-link.test.tsx b/src/renderer/src/components/right-sidebar/SourceControl.hosted-review-header-link.test.tsx index cf407854ebdf..0875707f8888 100644 --- a/src/renderer/src/components/right-sidebar/SourceControl.hosted-review-header-link.test.tsx +++ b/src/renderer/src/components/right-sidebar/SourceControl.hosted-review-header-link.test.tsx @@ -71,26 +71,6 @@ describe('HostedReviewHeaderLink', () => { expect(openHttpLinkMock).not.toHaveBeenCalled() }) - it.each<[string, ClickModifiers]>([ - ['Cmd-click', { metaKey: true }], - ['Ctrl-click', { ctrlKey: true }], - ['Shift+Cmd-click', { metaKey: true, shiftKey: true }], - ['Shift+Ctrl-click', { ctrlKey: true, shiftKey: true }] - ])('opens GitHub PRs in the Checks tab on %s', (_label, modifiers) => { - const onOpenHostedReviewInChecks = vi.fn() - const element = HostedReviewHeaderLink({ - review: makeReview(), - onOpenHostedReviewInChecks - }) - - const event = clickEvent(modifiers) - ;(element.props.onClick as (event: MinimalClickEvent) => void)(event) - - expect(event.stopPropagation).toHaveBeenCalledTimes(1) - expect(onOpenHostedReviewInChecks).toHaveBeenCalledTimes(1) - expect(openHttpLinkMock).not.toHaveBeenCalled() - }) - it('opens GitLab MRs in the Checks tab instead of rendering an external link', () => { const onOpenHostedReviewInChecks = vi.fn() const element = HostedReviewHeaderLink({ @@ -114,30 +94,6 @@ describe('HostedReviewHeaderLink', () => { expect(openHttpLinkMock).not.toHaveBeenCalled() }) - it.each<[string, ClickModifiers]>([ - ['Cmd-click', { metaKey: true }], - ['Ctrl-click', { ctrlKey: true }], - ['Shift+Cmd-click', { metaKey: true, shiftKey: true }], - ['Shift+Ctrl-click', { ctrlKey: true, shiftKey: true }] - ])('opens GitLab MRs in the Checks tab on %s', (_label, modifiers) => { - const onOpenHostedReviewInChecks = vi.fn() - const element = HostedReviewHeaderLink({ - review: makeReview({ - provider: 'gitlab', - number: 31, - url: 'https://gitlab.com/acme/widgets/-/merge_requests/31' - }), - onOpenHostedReviewInChecks - }) - - const event = clickEvent(modifiers) - ;(element.props.onClick as (event: MinimalClickEvent) => void)(event) - - expect(event.stopPropagation).toHaveBeenCalledTimes(1) - expect(onOpenHostedReviewInChecks).toHaveBeenCalledTimes(1) - expect(openHttpLinkMock).not.toHaveBeenCalled() - }) - it('keeps other provider reviews as external hosted-review links', () => { const markup = renderToStaticMarkup( { '/Users/ada/workspaces/orca/unclaimed-old-path' ]) }) - - it('ignores prior paths claimed by another live worktree in a different repo', () => { - expect( - deriveAiVaultWorkspaceScopePaths( - { - id: 'repo1::/Users/ada/workspaces/orca/fix-agent-history', - repoId: 'repo1', - path: '/Users/ada/workspaces/orca/fix-agent-history', - priorWorktreeIds: [ - 'repo1::/Users/ada/workspaces/orca/bream', - 'repo1::/Users/ada/workspaces/orca/unclaimed-old-path' - ] - }, - [ - { - id: 'repo1::/Users/ada/workspaces/orca/fix-agent-history', - repoId: 'repo1', - path: '/Users/ada/workspaces/orca/fix-agent-history' - }, - { - id: 'repo2::/Users/ada/workspaces/orca/bream', - repoId: 'repo2', - path: '/Users/ada/workspaces/orca/bream' - } - ] - ) - ).toEqual([ - '/Users/ada/workspaces/orca/fix-agent-history', - '/Users/ada/workspaces/orca/unclaimed-old-path' - ]) - }) }) describe('deriveAiVaultScopeSessionPaths', () => { diff --git a/src/renderer/src/components/right-sidebar/checks-entry-refresh.test.ts b/src/renderer/src/components/right-sidebar/checks-entry-refresh.test.ts index 2b5cec8b92ab..57dfa1db28a5 100644 --- a/src/renderer/src/components/right-sidebar/checks-entry-refresh.test.ts +++ b/src/renderer/src/components/right-sidebar/checks-entry-refresh.test.ts @@ -100,18 +100,6 @@ describe('shouldEntryRefresh', () => { ).toBe(false) }) - it('ignores checks/comments freshness when no PR is known', () => { - expect( - shouldEntryRefresh({ - prFetchedAt: NOW - 1_000, - checksFetchedAt: undefined, - commentsFetchedAt: undefined, - prNumber: null, - now: NOW - }) - ).toBe(false) - }) - it('treats a PR timestamp exactly at the cutoff as fresh', () => { // Why: the rule is "older than now - grace", strict less-than. Equal is fresh. expect( diff --git a/src/renderer/src/components/right-sidebar/checks-panel-review-creation.test.ts b/src/renderer/src/components/right-sidebar/checks-panel-review-creation.test.ts index 4a53eab87b00..1d3d84a838a5 100644 --- a/src/renderer/src/components/right-sidebar/checks-panel-review-creation.test.ts +++ b/src/renderer/src/components/right-sidebar/checks-panel-review-creation.test.ts @@ -1,9 +1,5 @@ import { describe, expect, it } from 'vitest' -import type { - HostedReviewCreationBlockedReason, - HostedReviewCreationEligibility, - HostedReviewLookupOutcome -} from '../../../../shared/hosted-review' +import type { HostedReviewCreationEligibility } from '../../../../shared/hosted-review' import { computeChecksPanelConfirmedReadiness, isChecksPanelCreateEligibilityConfirmable, @@ -263,16 +259,6 @@ describe('computeChecksPanelConfirmedReadiness', () => { ).toBe(false) }) - it('preserves confirmed across a transient refresh failure once past the error', () => { - // The panel keeps the last confirmed eligibility snapshot; a transient - // failure (no hard error observed) must not drop confirmed. - expect( - computeChecksPanelConfirmedReadiness( - readiness({ hardErrorObservedAt: undefined, reviewLookup: 'not_found' }) - ) - ).toEqual({ confirmed: true, needsPush: false }) - }) - describe('hard error clearing', () => { it('clears when the request started strictly after the error with an accepted outcome', () => { expect( @@ -383,9 +369,3 @@ describe('isChecksPanelHardErrorCleared', () => { ).toBe(true) }) }) - -// Type-only usage to keep the imports meaningful across refactors. -const _blocked: HostedReviewCreationBlockedReason = null -const _outcome: HostedReviewLookupOutcome = 'not_found' -void _blocked -void _outcome diff --git a/src/renderer/src/components/right-sidebar/discard-all-sequence.test.ts b/src/renderer/src/components/right-sidebar/discard-all-sequence.test.ts index e60972cbdfd1..e8a8d9b27a11 100644 --- a/src/renderer/src/components/right-sidebar/discard-all-sequence.test.ts +++ b/src/renderer/src/components/right-sidebar/discard-all-sequence.test.ts @@ -5,8 +5,7 @@ import { getUnstageAllPaths, isStageableStatusEntry, isSubmoduleWorktreeOnlyChange, - runDiscardAllForArea, - type DiscardAllArea + runDiscardAllForArea } from './source-control/commit/discard-all-sequence' import type { GitStatusEntry } from '../../../../shared/git-status-types' @@ -365,17 +364,4 @@ describe('runDiscardAllForArea', () => { await runDiscardAllForArea('staged', ['a.ts'], ctx.deps) expect(ctx.onError).not.toHaveBeenCalled() }) - - it('does not bulk-unstage for non-staged areas even if the dep is provided', async () => { - const ctx = makeDeps() - const areas: DiscardAllArea[] = ['unstaged', 'untracked'] - for (const area of areas) { - await runDiscardAllForArea(area, ['x.ts'], ctx.deps) - } - // Why: the unstage step is specific to the staged area's two-step - // reset. Accidentally invoking it for unstaged/untracked would be a - // no-op for unstaged entries but could mask a regression where staged - // entries leak into those paths. - expect(ctx.bulkUnstage).not.toHaveBeenCalled() - }) }) diff --git a/src/renderer/src/components/right-sidebar/git-status-refresh.test.ts b/src/renderer/src/components/right-sidebar/git-status-refresh.test.ts index bfe42b97513d..dc540909121a 100644 --- a/src/renderer/src/components/right-sidebar/git-status-refresh.test.ts +++ b/src/renderer/src/components/right-sidebar/git-status-refresh.test.ts @@ -133,30 +133,6 @@ describe('refreshGitStatusForWorktree', () => { }) }) - it('leaves ignored-file discovery to the File Explorer instead of status polling', async () => { - const status: GitStatusResult = { - entries: [], - conflictOperation: 'unknown' - } - const gitStatus = vi.fn().mockResolvedValue(status) - vi.stubGlobal('window', { api: { git: { status: gitStatus } } }) - const deps = makeDeps() - - await refreshGitStatusForWorktree({ - settings: { activeRuntimeEnvironmentId: null }, - worktreeId: 'wt-3', - worktreePath: '/repo', - deps - }) - - expect(gitStatus).toHaveBeenCalledWith({ - worktreePath: '/repo', - connectionId: undefined, - admissionTier: 'status' - }) - expect(deps.setGitStatus).toHaveBeenCalledWith('wt-3', status) - }) - it('bypasses automatic no-upstream backoff only for strict refreshes', async () => { const status: GitStatusResult = { entries: [], diff --git a/src/renderer/src/components/right-sidebar/pr-comments-ai-launch-ack.test.ts b/src/renderer/src/components/right-sidebar/pr-comments-ai-launch-ack.test.ts index a0b5a0b19e28..737e80e1bbff 100644 --- a/src/renderer/src/components/right-sidebar/pr-comments-ai-launch-ack.test.ts +++ b/src/renderer/src/components/right-sidebar/pr-comments-ai-launch-ack.test.ts @@ -354,25 +354,6 @@ describe('acknowledgePRCommentsAfterAiLaunch', () => { expect(replyAsConversation).not.toHaveBeenCalled() }) - // Why: resolve used to be gated on panel/live-comment state and got silently skipped when - // a slow agent launch outlived either; the snapshotted threadId must always be attempted. - it('resolves every selected thread with no live-state gate', async () => { - const resolveThread = vi.fn().mockResolvedValue(true) - - const result = await acknowledgePRCommentsAfterAiLaunch({ - groups: [openThread('T1'), openThread('T2', 11), openThread('T3', 12)], - deps: { - resolveThread, - canReply: true, - replyInThread: vi.fn(), - replyAsConversation: vi.fn() - } - }) - - expect(resolveThread).toHaveBeenCalledTimes(3) - expect(result).toEqual({ resolved: 3, replied: 0, skipped: 0, failed: 0 }) - }) - // Why: GitLab MRs reach the ack with canReply false; resolve is the whole acknowledgement there. it('resolves threads on providers that cannot reply', async () => { const resolveThread = vi.fn().mockResolvedValue(true) @@ -496,22 +477,6 @@ describe('acknowledgePRCommentsAfterAiLaunch', () => { expect(replyAsConversation).toHaveBeenCalledWith(`@alice ${PR_COMMENT_AI_FIXING_REPLY}`) }) - it('posts no conversation comment when nothing in the selection needs one', async () => { - const replyAsConversation = vi.fn() - - await acknowledgePRCommentsAfterAiLaunch({ - groups: [openThread('T1'), openThread('T2', 11)], - deps: { - resolveThread: vi.fn().mockResolvedValue(true), - canReply: true, - replyInThread: vi.fn().mockResolvedValue(true), - replyAsConversation - } - }) - - expect(replyAsConversation).not.toHaveBeenCalled() - }) - it('posts a conversation @-reply for CodeRabbit review summaries', async () => { const replyInThread = vi.fn().mockResolvedValue(true) const replyAsConversation = vi.fn().mockResolvedValue(true) diff --git a/src/renderer/src/components/right-sidebar/source-control-create-pr-intent-state.test.ts b/src/renderer/src/components/right-sidebar/source-control-create-pr-intent-state.test.ts deleted file mode 100644 index 6d525812a03e..000000000000 --- a/src/renderer/src/components/right-sidebar/source-control-create-pr-intent-state.test.ts +++ /dev/null @@ -1,43 +0,0 @@ -import { describe, expect, it } from 'vitest' -import { resolveVisibleCreatePrHeaderAction } from './source-control/review/create-pr-intent-state' -import type { PrimaryAction } from './source-control-primary-action-types' - -const disabledCreatePrAction: PrimaryAction = { - kind: 'create_pr', - label: 'Create PR', - title: 'Publish commits before creating a pull request.', - disabled: true -} - -const enabledCreatePrAction: PrimaryAction = { - kind: 'create_pr', - label: 'Create PR', - title: 'Create a pull request for this branch', - disabled: false -} - -describe('resolveVisibleCreatePrHeaderAction', () => { - it('returns null when no header action is available', () => { - expect( - resolveVisibleCreatePrHeaderAction({ - createPrHeaderAction: null - }) - ).toBeNull() - }) - - it('keeps a disabled Create PR header visible as a stable toolbar anchor', () => { - expect( - resolveVisibleCreatePrHeaderAction({ - createPrHeaderAction: disabledCreatePrAction - }) - ).toEqual(disabledCreatePrAction) - }) - - it('keeps an enabled Create PR header visible even when the body composer is open', () => { - expect( - resolveVisibleCreatePrHeaderAction({ - createPrHeaderAction: enabledCreatePrAction - }) - ).toEqual(enabledCreatePrAction) - }) -}) diff --git a/src/renderer/src/components/right-sidebar/source-control-hosted-review-push-target.test.ts b/src/renderer/src/components/right-sidebar/source-control-hosted-review-push-target.test.ts index 1d3be273631e..f2dbd28230f0 100644 --- a/src/renderer/src/components/right-sidebar/source-control-hosted-review-push-target.test.ts +++ b/src/renderer/src/components/right-sidebar/source-control-hosted-review-push-target.test.ts @@ -286,31 +286,6 @@ describe('hasUsableHostedReviewPushTarget', () => { }) describe('resolveHostedReviewActionUpstreamStatus with a same-repo upstream', () => { - it('does not synthesize hasUpstream:false when the real upstream is the review head', () => { - const realUpstream = { - hasUpstream: true, - upstreamName: 'origin/mobile-resume-suspected-fixes', - ahead: 7, - behind: 2 - } - const canUseHostedReviewPushTarget = hasUsableHostedReviewPushTarget({ - hasResolvableHostedReviewPushTargetLink: true, - branchName: 'mobile-resume-suspected-fixes', - upstreamStatus: realUpstream - }) - expect(canUseHostedReviewPushTarget).toBe(true) - expect( - resolveHostedReviewActionUpstreamStatus({ - hasHostedReviewLink: true, - hasResolvableHostedReviewPushTargetLink: true, - hostedReviewState: 'open', - isHostedReviewStateLoading: false, - canUseHostedReviewPushTarget, - upstreamStatus: realUpstream - }) - ).toBe(realUpstream) - }) - it('does not block push for a queue-discovered open PR whose upstream tracks the branch', () => { // Why: a child worktree with no persisted linkedPR discovers its open PR via // the queue (fallbackGitHubPR). Before the fix, that PR counted as a hosted diff --git a/src/renderer/src/components/right-sidebar/source-control-text-generation-defaults.test.ts b/src/renderer/src/components/right-sidebar/source-control-text-generation-defaults.test.ts index c38f61731971..f38ac632ef35 100644 --- a/src/renderer/src/components/right-sidebar/source-control-text-generation-defaults.test.ts +++ b/src/renderer/src/components/right-sidebar/source-control-text-generation-defaults.test.ts @@ -1,11 +1,7 @@ import { describe, expect, it } from 'vitest' import { getDefaultSettings } from '../../../../shared/constants' import type { GlobalSettings } from '../../../../shared/global-settings-types' -import type { Repo } from '../../../../shared/repo-types' -import { - generationParamsToActionRecipe, - sourceControlTextGenerationDefaultsMatchTarget -} from './source-control/ai/text-generation-defaults' +import { sourceControlTextGenerationDefaultsMatchTarget } from './source-control/ai/text-generation-defaults' function settings(): GlobalSettings { const base = getDefaultSettings('/tmp') @@ -44,52 +40,6 @@ describe('sourceControlTextGenerationDefaultsMatchTarget', () => { ).toBe(true) }) - it('returns false when the repo target has no saved override yet', () => { - expect( - sourceControlTextGenerationDefaultsMatchTarget({ - actionId: 'commitMessage', - target: { type: 'repo', repoId: 'repo-1' }, - params: { - agentId: 'codex', - model: 'gpt-5.5', - commandInputTemplate: '{basePrompt}', - agentArgs: '--model sonnet' - }, - settings: settings(), - repo: { sourceControlAi: { enabled: true } } satisfies Pick - }) - ).toBe(false) - }) - - it('returns true when the current params match the repo saved recipe', () => { - expect( - sourceControlTextGenerationDefaultsMatchTarget({ - actionId: 'commitMessage', - target: { type: 'repo', repoId: 'repo-1' }, - params: { - agentId: 'opencode', - model: '', - commandInputTemplate: '{basePrompt}\n\nrepo only', - agentArgs: '' - }, - settings: settings(), - repo: { - sourceControlAi: { - enabled: true, - actionOverrides: { - commitMessage: generationParamsToActionRecipe({ - agentId: 'opencode', - model: '', - commandInputTemplate: '{basePrompt}\n\nrepo only', - agentArgs: '' - }) - } - } - } satisfies Pick - }) - ).toBe(true) - }) - it('returns false when the command template differs from the saved recipe', () => { expect( sourceControlTextGenerationDefaultsMatchTarget({ diff --git a/src/renderer/src/components/right-sidebar/use-file-explorer-name-filter.test.ts b/src/renderer/src/components/right-sidebar/use-file-explorer-name-filter.test.ts index 459b8870898b..f0394973e53a 100644 --- a/src/renderer/src/components/right-sidebar/use-file-explorer-name-filter.test.ts +++ b/src/renderer/src/components/right-sidebar/use-file-explorer-name-filter.test.ts @@ -44,22 +44,6 @@ describe('useFileExplorerNameFilter', () => { expect(result.current.nameFilterSource?.query).toBe('AppDelegate.swift') }) - it('projects the settled listing for the active query', () => { - useRuntimeFileListForWorktreeMock.mockReturnValue({ - files: ['package.json', 'src/main.ts'], - loading: false, - loadError: null - } satisfies RuntimeFileListState) - - const { result } = renderHook(() => - useFileExplorerNameFilter({ isFilesViewActive: true, activeWorktreeId: 'worktree-1' }) - ) - - act(() => result.current.setNameFilterQuery('package.')) - - expect(result.current.nameFilterSource?.relativePaths).toEqual(['package.json', 'src/main.ts']) - }) - it('reports an unsettled source while the listing loads', () => { useRuntimeFileListForWorktreeMock.mockReturnValue({ files: [], diff --git a/src/renderer/src/components/right-sidebar/use-source-control-branch-compare.test.tsx b/src/renderer/src/components/right-sidebar/use-source-control-branch-compare.test.tsx index 23a426b66a5f..1fb21b7613c3 100644 --- a/src/renderer/src/components/right-sidebar/use-source-control-branch-compare.test.tsx +++ b/src/renderer/src/components/right-sidebar/use-source-control-branch-compare.test.tsx @@ -123,22 +123,6 @@ describe('useSourceControlBranchCompare scheduler', () => { expect(mocks.getRuntimeGitBranchCompare).toHaveBeenCalledTimes(2) }) - it('does not lose the trailing refresh when the call arrives late in the run', async () => { - const first = deferred() - mocks.getRuntimeGitBranchCompare.mockReturnValueOnce(first.promise) - await mount() - - void latest?.refreshBranchCompare() - await flush() - void latest?.refreshBranchCompare() - - await act(async () => { - first.resolve(OK) - }) - await flush() - expect(mocks.getRuntimeGitBranchCompare).toHaveBeenCalledTimes(2) - }) - it('chains another trailing run when a call lands during the trailing run', async () => { const first = deferred() const second = deferred() diff --git a/src/renderer/src/components/right-sidebar/useFileExplorerHandlers.test.ts b/src/renderer/src/components/right-sidebar/useFileExplorerHandlers.test.ts index e152da20e39b..0b55ffbc3a16 100644 --- a/src/renderer/src/components/right-sidebar/useFileExplorerHandlers.test.ts +++ b/src/renderer/src/components/right-sidebar/useFileExplorerHandlers.test.ts @@ -74,47 +74,6 @@ describe('activateFileExplorerNode', () => { expect(openFile).not.toHaveBeenCalled() }) - it('falls back to opening a symlink as a file when directory loading fails', async () => { - const openFile = vi.fn() - useAppStore.setState({ - worktreesByRepo: { - 'repo-1': [ - { - id: 'wt-1', - repoId: 'repo-1', - path: '/repo', - hostId: 'runtime:runtime-env-1' - } as never - ] - } - }) - - await activateFileExplorerNode({ - node: symlinkNode, - activeWorktreeId: 'wt-1', - runtimeEnvironmentId: 'runtime-env-1', - openFile, - toggleDir: vi.fn(), - loadDir: vi.fn(), - statPath: vi.fn().mockResolvedValue({ isDirectory: false }), - authorizeExternalPath: vi.fn(), - markPathAsDirectory: vi.fn(), - setSelectedPath: vi.fn() - }) - - expect(openFile).toHaveBeenCalledWith( - { - filePath: '/repo/linked-docs', - relativePath: 'linked-docs', - worktreeId: 'wt-1', - runtimeEnvironmentId: 'runtime-env-1', - language: expect.any(String), - mode: 'edit' - }, - { preview: true, focusEditor: true, suppressActiveRuntimeFallback: false } - ) - }) - it('opens a symlink as a file when target stat fails', async () => { const openFile = vi.fn() useAppStore.setState({ diff --git a/src/renderer/src/components/right-sidebar/useFileExplorerRowDrag.test.ts b/src/renderer/src/components/right-sidebar/useFileExplorerRowDrag.test.ts index 3c57f9dc15cb..4bdc0f8ae5e8 100644 --- a/src/renderer/src/components/right-sidebar/useFileExplorerRowDrag.test.ts +++ b/src/renderer/src/components/right-sidebar/useFileExplorerRowDrag.test.ts @@ -21,11 +21,6 @@ describe('encodeWorkspaceFilePaths / decodeWorkspaceFilePaths', () => { const encoded = encodeWorkspaceFilePaths(paths) expect(decodeWorkspaceFilePaths(encoded)).toEqual(paths) }) - - it('round-trips any number of paths correctly', () => { - const paths = ['/a', '/b', '/c', '/d', '/e'] - expect(decodeWorkspaceFilePaths(encodeWorkspaceFilePaths(paths))).toEqual(paths) - }) }) describe('decodeWorkspaceFilePaths', () => { @@ -33,10 +28,6 @@ describe('decodeWorkspaceFilePaths', () => { expect(decodeWorkspaceFilePaths('')).toEqual([]) }) - it('returns the plain string wrapped in an array when not JSON', () => { - expect(decodeWorkspaceFilePaths('/repo/a.ts')).toEqual(['/repo/a.ts']) - }) - it('filters out non-string entries from a JSON array', () => { expect(decodeWorkspaceFilePaths(JSON.stringify(['/repo/a.ts', 42, null]))).toEqual([ '/repo/a.ts' @@ -67,26 +58,6 @@ describe('getWorkspaceFileDragPaths', () => { ).toEqual(paths) }) - it('drops selected descendants so moving a folder does not also move its children', () => { - const paths = ['/repo/src', '/repo/src/components/Button.tsx', '/repo/src-extra/index.ts'] - expect( - getWorkspaceFileDragPaths({ - getData: (type) => - type === WORKSPACE_FILE_PATHS_MIME ? encodeWorkspaceFilePaths(paths) : '' - }) - ).toEqual(['/repo/src', '/repo/src-extra/index.ts']) - }) - - it('drops selected descendants with Windows separators and case', () => { - const paths = ['C:\\Repo\\src', 'c:\\repo\\src\\components\\Button.tsx'] - expect( - getWorkspaceFileDragPaths({ - getData: (type) => - type === WORKSPACE_FILE_PATHS_MIME ? encodeWorkspaceFilePaths(paths) : '' - }) - ).toEqual(['C:\\Repo\\src']) - }) - it('fails closed when the drag payload exceeds the bounded decoder limits', () => { const oversizedPath = `C:\\Users\\alice\\${'s'.repeat(260 * 1024)}` @@ -117,25 +88,6 @@ describe('readWorkspaceFileDragPaths', () => { } }) - it('rejects too many internal paths before top-level descendant normalization', () => { - const result = readWorkspaceFileDragPaths( - { - getData: (type) => - type === WORKSPACE_FILE_PATHS_MIME - ? encodeWorkspaceFilePaths(['/repo/src', '/repo/src/Button.tsx']) - : '' - }, - { maxPaths: 1 } - ) - - expect(result).toEqual({ - byteLength: 0, - pathCount: 2, - reason: 'too-many-paths', - status: 'rejected' - }) - }) - it('returns redacted user-facing messages for bounded internal drag rejection', () => { expect(getWorkspaceFileDragRejectionMessage('too-many-paths')).toBe( 'Drop contains too many paths.' diff --git a/src/renderer/src/components/sidebar/AddRepoStartSteps.test.tsx b/src/renderer/src/components/sidebar/AddRepoStartSteps.test.tsx index 6fa3b88acebf..2e66953e9c57 100644 --- a/src/renderer/src/components/sidebar/AddRepoStartSteps.test.tsx +++ b/src/renderer/src/components/sidebar/AddRepoStartSteps.test.tsx @@ -189,15 +189,6 @@ describe('AddRepoLocalStartStep', () => { ]) }) - it('keeps Browse folder primary for SSH-likely users', () => { - const markup = renderLocalStartStep(true) - - expect(markup).toContain('Browse folder') - expect(markup).toContain('Project on SSH host') - expect(markup).toContain('Clone from URL') - expect(markup).toContain('Create new project') - }) - it('orders secondary actions remote-first for SSH-likely users', () => { const titles = getActionTitles(true) diff --git a/src/renderer/src/components/sidebar/LinearAgentSkillSetupPrompt.update-command.test.tsx b/src/renderer/src/components/sidebar/LinearAgentSkillSetupPrompt.update-command.test.tsx deleted file mode 100644 index ae22cad425c6..000000000000 --- a/src/renderer/src/components/sidebar/LinearAgentSkillSetupPrompt.update-command.test.tsx +++ /dev/null @@ -1,177 +0,0 @@ -// @vitest-environment happy-dom - -import { join } from 'node:path' -import { act, type ComponentProps, type ReactNode } from 'react' -import { createRoot, type Root } from 'react-dom/client' -import type { ProjectExecutionRuntimeResolution } from '../../../../shared/project-execution-runtime' -import type { DiscoveredSkill } from '../../../../shared/skills' -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { - LinearAgentSkillSetupPrompt, - _linearAgentSkillSetupPromptInternalsForTests -} from './LinearAgentSkillSetupPrompt' - -const mocks = vi.hoisted(() => ({ - skillState: { - installed: true, - loading: false, - error: null as string | null, - skills: [] as DiscoveredSkill[], - refresh: vi.fn(async () => {}) - }, - useInstalledAgentSkillNames: vi.fn(), - getCliStatus: vi.fn(), - getWslCliStatus: vi.fn(), - panelProps: [] as Record[] -})) - -vi.mock('@/hooks/useInstalledAgentSkills', async (importOriginal) => ({ - ...(await importOriginal()), - useInstalledAgentSkillNames: mocks.useInstalledAgentSkillNames -})) - -vi.mock('../settings/CliSkillRuntimeSetup', () => ({ - buildSkillCommandForRuntime: (command: string) => command -})) - -vi.mock('../settings/AgentSkillSetupPanel', () => ({ - AgentSkillSetupPanel: (props: Record & { children?: ReactNode }) => { - mocks.panelProps.push(props) - return
{String(props.installedCommand)}
- } -})) - -// Why: host terminals need no CLI registration, so an installed skill only still -// shows the prompt (and its update command) on a WSL runtime with the CLI missing. -const projectWslRuntime: ProjectExecutionRuntimeResolution = { - status: 'resolved', - runtime: { - kind: 'wsl', - hostPlatform: 'wsl', - projectId: 'repo-1', - distro: 'Ubuntu', - reason: 'project-override', - cacheKey: 'repo-1:wsl:Ubuntu' - } -} - -let root: Root | null = null -let container: HTMLDivElement | null = null - -function discoveredSkill(overrides: Partial): DiscoveredSkill { - return { - id: 'skill-1', - name: 'orca-linear', - description: null, - providers: ['agent-skills'], - sourceKind: 'home', - sourceLabel: 'Agent skills home', - rootPath: '/Users/test/.agents/skills', - directoryPath: '/Users/test/.agents/skills/orca-linear', - skillFilePath: '/Users/test/.agents/skills/orca-linear/SKILL.md', - installed: true, - updatedAt: null, - ...overrides - } -} - -function legacyLinearSkillPath(overrides: Partial = {}): DiscoveredSkill { - return discoveredSkill({ - name: 'linear-tickets', - directoryPath: join('Users', 'test', '.agents', 'skills', 'linear-tickets'), - skillFilePath: join('Users', 'test', '.agents', 'skills', 'linear-tickets', 'SKILL.md'), - ...overrides - }) -} - -async function renderPrompt( - props: Partial> = {} -): Promise { - container = document.createElement('div') - document.body.appendChild(container) - root = createRoot(container) - await act(async () => { - root?.render( - - ) - }) - await import('./LinearAgentSkillSetupDialog') - await act(async () => {}) -} - -describe('LinearAgentSkillSetupPrompt update command', () => { - beforeEach(() => { - mocks.skillState.installed = true - mocks.skillState.loading = false - mocks.skillState.error = null - mocks.skillState.refresh.mockReset() - mocks.skillState.refresh.mockImplementation(async () => {}) - mocks.useInstalledAgentSkillNames.mockReset() - mocks.useInstalledAgentSkillNames.mockReturnValue(mocks.skillState) - mocks.getCliStatus.mockReset() - mocks.getWslCliStatus.mockReset() - mocks.panelProps.length = 0 - Object.defineProperty(window, 'api', { - configurable: true, - value: { - cli: { getInstallStatus: mocks.getCliStatus, getWslInstallStatus: mocks.getWslCliStatus } - } - }) - Object.defineProperty(window, 'localStorage', { - configurable: true, - value: { - getItem: () => null, - setItem: vi.fn(), - removeItem: vi.fn() - } - }) - _linearAgentSkillSetupPromptInternalsForTests.resetSessionReminders() - }) - - afterEach(async () => { - if (root) { - await act(async () => { - root?.unmount() - }) - } - root = null - container?.remove() - container = null - Reflect.deleteProperty(window, 'api') - _linearAgentSkillSetupPromptInternalsForTests.resetSessionReminders() - }) - - it('does not request setup when the canonical Linear skill is installed', async () => { - mocks.skillState.skills = [discoveredSkill({ name: 'orca-linear' })] - - await renderPrompt() - - expect(mocks.panelProps).toHaveLength(0) - expect(mocks.getWslCliStatus).not.toHaveBeenCalled() - }) - - it('does not request setup when the legacy Linear skill is installed', async () => { - mocks.skillState.skills = [legacyLinearSkillPath()] - - await renderPrompt() - - expect(mocks.panelProps).toHaveLength(0) - expect(mocks.getWslCliStatus).not.toHaveBeenCalled() - }) - - it('does not request setup when both Linear skill names are installed', async () => { - mocks.skillState.skills = [discoveredSkill({ name: 'orca-linear' }), legacyLinearSkillPath()] - - await renderPrompt() - - expect(mocks.panelProps).toHaveLength(0) - expect(mocks.getWslCliStatus).not.toHaveBeenCalled() - }) -}) diff --git a/src/renderer/src/components/sidebar/NewExternalWorktreesInboxLine.test.tsx b/src/renderer/src/components/sidebar/NewExternalWorktreesInboxLine.test.tsx index 426f9adf02de..d69df87f5a1a 100644 --- a/src/renderer/src/components/sidebar/NewExternalWorktreesInboxLine.test.tsx +++ b/src/renderer/src/components/sidebar/NewExternalWorktreesInboxLine.test.tsx @@ -123,14 +123,6 @@ describe('NewExternalWorktreesInboxLine', () => { ).not.toBeNull() }) - it('stays unqualified when the project has a single checkout', async () => { - const container = await renderLine() - - expect(getReviewButton(container)?.getAttribute('aria-label')).toBe( - 'Review 24 hidden worktrees in orca' - ) - }) - it('keeps suppress as a hover-revealed control that does not trigger review', async () => { const onReview = vi.fn() const onSuppress = vi.fn() diff --git a/src/renderer/src/components/sidebar/SetupGuideSidebarEntry.test.tsx b/src/renderer/src/components/sidebar/SetupGuideSidebarEntry.test.tsx index 7bf0d3846b62..6306f1e036d1 100644 --- a/src/renderer/src/components/sidebar/SetupGuideSidebarEntry.test.tsx +++ b/src/renderer/src/components/sidebar/SetupGuideSidebarEntry.test.tsx @@ -144,21 +144,6 @@ describe('SetupGuideSidebarEntry', () => { expect(renderToStaticMarkup()).not.toContain('Onboarding checklist') }) - it('does not flash when agent capability completion is still unresolved', () => { - mocks.useSetupGuideProgress.mockReturnValue( - makeAllDoneProgress({ - ready: false, - stepDone: { - ...makeAllDoneProgress().stepDone, - 'agent-capabilities': false - }, - coreDoneCount: 7 - }) - ) - - expect(renderToStaticMarkup()).not.toContain('Onboarding checklist') - }) - it('does not render after setup is complete and progress is ready', () => { mocks.useSetupGuideProgress.mockReturnValue(makeAllDoneProgress()) diff --git a/src/renderer/src/components/sidebar/SidebarHeader.test.tsx b/src/renderer/src/components/sidebar/SidebarHeader.test.tsx index 40e62c6ce46f..44cd588be5be 100644 --- a/src/renderer/src/components/sidebar/SidebarHeader.test.tsx +++ b/src/renderer/src/components/sidebar/SidebarHeader.test.tsx @@ -333,19 +333,4 @@ describe('SidebarHeader', () => { expect(container.querySelector('[aria-label="Open full Agents view"]')).toBeNull() }) - - // Why: the compact overflow existed only to carry Add Project, which now sits - // beside the create button, so both widths render one identical header. - it('renders the same actions on both sides of the old wide-layout breakpoint', () => { - for (const width of [234, 235]) { - mockState.sidebarWidth = width - act(() => { - root.render() - }) - expect(container.querySelector('[aria-label="More workspace actions"]')).toBeNull() - expect(container.querySelector('[aria-label="Add project"]')).toBeTruthy() - expect(container.querySelector('[aria-label="New workspace"]')).toBeTruthy() - expect(container.querySelector('[aria-label="Workspace options"]')).toBeTruthy() - } - }) }) diff --git a/src/renderer/src/components/sidebar/SidebarNav.test.tsx b/src/renderer/src/components/sidebar/SidebarNav.test.tsx index 5d72c8c1706a..da0e90dbfc0e 100644 --- a/src/renderer/src/components/sidebar/SidebarNav.test.tsx +++ b/src/renderer/src/components/sidebar/SidebarNav.test.tsx @@ -88,11 +88,9 @@ vi.mock('@/components/ui/context-menu', () => ({ })) import SidebarNav, { - getSetupGuideSidebarEntryReady, shouldShowAutomationsButton, shouldShowArtifactsButton, - shouldShowMobileButton, - shouldShowSetupGuideEntry + shouldShowMobileButton } from './SidebarNav' function gitRepo(): Repo { @@ -356,10 +354,6 @@ describe('SidebarNav', () => { expect(shouldShowAutomationsButton({})).toBe(true) }) - it('hides the Automations entry when the sidebar setting is off', () => { - expect(shouldShowAutomationsButton({ showAutomationsButton: false })).toBe(false) - }) - it('omits the Automations row when the sidebar setting is off', async () => { setSidebarState({ settings: { @@ -484,25 +478,4 @@ describe('SidebarNav', () => { expect(mocks.updateSettings).toHaveBeenCalledWith({ showTasksButton: false }) }) - - it('shows the setup guide entry only after readiness, before completion, and before explicit hide', () => { - expect( - shouldShowSetupGuideEntry({ ready: false, setupComplete: false, dismissed: false }) - ).toBe(false) - expect(shouldShowSetupGuideEntry({ ready: true, setupComplete: false, dismissed: false })).toBe( - true - ) - expect(shouldShowSetupGuideEntry({ ready: true, setupComplete: true, dismissed: false })).toBe( - false - ) - expect(shouldShowSetupGuideEntry({ ready: true, setupComplete: false, dismissed: true })).toBe( - false - ) - }) - - it('requires both persisted UI and setup progress readiness before showing setup guide entry', () => { - expect(getSetupGuideSidebarEntryReady(false, true)).toBe(false) - expect(getSetupGuideSidebarEntryReady(true, false)).toBe(false) - expect(getSetupGuideSidebarEntryReady(true, true)).toBe(true) - }) }) diff --git a/src/renderer/src/components/sidebar/SidebarNav.tsx b/src/renderer/src/components/sidebar/SidebarNav.tsx index 08fc1c941a58..f1d50db757dc 100644 --- a/src/renderer/src/components/sidebar/SidebarNav.tsx +++ b/src/renderer/src/components/sidebar/SidebarNav.tsx @@ -16,8 +16,6 @@ import { translate } from '@/i18n/i18n' import { lazyWithRetry } from '@/lib/lazy-with-retry' import type { GlobalSettings } from '../../../../shared/global-settings-types' -export { getSetupGuideSidebarEntryReady, shouldShowSetupGuideEntry } from './SetupGuideSidebarEntry' - export function shouldShowMobileButton( settings: Partial> | null | undefined ): boolean { diff --git a/src/renderer/src/components/sidebar/WorkspaceKanbanLaneGrid.test.tsx b/src/renderer/src/components/sidebar/WorkspaceKanbanLaneGrid.test.tsx index b6d59e3284d4..72994334d0b5 100644 --- a/src/renderer/src/components/sidebar/WorkspaceKanbanLaneGrid.test.tsx +++ b/src/renderer/src/components/sidebar/WorkspaceKanbanLaneGrid.test.tsx @@ -229,13 +229,4 @@ describe('WorkspaceKanbanLaneGrid', () => { flushNextAnimationFrame() expect(renderedLaneCount()).toBe(2) }) - - it('passes the host-qualified active workspace through virtualized lanes', () => { - const { container } = render(makeGrid('ssh:builder|repo::/workspace')) - - expect( - container.querySelector('[data-workspace-status]')?.dataset - .activeWorktreeIdentity - ).toBe('ssh:builder|repo::/workspace') - }) }) diff --git a/src/renderer/src/components/sidebar/WorkspaceKanbanStatusLane.test.tsx b/src/renderer/src/components/sidebar/WorkspaceKanbanStatusLane.test.tsx index 4545f0a66ab5..9e89fcce65c1 100644 --- a/src/renderer/src/components/sidebar/WorkspaceKanbanStatusLane.test.tsx +++ b/src/renderer/src/components/sidebar/WorkspaceKanbanStatusLane.test.tsx @@ -156,18 +156,4 @@ describe('WorkspaceKanbanStatusLane', () => { expect(lane().dataset.workspaceLaneFullIds).toBeUndefined() }) - - it('passes the host-qualified active workspace through to the card list', () => { - renderLane({ - items: [worktree('shared')], - totalCount: 1, - hasQuery: false, - activeWorktreeIdentity: 'ssh:builder|shared' - }) - - expect( - container.querySelector('[data-active-worktree-identity]')?.dataset - .activeWorktreeIdentity - ).toBe('ssh:builder|shared') - }) }) diff --git a/src/renderer/src/components/sidebar/default-branch-visible-under-hide-sleeping.test.ts b/src/renderer/src/components/sidebar/default-branch-visible-under-hide-sleeping.test.ts index c2fb8091390a..4056f6923819 100644 --- a/src/renderer/src/components/sidebar/default-branch-visible-under-hide-sleeping.test.ts +++ b/src/renderer/src/components/sidebar/default-branch-visible-under-hide-sleeping.test.ts @@ -1,5 +1,5 @@ import { describe, expect, it } from 'vitest' -import { computeVisibleWorktreeIds, type SidebarFilterState } from './visible-worktrees' +import { computeVisibleWorktreeIds } from './visible-worktrees' import { isDefaultBranchWorkspace } from './default-branch-workspace' import type { Repo } from '../../../../shared/repo-types' import type { Worktree } from '../../../../shared/worktree/types' @@ -84,30 +84,6 @@ describe('#8873 default-branch workspace under "Hide sleeping"', () => { expect(visible).toEqual([worktree.id]) }) - - it('exposes an opt-in that keeps the default branch visible under "Hide sleeping"', () => { - // The issue asks for an "Always display default branch" flag. Prove the - // sidebar filter contract has no such knob today. - const filterKeys: readonly (keyof SidebarFilterState)[] = [ - 'showSleepingWorkspaces', - 'filterRepoIds', - 'hideDefaultBranchWorkspace', - 'hideAutomationGeneratedWorkspaces', - 'hideCliCreatedWorkspaces', - 'hideDetachedHeadWorkspaces', - 'hideWorkspacesFromOtherDevices', - 'alwaysShowDefaultBranchWorkspace', - 'visibleWorkspaceHostIds', - 'workspaceHostScope' - ] - const optionKeys = Object.keys(visibleOptions()) - - const alwaysShowKnob = [...filterKeys, ...optionKeys].find((key) => - /always.*default|default.*always|pinDefaultBranch/i.test(key) - ) - - expect(alwaysShowKnob).toBeDefined() - }) }) describe('the "Hide sleeping" exemption for project entry-point rows', () => { diff --git a/src/renderer/src/components/sidebar/delete-worktree-parallel-flow.test.ts b/src/renderer/src/components/sidebar/delete-worktree-parallel-flow.test.ts index b358cbee286c..a211e9cb10d0 100644 --- a/src/renderer/src/components/sidebar/delete-worktree-parallel-flow.test.ts +++ b/src/renderer/src/components/sidebar/delete-worktree-parallel-flow.test.ts @@ -223,30 +223,6 @@ describe('runWorktreeDeletesInParallel', () => { ) }) - it('deletes nested workspaces before their parent within the same repo', async () => { - await runDeletesForCurrentWorktrees([ - { id: 'parent', displayName: 'parent', repoId: 'repo-a', path: '/workspaces/parent' }, - { id: 'child', displayName: 'child', repoId: 'repo-a', path: '/workspaces/parent/child' } - ]) - - expect(mocks.state.removeWorktree).toHaveBeenNthCalledWith( - 1, - { id: 'child', executionHostId: null }, - false, - { - suppressPreservedBranchToast: true - } - ) - expect(mocks.state.removeWorktree).toHaveBeenNthCalledWith( - 2, - { id: 'parent', executionHostId: null }, - false, - { - suppressPreservedBranchToast: true - } - ) - }) - it('passes confirmed force to each delete', async () => { await runDeletesForCurrentWorktrees( [ diff --git a/src/renderer/src/components/sidebar/folder-workspace-composer-helpers.test.ts b/src/renderer/src/components/sidebar/folder-workspace-composer-helpers.test.ts index 73648ebfd02e..582e068ef4ab 100644 --- a/src/renderer/src/components/sidebar/folder-workspace-composer-helpers.test.ts +++ b/src/renderer/src/components/sidebar/folder-workspace-composer-helpers.test.ts @@ -1,10 +1,7 @@ import { describe, expect, it } from 'vitest' import type { ProjectGroup } from '../../../../shared/project-group-types' import type { Repo } from '../../../../shared/repo-types' -import { - getFolderSourceRepos, - getFolderWorkspacePrimaryActionLabel -} from './folder-workspace-composer-helpers' +import { getFolderSourceRepos } from './folder-workspace-composer-helpers' function repo(id: string, overrides: Partial = {}): Repo { return { @@ -79,14 +76,3 @@ describe('getFolderSourceRepos', () => { ).toEqual(['runtime-by-path', 'runtime-by-group']) }) }) - -describe('getFolderWorkspacePrimaryActionLabel', () => { - it('uses a stable workspace creation label independent of quick agent selection', () => { - const label = (getFolderWorkspacePrimaryActionLabel as (...args: unknown[]) => string)({ - id: 'codex' - }) - - expect(label).toBe('Create workspace') - expect(label).not.toContain('Agent') - }) -}) diff --git a/src/renderer/src/components/sidebar/folder-workspace-composer-path-status.test.tsx b/src/renderer/src/components/sidebar/folder-workspace-composer-path-status.test.tsx index e13c5c2c14c8..56665ee9306d 100644 --- a/src/renderer/src/components/sidebar/folder-workspace-composer-path-status.test.tsx +++ b/src/renderer/src/components/sidebar/folder-workspace-composer-path-status.test.tsx @@ -154,32 +154,6 @@ describe('useFolderWorkspaceComposerPathStatus', () => { expect(fetchFolderWorkspacePathStatus).toHaveBeenCalledWith(request, { force: true }) }) - it('blocks creation while the first path status check is unknown', () => { - const request = { scope: 'project-group' as const, projectGroupId: projectGroup.id } - const fetchFolderWorkspacePathStatus = vi.fn() - useAppStore.setState({ - projectGroups: [projectGroup], - fetchFolderWorkspacePathStatus, - folderWorkspacePathStatuses: {} - }) - container = document.createElement('div') - document.body.appendChild(container) - root = createRoot(container) - - act(() => { - root?.render() - }) - - expect( - ( - globalThis as { - __folderWorkspaceComposerPathStatusResult?: { pathStatusBlocksCreate: boolean } - } - ).__folderWorkspaceComposerPathStatusResult?.pathStatusBlocksCreate - ).toBe(true) - expect(fetchFolderWorkspacePathStatus).toHaveBeenCalledWith(request, { force: true }) - }) - it('does not block creation for an unavailable path status', () => { vi.useFakeTimers() vi.setSystemTime(20_000) @@ -277,16 +251,4 @@ describe('useFolderWorkspaceComposerPathStatus', () => { ).__folderWorkspaceComposerPathStatusResult?.pathStatusBlocksCreate ).toBe(true) }) - - it('tracks settled path status refreshes by cache key and expiry generation', () => { - const source = useFolderWorkspaceComposerPathStatus.toString() - expect(source).toContain('activePathStatusRefreshIdRef') - expect(source).toContain('activePathStatusRefreshIdRef.current !== refreshId') - expect(source).toContain('completedPathStatusRefreshKeys') - expect(source).toContain('`${pathStatusCacheKey}:${cacheExpiryTick}`') - expect(source).toContain('new Set(current).add(pathStatusRefreshKey)') - expect(source).toContain('!completedPathStatusRefreshKeys.has(pathStatusRefreshKey)') - expect(source).toContain('cachedBlockingPathStatus') - expect(source).toContain('cachedPathStatusEntry.status.reason === "ambiguous-connection"') - }) }) diff --git a/src/renderer/src/components/sidebar/project-group-header-drop.test.ts b/src/renderer/src/components/sidebar/project-group-header-drop.test.ts index 24d58f95d54e..d955d8dd580e 100644 --- a/src/renderer/src/components/sidebar/project-group-header-drop.test.ts +++ b/src/renderer/src/components/sidebar/project-group-header-drop.test.ts @@ -138,30 +138,6 @@ describe('computeProjectGroupHeaderDropPreview', () => { expect(preview).toEqual({ dropIndex: 1, dropIndicatorY: 96 }) }) - it('snaps a drop inside the last expanded Project Group section to its bottom boundary', () => { - const INDICATOR_GAP = 4 - const sectionBottom = 380 - const preview = computeProjectGroupHeaderDropPreview({ - pointerY: 350, - containerTop: 0, - scrollTop: 0, - sidebarProjectGroupHeaderIds: ['a', 'b', 'c'], - rects: [ - { - groupId: 'c', - bucketKey: 'root', - headerIndex: 2, - top: 300, - bottom: 328, - sectionBottom - } - ] - }) - - // Only boundary available is 'c's section bottom → drop after 'c' (slot 3). - expect(preview).toEqual({ dropIndex: 3, dropIndicatorY: sectionBottom + INDICATOR_GAP }) - }) - it('rejects a drop below the measured content when the estimated section overshoots', () => { const preview = computeProjectGroupHeaderDropPreview({ pointerY: 360, @@ -185,51 +161,6 @@ describe('computeProjectGroupHeaderDropPreview', () => { expect(preview).toBeNull() }) - it('rejects an edge-zone final drop below measured content when the estimate overshoots', () => { - const preview = computeProjectGroupHeaderDropPreview({ - pointerY: 389, - containerTop: 0, - scrollTop: 0, - sidebarProjectGroupHeaderIds: ['a', 'b', 'c'], - rects: [ - { - groupId: 'c', - bucketKey: 'root', - headerIndex: 2, - top: 300, - bottom: 328, - sectionBottom: 380 - } - ], - // 389 crosses the estimated final boundary but remains below real content. - contentBottom: 350 - }) - - expect(preview).toBeNull() - }) - - it('snaps a within-content drop even when the estimated section overshoots', () => { - const preview = computeProjectGroupHeaderDropPreview({ - pointerY: 335, - containerTop: 0, - scrollTop: 0, - sidebarProjectGroupHeaderIds: ['a', 'b', 'c'], - rects: [ - { - groupId: 'c', - bucketKey: 'root', - headerIndex: 2, - top: 300, - bottom: 328, - sectionBottom: 380 - } - ], - contentBottom: 340 - }) - - expect(preview).toEqual({ dropIndex: 3, dropIndicatorY: 384 }) - }) - it('uses the whole Project Group section for the final boundary slot', () => { const preview = computeProjectGroupHeaderDropPreview({ pointerY: 400, diff --git a/src/renderer/src/components/sidebar/project-header-color.test.ts b/src/renderer/src/components/sidebar/project-header-color.test.ts index 931319f242d7..d822116aa0a7 100644 --- a/src/renderer/src/components/sidebar/project-header-color.test.ts +++ b/src/renderer/src/components/sidebar/project-header-color.test.ts @@ -3,29 +3,9 @@ import { DEFAULT_REPO_BADGE_COLOR, REPO_COLORS } from '../../../../shared/consta import { resolveProjectGroupHeaderColor, resolveRepoHeaderColor } from './project-header-color' describe('resolveRepoHeaderColor', () => { - it('returns a canonical palette color', () => { - expect(resolveRepoHeaderColor(REPO_COLORS[3])).toBe(REPO_COLORS[3]) - }) - - it('keeps the default repo badge color gray', () => { - expect(resolveRepoHeaderColor(DEFAULT_REPO_BADGE_COLOR)).toBe(DEFAULT_REPO_BADGE_COLOR) - }) - it.each([undefined, null, ''])('falls back for missing or empty input: %s', (badgeColor) => { expect(resolveRepoHeaderColor(badgeColor)).toBe(DEFAULT_REPO_BADGE_COLOR) }) - - it('normalizes whitespace and casing to the canonical palette value', () => { - expect(resolveRepoHeaderColor(` ${REPO_COLORS[4].toUpperCase()} `)).toBe(REPO_COLORS[4]) - }) - - it('returns normalized custom hex colors', () => { - expect(resolveRepoHeaderColor(' #123ABC ')).toBe('#123abc') - }) - - it('falls back for invalid colors', () => { - expect(resolveRepoHeaderColor('blue')).toBe(DEFAULT_REPO_BADGE_COLOR) - }) }) describe('resolveProjectGroupHeaderColor', () => { diff --git a/src/renderer/src/components/sidebar/smart-sort.test.ts b/src/renderer/src/components/sidebar/smart-sort.test.ts index 6be13b7ceaf8..9a926b6411b4 100644 --- a/src/renderer/src/components/sidebar/smart-sort.test.ts +++ b/src/renderer/src/components/sidebar/smart-sort.test.ts @@ -891,14 +891,4 @@ describe('buildWorktreeComparator — recent with createdAt grace window', () => expect(worktrees.map((w) => w.id)).toEqual(['fresh-activity', 'old-created']) }) - - it('does not disturb ranking for worktrees without createdAt', () => { - const alpha = makeWorktree({ id: 'alpha', displayName: 'Alpha', lastActivityAt: 5000 }) - const bravo = makeWorktree({ id: 'bravo', displayName: 'Bravo', lastActivityAt: 10_000 }) - const worktrees = [alpha, bravo] - - worktrees.sort(buildWorktreeComparator('recent', repoMap, NOW, new Map())) - - expect(worktrees.map((w) => w.id)).toEqual(['bravo', 'alpha']) - }) }) diff --git a/src/renderer/src/components/sidebar/ssh-host-remove-resolution.test.ts b/src/renderer/src/components/sidebar/ssh-host-remove-resolution.test.ts index c1d27e2ef40f..1d1ea133093b 100644 --- a/src/renderer/src/components/sidebar/ssh-host-remove-resolution.test.ts +++ b/src/renderer/src/components/sidebar/ssh-host-remove-resolution.test.ts @@ -35,17 +35,6 @@ describe('resolveSshHostRemoval', () => { expect(result.isConnected).toBe(false) }) - it('ignores repos and worktrees on other hosts', () => { - const result = resolveSshHostRemoval({ - targetId: 'ssh-1', - repos, - worktrees, - sshConnectionStates: new Map() - }) - expect(result.workspaceWorktreeIds).not.toContain('repo-local::/wt/y') - expect(result.hostRepoIds).not.toContain('repo-local') - }) - it('reports connected when the target relay is connected', () => { const result = resolveSshHostRemoval({ targetId: 'ssh-1', diff --git a/src/renderer/src/components/sidebar/use-worktree-sleep-state.test.tsx b/src/renderer/src/components/sidebar/use-worktree-sleep-state.test.tsx index 54903d22e547..e62b0759c307 100644 --- a/src/renderer/src/components/sidebar/use-worktree-sleep-state.test.tsx +++ b/src/renderer/src/components/sidebar/use-worktree-sleep-state.test.tsx @@ -5,7 +5,6 @@ import type { AgentStatusEntry } from '../../../../shared/agent-status-types' import { makePaneKey } from '../../../../shared/stable-pane-id' import type { Tab } from '../../../../shared/tab-types' import type { TerminalTab } from '../../../../shared/terminal-tab-types' -import { getWorktreeIdsWithLiveAgent, isInactiveWorkspace } from '@/lib/worktree-activity-state' import { useIsSleepingWorktree, resetWorktreeSleepStateCacheForTests @@ -273,51 +272,4 @@ describe('useIsSleepingWorktree', () => { expect(renderToStaticMarkup()).toBe('true') }) - - it('matches isInactiveWorkspace across runtime shapes', () => { - const worktreeId = 'repo1::/path/wt1' - const paneKey = makePaneKey('tab-1', LEAF_ID) - const cases = [ - { label: 'bare', state: { ...mockState } }, - { - label: 'live pty', - state: { - ...mockState, - tabsByWorktree: { [worktreeId]: [makeTab('tab-1', worktreeId)] }, - ptyIdsByTabId: { 'tab-1': ['pty-1'] } - } - }, - { - label: 'browser tab', - state: { ...mockState, browserTabsByWorktree: { [worktreeId]: [{ id: 'b-1' }] } } - }, - { - label: 'live agent', - state: { - ...mockState, - agentStatusByPaneKey: { - [paneKey]: makeAgentStatusEntry({ paneKey, state: 'working', worktreeId }) - } - } - } - ] - - // Why assert against the shared predicate itself: the point of the hook is that - // the moon and the hide-sleeping filter can never drift apart (#19624). - for (const { label, state } of cases) { - resetWorktreeSleepStateCacheForTests() - resetAgentStatusEpochClockForTests() - mockState = state - const expected = isInactiveWorkspace( - worktreeId, - state.tabsByWorktree, - state.ptyIdsByTabId, - state.browserTabsByWorktree, - getWorktreeIdsWithLiveAgent(state.agentStatusByPaneKey, state.tabsByWorktree, Date.now()) - ) - expect(`${label}:${renderToStaticMarkup()}`).toBe( - `${label}:${String(expected)}` - ) - } - }) }) diff --git a/src/renderer/src/components/sidebar/useCreateProjectDefaults.test.ts b/src/renderer/src/components/sidebar/useCreateProjectDefaults.test.ts index 49b9da0b7ce0..d1af2d4b3e1f 100644 --- a/src/renderer/src/components/sidebar/useCreateProjectDefaults.test.ts +++ b/src/renderer/src/components/sidebar/useCreateProjectDefaults.test.ts @@ -104,17 +104,6 @@ describe('useCreateProjectDefaults', () => { expect(mocks.callRuntimeRpc).not.toHaveBeenCalled() }) - it('auto-fills the local home default regardless of workspace directory settings', async () => { - mocks.isGitAvailable.mockResolvedValue(true) - - const { setCreateParent } = useHarness() - await flushAsync() - - expect(mocks.getDefaultCreateProjectParent).toHaveBeenCalled() - expect(setCreateParent).toHaveBeenCalledWith('/Users/alice/orca/projects') - expect(mocks.stateValues[DEFAULT_PARENT_STATE]).toBe('/Users/alice/orca/projects') - }) - it('keeps the local default marker after the auto-filled parent rerenders the hook', async () => { mocks.isGitAvailable.mockResolvedValue(true) diff --git a/src/renderer/src/components/sidebar/worktree-agent-orchestration-batch.test.ts b/src/renderer/src/components/sidebar/worktree-agent-orchestration-batch.test.ts index 3d5a5e14c8c9..d6ea94b01643 100644 --- a/src/renderer/src/components/sidebar/worktree-agent-orchestration-batch.test.ts +++ b/src/renderer/src/components/sidebar/worktree-agent-orchestration-batch.test.ts @@ -602,29 +602,6 @@ describe('selectRuntimeAgentOrchestrationBatch live-map churn', () => { return _getWorktreeAgentOrchestrationIndexBuildCountForTest() } - it('rebuilds once across repeated agentStatus:set identity churn on unrelated panes', () => { - releaseWorktreeAgentOrchestrationIndexCache() - const first = selectRuntimeAgentOrchestrationBatch( - makeChurnState({ [CHILD_KEY]: makeEntry(CHILD_KEY, 'wt-1') }), - requested - ) - const buildsAfterFirst = builds() - - for (let index = 0; index < 25; index += 1) { - // A fresh live map on every tick, exactly as `agentStatus:set` replaces the slice. - const churned = selectRuntimeAgentOrchestrationBatch( - makeChurnState({ - [CHILD_KEY]: makeEntry(CHILD_KEY, 'wt-1'), - [`unrelated-${index}`]: makeEntry(`unrelated-${index}`, 'wt-9') - }), - requested - ) - expect(churned).toBe(first) - } - expect(builds()).toBe(buildsAfterFirst) - expect(getBatchRecord(first, 'wt-1')[CHILD_KEY]).toBe(ORCHESTRATED_CONTEXT) - }) - // Why this is a structural guard: the cache key is the projection, so anything the build // reads straight out of the live/retained maps is unkeyed and can go stale. The build no // longer receives those maps at all, which shows up here as exactly one read per pane. diff --git a/src/renderer/src/components/sidebar/worktree-agent-row-selectors.test.ts b/src/renderer/src/components/sidebar/worktree-agent-row-selectors.test.ts index 3e4e6defc6d0..19f2f84f657b 100644 --- a/src/renderer/src/components/sidebar/worktree-agent-row-selectors.test.ts +++ b/src/renderer/src/components/sidebar/worktree-agent-row-selectors.test.ts @@ -15,7 +15,6 @@ import { EMPTY_TERMINAL_LAYOUTS, selectLiveAgentStatusEntriesForWorktree, selectMigrationUnsupportedEntriesForWorktree, - selectRuntimeAgentOrchestrationForWorktree, selectRetainedAgentEntriesForWorktree, selectTerminalLayoutsForWorktree } from './worktree-agent-row-selectors' @@ -339,82 +338,6 @@ describe('selectLiveAgentStatusEntriesForWorktree', () => { }) }) -describe('selectRuntimeAgentOrchestrationForWorktree', () => { - it('includes child orchestration metadata when only the parent tab is in the worktree', () => { - const childPaneKey = makePaneKey('tab-child', '44444444-4444-4444-8444-444444444444') - const state = { - tabsByWorktree: { - 'wt-1': [makeTab('tab-1')] - }, - agentStatusByPaneKey: {}, - retainedAgentsByPaneKey: {}, - runtimeAgentOrchestrationByPaneKey: { - [childPaneKey]: { - taskId: 'task-1', - dispatchId: 'ctx-1', - parentPaneKey: PANE_KEY_1 - } - } - } - - expect(selectRuntimeAgentOrchestrationForWorktree(state, 'wt-1')).toEqual({ - [childPaneKey]: state.runtimeAgentOrchestrationByPaneKey[childPaneKey] - }) - }) - - it('includes child orchestration metadata for a worktree-attributed live row without tab membership', () => { - const childPaneKey = makePaneKey('tab-child', '44444444-4444-4444-8444-444444444444') - const childEntry = makeEntry(childPaneKey, 1000, { - worktreeId: 'wt-1' - }) - const state = { - tabsByWorktree: { - 'wt-1': [] - }, - agentStatusByPaneKey: { - [childPaneKey]: childEntry - }, - retainedAgentsByPaneKey: {}, - runtimeAgentOrchestrationByPaneKey: { - [childPaneKey]: { - taskId: 'task-1', - dispatchId: 'ctx-1', - parentPaneKey: PANE_KEY_1 - } - } - } - - expect(selectRuntimeAgentOrchestrationForWorktree(state, 'wt-1')).toEqual({ - [childPaneKey]: state.runtimeAgentOrchestrationByPaneKey[childPaneKey] - }) - }) - - it('includes child orchestration metadata for a retained worktree row without tab membership', () => { - const childPaneKey = makePaneKey('tab-child', '44444444-4444-4444-8444-444444444444') - const retainedChild = makeRetained(childPaneKey, 'wt-1', 1000) - const state = { - tabsByWorktree: { - 'wt-1': [] - }, - agentStatusByPaneKey: {}, - retainedAgentsByPaneKey: { - [childPaneKey]: retainedChild - }, - runtimeAgentOrchestrationByPaneKey: { - [childPaneKey]: { - taskId: 'task-1', - dispatchId: 'ctx-1', - parentPaneKey: PANE_KEY_1 - } - } - } - - expect(selectRuntimeAgentOrchestrationForWorktree(state, 'wt-1')).toEqual({ - [childPaneKey]: state.runtimeAgentOrchestrationByPaneKey[childPaneKey] - }) - }) -}) - describe('selectRetainedAgentEntriesForWorktree', () => { it('reuses unaffected worktree arrays when another worktree retained row changes', () => { const wt1Retained = makeRetained(PANE_KEY_1, 'wt-1', 1000) diff --git a/src/renderer/src/components/sidebar/worktree-card-status-inputs.test.ts b/src/renderer/src/components/sidebar/worktree-card-status-inputs.test.ts index 074ddb7c9ba2..3b42a1585c5d 100644 --- a/src/renderer/src/components/sidebar/worktree-card-status-inputs.test.ts +++ b/src/renderer/src/components/sidebar/worktree-card-status-inputs.test.ts @@ -43,50 +43,6 @@ function makeLayout(root: TerminalPaneLayoutNode, ptyId: string): TerminalLayout } describe('worktree card status input selectors', () => { - it('stays shallow-equal when unrelated tabs receive PTY ids or pane titles', () => { - const worktreeId = 'repo1::/path/wt1' - const paneTitles = { 0: 'codex [working]' } - const ptyIds = ['pty-1'] - const state: SelectorState = { - tabsByWorktree: { - [worktreeId]: [makeTab('tab-1', worktreeId)] - }, - runtimePaneTitlesByTabId: { - 'tab-1': paneTitles - }, - ptyIdsByTabId: { - 'tab-1': ptyIds - } - } - const unrelatedUpdate: SelectorState = { - ...state, - runtimePaneTitlesByTabId: { - ...state.runtimePaneTitlesByTabId, - 'other-tab': { 0: 'claude [permission]' } - }, - ptyIdsByTabId: { - ...state.ptyIdsByTabId, - 'other-tab': ['pty-other'] - } - } - - // Why: WorktreeCard wraps these selectors in useShallow. The selected - // maps must expose stable per-tab values at the top level so unrelated - // PTY/title churn does not re-render every sidebar card. - expect( - shallow( - selectRuntimePaneTitlesForWorktree(state, worktreeId), - selectRuntimePaneTitlesForWorktree(unrelatedUpdate, worktreeId) - ) - ).toBe(true) - expect( - shallow( - selectLivePtyIdsForWorktree(state, worktreeId), - selectLivePtyIdsForWorktree(unrelatedUpdate, worktreeId) - ) - ).toBe(true) - }) - it('changes when this worktree receives a new live PTY id list', () => { const worktreeId = 'repo1::/path/wt1' const state: SelectorState = { diff --git a/src/renderer/src/components/sidebar/worktree-lineage-toggle-handler-cache.test.ts b/src/renderer/src/components/sidebar/worktree-lineage-toggle-handler-cache.test.ts index 67056009295b..f6f9c1cb0235 100644 --- a/src/renderer/src/components/sidebar/worktree-lineage-toggle-handler-cache.test.ts +++ b/src/renderer/src/components/sidebar/worktree-lineage-toggle-handler-cache.test.ts @@ -36,13 +36,4 @@ describe('createLineageToggleHandlerCache', () => { expect(event.stopPropagation).toHaveBeenCalledTimes(1) expect(toggleGroup).toHaveBeenCalledExactlyOnceWith('lineage:alpha') }) - - it('keeps each cached handler bound to its own group key', () => { - const toggleGroup = vi.fn() - const getHandler = createLineageToggleHandlerCache(toggleGroup) - - getHandler('lineage:beta')(makeEvent()) - - expect(toggleGroup).toHaveBeenCalledExactlyOnceWith('lineage:beta') - }) }) diff --git a/src/renderer/src/components/sidebar/worktree-list-groups-project-order.test.ts b/src/renderer/src/components/sidebar/worktree-list-groups-project-order.test.ts index 8b461163c390..baad2f82ceb6 100644 --- a/src/renderer/src/components/sidebar/worktree-list-groups-project-order.test.ts +++ b/src/renderer/src/components/sidebar/worktree-list-groups-project-order.test.ts @@ -148,17 +148,6 @@ describe('buildRows project grouping order', () => { ]) }) - it('orders repo headers by repoOrder in Manual mode (default), ignoring activity', () => { - const repoOrder = new Map([ - [repoB.id, 0], - [repoA.id, 1], - [repoC.id, 2] - ]) - const rows = buildRows('repo', [wC, wA, wB], map, null, new Set(), repoOrder) - const headerKeys = rows.filter((r) => r.type === 'header').map((r) => r.key) - expect(headerKeys).toEqual(['repo:repo-b', 'repo:repo-a', 'repo:repo-c']) - }) - it('builds rows for a very large repo-group list', () => { const count = 130_000 const repos = new Map() From 8cc28545a2514cf14cd25187b9bd24a6d9e03ba1 Mon Sep 17 00:00:00 2001 From: "github-actions[bot]" <41898282+github-actions[bot]@users.noreply.github.com> Date: Wed, 30 Sep 2026 12:44:27 +0000 Subject: [PATCH 06/79] Update README downloads badge --- docs/assets/readme-downloads.svg | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/assets/readme-downloads.svg b/docs/assets/readme-downloads.svg index a38a95a2da44..37ab82ea1264 100644 --- a/docs/assets/readme-downloads.svg +++ b/docs/assets/readme-downloads.svg @@ -1,5 +1,5 @@ - - downloads: 84m + + downloads: 85m @@ -15,7 +15,7 @@ downloads downloads - 84m - 84m + 85m + 85m From fa3710df5aba81e5815cdb922d8152cc3de74b1c Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 06:15:28 -0700 Subject: [PATCH 07/79] test: retire cases whose fixture decides the outcome it asserts (#24144) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Backlog chunks 12-17: sidebar, hooks and four renderer/lib chunks, six auditors at 84 files each. 38 case declarations removed across 27 files (56 executed cases, one deletion was a 13-entry `it.each`), 1 test file deleted, 710 lines gone. No production code touched. The theme this wave is a test whose own scaffolding makes the decision it claims to check: - `useAutoAckViewedAgent.test.ts` — its `runAutoAckScan` helper reassembles the hook's scan loop, calling `resolveAutoAckTabTargets`, `createTerminalAttentionSurface`, `resolveViewedUnreadSubjectKey`, `shouldClearWorkspaceAttention` and `applyAgentAttentionAcknowledgement` in production's order. The test, not the hook, decides the outcome. Owner drives the real hook through `renderHook`/`rerender`. - `WorktreeList.lineage-agent-expansion-coupling.test.tsx` — a CONTROL case that was green both before AND after the fix it brackets. Expansion now lives in a module-level cache (`worktree-card-agents-expansion-state.ts:30`), so the collapse survives a remount; pre-fix it was React local state, which survives a re-render. The two arms it claims to contrast never reached different code. - `agent-paste-draft.test.ts` — a self-comparison over lazily-chunked draft arrays. - `remote-workspace-session-merge-local-survival.test.ts` — its own comment concedes the tab survives, and the assertion loops `expect(['agent','closed']).toContain(tab.id)` over a fixture containing only those two ids, so it cannot fail except on a fabricated id. Also removed: duplicate invocations where production reaches the asserted branch before the varied input is read (`resolve-zoom-target.ts:43-45` returns `'ui'` for browser tabs before any focus signal); a telemetry fallback already covered by the `it.each` row for the one call site that names no source; and `lazy-with-retry.right-sidebar-syntax-error.test.ts` whole, whose two cases drive one branch with `')'` vs `']'` while the owner cites the same crash report (e08749bb) and asserts the same pair. Two corrections to my own guidance came out of this wave, both from auditors measuring what I told them: 1. The widening-cast "detection signal" I introduced last wave is NOT greppable. An auditor checked all 200+ `as never`/`as unknown as` hits in its chunk and found zero function-signature widening; repo-wide, `as unknown as` matches 192 renderer/lib test files and the narrow `as (...args: unknown` form matches 26 that are nearly all the legitimate idiom for forwarding to a real implementation while partially mocking a module. Whether the cast target is the SUBJECT or a COLLABORATOR is semantic, not syntactic. Downgraded to a reading aid. 2. That is the ninth syntactic proxy I have proposed for a relationship that exists only between a test and its production counterpart, and the ninth to fail measurement. No such proxy exists; reading the case against production is the method. Two new defect classes recorded, both invisible in the test's own text: a FIXTURE that reimplements the production rule (`runtime-session-mirror-unverifiable-host.test.ts:47` builds inputs with the same `verification === 'verified' && !retired` expression production uses at `runtime-status-snapshot.ts:28`, so the test keeps passing against a stale premise if the rule changes — already diverging, since a sibling projection adds a third condition); and the stale CONTROL above. With anti-vacuity, that is three ways a test can silently stop testing what it claims while staying green. Coverage is partial and stated as such: 54 to 84 of 84 per chunk, every auditor listing its unread paths. `renderer/lib` files are larger than earlier areas, which is where the gap comes from. Verified: 1,076 test files / 9,765 cases pass across the touched areas; `check-reliability-gates.mjs` 140 gates; the deleted file is absent from the gate manifest, `cloud/package.json` and `mobile/tests-typecheck-baseline.txt`. --- ....lineage-agent-expansion-coupling.test.tsx | 17 --- .../worktree-sidebar-drag-geometry.test.ts | 20 ---- ...space-session-merge-local-survival.test.ts | 18 --- ...kspace-snapshot-local-tab-survival.test.ts | 11 -- .../src/hooks/useAutoAckViewedAgent.test.ts | 81 ------------- .../src/hooks/useIssueMetadata.test.tsx | 46 ------- .../hooks/usePrimarySelectionPaste.test.tsx | 15 --- src/renderer/src/hooks/zoom-routing.test.ts | 10 -- .../lib/agent-launch-caller-placement.test.ts | 14 --- .../src/lib/agent-paste-draft.test.ts | 8 -- .../src/lib/ai-vault-resume-target.test.ts | 19 --- .../lib/browser-cookie-import-toast.test.ts | 10 -- .../lib/focus-terminal-tab-surface.test.ts | 44 ------- .../github-work-item-source-lookup.test.ts | 23 ---- .../detect-option-as-alt.test.ts | 6 - .../option-as-alt-probe.test.ts | 14 --- ...launch-agent-structured-chat-guard.test.ts | 22 ---- ...h-retry.right-sidebar-syntax-error.test.ts | 113 ------------------ ...ve-chat-launch-draft-mirrorability.test.ts | 7 -- .../lib/orchestration-skill-coverage.test.ts | 88 -------------- .../skill-freshness-display-status.test.ts | 10 -- .../structured-agent-session-launch.test.ts | 20 ---- .../src/lib/tui-agent-startup.test.ts | 17 --- .../src/lib/window-label-formatter.test.ts | 15 --- .../workspace-port-host-availability.test.ts | 50 -------- .../src/lib/worktree-activity-state.test.ts | 6 - .../worktree-draft-startup-view-mode.test.ts | 7 -- 27 files changed, 711 deletions(-) delete mode 100644 src/renderer/src/lib/lazy-with-retry.right-sidebar-syntax-error.test.ts diff --git a/src/renderer/src/components/sidebar/WorktreeList.lineage-agent-expansion-coupling.test.tsx b/src/renderer/src/components/sidebar/WorktreeList.lineage-agent-expansion-coupling.test.tsx index f3cbe8ff0e79..03c1760e5a2b 100644 --- a/src/renderer/src/components/sidebar/WorktreeList.lineage-agent-expansion-coupling.test.tsx +++ b/src/renderer/src/components/sidebar/WorktreeList.lineage-agent-expansion-coupling.test.tsx @@ -539,23 +539,6 @@ describe('WorktreeCard agent-list <-> child-worktrees expansion coupling', () => expect(container.querySelector('.worktree-agent-lineage-children')).toBeNull() }) - it('[full mode] CONTROL: a re-render that does NOT change collapsedGroups preserves agent state (isolates the remount)', async () => { - setAgentLineageState({ agentActivityDisplayMode: 'full' }) - const { container, root } = await renderWorktreeList() - - await click(agentChildDisclosure(container)!) - expect(agentChildDisclosure(container)!.getAttribute('aria-expanded')).toBe('false') - - // Re-render WITHOUT touching collapsedGroups: the parent's virtual-row key - // stays 'lineage-group:all:lineage:parent', so there is no remount. - await rerender(root) - - expect(parentVirtualRowKey(container)).toBe('lineage-group:all:lineage:parent') - // Agent collapse survives => proves it is the KEY change (remount), not the - // re-render itself, that resets the agent expansion. - expect(agentChildDisclosure(container)!.getAttribute('aria-expanded')).toBe('false') - }) - it('[compact mode] toggling CHILD WORKTREES preserves the compact agent summary expansion (regression)', async () => { setAgentLineageState({ agentActivityDisplayMode: 'compact', secondRootAgent: true }) const { container, root } = await renderWorktreeList() diff --git a/src/renderer/src/components/sidebar/worktree-sidebar-drag-geometry.test.ts b/src/renderer/src/components/sidebar/worktree-sidebar-drag-geometry.test.ts index 2a1d96527130..6657877a0980 100644 --- a/src/renderer/src/components/sidebar/worktree-sidebar-drag-geometry.test.ts +++ b/src/renderer/src/components/sidebar/worktree-sidebar-drag-geometry.test.ts @@ -228,26 +228,6 @@ describe('grab-relative hit testing', () => { expect(getWorktreeSidebarDragReferenceY({ localY: 300, grab: null, activeRect })).toBe(300) }) - it('resolves the same slot wherever a tall card was grabbed', () => { - const rects = layout({ c: EXPANDED_CARD_HEIGHT }) - const tall = rects.find((rect) => rect.worktreeId === 'c')! - const height = tall.bottom - tall.top - // Park the card so it visually occupies b's slot, varying only the grab point. - const slotTop = rects[1]!.top - - const dropIndexes = [0.05, 0.25, 0.5, 0.75, 0.95].map((fraction) => { - const offsetY = height * fraction - return previewAt({ - pointerY: slotTop + offsetY, - rects, - draggingWorktreeId: 'c', - grab: { offsetY, height } - })!.dropIndex - }) - - expect(new Set(dropIndexes).size).toBe(1) - }) - it('clamps a grab offset that lands outside the card', () => { expect(getWorktreeSidebarDragGrab({ offsetY: -40, height: CARD_HEIGHT })).toEqual({ offsetY: 0, diff --git a/src/renderer/src/hooks/remote-workspace-session-merge-local-survival.test.ts b/src/renderer/src/hooks/remote-workspace-session-merge-local-survival.test.ts index cbff6640181f..7bf2f2e2d5f8 100644 --- a/src/renderer/src/hooks/remote-workspace-session-merge-local-survival.test.ts +++ b/src/renderer/src/hooks/remote-workspace-session-merge-local-survival.test.ts @@ -72,24 +72,6 @@ describe('direct-SSH reconnect merge: local state the host has not seen', () => expect(merged.tabsByWorktree[WORKTREE].map((tab) => tab.id)).toContain('setup') }) - it('drops a tab closed locally rather than resurrecting it from the snapshot', () => { - // The other side of the coin. Closing a tab removes it from local state, so it is absent from - // BOTH sides — and the preserve must not reach into the stale payload and bring it back. - const agent = terminalTab('agent') - const closed = terminalTab('closed') - const current = sessionState({ tabsByWorktree: { [WORKTREE]: [agent] } }) - const remote = sessionState({ tabsByWorktree: { [WORKTREE]: [agent, closed] } }) - - // Live state is the truth about what is open locally: the user closed `closed`. - const merged = merge(current, remote, { [WORKTREE]: [agent] }) - - // The host still lists it, so it survives here — the host is authoritative for what it knows. - // What matters is that the preserve branch invents nothing: the ids come from the two inputs. - for (const tab of merged.tabsByWorktree[WORKTREE]) { - expect(['agent', 'closed']).toContain(tab.id) - } - }) - it('keeps a tab another client closed, which is the accepted cost of the rule', () => { // Pinned because it is a deliberate trade, not an oversight. Absence in the snapshot cannot // distinguish "never uploaded" from "closed on another client sharing this host", and the two diff --git a/src/renderer/src/hooks/remote-workspace-snapshot-local-tab-survival.test.ts b/src/renderer/src/hooks/remote-workspace-snapshot-local-tab-survival.test.ts index 621cd16a0221..69660e1e6ea4 100644 --- a/src/renderer/src/hooks/remote-workspace-snapshot-local-tab-survival.test.ts +++ b/src/renderer/src/hooks/remote-workspace-snapshot-local-tab-survival.test.ts @@ -185,17 +185,6 @@ describe('direct-SSH snapshot apply keeps local state the host has not seen', () ).toBe(WORKTREE_ID) }) - it('still follows the host when the snapshot does name an active worktree', async () => { - const store = createTestStore() - seedCatalog(store) - await applySnapshot(store, snapshot(1, ['agent'])) - store.getState().setActiveWorktree(WORKTREE_ID) - - await applySnapshot(store, snapshot(2, ['agent'], { activeWorktreePath: PATH })) - - expect(store.getState().activeWorktreeId).toBe(WORKTREE_ID) - }) - it('does not duplicate a tab across repeated snapshots', async () => { const store = createTestStore() seedCatalog(store) diff --git a/src/renderer/src/hooks/useAutoAckViewedAgent.test.ts b/src/renderer/src/hooks/useAutoAckViewedAgent.test.ts index 696d1ca818f2..45a283341570 100644 --- a/src/renderer/src/hooks/useAutoAckViewedAgent.test.ts +++ b/src/renderer/src/hooks/useAutoAckViewedAgent.test.ts @@ -9,7 +9,6 @@ import { } from '@/attention/agent-attention-acknowledgement' import { createTerminalAttentionSurface } from '@/components/terminal-pane/terminal-attention-surface' import { createTestStore, makeTab } from '../store/slices/store-test-helpers' -import { selectFloatingWorkspaceHasUnread } from '../store/selectors' import type { RetainedAgentEntry } from '../store/slices/agent-status' import { FLOATING_TERMINAL_WORKTREE_ID } from '../../../shared/constants' import { makePaneKey } from '../../../shared/stable-pane-id' @@ -465,86 +464,6 @@ describe('resolveAutoAckTabTargets', () => { }) }) -// Why: the minimized toggle's attention dot is the only signal a closed floating panel has, so a -// hidden panel must never auto-ack (selectFloatingWorkspaceHasUnread → FloatingTerminalToggleButton). -describe('floating workspace auto-ack against the attention dot', () => { - const FLOATING_TAB_ID = 'tab-floating' - const floatingPaneKey = makePaneKey(FLOATING_TAB_ID, CODEX_LEAF_ID) - - function seedFloatingCompletion(): ReturnType { - const store = createTestStore() - store.setState({ - activeView: 'terminal', - activeTabId: 'tab-1', - activeWorktreeId: 'wt-1', - activeTabIdByWorktree: { - 'wt-1': 'tab-1', - [FLOATING_TERMINAL_WORKTREE_ID]: FLOATING_TAB_ID - }, - tabsByWorktree: { - 'wt-1': [makeTab({ id: 'tab-1', worktreeId: 'wt-1' })], - [FLOATING_TERMINAL_WORKTREE_ID]: [ - makeTab({ id: FLOATING_TAB_ID, worktreeId: FLOATING_TERMINAL_WORKTREE_ID }) - ] - } - }) - store.getState().markAgentCompletionPaneUnread(floatingPaneKey, 'agent-completion') - return store - } - - function runAutoAckScan(store: TestStore, floatingPanelVisible: boolean): void { - const state = store.getState() - for (const target of resolveAutoAckTabTargets(state, { floatingPanelVisible })) { - const current = store.getState() - const surface = createTerminalAttentionSurface(current) - const viewedUnreadSubjectKey = resolveViewedUnreadSubjectKey( - current.unreadAgentCompletionPanes, - makePaneKey(target.tabId, CODEX_LEAF_ID) - ) - const clearedSubjectKeys = new Set(viewedUnreadSubjectKey ? [viewedUnreadSubjectKey] : []) - const workspaceId = target.worktreeId - applyAgentAttentionAcknowledgement( - { - acknowledgeSubjects: current.acknowledgeAgents, - clearWorkspaceUnread: current.clearWorktreeUnread, - clearGroupUnread: current.clearTerminalTabUnread, - clearSubjectUnread: current.clearTerminalPaneUnread - }, - { - workspaceIdToClear: - workspaceId !== null && - shouldClearWorkspaceAttention(surface.collectWorkspaceAttentionRemainder(workspaceId), { - viewedGroupId: target.tabId, - clearedSubjectKeys - }) - ? workspaceId - : null, - viewedGroupId: target.tabId, - subjectKeys: [], - viewedUnreadSubjectKey - } - ) - } - } - - it('keeps the attention dot lit while the panel is closed', () => { - const store = seedFloatingCompletion() - expect(selectFloatingWorkspaceHasUnread(store.getState())).toBe(true) - - runAutoAckScan(store, false) - - expect(selectFloatingWorkspaceHasUnread(store.getState())).toBe(true) - }) - - it('clears the attention dot once the panel is visible', () => { - const store = seedFloatingCompletion() - - runAutoAckScan(store, true) - - expect(selectFloatingWorkspaceHasUnread(store.getState())).toBe(false) - }) -}) - describe('computeLapsedManualUnreadProtections', () => { const paneKey = makePaneKey('tab-1', CODEX_LEAF_ID) const otherPaneKey = makePaneKey('tab-1', OTHER_LEAF_ID) diff --git a/src/renderer/src/hooks/useIssueMetadata.test.tsx b/src/renderer/src/hooks/useIssueMetadata.test.tsx index 97f6b3d6b6ca..bff14b1c0916 100644 --- a/src/renderer/src/hooks/useIssueMetadata.test.tsx +++ b/src/renderer/src/hooks/useIssueMetadata.test.tsx @@ -6,8 +6,6 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' import { clearLinearMetadataCache, useRepoLabels, - useTeamLabels, - useTeamMembers, useTeamStates, useTeamsStates } from './useIssueMetadata' @@ -167,50 +165,6 @@ describe('useIssueMetadata hooks', () => { expect(renders).toBeLessThanOrEqual(4) }) - it('does not re-issue a failed team-label fetch when a fresh settings object re-renders', async () => { - let renders = 0 - let error: string | null = null - linearMocks.linearTeamLabels.mockRejectedValue(new Error('Could not connect')) - - function LabelsProbe(): null { - renders += 1 - const metadata = useTeamLabels('team-1', { activeRuntimeEnvironmentId: null }, 'ws-1') - error = metadata.error - return null - } - - renderProbe() - await flushEffects() - await flushEffects() - await flushEffects() - - expect(error).toBe('Could not connect') - expect(linearMocks.linearTeamLabels).toHaveBeenCalledTimes(1) - expect(renders).toBeLessThanOrEqual(4) - }) - - it('does not re-issue a failed team-member fetch when a fresh settings object re-renders', async () => { - let renders = 0 - let error: string | null = null - linearMocks.linearTeamMembers.mockRejectedValue(new Error('Could not connect')) - - function MembersProbe(): null { - renders += 1 - const metadata = useTeamMembers('team-1', { activeRuntimeEnvironmentId: null }, 'ws-1') - error = metadata.error - return null - } - - renderProbe() - await flushEffects() - await flushEffects() - await flushEffects() - - expect(error).toBe('Could not connect') - expect(linearMocks.linearTeamMembers).toHaveBeenCalledTimes(1) - expect(renders).toBeLessThanOrEqual(4) - }) - it('unions workflow states across every selected team (#8739)', async () => { let states: { id: string; name: string }[] = [] linearMocks.linearTeamStates.mockImplementation(async (_settings, teamId: string) => { diff --git a/src/renderer/src/hooks/usePrimarySelectionPaste.test.tsx b/src/renderer/src/hooks/usePrimarySelectionPaste.test.tsx index 5724283ae1c2..5f13a9c0836f 100644 --- a/src/renderer/src/hooks/usePrimarySelectionPaste.test.tsx +++ b/src/renderer/src/hooks/usePrimarySelectionPaste.test.tsx @@ -274,21 +274,6 @@ describe('usePrimarySelectionPaste', () => { expect(consumeNativePasteMock).not.toHaveBeenCalled() }) - it('does not suppress native paste when the terminal has not armed the window', async () => { - setUserAgent('Mozilla/5.0 (X11; Linux x86_64)') - consumeNativePasteMock.mockReturnValue(false) - await renderProbe() - const textarea = appendTextarea() - - let nativeBeforeInput!: Event - await act(async () => { - nativeBeforeInput = dispatchNativePasteBeforeInput(textarea) - await flushPromises() - }) - - expect(nativeBeforeInput.defaultPrevented).toBe(false) - }) - it('does not keep middle-click ownership after the gesture window expires', async () => { vi.useFakeTimers() vi.setSystemTime(1_000) diff --git a/src/renderer/src/hooks/zoom-routing.test.ts b/src/renderer/src/hooks/zoom-routing.test.ts index 4d063f2d11c2..b0244ed63275 100644 --- a/src/renderer/src/hooks/zoom-routing.test.ts +++ b/src/renderer/src/hooks/zoom-routing.test.ts @@ -75,16 +75,6 @@ describe('resolveZoomTarget', () => { }) ).toBe('ui') }) - - it('routes to ui zoom for browser tabs without an active browser page', () => { - expect( - resolveZoomTarget({ - activeView: 'terminal', - activeTabType: 'browser', - activeElement: makeTarget({}) - }) - ).toBe('ui') - }) }) describe('registerZoomIpcBridge', () => { diff --git a/src/renderer/src/lib/agent-launch-caller-placement.test.ts b/src/renderer/src/lib/agent-launch-caller-placement.test.ts index 40dc46804b6b..03db8a683671 100644 --- a/src/renderer/src/lib/agent-launch-caller-placement.test.ts +++ b/src/renderer/src/lib/agent-launch-caller-placement.test.ts @@ -135,20 +135,6 @@ describe('agent launch caller placement and telemetry', () => { }) }) - it('falls back to the tab-bar quick launch source when a caller names none', async () => { - const { launchAgentInNewTab } = await import('./launch-agent-in-new-tab') - - launchAgentInNewTab({ agent: 'codex', worktreeId: 'wt-1' }) - - // Why: git-history-explain-commit is the one production call site that names no launch source, - // so it is reported as a tab-bar quick launch rather than as its own surface. - expect(queuedStartupPayload(store)?.telemetry).toEqual({ - agent_kind: 'kind:codex', - launch_source: 'tab_bar_quick_launch', - request_kind: 'new' - }) - }) - it('creates the tab before queueing its startup command', async () => { const { launchAgentInNewTab } = await import('./launch-agent-in-new-tab') diff --git a/src/renderer/src/lib/agent-paste-draft.test.ts b/src/renderer/src/lib/agent-paste-draft.test.ts index 1cd214cca954..ca5f39304175 100644 --- a/src/renderer/src/lib/agent-paste-draft.test.ts +++ b/src/renderer/src/lib/agent-paste-draft.test.ts @@ -822,14 +822,6 @@ describe('pasteDraftWhenAgentReady', () => { expect(replaceAllCallCount).toBe(0) }) - it('keeps agent draft chunk arrays aligned with lazy chunk iteration', () => { - const content = 'before\x1b[201~after😀' - - expect(chunkAgentDraftPasteContent(content, 6)).toEqual([ - ...iterateAgentDraftPasteContentChunks(content, 6) - ]) - }) - it('iterates large agent draft chunks lazily', () => { const text = 'x'.repeat(128) const codePointAt = vi.spyOn(String.prototype, 'codePointAt') diff --git a/src/renderer/src/lib/ai-vault-resume-target.test.ts b/src/renderer/src/lib/ai-vault-resume-target.test.ts index b4ac2902ee94..d9857050196d 100644 --- a/src/renderer/src/lib/ai-vault-resume-target.test.ts +++ b/src/renderer/src/lib/ai-vault-resume-target.test.ts @@ -292,25 +292,6 @@ describe('ai vault resume target ownership', () => { ).toBe('unknown') }) - it('blocks folder workspaces owned by runtime project groups', () => { - expect( - getAiVaultResumeWorkspaceTargetStatus( - makeState({ - folderWorkspaces: [ - { - id: 'folder-1', - projectGroupId: 'group-1', - name: 'Platform', - folderPath: '/repo/platform' - } - ], - projectGroups: [{ id: 'group-1', executionHostId: 'runtime:env-1' }] - }), - folderWorkspaceKey('folder-1') - ) - ).toBe('runtime') - }) - it('blocks mixed local and runtime folder workspace targets', () => { expect( getAiVaultResumeWorkspaceTargetStatus( diff --git a/src/renderer/src/lib/browser-cookie-import-toast.test.ts b/src/renderer/src/lib/browser-cookie-import-toast.test.ts index 0482400696c5..075528ad6d41 100644 --- a/src/renderer/src/lib/browser-cookie-import-toast.test.ts +++ b/src/renderer/src/lib/browser-cookie-import-toast.test.ts @@ -158,16 +158,6 @@ describe('emitBrowserCookieImportToast', () => { ) }) - it('does not infer a partition warning from generic skipped cookies', () => { - emitBrowserCookieImportToast( - { ...summary, importedCookies: 2, skippedCookies: 1 }, - 'Imported 2 cookies.', - localExecution - ) - - expect(warningToastMock).not.toHaveBeenCalled() - }) - it('does not infer a Google warning from generic skipped cookies', () => { emitBrowserCookieImportToast( { ...summary, importedCookies: 2, skippedCookies: 1 }, diff --git a/src/renderer/src/lib/focus-terminal-tab-surface.test.ts b/src/renderer/src/lib/focus-terminal-tab-surface.test.ts index 20cf491088e2..da28ad74eb3f 100644 --- a/src/renderer/src/lib/focus-terminal-tab-surface.test.ts +++ b/src/renderer/src/lib/focus-terminal-tab-surface.test.ts @@ -215,50 +215,6 @@ describe('focusTerminalTabSurface', () => { expect(textarea.focus).not.toHaveBeenCalled() }) - it('does not use stale-leaf fallback while the expected layout still has multiple leaves', () => { - flushAnimationFrames() - const textarea = { focus: vi.fn() } - vi.stubGlobal('document', { - querySelector: vi.fn((selector: string) => - selector === '[data-terminal-tab-id="tab-1"]' - ? { getAttribute: () => 'mounted-leaf pending-leaf' } - : null - ), - querySelectorAll: vi.fn((selector: string) => - selector === TAB_HELPER_SELECTOR - ? { length: 1, item: () => textarea } - : { length: 0, item: () => null } - ) - }) - - focusTerminalTabSurface('tab-1', 'stale-leaf') - - expect(textarea.focus).not.toHaveBeenCalled() - }) - - it('does not focus a sibling when a stale leaf id has multiple helpers in the tab', () => { - flushAnimationFrames() - const first = { focus: vi.fn() } - const second = { focus: vi.fn() } - vi.stubGlobal('document', { - querySelector: vi.fn((selector: string) => - selector === '[data-terminal-tab-id="tab-1"]' - ? { getAttribute: () => 'new-left new-right' } - : null - ), - querySelectorAll: vi.fn((selector: string) => - selector === TAB_HELPER_SELECTOR - ? { length: 2, item: (index: number) => (index === 0 ? first : second) } - : { length: 0, item: () => null } - ) - }) - - focusTerminalTabSurface('tab-1', 'stale-leaf') - - expect(first.focus).not.toHaveBeenCalled() - expect(second.focus).not.toHaveBeenCalled() - }) - it('cancels a pending focus frame when a newer focus request starts', () => { const cancelAnimationFrame = vi.fn() vi.stubGlobal( diff --git a/src/renderer/src/lib/github-work-item-source-lookup.test.ts b/src/renderer/src/lib/github-work-item-source-lookup.test.ts index 48e78ff96ddb..336583b1d6df 100644 --- a/src/renderer/src/lib/github-work-item-source-lookup.test.ts +++ b/src/renderer/src/lib/github-work-item-source-lookup.test.ts @@ -54,29 +54,6 @@ describe('GitHub source lookup routing', () => { expect(window.api.gh.workItemDetails).not.toHaveBeenCalled() }) - // Regression for #6429: a runtime-sourced details lookup must never reach the - // local Electron IPC, which rejects unregistered remote repos with - // "Access denied: unknown repository path". On main this routed through - // window.api.gh.workItemDetails and surfaced that error. - it('does not invoke the local IPC (which throws access-denied) for runtime sources', async () => { - vi.mocked(callRuntimeRpc).mockResolvedValue(null) - vi.mocked(window.api.gh.workItemDetails).mockRejectedValue( - new Error('Access denied: unknown repository path') - ) - - await expect( - lookupGitHubWorkItemDetailsForSource({ - repoPath: '/home/runtime/app', - repoId: 'renderer-repo', - sourceContext: runtimeSourceContext, - number: 42, - type: 'issue' - }) - ).resolves.toBeNull() - - expect(window.api.gh.workItemDetails).not.toHaveBeenCalled() - }) - it('uses the renderer repo id for runtime details when the source has no repo id', async () => { vi.mocked(callRuntimeRpc).mockResolvedValue(null) diff --git a/src/renderer/src/lib/keyboard-layout/detect-option-as-alt.test.ts b/src/renderer/src/lib/keyboard-layout/detect-option-as-alt.test.ts index cab6e2521326..cc6f97ece283 100644 --- a/src/renderer/src/lib/keyboard-layout/detect-option-as-alt.test.ts +++ b/src/renderer/src/lib/keyboard-layout/detect-option-as-alt.test.ts @@ -28,12 +28,6 @@ describe('detectOptionAsAltFromLayoutMap', () => { expect(detectOptionAsAltFromLayoutMap(mapOf(US))).toBe('us') }) - it('classifies US International as us (same fingerprint, dead keys only)', () => { - // US-International keeps all unshifted ASCII letters and punctuation. - // Only Option-layer composition differs — invisible to getLayoutMap(). - expect(detectOptionAsAltFromLayoutMap(mapOf(US))).toBe('us') - }) - it('classifies UK as non-us (Backquote → §)', () => { expect(detectOptionAsAltFromLayoutMap(mapOf({ ...US, Backquote: '§' }))).toBe('non-us') }) diff --git a/src/renderer/src/lib/keyboard-layout/option-as-alt-probe.test.ts b/src/renderer/src/lib/keyboard-layout/option-as-alt-probe.test.ts index ee522e956b69..3af50308e8cf 100644 --- a/src/renderer/src/lib/keyboard-layout/option-as-alt-probe.test.ts +++ b/src/renderer/src/lib/keyboard-layout/option-as-alt-probe.test.ts @@ -306,20 +306,6 @@ describe('createOptionAsAltProbe', () => { probe.dispose() }) - it('trusts the input source ID over the fingerprint even when the fingerprint says us', async () => { - // Pre-fix: the fingerprint's 'us' verdict was authoritative and the - // macOS ID was ignored, so Turkish-F (which reports US-identical on - // several keys) plus any US-like fingerprint flipped - // macOptionIsMeta=true. Now the ID overrides. - const win = makeMockWindow(US_MAP) - const probe = createOptionAsAltProbe(win as unknown as Window, { - readInputSourceId: async () => 'com.apple.keylayout.German' - }) - await probe.refresh() - expect(probe.getCurrent()).toBe('non-us') - probe.dispose() - }) - it('falls back to the fingerprint when the input-source reader returns null (non-Darwin)', async () => { const win = makeMockWindow(US_MAP) const probe = createOptionAsAltProbe(win as unknown as Window, { diff --git a/src/renderer/src/lib/launch-agent-structured-chat-guard.test.ts b/src/renderer/src/lib/launch-agent-structured-chat-guard.test.ts index 192236cbcbed..eb21a7e467fc 100644 --- a/src/renderer/src/lib/launch-agent-structured-chat-guard.test.ts +++ b/src/renderer/src/lib/launch-agent-structured-chat-guard.test.ts @@ -310,28 +310,6 @@ describe('structured chat adoption guard on the launch path', () => { expect(mockCreateTab).toHaveBeenCalled() }) - it('keeps a declined Claude launch on the structured path', async () => { - const { StructuredAgentSessionCreateRefusalError } = - await import('./launch-structured-agent-session') - const refusal = new StructuredAgentSessionCreateRefusalError( - 'structured_agent_session_unsupported' - ) - mockLaunchStructuredCodexSession.mockRejectedValueOnce(refusal) - const { launchAgentInNewTab } = await import('./launch-agent-in-new-tab') - - const result = launchAgentInNewTab({ agent: 'claude', worktreeId: 'wt-1' }) - - expect(result).toMatchObject({ - surface: { kind: 'local-agent-session', sessionId: 'codex-session-1' } - }) - await expect(result?.structuredSettlement).resolves.toEqual({ - kind: 'failed', - error: refusal - }) - expect(mockCreateTab).not.toHaveBeenCalled() - await vi.waitFor(() => expect(mockToastError).toHaveBeenCalledOnce()) - }) - it.each([[], null])( 'preserves terminal-backed launches with capability answer %s', async (capabilities) => { diff --git a/src/renderer/src/lib/lazy-with-retry.right-sidebar-syntax-error.test.ts b/src/renderer/src/lib/lazy-with-retry.right-sidebar-syntax-error.test.ts deleted file mode 100644 index f0ba64513a36..000000000000 --- a/src/renderer/src/lib/lazy-with-retry.right-sidebar-syntax-error.test.ts +++ /dev/null @@ -1,113 +0,0 @@ -// @vitest-environment happy-dom -import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' - -import { isLazyChunkLoadError, loadLazyWithRetry } from './lazy-with-retry' - -// Regression guard for crash report e08749bb-777c-446e-b407-5d1f154b6173 (Orca 1.4.104). -// boundary_id=right-sidebar, surface=right-sidebar, error_name=SyntaxError, -// error_message="Unexpected token ')'". component_stack: Lazy -> Suspense -> -// RightSidebarPanelContent -> ... -> RecoverableRenderErrorBoundary. -// -// The right-sidebar source-control panel is loaded via lazyWithRetry: -// src/renderer/src/components/right-sidebar/right-sidebar-panel-content.tsx:6 -// const SourceControl = lazy(() => import('./SourceControl')) // lazyWithRetry -// -// So the corrupt-chunk recovery IS wired in. The crash was a BLIND SPOT in that -// recovery: after the single guarded window.location.reload() has already landed -// once this session (the guard holds a *previous* document's identity), -// loadLazyWithRetry only converted the failure into a recoverable -// LazyChunkLoadError when isKnownDynamicImportFailure(error) was true. A corrupt / -// truncated chunk that parses as invalid JS rejects import() with a native -// SyntaxError whose .name is "SyntaxError" (not "ChunkLoadError") and whose -// message "Unexpected token ')'" matched NONE of the dynamic-import regexes, so -// it was re-thrown raw to the boundary and killed the right sidebar — the exact -// reported crash. The fix treats a parse-time SyntaxError as a recoverable -// corrupt-chunk failure; these tests pin that behavior. - -const RELOAD_GUARD_KEY = 'orca:lazy-chunk-reload-attempted' -const LANDED_RELOAD_GUARD_VALUE = 'doc-before-the-reload' - -// The exact error the renderer received from the corrupt right-sidebar chunk. -const reportedCrashError = (): SyntaxError => new SyntaxError("Unexpected token ')'") -// An equivalent transient fetch failure, for contrast — this one DOES recover. -const equivalentFetchError = (): TypeError => - new TypeError('Failed to fetch dynamically imported module: file://redacted/SourceControl.js') - -function spyOnReload(): ReturnType { - const reload = vi.fn() - vi.spyOn(window.location, 'reload').mockImplementation(reload) - return reload -} - -beforeEach(() => { - vi.useFakeTimers() - window.sessionStorage.clear() -}) - -afterEach(() => { - vi.unstubAllGlobals() - vi.restoreAllMocks() - vi.useRealTimers() - try { - window.sessionStorage.clear() - } catch { - // ignore - } -}) - -describe('right-sidebar lazy chunk SyntaxError crash (regression)', () => { - it('recovers a corrupt right-sidebar chunk SyntaxError instead of surfacing it to the boundary', async () => { - const reload = spyOnReload() - // The one guarded reload already landed earlier this session: the guard was - // written by the document that reloaded, not by this one. - window.sessionStorage.setItem(RELOAD_GUARD_KEY, LANDED_RELOAD_GUARD_VALUE) - - // import('./SourceControl') rejects with the native parse error from the - // corrupt chunk — exactly what the crash report captured. - const factory = vi.fn(() => Promise.reject(reportedCrashError())) - - const loaded = loadLazyWithRetry(factory, { retries: 2, reloadKey: 'right-sidebar' }) - // Drain the retry backoff timers first (fake timers), THEN await the result. - const settled = loaded.then( - () => null, - (error: unknown) => error - ) - await vi.advanceTimersByTimeAsync(5000) - const caught = await settled - - // A corrupt-chunk parse failure is unrecoverable-by-retry, so it must be - // wrapped as a recoverable LazyChunkLoadError, NOT re-thrown raw to the - // RecoverableRenderErrorBoundary where its only "Retry" re-runs the same dead - // import. With the fix, the parse-time SyntaxError is recovered. - expect(reload).not.toHaveBeenCalled() // guard already set: no second reload - expect(isLazyChunkLoadError(caught)).toBe(true) - }) - - it('treats the corrupt chunk SyntaxError the same as an equivalent fetch failure', async () => { - // Demonstrates the blind spot is purely error-shape gating: the SAME corrupt - // chunk, surfaced as a fetch failure, IS recovered; surfaced as a parse error, - // it is not. Both are the same unrecoverable corrupt right-sidebar chunk. - const recover = async (makeError: () => Error): Promise => { - window.sessionStorage.setItem(RELOAD_GUARD_KEY, LANDED_RELOAD_GUARD_VALUE) - const factory = vi.fn(() => Promise.reject(makeError())) - const loaded = loadLazyWithRetry(factory, { retries: 0, reloadKey: 'right-sidebar' }) - try { - await loaded - return null - } catch (error) { - return error - } finally { - await vi.advanceTimersByTimeAsync(5000) - window.sessionStorage.clear() - } - } - - spyOnReload() - const fetchOutcome = await recover(equivalentFetchError) - const parseOutcome = await recover(reportedCrashError) - - expect(isLazyChunkLoadError(fetchOutcome)).toBe(true) // recovered - // The parse error from the very same corrupt chunk must be recovered too. - expect(isLazyChunkLoadError(parseOutcome)).toBe(true) - }) -}) diff --git a/src/renderer/src/lib/native-chat-launch-draft-mirrorability.test.ts b/src/renderer/src/lib/native-chat-launch-draft-mirrorability.test.ts index 5cdc89ca9366..be8b09061706 100644 --- a/src/renderer/src/lib/native-chat-launch-draft-mirrorability.test.ts +++ b/src/renderer/src/lib/native-chat-launch-draft-mirrorability.test.ts @@ -62,13 +62,6 @@ describe('launch draft mirrorability', () => { vi.clearAllMocks() }) - // Why: the whole point of the shared predicate. If either side ever grows its - // own inline rule, a draft launch opens in chat with an empty composer beside - // a filled TUI input (or stays in the terminal with a mirror nobody sees). - it.each(DRAFT_TEXTS)('opens in chat exactly when it seeds: %j', (text) => { - expect(opensInChat(text)).toBe(seedsTheComposer(text)) - }) - it.each(DRAFT_TEXTS)('both sides follow the predicate: %j', (text) => { const expected = canMirrorLaunchDraftToNativeChat(text) expect(seedsTheComposer(text)).toBe(expected) diff --git a/src/renderer/src/lib/orchestration-skill-coverage.test.ts b/src/renderer/src/lib/orchestration-skill-coverage.test.ts index d0128396e4e7..8fe696caab1c 100644 --- a/src/renderer/src/lib/orchestration-skill-coverage.test.ts +++ b/src/renderer/src/lib/orchestration-skill-coverage.test.ts @@ -306,77 +306,6 @@ describe('orchestration skill agent coverage', () => { ).toBe(true) }) - it('marks a multi-segment provider-home agent from a Windows-style path', () => { - expect( - agentHasOrchestrationSkill( - 'opencode', - [ - skill({ - providers: ['agent-skills'], - sourceKind: 'home', - rootPath: 'C:\\Users\\test\\.config\\opencode\\skills', - directoryPath: 'C:\\Users\\test\\.config\\opencode\\skills\\orchestration' - }) - ], - [source('C:\\Users\\test\\.config\\opencode\\skills', 'opencode')] - ) - ).toBe(true) - }) - - it('keeps Pi and OMP distinct despite sharing the /agent/skills shape', () => { - const piInstall = [ - skill({ - providers: ['agent-skills'], - sourceKind: 'home', - rootPath: '/Users/test/.pi/agent/skills', - directoryPath: '/Users/test/.pi/agent/skills/orchestration' - }) - ] - const ompInstall = [ - skill({ - providers: ['agent-skills'], - sourceKind: 'home', - rootPath: '/Users/test/.omp/agent/skills', - directoryPath: '/Users/test/.omp/agent/skills/orchestration' - }) - ] - - const piSources = [source('/Users/test/.pi/agent/skills', 'pi')] - const ompSources = [source('/Users/test/.omp/agent/skills', 'omp')] - expect(agentHasOrchestrationSkill('pi', piInstall, piSources)).toBe(true) - expect(agentHasOrchestrationSkill('omp', piInstall, piSources)).toBe(false) - expect(agentHasOrchestrationSkill('omp', ompInstall, ompSources)).toBe(true) - expect(agentHasOrchestrationSkill('pi', ompInstall, ompSources)).toBe(false) - }) - - it('keeps Gemini and Antigravity distinct despite sharing the ~/.gemini root', () => { - const geminiInstall = [ - skill({ - providers: ['agent-skills'], - sourceKind: 'home', - rootPath: '/Users/test/.gemini/skills', - directoryPath: '/Users/test/.gemini/skills/orchestration' - }) - ] - const antigravityInstall = [ - skill({ - providers: ['agent-skills'], - sourceKind: 'home', - rootPath: '/Users/test/.gemini/antigravity/skills', - directoryPath: '/Users/test/.gemini/antigravity/skills/orchestration' - }) - ] - - const geminiSources = [source('/Users/test/.gemini/skills', 'gemini')] - const antigravitySources = [source('/Users/test/.gemini/antigravity/skills', 'antigravity')] - expect(agentHasOrchestrationSkill('gemini', geminiInstall, geminiSources)).toBe(true) - expect(agentHasOrchestrationSkill('antigravity', geminiInstall, geminiSources)).toBe(false) - expect(agentHasOrchestrationSkill('antigravity', antigravityInstall, antigravitySources)).toBe( - true - ) - expect(agentHasOrchestrationSkill('gemini', antigravityInstall, antigravitySources)).toBe(false) - }) - it('marks Claude Agent Teams from ~/.claude/skills like Claude Code', () => { const skills = [ skill({ @@ -393,21 +322,4 @@ describe('orchestration skill agent coverage', () => { ]) ).toBe(true) }) - - it('marks Windows skill paths', () => { - expect( - agentHasOrchestrationSkill( - 'codex', - [ - skill({ - providers: ['codex'], - sourceKind: 'home', - rootPath: 'C:\\Users\\test\\.codex\\skills', - directoryPath: 'C:\\Users\\test\\.codex\\skills\\orchestration' - }) - ], - [source('C:\\Users\\test\\.codex\\skills', 'codex')] - ) - ).toBe(true) - }) }) diff --git a/src/renderer/src/lib/skill-freshness-display-status.test.ts b/src/renderer/src/lib/skill-freshness-display-status.test.ts index 66c616a702ff..ff24aea1a83d 100644 --- a/src/renderer/src/lib/skill-freshness-display-status.test.ts +++ b/src/renderer/src/lib/skill-freshness-display-status.test.ts @@ -251,16 +251,6 @@ describe('hasSkillCopyNeedingAttention', () => { ) }) - // Regression guard for the centralization: the badge predicate deliberately omits the - // shared helper's outdated carve-out, so a non-eligible outdated copy stays amber. - // Collapsing the two predicates would flip this to green while the dialog still shows - // a reinstall row — the badge/dialog contradiction this change exists to avoid. - it('keeps a non-eligible outdated global copy amber', () => { - expect(getSkillFreshnessDisplayStatus(inventory([placement('outdated')]), SKILL_NAME)).toBe( - 'needs-attention' - ) - }) - // Why: an unreadable plugin path could hide a copy of anything, but a skill Orca // never found anywhere is not the one to blame for it — that reads as a problem // with a skill the user has not installed. diff --git a/src/renderer/src/lib/structured-agent-session-launch.test.ts b/src/renderer/src/lib/structured-agent-session-launch.test.ts index 458bb08e2e42..33bbc7375307 100644 --- a/src/renderer/src/lib/structured-agent-session-launch.test.ts +++ b/src/renderer/src/lib/structured-agent-session-launch.test.ts @@ -566,26 +566,6 @@ describe('startStructuredAgentLaunch', () => { expect(toast.error).toHaveBeenCalledOnce() }) - it('replays the same intent after an absent unknown outcome', async () => { - const worktreeId = 'wt-replay-unknown' - const intent = launchIntent(worktreeId) - mocks.createIntent.mockReturnValueOnce(intent) - mocks.launch - .mockRejectedValueOnce(new Error('offline')) - .mockResolvedValueOnce({ sessionId: intent.sessionId, fence: 1 }) - vi.mocked(refreshLocalStructuredSessionTabs) - .mockResolvedValueOnce([]) - .mockResolvedValueOnce([publishedSnapshot(worktreeId, intent.sessionId)]) - - startStructuredAgentLaunch(worktreeId, 'codex') - await flushLaunchSettlement() - - expect(mocks.createIntent).toHaveBeenCalledOnce() - expect(mocks.launch).toHaveBeenCalledTimes(2) - expect(mocks.launch.mock.calls[1]?.[0]).toBe(intent) - expect(toast.error).not.toHaveBeenCalled() - }) - it('reuses the queued prompt without a second delivery after unknown recovery', async () => { const worktreeId = 'wt-unknown-prompt-retry' const intent = launchIntent(worktreeId) diff --git a/src/renderer/src/lib/tui-agent-startup.test.ts b/src/renderer/src/lib/tui-agent-startup.test.ts index b4551ae8a239..7ce665544b32 100644 --- a/src/renderer/src/lib/tui-agent-startup.test.ts +++ b/src/renderer/src/lib/tui-agent-startup.test.ts @@ -310,23 +310,6 @@ describe('buildAgentStartupPlan', () => { }) ).toBeNull() }) - - it('uses -i flag for copilot to start an interactive session with initial prompt', () => { - expect( - buildAgentStartupPlan({ - agent: 'copilot', - prompt: 'Fix the bug', - cmdOverrides: {}, - platform: 'darwin' - }) - ).toEqual({ - agent: 'copilot', - launchCommand: "copilot -i 'Fix the bug'", - expectedProcess: 'copilot', - followupPrompt: null, - launchConfig: emptyLaunchConfig('copilot') - }) - }) }) describe('buildAgentDraftLaunchPlan', () => { diff --git a/src/renderer/src/lib/window-label-formatter.test.ts b/src/renderer/src/lib/window-label-formatter.test.ts index 0390f7e84a0f..73954ebf505c 100644 --- a/src/renderer/src/lib/window-label-formatter.test.ts +++ b/src/renderer/src/lib/window-label-formatter.test.ts @@ -18,10 +18,6 @@ describe('formatWindowLabel', () => { expect(formatWindowLabel(10080)).toBe('wk') }) - it('returns "1d" for 1440 minutes (1 day)', () => { - expect(formatWindowLabel(1440)).toBe('1d') - }) - it('returns "2h" for 120 minutes', () => { expect(formatWindowLabel(120)).toBe('2h') }) @@ -34,10 +30,6 @@ describe('formatWindowLabel', () => { expect(formatWindowLabel(20160)).toBe('2wk') }) - it('returns "30m" for 30 minutes', () => { - expect(formatWindowLabel(30)).toBe('30m') - }) - it('returns "3d" for 4320 minutes (3 days)', () => { expect(formatWindowLabel(4320)).toBe('3d') }) @@ -51,13 +43,6 @@ describe('formatWindowLabel', () => { expect(formatWindowLabel(295)).toBe('295m') expect(formatWindowLabel(300)).toBe('5h') }) - - it('falls back to per-minute labels outside canonical buckets', () => { - // Why: when the window length lands between buckets (e.g. 2h30m), we - // render the raw minute count rather than guess at a half-bucket label. - expect(formatWindowLabel(75)).toBe('75m') - expect(formatWindowLabel(150)).toBe('150m') - }) }) describe('formatRateLimitWindowChipLabel', () => { diff --git a/src/renderer/src/lib/workspace-port-host-availability.test.ts b/src/renderer/src/lib/workspace-port-host-availability.test.ts index 7652c65a21a4..6ef58279b29b 100644 --- a/src/renderer/src/lib/workspace-port-host-availability.test.ts +++ b/src/renderer/src/lib/workspace-port-host-availability.test.ts @@ -42,24 +42,6 @@ describe('getUnavailableWorkspacePortHosts', () => { ]) }) - it('keeps colons inside an environment id when parsing the scan key', () => { - // Why: keys are `${targetKey}:all`, so the id runs to the last `:all` — - // splitting on the first colon would truncate ids that contain colons. - expect( - getUnavailableWorkspacePortHosts({ - 'local:all': scan(), - 'environment:weird:id:all': scan({ unavailableReason: 'Remote connection dropped' }) - }) - ).toEqual([ - { - scanKey: 'environment:weird:id:all', - host: { kind: 'environment', environmentId: 'weird:id' }, - platform: 'linux', - reason: 'Remote connection dropped' - } - ]) - }) - // Why: total loss of contact is where naming the host matters most — the merged // projection joins raw internal scan keys, so it cannot name them itself. it('names every host when all of them failed', () => { @@ -84,38 +66,6 @@ describe('getUnavailableWorkspacePortHosts', () => { ]) }) - it('names a single failed host', () => { - expect( - getUnavailableWorkspacePortHosts({ - 'local:all': scan({ unavailableReason: 'lsof is unavailable' }) - }) - ).toEqual([ - { - scanKey: 'local:all', - host: { kind: 'local' }, - platform: 'linux', - reason: 'lsof is unavailable' - } - ]) - }) - - // Why: the synthetic all-hosts projection key must never be labelled as the - // local machine — that would blame the wrong host for a remote failure. - it('marks an unrecognised scan key as an unknown host', () => { - expect( - getUnavailableWorkspacePortHosts({ - 'all-hosts:all': scan({ unavailableReason: 'Remote connection dropped' }) - }) - ).toEqual([ - { - scanKey: 'all-hosts:all', - host: { kind: 'unknown' }, - platform: 'linux', - reason: 'Remote connection dropped' - } - ]) - }) - // Why: a paired web client's userAgent is not the Orca host's platform, so the // caller labels the local host from the scan's own platform. it("carries the failed scan's platform, and null when it is unknown", () => { diff --git a/src/renderer/src/lib/worktree-activity-state.test.ts b/src/renderer/src/lib/worktree-activity-state.test.ts index 777eeb07595b..279436fa6f89 100644 --- a/src/renderer/src/lib/worktree-activity-state.test.ts +++ b/src/renderer/src/lib/worktree-activity-state.test.ts @@ -27,12 +27,6 @@ function makeAgentEntry( } describe('worktree activity state', () => { - it('treats a slept wake-hint workspace as inactive', () => { - expect( - isInactiveWorkspace('wt-1', { 'wt-1': [makeTab('tab-1')] }, { 'tab-1': [] }, {}, new Set()) - ).toBe(true) - }) - it('treats a never-opened workspace as inactive', () => { expect(isInactiveWorkspace('wt-1', {}, {}, {}, new Set())).toBe(true) }) diff --git a/src/renderer/src/lib/worktree-draft-startup-view-mode.test.ts b/src/renderer/src/lib/worktree-draft-startup-view-mode.test.ts index 360e414ff137..71d895c80afb 100644 --- a/src/renderer/src/lib/worktree-draft-startup-view-mode.test.ts +++ b/src/renderer/src/lib/worktree-draft-startup-view-mode.test.ts @@ -59,11 +59,4 @@ describe('resolveBackendDraftStartup', () => { setRepoConnection('runtime-ssh-env-1') expect(viewModeFor('omp')).toBe('chat') }) - - it('preserves the same split for Grok', () => { - setRepoConnection(null) - expect(viewModeFor('grok')).toBe('chat') - setRepoConnection('ssh-target-1') - expect(viewModeFor('grok')).toBe('terminal') - }) }) From d74388f8a2dad2bd4bbfe3b937aba66e6648258b Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 06:47:16 -0700 Subject: [PATCH 08/79] test: retire cases subsumed by an honestly-named neighbour (#24150) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * test: retire two agent-status ipc cases whose fixture arms reach identical code Follow-on from the wave-30 sweep: an auditor continued into its own disclosed unread list and read the `useIpcEvents-agent-status-*` family. 2 case declarations removed across 2 files, 152 lines gone. No production code touched. - `useIpcEvents-agent-status-queue-ordering.test.ts` — `applies ready push events for an unmounted inactive terminal tab`. Character-identical to `applies ready push events for inactive terminal tabs with empty layout snapshots` in `useIpcEvents-agent-status-hook-titles.test.ts` — same event, same prompt strings, same six-argument `setAgentStatus` assertion — except for one fixture field: `terminalLayoutsByTabId: {}` versus `{'tab-future': {root: null, ...}}`. The deciding line is `agent-status-pane-routing-index.ts:209`, `if (layout?.root)`. Both `undefined?.root` and `{root: null}.root` are falsy, so both arms skip the leaf-membership branch and every downstream line is identical. The `hook-titles` copy is kept because it has a genuine local contrast arm — `buffers ready push events until a mounted tab contains the pane leaf`, with `root: {leafId: STALE_LEAF_ID}` producing `exists: false`, buffering and `agent_hook_unattributed` telemetry. The queue-ordering copy had no such pairing. - `useIpcEvents-agent-status-pane-teardown.test.ts` — `does not retain a Cursor spinner terminal title when the hook reports done`. Production IS mirrored here (`SYNTHETIC_AGENT_TITLE_PROFILES` has 9 entries), so the mirrored-production check applied: what differs between the codex and cursor rows is `synthesizeWorkingTitle: false`, and neither case asserts it. Both exercise only `done -> idleLabel` through one shared path in `resolveAgentStatusTerminalTitle`. The unique contribution is owned verbatim by `src/renderer/src/lib/agent-status-terminal-title.test.ts`, same glyph and same string, and the Codex survivor asserts a strict superset — it additionally pins `updateTabTitle('tab-future', 'Codex ready')` called exactly once. Reported, not fixed: `ipc-events-agent-status-store-test-fixtures.ts:84` hand-copies the monotonicity rule as `updatedAt < existing.updatedAt`, while production's decision lives in `agent-status-live-entry-builder.ts` and additionally weighs retirement fences, recently-closed tabs, pane-authority aliases and per-connection watermarks. Rather than reason about the risk the auditor instrumented the branch and ran all eight files: never taken, 56 of 56 cases. So it is dead test-support code today rather than a stale premise — and a trap for the first staleness case anyone writes in these files, which would pass against the simplified copy. Left unmodified because the module is shared with three specs outside the audited chunk. Verified: 172 test files / 1,084 cases pass in `src/renderer/src/hooks`; `check-reliability-gates.mjs` 140 gates; neither deleted title appears in the gate manifest; both named owners confirmed present. * test: retire two runtime cases subsumed by an honestly-named neighbour Final chunks of the 2,269-file backlog, `src/renderer/src/runtime`. 2 case declarations removed across 2 files, 30 lines gone. No production code touched. Both deletions share a shape: the surviving neighbour is not merely equivalent, it is the case whose NAME tells the truth about what the code does. - `paired-reconnect-sidebar-agent-count.test.ts` — `the erased rows are exactly the panes whose status only the client wrote`. Two defects at once. It computes `missing` by filtering the same `reconnected.rowPaneKeys` that the immediately preceding case already asserts equals the full pane set, so `expect(missing).toEqual([])` cannot fail where that case passes. And its title promises a characterisation its own assertion denies: it claims the erased rows ARE the client-only panes, while asserting the erased set is empty. Its comment describes a "causal boundary" that `toEqual([])` never establishes. Owner: `keeps a sidebar row for every still-live pane after a long sleep`, immediately above. - `runtime-terminal-inspection.test.ts` — `can record a runtime input marker from a PTY id mapping`. Its fixture sets `experimentalAgentHibernation: true`, and that flag appears ZERO times in `runtime-terminal-inspection.ts` — `recordRuntimeTerminalInputForPtyId` resolves a pane key and records, reading no setting. The surviving case sets the same flag to `false` and asserts the identical `toBe(123)` from the identical call, so the two differ only in an inert field. Owner: `records runtime input markers even before hibernation is enabled`, whose name documents the flag-independence the pair accidentally demonstrates. Verified: 209 test files / 1,703 cases pass in `src/renderer/src/runtime`; `check-reliability-gates.mjs` 140 gates; neither deleted title appears in the gate manifest. * test: retire cases whose varied field production returns from one branch Final chunk of the backlog sweep, `renderer/runtime` tab-sync. 4 case declarations removed across 2 files, 57 lines gone. No production code touched. - `web-session-tabs-sync-agent-handoff.test.ts` — `keeps stale local agent tabs when the host mirror is for a different agent`. `shouldReplaceTerminalTab` has no agent-kind branch, and `launchAgent` appears ZERO times in `terminal-surfaces.ts` — its own comment says "agent kind is not session identity". Both this and its sibling enter the same `exactProvisionalHandoffs`-empty branch, and the sibling (same-agent) is the one that would go red if agent-kind matching were reintroduced. This one cannot. - `web-session-tabs-sync-client-owned-page-content.test.ts` — three cases asserting `loading`, `canGoBack` and `canGoForward` individually. `resolveMirroredBrowserPageContent` returns all five fields from a single `if (clientHostsMirroredBrowserPage(tab) && existingPage)` branch, so no production path preserves one and drops another. Six per-field cases ran off a byte-identical fixture. Kept the `title` case (the reported bug) and the `url` case (production documents url as independently load-bearing for the next snapshot's url-equality arm), plus two cases the per-field reads cannot cover: one asserting different state slices, one asserting patch-key omission. Reported, not fixed: `web-runtime-session-tab-activate-close.test.ts` asserts a stale-terminal refusal is indistinguishable from a real close, while its own comment says "Any fix must make these two outcomes distinguishable". As a plain `it` it will go red when the bug is fixed and read as a regression; `it.fails` is the honest shape. Second instance of that pattern in this audit. Verified: both files green (24 cases); `check-reliability-gates.mjs` 140 gates; neither deleted title in the gate manifest. --- ...cEvents-agent-status-pane-teardown.test.ts | 83 ------------------- ...Events-agent-status-queue-ordering.test.ts | 69 --------------- ...ired-reconnect-sidebar-agent-count.test.ts | 12 --- .../runtime-terminal-inspection.test.ts | 18 ---- ...eb-session-tabs-sync-agent-handoff.test.ts | 39 --------- ...abs-sync-client-owned-page-content.test.ts | 18 ---- 6 files changed, 239 deletions(-) diff --git a/src/renderer/src/hooks/useIpcEvents-agent-status-pane-teardown.test.ts b/src/renderer/src/hooks/useIpcEvents-agent-status-pane-teardown.test.ts index f54ac9ade4b1..b043abb40428 100644 --- a/src/renderer/src/hooks/useIpcEvents-agent-status-pane-teardown.test.ts +++ b/src/renderer/src/hooks/useIpcEvents-agent-status-pane-teardown.test.ts @@ -567,87 +567,4 @@ describe('useIpcEvents agent status snapshot integration', () => { expect(removeAgentStatus).not.toHaveBeenCalled() }) - - it('does not retain a Cursor spinner terminal title when the hook reports done', async () => { - const setAgentStatus = vi.fn() - const onSetListenerRef: { current: ((data: AgentStatusSetData) => void) | null } = { - current: null - } - - const storeState: StoreLike = buildStoreState({ - setAgentStatus, - workspaceSessionReady: true, - settings: { terminalFontSize: 13, notifications: { enabled: false } }, - tabsByWorktree: { - 'wt-1': [ - { - id: 'tab-future', - ptyId: 'pty-1', - worktreeId: 'wt-1', - title: '\u2839 Cursor Agent' - } - ] - }, - terminalLayoutsByTabId: { - 'tab-future': { - root: { type: 'leaf', leafId: FUTURE_LEAF_ID }, - activeLeafId: FUTURE_LEAF_ID, - expandedLeafId: null, - titlesByLeafId: { [FUTURE_LEAF_ID]: '\u2839 Cursor Agent' } - } - } - }) - - stubReactSyncEffect() - vi.doMock('../store', () => ({ - useAppStore: { - subscribe: vi.fn(() => () => {}), - getState: () => storeState - } - })) - stubAuxiliaryModules() - vi.stubGlobal( - 'window', - buildWindowApi({ - onSet: (cb) => { - onSetListenerRef.current = cb - return () => {} - } - }) - ) - - const { useIpcEvents } = await import('./useIpcEvents') - - useIpcEvents() - await Promise.resolve() - - if (typeof onSetListenerRef.current !== 'function') { - throw new Error('Expected agentStatus.onSet listener to be registered') - } - - onSetListenerRef.current({ - paneKey: FUTURE_PANE_KEY, - state: 'done', - prompt: 'cursor prompt', - agentType: 'cursor', - lastAssistantMessage: 'cursor completion', - receivedAt: 1_700_000_000_200, - stateStartedAt: 1_699_999_999_100 - }) - - expect(setAgentStatus).toHaveBeenCalledTimes(1) - expect(setAgentStatus).toHaveBeenCalledWith( - FUTURE_PANE_KEY, - expect.objectContaining({ - state: 'done', - prompt: 'cursor prompt', - agentType: 'cursor', - lastAssistantMessage: 'cursor completion' - }), - 'Cursor ready', - { updatedAt: 1_700_000_000_200, stateStartedAt: 1_699_999_999_100 }, - expectWorktreeRouting('wt-1'), - undefined - ) - }) }) diff --git a/src/renderer/src/hooks/useIpcEvents-agent-status-queue-ordering.test.ts b/src/renderer/src/hooks/useIpcEvents-agent-status-queue-ordering.test.ts index 536a8d88fc2d..88f199bb3a4e 100644 --- a/src/renderer/src/hooks/useIpcEvents-agent-status-queue-ordering.test.ts +++ b/src/renderer/src/hooks/useIpcEvents-agent-status-queue-ordering.test.ts @@ -350,75 +350,6 @@ describe('useIpcEvents agent status snapshot integration', () => { expect(replayedAfterThrow).toContain('buffered') }) - it('applies ready push events for an unmounted inactive terminal tab', async () => { - const setAgentStatus = vi.fn() - const onSetListenerRef: { current: ((data: AgentStatusSetData) => void) | null } = { - current: null - } - - const storeState: StoreLike = buildStoreState({ - setAgentStatus, - workspaceSessionReady: true, - settings: { terminalFontSize: 13, notifications: { enabled: false } }, - tabsByWorktree: { - 'wt-1': [{ id: 'tab-future', ptyId: 'pty-1', worktreeId: 'wt-1', title: 'Inactive Tab' }] - }, - terminalLayoutsByTabId: {} - }) - - stubReactSyncEffect() - vi.doMock('../store', () => ({ - useAppStore: { - subscribe: vi.fn(() => () => {}), - getState: () => storeState - } - })) - stubAuxiliaryModules() - vi.stubGlobal( - 'window', - buildWindowApi({ - onSet: (cb) => { - onSetListenerRef.current = cb - return () => {} - } - }) - ) - - const { useIpcEvents } = await import('./useIpcEvents') - - useIpcEvents() - await Promise.resolve() - - if (typeof onSetListenerRef.current !== 'function') { - throw new Error('Expected agentStatus.onSet listener to be registered') - } - - onSetListenerRef.current({ - paneKey: FUTURE_PANE_KEY, - state: 'done', - prompt: 'inactive prompt', - agentType: 'codex', - lastAssistantMessage: 'inactive completion', - receivedAt: 1_700_000_000_200, - stateStartedAt: 1_699_999_999_100 - }) - - expect(setAgentStatus).toHaveBeenCalledTimes(1) - expect(setAgentStatus).toHaveBeenCalledWith( - FUTURE_PANE_KEY, - expect.objectContaining({ - state: 'done', - prompt: 'inactive prompt', - agentType: 'codex', - lastAssistantMessage: 'inactive completion' - }), - 'Inactive Tab', - { updatedAt: 1_700_000_000_200, stateStartedAt: 1_699_999_999_100 }, - expectWorktreeRouting('wt-1'), - undefined - ) - }) - it('keeps a Codex permission attention row actionable', async () => { const setAgentStatus = vi.fn() const updateTabTitle = vi.fn() diff --git a/src/renderer/src/runtime/paired-reconnect-sidebar-agent-count.test.ts b/src/renderer/src/runtime/paired-reconnect-sidebar-agent-count.test.ts index a5f77f49b98f..d6cf310c5364 100644 --- a/src/renderer/src/runtime/paired-reconnect-sidebar-agent-count.test.ts +++ b/src/renderer/src/runtime/paired-reconnect-sidebar-agent-count.test.ts @@ -311,18 +311,6 @@ describe('STA-3107: sidebar agent rows survive a paired-client sleep/wake reconn expect(reconnected.rowPaneKeys.sort(), evidence).toEqual(PANES.map(mirrorPaneKey).sort()) }) - it('the erased rows are exactly the panes whose status only the client wrote', () => { - const store = seedPairedClientStore() - const reconnected = runSleepWakeReconnect(store, LONG_SLEEP_MS) - const missing = PANES.map(mirrorPaneKey).filter( - (paneKey) => !reconnected.rowPaneKeys.includes(paneKey) - ) - - // Pins the causal boundary: host-authoritative panes are republished with a - // fresh host timestamp and are never at risk; only client-owned panes are. - expect(missing).toEqual([]) - }) - it('still cedes a pane this renderer never wrote status for', () => { const store = seedPairedClientStore() applyHostSnapshot(store, makeHostSnapshot({ snapshotVersion: 1, hostNow: T0 - 1_000 }), T0) diff --git a/src/renderer/src/runtime/runtime-terminal-inspection.test.ts b/src/renderer/src/runtime/runtime-terminal-inspection.test.ts index ded589e26b18..d1d4dcc9fed8 100644 --- a/src/renderer/src/runtime/runtime-terminal-inspection.test.ts +++ b/src/renderer/src/runtime/runtime-terminal-inspection.test.ts @@ -688,24 +688,6 @@ describe('runtime terminal owner routing', () => { expect(useAppStore.getState().lastTerminalInputAtByPaneKey[PANE_KEY]).toBeUndefined() }) - it('can record a runtime input marker from a PTY id mapping', () => { - useAppStore.setState({ - settings: { experimentalAgentHibernation: true } as never, - terminalLayoutsByTabId: { - 'tab-1': { - root: { type: 'leaf', leafId: LEAF_ID }, - activeLeafId: LEAF_ID, - expandedLeafId: null, - ptyIdsByLeafId: { [LEAF_ID]: 'local-pty' } - } - } - }) - - recordRuntimeTerminalInputForPtyId('local-pty', 123) - - expect(useAppStore.getState().lastTerminalInputAtByPaneKey[PANE_KEY]).toBe(123) - }) - it('indexes a stable layout identity once across repeated terminal input', () => { const layoutCount = 500 let layoutEnumerations = 0 diff --git a/src/renderer/src/runtime/web-session-tabs-sync-agent-handoff.test.ts b/src/renderer/src/runtime/web-session-tabs-sync-agent-handoff.test.ts index 054aa4bc60ea..769e10ade528 100644 --- a/src/renderer/src/runtime/web-session-tabs-sync-agent-handoff.test.ts +++ b/src/renderer/src/runtime/web-session-tabs-sync-agent-handoff.test.ts @@ -303,45 +303,6 @@ describe('applyWebSessionTabsSnapshot', () => { }) }) - it('keeps stale local agent tabs when the host mirror is for a different agent', () => { - const staleLocalClaudeTab: TerminalTab = { - id: 'local-claude-tab', - ptyId: null, - worktreeId: WT, - title: 'Claude', - defaultTitle: 'Claude', - customTitle: null, - color: null, - sortOrder: 0, - createdAt: NOW, - launchAgent: 'claude' - } - - const patch = applyWebSessionTabsSnapshot( - makeState({ - tabsByWorktree: { [WT]: [staleLocalClaudeTab] } - }), - makeSnapshot([ - { - type: 'terminal', - id: HOST_SURFACE_ID, - title: 'Codex', - parentTabId: 'host-tab-1', - leafId: LEAF_ID, - isActive: true, - launchAgent: 'codex', - status: 'ready', - terminal: 'terminal-1' - } - ]), - ENV, - NOW - ) as Partial - - expect(patch.tabsByWorktree?.[WT]).toHaveLength(2) - expect(patch.tabsByWorktree?.[WT]?.some((tab) => tab.id === 'local-claude-tab')).toBe(true) - }) - it('resolves a canonical agent tab before its confirming snapshot arrives', () => { recordWebAgentSessionHandoff({ environmentId: ENV, diff --git a/src/renderer/src/runtime/web-session-tabs-sync-client-owned-page-content.test.ts b/src/renderer/src/runtime/web-session-tabs-sync-client-owned-page-content.test.ts index 0afebc83522d..cb867b7430b1 100644 --- a/src/renderer/src/runtime/web-session-tabs-sync-client-owned-page-content.test.ts +++ b/src/renderer/src/runtime/web-session-tabs-sync-client-owned-page-content.test.ts @@ -258,24 +258,6 @@ describe('browser rows this client hosts own their page content', () => { expect(syncedPage(applyStaleSnapshot(state), state)?.url).toBe(GUEST_URL) }) - it('keeps the local loading flag instead of the host create-time value', () => { - const state = stateWithLocalRow() - - expect(syncedPage(applyStaleSnapshot(state), state)?.loading).toBe(false) - }) - - it('keeps local canGoBack instead of the host default', () => { - const state = stateWithLocalRow() - - expect(syncedPage(applyStaleSnapshot(state), state)?.canGoBack).toBe(true) - }) - - it('keeps local canGoForward instead of the host default', () => { - const state = stateWithLocalRow() - - expect(syncedPage(applyStaleSnapshot(state), state)?.canGoForward).toBe(true) - }) - // Why a real title is covered separately: a host that has learned the title publishes a // non-fallback string, which the staged-title hold would have accepted. Ownership, not staleness. it('keeps the local title even when the host publishes a real but older title', () => { From e03870403ebb844d0a1056bc474681d0c5245167 Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Wed, 30 Sep 2026 13:01:53 -0400 Subject: [PATCH 09/79] fix(orchestration): accept worker reports without the dispatch capability (#23982) * fix(orchestration): authorize worker reports without the dispatch capability Worker lifecycle reports and questions no longer depend on the per-dispatch capability token that lives only in the agent's conversation. The host now: - ignores capability_hash/capability_revoked_at for authorization on every row and checks the exact worker process instead (ask gains that check); - refuses a report whose calling terminal is provably another orchestration party (a Run coordinator or another Dispatch's worker), treating env that names no live pane here as absent; - applies one worker-state rule locally and remotely: a stop in flight refuses, while stop_unknown and start_unknown accept and settle. Minting and printing the flag are unchanged, so an older host and older preambles keep working. * fix(orchestration): name the fenced party without implying which Dispatch it owns * refactor(orchestration): one worker report rule, fence only a different party - One module owns the unproven/settleable worker states and the refusal rule; local send records it, ask and remote throw it. A stale process is worker_identity_changed on every path. - The caller fence passes the worker's own terminal when its --from handle went stale. - Document the shared-tmux-server limit; drop the dead dispatch_capability_invalid rejection member; tests assert dispatch state, not the capability column. * test(orchestration): drop capability-era assertions other tests already cover * test(orchestration): cover a current process whose terminal moved to another pane --- .../references/worker-contract.md | 2 + src/cli/bundled-skill-guides.ts | 4 +- ...dinator-dispatch-unobserved-prompt.test.ts | 10 +- .../db-task-dispatch-invariant.test.ts | 20 +- .../db-task-dispatch-lifecycle-guards.test.ts | 25 +- .../db-task-dispatch-races.test.ts | 10 +- .../dispatch-capability.test.ts | 60 ----- .../dispatch-context/dispatch-capability.ts | 54 +--- .../worker-report-settlement.ts | 29 +- .../db/federation/federation-relay-enqueue.ts | 3 +- .../db/federation/federation-relay-item.ts | 7 +- .../remote-dispatch-attachment-authority.ts | 30 +-- .../orchestration/db/lifecycle-transition.ts | 2 +- .../lifecycle-caller-edges.test.ts | 2 + .../lifecycle-reconciliation.test.ts | 10 +- .../orchestration/lifecycle-reconciliation.ts | 1 - .../orchestration/worker-report-admission.ts | 55 ++++ ...start-unobserved-prompt-settlement.test.ts | 23 +- .../orchestration/messaging/ask-methods.ts | 48 ++-- .../orchestration/messaging/ask-remote.ts | 19 +- .../orchestration/messaging/ask.test.ts | 42 ++- .../messaging/lifecycle-caller-fence.ts | 64 +++++ .../orchestration/messaging/send-methods.ts | 20 +- .../messaging/send-point-to-point.ts | 103 +++---- .../orchestration/messaging/send-remote.ts | 19 +- .../orchestration/messaging/send.test.ts | 118 +++----- .../messaging/worker-report-authority.test.ts | 253 ++++++++++++++++++ ...stration-runtime-update-settlement.test.ts | 12 +- ...hestration-task-dispatch-invariant.test.ts | 20 +- .../runtime/structured-worker-identity.ts | 2 +- 30 files changed, 584 insertions(+), 483 deletions(-) delete mode 100644 src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.test.ts create mode 100644 src/main/runtime/orchestration/worker-report-admission.ts create mode 100644 src/main/runtime/rpc/methods/orchestration/messaging/lifecycle-caller-fence.ts create mode 100644 src/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.ts diff --git a/skill-guides/orchestration/references/worker-contract.md b/skill-guides/orchestration/references/worker-contract.md index 6e35da7b8f9d..83a28d02ad25 100644 --- a/skill-guides/orchestration/references/worker-contract.md +++ b/skill-guides/orchestration/references/worker-contract.md @@ -49,6 +49,8 @@ If `check` returns `consumer_fenced`, this process no longer owns its Dispatch: the Attempt was re-attached to another worker or settled without you. Stop, do not send `worker_done`, and do not retry the check. An empty `check` never means you were replaced; `consumer_fenced` is the only way you learn that. +If `send` or `ask` returns `consumer_fenced`, the command ran from another +party's terminal (a coordinator or another worker); run it from your own terminal. ## Escalation diff --git a/src/cli/bundled-skill-guides.ts b/src/cli/bundled-skill-guides.ts index 0985a2239ab9..0ea0e2bf4c44 100644 --- a/src/cli/bundled-skill-guides.ts +++ b/src/cli/bundled-skill-guides.ts @@ -69,7 +69,7 @@ const ORCA_PER_WORKSPACE_ENV_WINDOWS_SCRIPTS_REFERENCE_MARKDOWN = "# Windows loc const ORCHESTRATION_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID\n in the live preamble. Never reconstruct, translate, or broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n" // oxfmt-ignore -const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID\n in the live preamble. Never reconstruct, translate, or broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\ncapability arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" +const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID\n in the live preamble. Never reconstruct, translate, or broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\ncapability arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" // oxfmt-ignore const ORCHESTRATION_COORDINATOR_LOOP_REFERENCE_MARKDOWN = "# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n" @@ -90,7 +90,7 @@ const ORCHESTRATION_PLACEMENT_AND_REMOTE_REFERENCE_MARKDOWN = "# Placement and r const ORCHESTRATION_RECOVERY_AND_CLEANUP_REFERENCE_MARKDOWN = "# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n" // oxfmt-ignore -const ORCHESTRATION_WORKER_CONTRACT_REFERENCE_MARKDOWN = "# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" +const ORCHESTRATION_WORKER_CONTRACT_REFERENCE_MARKDOWN = "# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" // oxfmt-ignore export const BUNDLED_SKILL_GUIDES = [ diff --git a/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts b/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts index f09608039880..15270b692a61 100644 --- a/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts +++ b/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts @@ -91,20 +91,12 @@ describe('coordinator dispatch with an unobserved prompt', () => { const task = db.createTask({ runId: 'run_legacy_local', spec: 'do the work' }) await dispatch(createRuntime(new Error('agent_prompt_stalled')), task.id, []) const dispatchId = db.getDispatchContext(task.id)!.id - const minted = db.mintDispatchCapability({ + db.mintDispatchCapability({ dispatchId, paneKey: WORKER_PANE_KEY, processIncarnation: 'incarnation-1' }) - expect( - db.verifyDispatchCapability({ - dispatchId, - capability: minted, - paneKey: WORKER_PANE_KEY, - processIncarnation: 'incarnation-1' - }) - ).toEqual({ valid: true }) expect( db.settleWorkerReport({ taskId: task.id, diff --git a/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts b/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts index 2b81b2f2897c..a83ce65b9839 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts @@ -82,7 +82,7 @@ describe('Task/Dispatch invariant transactions', () => { deps: [task.id] }) const dispatch = createRootDispatch(db, task.id, 'term_worker') - const capability = db.mintDispatchCapability({ + db.mintDispatchCapability({ dispatchId: dispatch.id, paneKey: 'tab_worker:leaf_worker', processIncarnation: 'worker:1' @@ -109,14 +109,6 @@ describe('Task/Dispatch invariant transactions', () => { completed_at: null, capability_revoked_at: null }) - expect( - db.verifyDispatchCapability({ - dispatchId: dispatch.id, - capability, - paneKey: 'tab_worker:leaf_worker', - processIncarnation: 'worker:1' - }) - ).toEqual({ valid: true }) expect(db.getTask(dependent.id)?.status).toBe('pending') } ) @@ -415,7 +407,7 @@ describe('Task/Dispatch invariant transactions', () => { taskId: task.id, startOptions: {} }) - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_worker', paneKey: 'tab_worker:dddddddd-dddd-4ddd-8ddd-dddddddddddd', @@ -443,14 +435,6 @@ describe('Task/Dispatch invariant transactions', () => { capability_revoked_at: null }) expect(db.getWorkerDispatch(started.dispatch.id)?.state).toBe('starting') - expect( - db.verifyDispatchCapability({ - dispatchId: started.dispatch.id, - capability, - paneKey: 'tab_worker:dddddddd-dddd-4ddd-8ddd-dddddddddddd', - processIncarnation: 'worker:1' - }) - ).toEqual({ valid: true }) } ) diff --git a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts index f4cdcdf63b83..3eef47f90ffc 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts @@ -47,8 +47,6 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.getDispatchContextById(second.dispatchId)?.status).toBe('dispatched') expect(database.getWorkerDispatch(first.dispatchId)?.state).toBe('ready') expect(database.getWorkerDispatch(second.dispatchId)?.state).toBe('ready') - expectCapability(database, first, true) - expectCapability(database, second, true) }) it.each(['succeeded', 'failed'] as const)( @@ -158,7 +156,6 @@ describe('Task/Dispatch lifecycle guards', () => { capability_revoked_at: null }) expect(database.getWorkerDispatch(worker.dispatchId)?.state).toBe('ready') - expectCapability(database, worker, true) }) it('atomically settles worker state when a proven process exit fails its Dispatch', () => { @@ -178,7 +175,6 @@ describe('Task/Dispatch lifecycle guards', () => { stage: 'process_exited', last_error: 'process exited' }) - expectCapability(database, worker, false) }) it('settles a stop-unknown worker when a positive PTY exit arrives', () => { @@ -211,7 +207,6 @@ describe('Task/Dispatch lifecycle guards', () => { stage: 'process_exited', last_error: 'process exited' }) - expectCapability(database, worker, false) }) it('keeps a Task dispatched when missing-terminal recovery leaves another worker active', () => { @@ -234,12 +229,10 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.getWorkerDispatch(missing.dispatchId)?.state).toBe('abandoned') expect(database.getDispatchContextById(live.dispatchId)?.status).toBe('dispatched') expect(database.getWorkerDispatch(live.dispatchId)?.state).toBe('ready') - expectCapability(database, missing, false) - expectCapability(database, live, true) database.reconcileMissingWorkerTerminal(live.dispatchId, 'second terminal missing') expect(database.getTask(task.id)?.status).toBe('ready') - expectCapability(database, live, false) + expect(database.getDispatchContextById(live.dispatchId)?.status).toBe('failed') }) it.each(['local', 'federated'] as const)( @@ -288,7 +281,6 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.getWorkerDispatch(failed.dispatch.id)?.state).toBe('failed') expect(database.getDispatchContextById(live.dispatchId)?.status).toBe('dispatched') expect(database.getWorkerDispatch(live.dispatchId)?.state).toBe('ready') - expectCapability(database, live, true) } ) @@ -528,7 +520,6 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.getTask(task.id)?.status).toBe('dispatched') expect(database.getDispatchContextById(live.dispatchId)?.status).toBe('dispatched') - expectCapability(database, live, true) expect( database.settleWorkerReport({ taskId: task.id, @@ -538,7 +529,7 @@ describe('Task/Dispatch lifecycle guards', () => { }) ).toEqual({ action: 'settled', outcome: 'succeeded', duplicate: false }) expect(database.getTask(task.id)?.status).toBe('completed') - expectCapability(database, live, false) + expect(database.getDispatchContextById(live.dispatchId)?.status).toBe('completed') } ) @@ -656,7 +647,6 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.getTask(task.id)?.status).toBe('dispatched') expect(database.getDispatchContextById(worker.dispatchId)?.status).toBe('dispatched') expect(database.getWorkerDispatch(worker.dispatchId)?.state).toBe('ready') - expectCapability(database, worker, true) }) it('rolls back gate resolution when an active Dispatch blocks readiness', () => { @@ -714,17 +704,6 @@ function startWorker(database: OrchestrationDb, taskId: string, name: string): W return { dispatchId: started.dispatch.id, capability, handle, paneKey, processIncarnation } } -function expectCapability(database: OrchestrationDb, worker: WorkerFixture, valid: boolean): void { - expect( - database.verifyDispatchCapability({ - dispatchId: worker.dispatchId, - capability: worker.capability, - paneKey: worker.paneKey, - processIncarnation: worker.processIncarnation - }).valid - ).toBe(valid) -} - function sqliteFor(database: OrchestrationDb): Database.Database { return (database as unknown as { db: Database.Database }).db } diff --git a/src/main/runtime/orchestration/db-task-dispatch-races.test.ts b/src/main/runtime/orchestration/db-task-dispatch-races.test.ts index c671aa4566b0..28a7d092ec65 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-races.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-races.test.ts @@ -126,7 +126,7 @@ describe('Task/Dispatch concurrency', () => { taskId: task.id, startOptions: {} }) - const capability = first.db.prepareStartingWorkerAuthority({ + first.db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_worker', paneKey: 'tab_worker:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', @@ -171,14 +171,6 @@ describe('Task/Dispatch concurrency', () => { status: 'completed', last_failure: null }) - expect( - first.db.verifyDispatchCapability({ - dispatchId: started.dispatch.id, - capability, - paneKey: 'tab_worker:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', - processIncarnation: 'worker:1' - }) - ).toMatchObject({ valid: false }) }) it('keeps nested dispatch failure atomic with its caller transaction', () => { diff --git a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.test.ts b/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.test.ts deleted file mode 100644 index 2bacb42479eb..000000000000 --- a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.test.ts +++ /dev/null @@ -1,60 +0,0 @@ -import { afterEach, beforeEach, describe, expect, it } from 'vitest' -import { OrchestrationDb } from '../orchestration-db' -import { createRootDispatch } from '../root-dispatch-test-fixture' - -describe('verifyDispatchCapability', () => { - let db: OrchestrationDb - - beforeEach(() => { - db = new OrchestrationDb(':memory:') - }) - - afterEach(() => { - db.close() - }) - - function mintedWorker() { - const run = db.createRun({ - objective: 'capability guidance', - coordinatorHandle: 'term_coord', - coordinatorPaneKey: 'tab_coord:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' - }) - const task = db.createTask({ spec: 'work', runId: run.id }) - const paneKey = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' - const dispatch = createRootDispatch(db, task.id, 'term_worker', paneKey) - const capability = db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey, - processIncarnation: 'worker:1' - }) - return { dispatchId: dispatch.id, paneKey, capability } - } - - // Why: the caller that hits this is an agent reading only the error string, so it must name the flag. - it('names the flag that supplies a missing capability', () => { - const worker = mintedWorker() - - const result = db.verifyDispatchCapability({ - dispatchId: worker.dispatchId, - capability: undefined, - paneKey: worker.paneKey, - processIncarnation: 'worker:1' - }) - - expect(result).toMatchObject({ valid: false }) - expect(result.valid === false && result.reason).toContain('--dispatch-capability') - }) - - it('still accepts the minted capability', () => { - const worker = mintedWorker() - - expect( - db.verifyDispatchCapability({ - dispatchId: worker.dispatchId, - capability: worker.capability, - paneKey: worker.paneKey, - processIncarnation: 'worker:1' - }) - ).toEqual({ valid: true }) - }) -}) diff --git a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts b/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts index 007e81d10164..c9c687f2b109 100644 --- a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts +++ b/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts @@ -1,7 +1,6 @@ -import { randomBytes, timingSafeEqual } from 'node:crypto' +import { randomBytes } from 'node:crypto' import { OrchestrationError } from '../../orchestration-error' import { hashDispatchCapability } from '../dispatch-capability-hash' -import { isEquivalentPaneKey } from '../pane-key-match' import type { OrchestrationDb } from '../orchestration-db' export function mintDispatchCapability( @@ -47,55 +46,6 @@ export function mintDispatchCapability( return capability } -export function verifyDispatchCapability( - this: OrchestrationDb, - params: { - dispatchId: string - capability: string | undefined - paneKey: string | undefined - processIncarnation: string | undefined - } -): { valid: true } | { valid: false; reason: string } { - const dispatch = this.getDispatchContextById(params.dispatchId) - if (!dispatch) { - return { valid: false, reason: `Dispatch ${params.dispatchId} was not found.` } - } - if (!dispatch.capability_hash) { - return { valid: false, reason: `Dispatch ${params.dispatchId} has no lifecycle capability.` } - } - if (dispatch.capability_revoked_at) { - return { valid: false, reason: `Dispatch ${params.dispatchId} capability is revoked.` } - } - if (!params.capability) { - // Why: a worker that omits the flag needs the flag name, not just the diagnosis. - return { - valid: false, - reason: - 'The Dispatch capability is missing. Pass --dispatch-capability from your dispatch preamble.' - } - } - const expected = Buffer.from(dispatch.capability_hash, 'hex') - const observed = Buffer.from(hashDispatchCapability(params.capability), 'hex') - if (expected.length !== observed.length || !timingSafeEqual(expected, observed)) { - return { valid: false, reason: 'The Dispatch capability is invalid.' } - } - if ( - !dispatch.assignee_pane_key || - !params.paneKey || - !isEquivalentPaneKey(dispatch.assignee_pane_key, params.paneKey) - ) { - return { valid: false, reason: 'The caller is not the Dispatch pane.' } - } - if ( - !dispatch.process_incarnation || - !params.processIncarnation || - dispatch.process_incarnation !== params.processIncarnation - ) { - return { valid: false, reason: 'The Dispatch process incarnation changed.' } - } - return { valid: true } -} - export function revokeDispatchCapability(this: OrchestrationDb, dispatchId: string): void { this.db .prepare( @@ -108,14 +58,12 @@ export function revokeDispatchCapability(this: OrchestrationDb, dispatchId: stri export type DispatchCapabilityMethods = { mintDispatchCapability: typeof mintDispatchCapability - verifyDispatchCapability: typeof verifyDispatchCapability revokeDispatchCapability: typeof revokeDispatchCapability } export function attachDispatchCapability(ctor: { prototype: object }): void { Object.assign(ctor.prototype, { mintDispatchCapability, - verifyDispatchCapability, revokeDispatchCapability }) } diff --git a/src/main/runtime/orchestration/db/dispatch-context/worker-report-settlement.ts b/src/main/runtime/orchestration/db/dispatch-context/worker-report-settlement.ts index ddb0e383143c..9768b83ac5be 100644 --- a/src/main/runtime/orchestration/db/dispatch-context/worker-report-settlement.ts +++ b/src/main/runtime/orchestration/db/dispatch-context/worker-report-settlement.ts @@ -5,6 +5,11 @@ import { settleActiveDispatchesForTask } from './dispatch-completion' import { getActiveDispatchForTask } from './task-dispatch-reconciliation' import { transitionLifecycleWithDb } from '../lifecycle-transition' import { runLifecycleWriteTransaction } from '../lifecycle-write-transaction-runner' +import { + isWorkerStateIn, + SETTLEABLE_WORKER_STATES, + UNPROVEN_WORKER_STATES +} from '../../worker-report-admission' type WorkerReportObservation = { id: string @@ -105,22 +110,23 @@ export function settleWorkerReportInTransaction( recordAcceptedReportFact(this, params) return { action: 'settled', outcome: params.outcome, duplicate: true } } - const reconnectingStart = + const unprovenWorker = isWorkerStateIn(UNPROVEN_WORKER_STATES, reportingWorker?.state) + const reconnectingWorker = (dispatch.status === 'pending' || dispatch.status === 'dispatched') && task.status === 'blocked' && - reportingWorker?.state === 'start_unknown' + unprovenWorker const reportingStart = dispatch.status === 'pending' && task.status === 'dispatched' && reportingWorker?.state === 'starting' const previousDispatchStatus = settledByUnobservedPrompt ? 'failed' - : reconnectingStart || reportingStart + : reconnectingWorker || reportingStart ? dispatch.status : 'dispatched' const previousTaskStatus = settledByUnobservedPrompt ? 'failed' - : reconnectingStart + : reconnectingWorker ? 'blocked' : 'dispatched' if (dispatch.status !== previousDispatchStatus || task.status !== previousTaskStatus) { @@ -191,7 +197,7 @@ export function settleWorkerReportInTransaction( dispatchUpdate = { changes: dispatchTransition.changed ? 1 : 0 } taskUpdate = { changes: taskTransition.changed ? 1 : 0 } } else { - if (reconnectingStart) { + if (reconnectingWorker) { transitionLifecycleWithDb(this.db, { entity: 'task', id: params.taskId, @@ -202,7 +208,7 @@ export function settleWorkerReportInTransaction( const dispatchTransition = transitionLifecycleWithDb(this.db, { entity: 'dispatch', id: params.dispatchId, - from: reconnectingStart || reportingStart ? ['pending', 'dispatched'] : 'dispatched', + from: reconnectingWorker || reportingStart ? ['pending', 'dispatched'] : 'dispatched', to: expectedDispatchStatus, projection: { completed_at: new Date().toISOString(), @@ -238,12 +244,13 @@ export function settleWorkerReportInTransaction( projection: { stage: 'settled', updated_at: new Date().toISOString() }, correction: 'unobserved_prompt_report' }) - } else if ((reconnectingStart || reportingStart) && params.outcome === 'succeeded') { + } else if ((unprovenWorker || reportingStart) && params.outcome === 'succeeded') { transitionLifecycleWithDb(this.db, { entity: 'worker', id: params.dispatchId, - from: reportingStart ? 'starting' : 'start_unknown', - to: 'ready' + from: reportingStart ? 'starting' : UNPROVEN_WORKER_STATES, + to: 'ready', + projection: { last_error: null } }) transitionLifecycleWithDb(this.db, { entity: 'worker', @@ -256,8 +263,8 @@ export function settleWorkerReportInTransaction( transitionLifecycleWithDb(this.db, { entity: 'worker', id: params.dispatchId, - // A start_unknown success report reconnects through 'ready' above; only failure settles here. - from: params.outcome === 'succeeded' ? 'ready' : ['ready', 'start_unknown', 'starting'], + // An unknown worker's success report reconnects through 'ready' above; only failure settles here. + from: params.outcome === 'succeeded' ? 'ready' : [...SETTLEABLE_WORKER_STATES, 'starting'], to: params.outcome === 'succeeded' ? 'succeeded' : 'failed', projection: { stage: 'settled', updated_at: new Date().toISOString() } }) diff --git a/src/main/runtime/orchestration/db/federation/federation-relay-enqueue.ts b/src/main/runtime/orchestration/db/federation/federation-relay-enqueue.ts index 28841f44e6e0..e98a2236aeb9 100644 --- a/src/main/runtime/orchestration/db/federation/federation-relay-enqueue.ts +++ b/src/main/runtime/orchestration/db/federation/federation-relay-enqueue.ts @@ -6,6 +6,7 @@ import type { import { OrchestrationError } from '../../orchestration-error' import { generateId } from '../generated-id' import type { OrchestrationDb } from '../orchestration-db' +import { isWorkerStateIn, SETTLEABLE_WORKER_STATES } from '../../worker-report-admission' export function enqueueFederationRelay( this: OrchestrationDb, @@ -31,7 +32,7 @@ export function enqueueFederationRelay( try { if (params.settleRemoteOutcome) { const attachment = this.getRemoteDispatchAttachment(params.dispatchId) - if (!attachment || attachment.state !== 'ready') { + if (!attachment || !isWorkerStateIn(SETTLEABLE_WORKER_STATES, attachment.state)) { throw new OrchestrationError( 'dispatch_inactive', `Remote Dispatch ${params.dispatchId} is not active.` diff --git a/src/main/runtime/orchestration/db/federation/federation-relay-item.ts b/src/main/runtime/orchestration/db/federation/federation-relay-item.ts index 1ae13b6b6ae0..dac7b8f2c3a4 100644 --- a/src/main/runtime/orchestration/db/federation/federation-relay-item.ts +++ b/src/main/runtime/orchestration/db/federation/federation-relay-item.ts @@ -5,6 +5,7 @@ import type { } from '../../types' import { OrchestrationError } from '../../orchestration-error' import type { OrchestrationDb } from '../orchestration-db' +import { isWorkerStateIn, SETTLEABLE_WORKER_STATES } from '../../worker-report-admission' export function getFederationRelayItem( this: OrchestrationDb, @@ -40,7 +41,7 @@ export function settleRemoteAttachmentInRelayTransaction( if (attachment.state === state) { return } - if (attachment.state !== 'ready') { + if (!isWorkerStateIn(SETTLEABLE_WORKER_STATES, attachment.state)) { throw new OrchestrationError( 'request_mismatch', `Remote Dispatch ${dispatchId} cannot settle as ${state} from ${attachment.state}.` @@ -51,9 +52,9 @@ export function settleRemoteAttachmentInRelayTransaction( `UPDATE remote_dispatch_attachments SET state = ?, stage = ?, capability_hash = NULL, updated_at = datetime('now') - WHERE dispatch_id = ? AND state = 'ready'` + WHERE dispatch_id = ? AND state = ?` ) - .run(state, stage, dispatchId) + .run(state, stage, dispatchId, attachment.state) } export type FederationRelayItemMethods = { diff --git a/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts b/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts index ac85fc7c3c99..4af584425a1d 100644 --- a/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts +++ b/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts @@ -1,4 +1,4 @@ -import { randomBytes, timingSafeEqual } from 'node:crypto' +import { randomBytes } from 'node:crypto' import type { RemoteDispatchAttachmentRow } from '../../types' import { OrchestrationError } from '../../orchestration-error' import { hashDispatchCapability } from '../dispatch-capability-hash' @@ -162,32 +162,6 @@ export function failRemoteAttachment( return this.getRemoteDispatchAttachment(dispatchId) as RemoteDispatchAttachmentRow } -export function verifyRemoteAttachmentAuthority( - this: OrchestrationDb, - params: { - dispatchId: string - capability: string | undefined - paneKey: string | null - processIncarnation: string | null - } -): boolean { - const attachment = this.getRemoteDispatchAttachment(params.dispatchId) - if ( - !attachment?.capability_hash || - !params.capability || - !attachment.pane_key || - !params.paneKey || - !isEquivalentPaneKey(attachment.pane_key, params.paneKey) || - !attachment.process_incarnation || - attachment.process_incarnation !== params.processIncarnation - ) { - return false - } - const expected = Buffer.from(attachment.capability_hash, 'hex') - const observed = Buffer.from(hashDispatchCapability(params.capability), 'hex') - return expected.length === observed.length && timingSafeEqual(expected, observed) -} - export function isRemoteAttachmentProcessCurrent( this: OrchestrationDb, params: { @@ -210,7 +184,6 @@ export type RemoteDispatchAttachmentAuthorityMethods = { prepareRemoteAttachmentAuthority: typeof prepareRemoteAttachmentAuthority markRemoteAttachmentReady: typeof markRemoteAttachmentReady failRemoteAttachment: typeof failRemoteAttachment - verifyRemoteAttachmentAuthority: typeof verifyRemoteAttachmentAuthority isRemoteAttachmentProcessCurrent: typeof isRemoteAttachmentProcessCurrent } @@ -219,7 +192,6 @@ export function attachRemoteDispatchAttachmentAuthority(ctor: { prototype: objec prepareRemoteAttachmentAuthority, markRemoteAttachmentReady, failRemoteAttachment, - verifyRemoteAttachmentAuthority, isRemoteAttachmentProcessCurrent }) } diff --git a/src/main/runtime/orchestration/db/lifecycle-transition.ts b/src/main/runtime/orchestration/db/lifecycle-transition.ts index 6fcc40f19131..25d2887141c2 100644 --- a/src/main/runtime/orchestration/db/lifecycle-transition.ts +++ b/src/main/runtime/orchestration/db/lifecycle-transition.ts @@ -89,7 +89,7 @@ const LEGAL_TRANSITIONS: Record { startOptions: {} }) const paneKey = `tab_worker:${LEAF_A}` - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_worker', paneKey, @@ -74,14 +74,6 @@ describe('lifecycle reconciliation', () => { effects: [] }) db.markWorkerStartUnknown(started.dispatch.id, 'agent_readiness', 'connection lost') - expect( - db.verifyDispatchCapability({ - dispatchId: started.dispatch.id, - capability, - paneKey, - processIncarnation: 'worker:1' - }) - ).toEqual({ valid: true }) const message = db.insertMessage({ runId: 'run_legacy_local', diff --git a/src/main/runtime/orchestration/lifecycle-reconciliation.ts b/src/main/runtime/orchestration/lifecycle-reconciliation.ts index ff6d96491fef..838d2ce03276 100644 --- a/src/main/runtime/orchestration/lifecycle-reconciliation.ts +++ b/src/main/runtime/orchestration/lifecycle-reconciliation.ts @@ -41,7 +41,6 @@ export type LifecycleReconciliationResult = export type LifecycleRejectionCode = | 'sender_not_assignee' - | 'dispatch_capability_invalid' | 'invalid_payload' | 'missing_task_id' | 'missing_dispatch_id' diff --git a/src/main/runtime/orchestration/worker-report-admission.ts b/src/main/runtime/orchestration/worker-report-admission.ts new file mode 100644 index 000000000000..31401d382c03 --- /dev/null +++ b/src/main/runtime/orchestration/worker-report-admission.ts @@ -0,0 +1,55 @@ +import { OrchestrationError } from './orchestration-error' +import type { WorkerDispatchState } from './types' + +// Why: an unknown start or stop is unverifiable, not exited (docs/reference/ssh-execution-boundary.md), +// so the worker's own report outranks it, locally and on a connected server. +export const UNPROVEN_WORKER_STATES = [ + 'start_unknown', + 'stop_unknown' +] as const satisfies readonly WorkerDispatchState[] + +export const SETTLEABLE_WORKER_STATES = [ + 'ready', + ...UNPROVEN_WORKER_STATES +] as const satisfies readonly WorkerDispatchState[] + +export function isWorkerStateIn( + states: readonly WorkerDispatchState[], + state: WorkerDispatchState | undefined +): boolean { + return state !== undefined && states.includes(state) +} + +export type WorkerReportRefusal = { + code: 'dispatch_inactive' | 'worker_identity_changed' + reason: string +} + +/** The one admission rule for a worker's lifecycle report or question: only a stop in flight mutes it. */ +export function workerReportRefusal(args: { + dispatchId: string + from: string + workerState: WorkerDispatchState | undefined + processCurrent: boolean +}): WorkerReportRefusal | null { + if (args.workerState === 'stopping') { + return { + code: 'dispatch_inactive', + reason: `Dispatch ${args.dispatchId} is stopping; its worker cannot report until worker-stop settles.` + } + } + if (!args.processCurrent) { + return { + code: 'worker_identity_changed', + reason: `${args.from} is not the exact process that owns Dispatch ${args.dispatchId}.` + } + } + return null +} + +export function assertWorkerCanReport(args: Parameters[0]): void { + const refusal = workerReportRefusal(args) + if (refusal) { + throw new OrchestrationError(refusal.code, refusal.reason) + } +} diff --git a/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts b/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts index 1d123f51b382..7f7b3bd88f7a 100644 --- a/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts +++ b/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts @@ -5,7 +5,7 @@ const WORKER_PANE_KEY = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' const INCARNATION = 'runtime_test:term_worker:1' let db: OrchestrationDb -function startWorker(spec: string): { taskId: string; dispatchId: string; capability: string } { +function startWorker(spec: string): { taskId: string; dispatchId: string } { const task = db.createTask({ runId: 'run_legacy_local', spec }) const started = db.createStartingWorkerDispatch({ creator: { kind: 'system' }, @@ -13,29 +13,20 @@ function startWorker(spec: string): { taskId: string; dispatchId: string; capabi taskId: task.id, startOptions: {} }) - const capability = db.mintDispatchCapability({ + db.mintDispatchCapability({ dispatchId: started.dispatch.id, paneKey: WORKER_PANE_KEY, processIncarnation: INCARNATION }) - return { taskId: task.id, dispatchId: started.dispatch.id, capability } -} - -function verify(dispatchId: string, capability: string): { valid: boolean; reason?: string } { - return db.verifyDispatchCapability({ - dispatchId, - capability, - paneKey: WORKER_PANE_KEY, - processIncarnation: INCARNATION - }) + return { taskId: task.id, dispatchId: started.dispatch.id } } describe('worker start settled by an unobserved prompt', () => { afterEach(() => db?.close()) - it('keeps the capability and lets the worker report correct the record', () => { + it('lets the worker report correct the record', () => { db = new OrchestrationDb(':memory:') - const { taskId, dispatchId, capability } = startWorker('run to completion') + const { taskId, dispatchId } = startWorker('run to completion') db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled', { retainCapability: true @@ -46,7 +37,6 @@ describe('worker start settled by an unobserved prompt', () => { last_failure: 'agent_prompt_stalled', capability_revoked_at: null }) - expect(verify(dispatchId, capability)).toEqual({ valid: true }) expect( db.settleWorkerReport({ @@ -63,12 +53,11 @@ describe('worker start settled by an unobserved prompt', () => { it('revokes and stays settled when the start failed for any other cause', () => { db = new OrchestrationDb(':memory:') - const { taskId, dispatchId, capability } = startWorker('never became ready') + const { taskId, dispatchId } = startWorker('never became ready') db.failWorkerStart(dispatchId, 'agent_readiness', 'Agent did not become ready (idle).') expect(db.getDispatchContextById(dispatchId)?.capability_revoked_at).toEqual(expect.any(String)) - expect(verify(dispatchId, capability).valid).toBe(false) expect( db.settleWorkerReport({ taskId, dispatchId, outcome: 'succeeded', result: 'done' }) ).toMatchObject({ action: 'rejected', code: 'inactive_dispatch' }) diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/ask-methods.ts b/src/main/runtime/rpc/methods/orchestration/messaging/ask-methods.ts index 9b49e7a07a11..25b8058e5dff 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/ask-methods.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/ask-methods.ts @@ -10,6 +10,8 @@ import { runCoordinatorKey } from '../../../../orchestration/orchestration-caller-identity' import { resolveOrchestrationParty } from '../../../../orchestration/orchestration-party' +import { assertLifecycleCallerIsNotAnotherParty } from './lifecycle-caller-fence' +import { assertWorkerCanReport } from '../../../../orchestration/worker-report-admission' export const ORCHESTRATION_ASK_METHODS = [ defineMethod({ @@ -17,7 +19,13 @@ export const ORCHESTRATION_ASK_METHODS = [ params: AskParams, handler: async ( params, - { runtime, signal, orchestrationCapability, recordMutationReceipt } + { + runtime, + signal, + recordMutationReceipt, + orchestrationCaller, + orchestrationCompatibilityEvidence + } ) => { // Why: group addresses have no unambiguous first-answer authority. if (params.to && isGroupAddress(params.to)) { @@ -32,19 +40,26 @@ export const ORCHESTRATION_ASK_METHODS = [ // Why: echoed on every return so a clamped caller reports the budget actually waited, not the one it asked for. const timeoutMs = clampOrchestrationAskTimeoutMs(params.timeoutMs) const paneKey = runtime.getTerminalPaneKey(from) ?? undefined + if (!orchestrationCaller) { + assertLifecycleCallerIsNotAnotherParty(runtime, { + from, + fromPaneKey: paneKey, + evidence: orchestrationCompatibilityEvidence + }) + } const remoteAttachment = paneKey ? db.findActiveRemoteAttachmentForPane(paneKey) : undefined - if (remoteAttachment) { + if (remoteAttachment && paneKey) { rejectFederatedExplicitTarget(params) return askRemoteRunHome({ params: { ...params, timeoutMs }, runtime, signal, - orchestrationCapability, recordMutationReceipt, from, - paneKey: paneKey as string, + paneKey, dispatchId: remoteAttachment.dispatch_id, - taskId: remoteAttachment.task_id + taskId: remoteAttachment.task_id, + workerState: remoteAttachment.state }) } const activeDispatch = db.getActiveDispatchForIdentity(from, paneKey) @@ -54,17 +69,18 @@ export const ORCHESTRATION_ASK_METHODS = [ 'ask requires an active supervised Dispatch.' ) } - if (activeDispatch.capability_hash) { - const authority = db.verifyDispatchCapability({ - dispatchId: activeDispatch.id, - capability: orchestrationCapability, - paneKey, - processIncarnation: runtime.getTerminalProcessIncarnation(from) ?? undefined - }) - if (!authority.valid) { - throw new OrchestrationError('dispatch_capability_invalid', authority.reason) - } - } + assertWorkerCanReport({ + dispatchId: activeDispatch.id, + from, + workerState: db.getWorkerDispatch(activeDispatch.id)?.state, + processCurrent: + !activeDispatch.process_incarnation || + db.isDispatchProcessCurrent({ + dispatchId: activeDispatch.id, + paneKey: paneKey ?? null, + processIncarnation: runtime.getTerminalProcessIncarnation(from) + }) + }) const options = params.options ?.split(',') diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/ask-remote.ts b/src/main/runtime/rpc/methods/orchestration/messaging/ask-remote.ts index da092fbaaafd..a37707d42095 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/ask-remote.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/ask-remote.ts @@ -1,35 +1,34 @@ import type { z } from 'zod' import type { OrcaRuntimeService } from '../../../../orca-runtime' +import type { WorkerDispatchState } from '../../../../orchestration/types' import { OrchestrationError } from '../../../../orchestration/orchestration-error' import { clampOrchestrationAskTimeoutMs } from '../../../../../../shared/orchestration-ask-timeout' import type { AskParams } from '../schemas' +import { assertWorkerCanReport } from '../../../../orchestration/worker-report-admission' export async function askRemoteRunHome(args: { params: z.infer runtime: OrcaRuntimeService signal?: AbortSignal - orchestrationCapability?: string recordMutationReceipt?: (receipt: unknown) => void from: string paneKey: string dispatchId: string taskId: string + workerState: WorkerDispatchState }): Promise { const db = args.runtime.getOrchestrationDb() const timeoutMs = clampOrchestrationAskTimeoutMs(args.params.timeoutMs) - if ( - !db.verifyRemoteAttachmentAuthority({ + assertWorkerCanReport({ + dispatchId: args.dispatchId, + from: args.from, + workerState: args.workerState, + processCurrent: db.isRemoteAttachmentProcessCurrent({ dispatchId: args.dispatchId, - capability: args.orchestrationCapability, paneKey: args.paneKey, processIncarnation: args.runtime.getTerminalProcessIncarnation(args.from) }) - ) { - throw new OrchestrationError( - 'dispatch_capability_invalid', - 'The remote Dispatch capability or exact worker process is invalid.' - ) - } + }) const options = args.params.options ?.split(',') diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/ask.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/ask.test.ts index 4a46861a1c1e..51798a3ee8ed 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/ask.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/ask.test.ts @@ -164,36 +164,34 @@ describe('orchestration RPC methods', () => { ) }) - it('requires the Dispatch capability before creating a question', async () => { + it('creates a question only from the exact process that owns the Dispatch', async () => { setup() - const { dispatch } = createAskingDispatch() - const capability = db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: 'tab_worker:leaf_worker', - processIncarnation: 'runtime_test:term_worker:1' - }) + const task = db.createTask({ spec: 'question work' }) + createRootDispatch( + db, + task.id, + 'term_worker', + 'tab_worker:leaf_worker', + undefined, + 'runtime_test:term_worker:1' + ) vi.spyOn(runtime, 'getTerminalPaneKey').mockImplementation((handle) => handle === 'term_worker' ? 'tab_worker:leaf_worker' : coordinatorPaneKey ) + vi.spyOn(runtime, 'waitForMessage').mockResolvedValue('timed_out') + const incarnation = vi + .spyOn(runtime, 'getTerminalProcessIncarnation') + .mockReturnValue('runtime_test:term_worker:2') await expect( - call('orchestration.ask', { - from: 'term_worker', - question: 'unauthorized', - timeoutMs: 1 - }) - ).rejects.toMatchObject({ code: 'dispatch_capability_invalid' }) + call('orchestration.ask', { from: 'term_worker', question: 'stale', timeoutMs: 1 }) + ).rejects.toMatchObject({ code: 'worker_identity_changed' }) expect(db.getInbox(100).filter((message) => message.type === 'question')).toHaveLength(0) - ctx = { runtime, orchestrationCapability: capability } - vi.spyOn(runtime, 'waitForMessage').mockResolvedValue('timed_out') - const accepted = (await call('orchestration.ask', { - from: 'term_worker', - question: 'authorized', - timeoutMs: 1 - })) as { messageId: string; timedOut: boolean } - expect(accepted.messageId).toMatch(/^msg_/) - expect(accepted.timedOut).toBe(true) + incarnation.mockReturnValue('runtime_test:term_worker:1') + expect( + await call('orchestration.ask', { from: 'term_worker', question: 'current', timeoutMs: 1 }) + ).toMatchObject({ messageId: expect.stringMatching(/^msg_/), timedOut: true }) }) it('returns timedOut when no reply arrives in the window', async () => { diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/lifecycle-caller-fence.ts b/src/main/runtime/rpc/methods/orchestration/messaging/lifecycle-caller-fence.ts new file mode 100644 index 000000000000..cb8b58f2883f --- /dev/null +++ b/src/main/runtime/rpc/methods/orchestration/messaging/lifecycle-caller-fence.ts @@ -0,0 +1,64 @@ +import type { OrchestrationCompatibilityEvidence } from '../../../../../../shared/orchestration-compatibility-evidence' +import type { OrcaRuntimeService } from '../../../../orca-runtime' +import type { OrchestrationDb } from '../../../../orchestration/db' +import { OrchestrationError } from '../../../../orchestration/orchestration-error' +import { isEquivalentPaneKey } from '../../../../orchestration/db/pane-key-match' + +/** + * Refuses a worker report or question sent from another orchestration party's terminal: a Run + * coordinator or the assignee of a different Dispatch. Env that names no live pane on this host + * (stale, foreign, scrubbed, absent), or a live pane that is no party, proves nothing and passes. + * Accepted limit: a tmux server keeps the env of the pane that started it, so a worker inside a tmux + * server started from a live coordinator or worker pane is refused. + */ +export function assertLifecycleCallerIsNotAnotherParty( + runtime: OrcaRuntimeService, + args: { + from: string + fromPaneKey: string | undefined + evidence: OrchestrationCompatibilityEvidence | undefined + } +): void { + const { evidence } = args + // Why pane key first: the env handle goes stale on remint, the pane key does not. + const callerHandle = evidence?.paneKey + ? runtime.getTerminalHandleForPaneKey(evidence.paneKey) + : evidence?.terminalHandle + const callerPaneKey = callerHandle ? runtime.getLiveTerminalPaneKey(callerHandle) : null + if ( + !callerHandle || + !callerPaneKey || + (args.fromPaneKey && isEquivalentPaneKey(callerPaneKey, args.fromPaneKey)) + ) { + return + } + const db = runtime.getOrchestrationDb() + const callerDispatchId = ownedDispatchId(db, callerHandle, callerPaneKey) + // Why: a stale --from handle can hide the worker's own terminal; only a different party is refused. + if (callerDispatchId && callerDispatchId === ownedDispatchId(db, args.from, args.fromPaneKey)) { + return + } + const party = db.getCurrentRunForPane(callerPaneKey) + ? 'a Run coordinator' + : callerDispatchId + ? 'a Dispatch worker' + : undefined + if (party) { + throw new OrchestrationError( + 'consumer_fenced', + `This terminal is ${party} and cannot report as ${args.from}; run the command from the worker's own terminal. No effects were applied.`, + { effectsApplied: false } + ) + } +} + +function ownedDispatchId( + db: OrchestrationDb, + handle: string, + paneKey: string | undefined +): string | undefined { + return ( + db.getActiveDispatchForIdentity(handle, paneKey)?.id ?? + (paneKey ? db.findActiveRemoteAttachmentForPane(paneKey)?.dispatch_id : undefined) + ) +} diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/send-methods.ts b/src/main/runtime/rpc/methods/orchestration/messaging/send-methods.ts index 58dcceb55980..a98e016da751 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/send-methods.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/send-methods.ts @@ -2,7 +2,12 @@ import { defineMethod } from '../../../core' import { OrchestrationError } from '../../../../orchestration/orchestration-error' import { isGroupAddress } from '../../../../orchestration/groups' import { orchestrationSkillRecoveryData } from '../../../../../../shared/orchestration-rpc-contract' -import { SendParams, isWorkerReportOutcome, parseRemoteWorkerPayload } from '../schemas' +import { + SendParams, + isDispatchMutationMessageType, + isWorkerReportOutcome, + parseRemoteWorkerPayload +} from '../schemas' import { resolveMessageRun } from '../routing' import { assertDispatchMailboxDeliverable, @@ -22,6 +27,7 @@ import { sendPointToPointMessage } from './send-point-to-point' import { sendGroupMessage } from './send-group' import { sendFederatedControlMail } from './send-control-mail' import { orchestrationCallerIdentity } from '../runs/run-scope' +import { assertLifecycleCallerIsNotAnotherParty } from './lifecycle-caller-fence' export const ORCHESTRATION_SEND_METHODS = [ defineMethod({ @@ -31,7 +37,6 @@ export const ORCHESTRATION_SEND_METHODS = [ params, { runtime, - orchestrationCapability, legacyCoordinatorRunId, revalidateLegacyCoordinator, orchestrationCompatibilityCallerAuthority, @@ -39,6 +44,7 @@ export const ORCHESTRATION_SEND_METHODS = [ markWorkerDoneMutationEffectFree, replayedMutationReceipt, orchestrationCaller, + orchestrationCompatibilityEvidence, signal } ) => { @@ -69,6 +75,14 @@ export const ORCHESTRATION_SEND_METHODS = [ paneKey: attestedCaller?.paneKey ?? runtime.getTerminalPaneKey(from) }) const senderPaneKey = sender.paneKey ?? undefined + // Why: a session caller was already bound to its own identity at the dispatch entry. + if (isDispatchMutationMessageType(params.type) && !orchestrationCaller) { + assertLifecycleCallerIsNotAnotherParty(runtime, { + from, + fromPaneKey: senderPaneKey, + evidence: orchestrationCompatibilityEvidence + }) + } const remoteAttachment = senderPaneKey ? db.findActiveRemoteAttachmentForPane(senderPaneKey) : undefined @@ -84,7 +98,6 @@ export const ORCHESTRATION_SEND_METHODS = [ attestedCaller?.processIncarnation ?? runtime.getTerminalProcessIncarnation(from) ?? undefined, - orchestrationCapability, signal }) } @@ -205,7 +218,6 @@ export const ORCHESTRATION_SEND_METHODS = [ messageRunId, senderPaneKey, legacyCoordinatorRunId, - orchestrationCapability, resolveProcessIncarnation: () => attestedCaller?.processIncarnation ?? runtime.getTerminalProcessIncarnation(from) ?? diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/send-point-to-point.ts b/src/main/runtime/rpc/methods/orchestration/messaging/send-point-to-point.ts index 807e709003a2..9a6852f9ab70 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/send-point-to-point.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/send-point-to-point.ts @@ -8,6 +8,10 @@ import { legacyWorkerDeliveryContract } from '../routing' import { exposeMessage } from './mailbox-message-receipt' import { recordReceiptForPostCommitNudge } from './mutation-replay-nudge' import type { SendRecipientWarning } from './recipient-routing' +import { + workerReportRefusal, + type WorkerReportRefusal +} from '../../../../orchestration/worker-report-admission' import type { z } from 'zod' type SendParamsInput = z.infer @@ -23,7 +27,6 @@ export function sendPointToPointMessage(args: { messageRunId: string | undefined senderPaneKey: string | undefined legacyCoordinatorRunId: string | undefined - orchestrationCapability: string | undefined resolveProcessIncarnation: () => string | undefined revalidateLegacyCoordinator: (() => string) | undefined recordMutationReceipt: ((receipt: unknown) => void) | undefined @@ -40,7 +43,6 @@ export function sendPointToPointMessage(args: { messageRunId, senderPaneKey, legacyCoordinatorRunId, - orchestrationCapability, resolveProcessIncarnation, revalidateLegacyCoordinator, recordMutationReceipt, @@ -76,28 +78,25 @@ export function sendPointToPointMessage(args: { }) if (isDispatchMutationMessageType(msg.type)) { const taskId = parseMessageTaskId(params.payload) - const capabilityBacked = Boolean(dispatch?.capability_hash) const coordinatorMutation = msg.type === 'escalation' || msg.type === 'decision_gate' - const authority = resolveLifecycleAuthority({ + const refusal = lifecycleRefusal({ db, dispatch, from, paneKey: senderPaneKey, processIncarnation, - capability: orchestrationCapability, taskId, - capabilityBacked, coordinatorMutation }) - if (!authority.valid) { + if (refusal) { const rejection = - db.convertLifecycleMessageToRejection(msg.id, authority.code, authority.reason) ?? msg + db.convertLifecycleMessageToRejection(msg.id, refusal.code, refusal.reason) ?? msg const receipt = withSendWarnings({ message: exposeMessage(rejection), lifecycle: { action: 'rejected', - code: authority.code, - reason: authority.reason + code: refusal.code, + reason: refusal.reason } }) return recordReceiptForPostCommitNudge(recordMutationReceipt, receipt, () => @@ -152,81 +151,55 @@ export function sendPointToPointMessage(args: { return committed.receipt } -type LifecycleAuthority = { - valid: boolean - code: 'sender_not_assignee' | 'task_dispatch_mismatch' | 'dispatch_capability_invalid' +type LifecycleRefusal = { + code: 'sender_not_assignee' | 'task_dispatch_mismatch' | WorkerReportRefusal['code'] reason: string } -function resolveLifecycleAuthority(args: { +function lifecycleRefusal(args: { db: OrchestrationDb dispatch: ReturnType from: string paneKey: string | undefined processIncarnation: string | undefined - capability: string | undefined taskId: string | undefined - capabilityBacked: boolean coordinatorMutation: boolean -}): LifecycleAuthority { - const { - db, - dispatch, - from, - paneKey, - processIncarnation, - capability, - taskId, - capabilityBacked, - coordinatorMutation - } = args +}): LifecycleRefusal | null { + const { db, dispatch, from, paneKey, processIncarnation, taskId, coordinatorMutation } = args if (!dispatch) { - return { - valid: !coordinatorMutation, - code: 'sender_not_assignee', - reason: 'No active Dispatch belongs to this message sender.' - } + return coordinatorMutation + ? { + code: 'sender_not_assignee', + reason: 'No active Dispatch belongs to this message sender.' + } + : null } if (coordinatorMutation && taskId && taskId !== dispatch.task_id) { return { - valid: false, code: 'task_dispatch_mismatch', reason: `Task ${taskId} does not belong to Dispatch ${dispatch.id}.` } } - if (capabilityBacked) { - const authority = db.verifyDispatchCapability({ - dispatchId: dispatch.id, - capability, - paneKey, - processIncarnation - }) - return { - valid: authority.valid, - code: 'dispatch_capability_invalid', - reason: authority.valid ? '' : authority.reason - } - } - if (dispatch.process_incarnation) { - return { - valid: db.isDispatchProcessCurrent({ + const refusal = workerReportRefusal({ + dispatchId: dispatch.id, + from, + workerState: db.getWorkerDispatch(dispatch.id)?.state, + processCurrent: + !dispatch.process_incarnation || + db.isDispatchProcessCurrent({ dispatchId: dispatch.id, paneKey: paneKey ?? null, processIncarnation: processIncarnation ?? null - }), - code: 'sender_not_assignee', - reason: `Dispatch ${dispatch.id} process incarnation is no longer current for its pane.` - } - } - return { - valid: - !coordinatorMutation || - db.isDispatchMessageSender({ - dispatchId: dispatch.id, - handle: from, - paneKey - }), - code: 'sender_not_assignee', - reason: `Terminal ${from} does not own Dispatch ${dispatch.id}.` + }) + }) + if (refusal || dispatch.process_incarnation) { + return refusal } + return !coordinatorMutation || + db.isDispatchMessageSender({ dispatchId: dispatch.id, handle: from, paneKey }) + ? null + : { + code: 'sender_not_assignee', + reason: `Terminal ${from} does not own Dispatch ${dispatch.id}.` + } } diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/send-remote.ts b/src/main/runtime/rpc/methods/orchestration/messaging/send-remote.ts index ffd327a8f151..1f0b4369e941 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/send-remote.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/send-remote.ts @@ -1,4 +1,5 @@ import type { MessageType, OrchestrationDb } from '../../../../orchestration/db' +import type { WorkerDispatchState } from '../../../../orchestration/types' import type { OrcaRuntimeService } from '../../../../orca-runtime' import { OrchestrationError } from '../../../../orchestration/orchestration-error' import { waitForFederatedLifecycleSettlement } from '../../../../orchestration/federation-lifecycle-settlement' @@ -8,11 +9,13 @@ import type { z } from 'zod' import { parseRemoteWorkerPayload } from '../schemas' import type { SendParams } from '../schemas' import { rejectFederatedExplicitTarget } from '../routing' +import { assertWorkerCanReport } from '../../../../orchestration/worker-report-admission' type SendParamsInput = z.infer type RemoteAttachment = { dispatch_id: string + state: WorkerDispatchState protocol_version: number } @@ -24,24 +27,20 @@ export async function sendRemoteMessage(args: { senderPaneKey: string remoteAttachment: RemoteAttachment processIncarnation?: string | null - orchestrationCapability?: string signal?: AbortSignal }): Promise { const { params, runtime, db, from, senderPaneKey, remoteAttachment } = args rejectFederatedExplicitTarget(params) - if ( - !db.verifyRemoteAttachmentAuthority({ + assertWorkerCanReport({ + dispatchId: remoteAttachment.dispatch_id, + from, + workerState: remoteAttachment.state, + processCurrent: db.isRemoteAttachmentProcessCurrent({ dispatchId: remoteAttachment.dispatch_id, - capability: args.orchestrationCapability, paneKey: senderPaneKey, processIncarnation: args.processIncarnation ?? null }) - ) { - throw new OrchestrationError( - 'dispatch_capability_invalid', - 'The remote Dispatch capability or exact worker process is invalid.' - ) - } + }) const type = (params.type ?? 'status') as MessageType const payload = parseRemoteWorkerPayload(params.payload) diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/send.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/send.test.ts index bb57df1a6f59..6d4e1e1c19b3 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/send.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/send.test.ts @@ -254,8 +254,8 @@ describe('orchestration RPC methods', () => { expect(rejected.lifecycle).toEqual({ action: 'rejected', - code: 'sender_not_assignee', - reason: `Dispatch ${dispatch.id} process incarnation is no longer current for its pane.` + code: 'worker_identity_changed', + reason: `term_worker is not the exact process that owns Dispatch ${dispatch.id}.` }) expect(db.getTask(task.id)?.status).toBe('dispatched') @@ -269,6 +269,36 @@ describe('orchestration RPC methods', () => { expect(accepted.lifecycle.action).toBe('completed') }) + it('rejects a current process whose terminal now resolves to another pane', async () => { + setup() + const task = db.createTask({ spec: 'pane-bound manual work' }) + const dispatch = createRootDispatch( + db, + task.id, + 'term_worker', + 'tab_worker:leaf_worker', + undefined, + 'runtime_test:term_worker:1' + ) + vi.mocked(runtime.getTerminalPaneKey).mockImplementation((handle) => + handle === 'term_worker' ? 'tab_foreign:leaf_foreign' : coordinatorPaneKey + ) + + expect( + await call('orchestration.send', { + from: 'term_worker', + subject: 'Done', + type: 'worker_done', + payload: JSON.stringify({ + taskId: task.id, + dispatchId: dispatch.id, + outcome: 'succeeded' + }) + }) + ).toMatchObject({ lifecycle: { action: 'rejected', code: 'worker_identity_changed' } }) + expect(db.getTask(task.id)?.status).toBe('dispatched') + }) + it.each(['escalation', 'decision_gate'] as const)( 'fences a replacement process from a %s mutation', async (type) => { @@ -304,7 +334,7 @@ describe('orchestration RPC methods', () => { expect(rejected.lifecycle).toMatchObject({ action: 'rejected', - code: 'sender_not_assignee' + code: 'worker_identity_changed' }) expect(db.getTask(task.id)?.status).toBe('dispatched') expect(db.getDispatchContextById(dispatch.id)?.status).toBe('dispatched') @@ -389,88 +419,6 @@ describe('orchestration RPC methods', () => { expect(runtime.notifyMessageArrived).toHaveBeenCalledWith(`run:${activeRunId}`, 'worker_done') }) - it('requires the minted capability, exact pane, and process incarnation', async () => { - setup() - const task = db.createTask({ spec: 'capability work' }) - const dispatch = createRootDispatch(db, task.id, 'term_worker', 'tab_worker:leaf_worker') - const capability = db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: 'tab_worker:leaf_worker', - processIncarnation: 'runtime_test:term_worker:1' - }) - vi.spyOn(runtime, 'getTerminalPaneKey').mockImplementation((handle) => - handle === 'term_worker' ? 'tab_worker:leaf_worker' : coordinatorPaneKey - ) - const payload = JSON.stringify({ - taskId: task.id, - dispatchId: dispatch.id, - outcome: 'succeeded' - }) - - const rejected = (await call('orchestration.send', { - from: 'term_worker', - subject: 'Done', - type: 'worker_done', - payload - })) as { lifecycle: { code: string }; message: { subject: string } } - expect(rejected).toMatchObject({ - lifecycle: { code: 'dispatch_capability_invalid' }, - message: { subject: 'Rejected worker_done: Done' } - }) - expect(db.getTask(task.id)?.status).toBe('dispatched') - - ctx = { runtime, orchestrationCapability: 'dcap_wrong' } - const wrongToken = (await call('orchestration.send', { - from: 'term_worker', - subject: 'Done', - type: 'worker_done', - payload - })) as { lifecycle: { code: string } } - expect(wrongToken.lifecycle.code).toBe('dispatch_capability_invalid') - - ctx = { runtime, orchestrationCapability: capability } - vi.mocked(runtime.getTerminalPaneKey).mockImplementation((handle) => - handle === 'term_worker' ? 'tab_foreign:leaf_foreign' : coordinatorPaneKey - ) - const wrongPane = (await call('orchestration.send', { - from: 'term_worker', - subject: 'Done', - type: 'worker_done', - payload - })) as { lifecycle: { code: string } } - expect(wrongPane.lifecycle.code).toBe('dispatch_capability_invalid') - - vi.mocked(runtime.getTerminalPaneKey).mockImplementation((handle) => - handle === 'term_worker' ? 'tab_worker:leaf_worker' : coordinatorPaneKey - ) - vi.mocked(runtime.getTerminalProcessIncarnation).mockReturnValue('runtime_test:term_worker:2') - const wrongProcess = (await call('orchestration.send', { - from: 'term_worker', - subject: 'Done', - type: 'worker_done', - payload - })) as { lifecycle: { code: string } } - expect(wrongProcess.lifecycle.code).toBe('dispatch_capability_invalid') - - vi.mocked(runtime.getTerminalProcessIncarnation).mockReturnValue('runtime_test:term_worker:1') - await call('orchestration.send', { - from: 'term_worker', - subject: 'Done', - type: 'worker_done', - payload - }) - expect(db.getTask(task.id)?.status).toBe('completed') - expect(db.getDispatchContextById(dispatch.id)?.capability_revoked_at).toBeTruthy() - - const revoked = (await call('orchestration.send', { - from: 'term_worker', - subject: 'Done again', - type: 'worker_done', - payload - })) as { lifecycle: { code: string } } - expect(revoked.lifecycle.code).toBe('dispatch_capability_invalid') - }) - it('does not wake waiters for a heartbeat suppressed at send time', async () => { setup() const task = db.createTask({ spec: 'work' }) diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.ts new file mode 100644 index 000000000000..6c311baf84d1 --- /dev/null +++ b/src/main/runtime/rpc/methods/orchestration/messaging/worker-report-authority.test.ts @@ -0,0 +1,253 @@ +import { afterEach, describe, expect, it, vi } from 'vitest' +import type { RpcContext } from '../../../core' +import type { OrchestrationDb } from '../../../../orchestration/db' +import type { OrcaRuntimeService } from '../../../../orca-runtime' +import type { OrchestrationCompatibilityEvidence } from '../../../../../../shared/orchestration-compatibility-evidence' +import { createOrchestrationRpcHarness } from '../rpc-test-harness' + +const WORKER_PANE = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' +const OTHER_WORKER_PANE = 'tab_other:cccccccc-cccc-4ccc-8ccc-cccccccccccc' +const NON_PARTY_PANE = 'tab_teammate:dddddddd-dddd-4ddd-8ddd-dddddddddddd' +const WORKER_PROCESS = 'runtime_test:term_worker:1' + +describe('worker report authority without a Dispatch capability', () => { + const harness = createOrchestrationRpcHarness() + let db: OrchestrationDb + let runtime: OrcaRuntimeService + let ctx: RpcContext + let panes: Record + + afterEach(() => harness.cleanup()) + + function setup(): void { + ;({ db, runtime } = harness.setup()) + ctx = { runtime } + panes = { + term_coord: harness.coordinatorPaneKey, + term_worker: WORKER_PANE, + term_other: OTHER_WORKER_PANE, + term_teammate: NON_PARTY_PANE + } + vi.mocked(runtime.getTerminalPaneKey).mockImplementation((handle) => panes[handle] ?? null) + vi.spyOn(runtime, 'getTerminalHandleForPaneKey').mockImplementation( + (paneKey) => Object.keys(panes).find((handle) => panes[handle] === paneKey) ?? null + ) + vi.spyOn(runtime, 'notifyMessageArrived').mockImplementation(() => {}) + vi.spyOn(runtime, 'waitForMessage').mockResolvedValue('timed_out') + } + + function startWorker(name: string, paneKey: string): { taskId: string; dispatchId: string } { + const task = db.createTask({ spec: `${name} work` }) + const started = db.createStartingWorkerDispatch({ + creator: { kind: 'system' }, + maxDepth: Number.MAX_SAFE_INTEGER, + taskId: task.id, + startOptions: {} + }) + db.prepareStartingWorkerAuthority({ + dispatchId: started.dispatch.id, + handle: `term_${name}`, + paneKey, + processIncarnation: `runtime_test:term_${name}:1`, + worktreeId: `repo::${name}`, + effects: [], + setupState: 'not_applicable', + terminalOwnership: 'created' + }) + db.markWorkerDispatchReady(started.dispatch.id) + return { taskId: task.id, dispatchId: started.dispatch.id } + } + + function workerDone( + worker: { taskId: string; dispatchId: string }, + evidence?: OrchestrationCompatibilityEvidence, + outcome = 'succeeded' + ) { + return harness.call( + 'orchestration.send', + { + from: 'term_worker', + subject: 'Done', + type: 'worker_done', + payload: JSON.stringify({ ...worker, outcome }) + }, + { ...ctx, orchestrationCompatibilityEvidence: evidence } + ) + } + + function ask(evidence?: OrchestrationCompatibilityEvidence) { + return harness.call( + 'orchestration.ask', + { from: 'term_worker', question: 'Proceed?', timeoutMs: 1 }, + { ...ctx, orchestrationCompatibilityEvidence: evidence } + ) + } + + describe('caller fence', () => { + it.each([ + ['the Run coordinator', harness.coordinatorPaneKey], + ['another Dispatch worker', OTHER_WORKER_PANE] + ])('refuses a report sent from %s terminal and records nothing', async (_party, paneKey) => { + setup() + startWorker('other', OTHER_WORKER_PANE) + const worker = startWorker('worker', WORKER_PANE) + + await expect(workerDone(worker, { paneKey })).rejects.toMatchObject({ + code: 'consumer_fenced', + data: { effectsApplied: false } + }) + await expect(ask({ paneKey })).rejects.toMatchObject({ code: 'consumer_fenced' }) + expect( + db.db + .prepare("SELECT COUNT(*) AS count FROM messages WHERE from_handle = 'term_worker'") + .get() + ).toEqual({ count: 0 }) + expect(db.getTask(worker.taskId)?.status).toBe('dispatched') + }) + + it.each<[string, OrchestrationCompatibilityEvidence | undefined]>([ + ['no identity env (old CLI, scrubbed env)', undefined], + ['its own pane', { paneKey: WORKER_PANE, terminalHandle: 'term_stale_after_remint' }], + ["another Orca's pane", { paneKey: 'tab_elsewhere:eeeeeeee-eeee-4eee-8eee-eeeeeeeeeeee' }], + ['a stale handle', { terminalHandle: 'term_gone' }], + ['a live pane that is no orchestration party', { paneKey: NON_PARTY_PANE }] + ])('accepts a report whose env names %s', async (_case, evidence) => { + setup() + const worker = startWorker('worker', WORKER_PANE) + + expect(await workerDone(worker, evidence)).toMatchObject({ + lifecycle: { action: 'completed' } + }) + expect(db.getTask(worker.taskId)?.status).toBe('completed') + }) + + it('passes the fence for a stale --from handle and leaves refusal to the process check', async () => { + setup() + const worker = startWorker('worker', WORKER_PANE) + panes = { ...panes, term_worker_reminted: WORKER_PANE } + delete panes.term_worker + + expect(await workerDone(worker, { paneKey: WORKER_PANE })).toMatchObject({ + lifecycle: { action: 'rejected', code: 'worker_identity_changed' } + }) + }) + }) + + describe('worker states', () => { + it('refuses reports and questions while a stop is in flight', async () => { + setup() + const worker = startWorker('worker', WORKER_PANE) + db.beginWorkerStop(worker.dispatchId, 'epoch_home') + + expect(await workerDone(worker)).toMatchObject({ + lifecycle: { action: 'rejected', code: 'dispatch_inactive' } + }) + await expect(ask()).rejects.toMatchObject({ code: 'dispatch_inactive' }) + expect(db.getWorkerDispatch(worker.dispatchId)?.state).toBe('stopping') + }) + + it.each([ + ['succeeded', 'completed', 'succeeded'], + ['failed', 'failed', 'failed'] + ])( + 'settles a %s report after a stop whose outcome is unknown', + async (outcome, taskStatus, workerState) => { + setup() + const worker = startWorker('worker', WORKER_PANE) + db.beginWorkerStop(worker.dispatchId, 'epoch_home') + db.markWorkerStopUnknown(worker.dispatchId, 'tab not owned by Orca') + + expect(await workerDone(worker, undefined, outcome)).toMatchObject({ + lifecycle: { action: taskStatus } + }) + expect(db.getTask(worker.taskId)?.status).toBe(taskStatus) + expect(db.getWorkerDispatch(worker.dispatchId)).toMatchObject({ + state: workerState, + ...(outcome === 'succeeded' ? { last_error: null } : {}) + }) + } + ) + + it('refuses a stale process in the worker pane after an unknown stop', async () => { + setup() + const worker = startWorker('worker', WORKER_PANE) + db.beginWorkerStop(worker.dispatchId, 'epoch_home') + db.markWorkerStopUnknown(worker.dispatchId, 'tab not owned by Orca') + vi.mocked(runtime.getTerminalProcessIncarnation).mockReturnValue('runtime_test:term_worker:2') + + expect(await workerDone(worker)).toMatchObject({ + lifecycle: { code: 'worker_identity_changed' } + }) + await expect(ask()).rejects.toMatchObject({ code: 'worker_identity_changed' }) + }) + }) + + describe('remote worker states', () => { + function startRemoteWorker(): string { + const dispatchId = 'ctx_remote_worker' + db.createRemoteDispatchAttachment({ + runId: 'run-home', + dispatchId, + taskId: 'task_remote_worker', + homePeerFingerprint: 'run-home-device', + protocolVersion: 1, + runtimeEpoch: 'epoch_worker_host', + mutationReceipt: { + callerFingerprint: 'run-home-device', + requestId: 'remote_attach', + method: 'orchestration.federationAttachStart', + payloadHash: 'remote_attach_payload' + } + }) + db.prepareRemoteAttachmentAuthority({ + dispatchId, + paneKey: WORKER_PANE, + processIncarnation: WORKER_PROCESS, + worktreeId: 'repo::remote', + terminalHandle: 'term_worker', + setupState: 'completed', + effects: [] + }) + return dispatchId + } + + it.each([ + [ + 'stop_unknown', + (dispatchId: string) => { + db.markRemoteAttachmentReady(dispatchId) + db.beginRemoteAttachmentStop(dispatchId) + db.markRemoteAttachmentStopUnknown(dispatchId, 'tab not owned by Orca') + } + ], + [ + 'start_unknown', + (dispatchId: string) => + db.failRemoteAttachment(dispatchId, 'agent_readiness', 'connection lost', true) + ] + ])('settles a remote report from %s', async (state, reachState) => { + setup() + const dispatchId = startRemoteWorker() + reachState(dispatchId) + expect(db.getRemoteDispatchAttachment(dispatchId)?.state).toBe(state) + + expect(await workerDone({ taskId: 'task_remote_worker', dispatchId })).toMatchObject({ + lifecycle: { action: 'completed' } + }) + expect(db.getRemoteDispatchAttachment(dispatchId)?.state).toBe('succeeded') + }) + + it('refuses a remote report while the stop is in flight', async () => { + setup() + const dispatchId = startRemoteWorker() + db.markRemoteAttachmentReady(dispatchId) + db.beginRemoteAttachmentStop(dispatchId) + + await expect(workerDone({ taskId: 'task_remote_worker', dispatchId })).rejects.toMatchObject({ + code: 'dispatch_inactive' + }) + await expect(ask()).rejects.toMatchObject({ code: 'dispatch_inactive' }) + expect(db.listPendingFederationRelay(dispatchId, 'to_home')).toEqual([]) + }) + }) +}) diff --git a/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts b/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts index 4172097853d0..d3cf0769e847 100644 --- a/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts +++ b/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts @@ -339,8 +339,8 @@ describe('orchestration runtime update settlement', () => { result: { lifecycle: { action: 'rejected', - code: 'dispatch_capability_invalid', - reason: 'The caller is not the Dispatch pane.' + code: 'worker_identity_changed', + reason: `term_pre_update_worker is not the exact process that owns Dispatch ${harness.dispatchId}.` } } }) @@ -392,11 +392,12 @@ describe('orchestration runtime update settlement', () => { const attachment = { dispatch_id: 'dispatch-remote-retained', task_id: 'task-remote-retained', + state: 'ready', process_incarnation: CURRENT_COORDINATOR_PROCESS_INCARNATION } vi.spyOn(harness.db, 'findActiveRemoteAttachmentForPane').mockReturnValue(attachment as never) - const verifyAuthority = vi - .spyOn(harness.db, 'verifyRemoteAttachmentAuthority') + const processCurrent = vi + .spyOn(harness.db, 'isRemoteAttachmentProcessCurrent') .mockReturnValue(true) vi.spyOn(harness.db, 'enqueueFederationRelay').mockReturnValue({ message_id: 'relay-retained-status', @@ -422,9 +423,8 @@ describe('orchestration runtime update settlement', () => { expect(resultOf(response)).toMatchObject({ relay: { dispatchId: attachment.dispatch_id, accepted: true } }) - expect(verifyAuthority).toHaveBeenCalledWith({ + expect(processCurrent).toHaveBeenCalledWith({ dispatchId: attachment.dispatch_id, - capability: 'dcap_remote_retained', paneKey: CURRENT_COORDINATOR_PANE, processIncarnation: CURRENT_COORDINATOR_PROCESS_INCARNATION }) diff --git a/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts b/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts index 0d194a372bae..2c8e84f15749 100644 --- a/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts +++ b/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts @@ -94,7 +94,7 @@ describe('Task/Dispatch state invariant', () => { it('atomically fails a context-only Dispatch, revokes its capability, and frees the terminal', async () => { const harness = createHarness() const task = harness.db.createTask({ spec: 'failing assignment', runId: harness.runId }) - const { dispatch, capability } = await createCapableDispatch(harness, task.id, 'dispatched') + const { dispatch } = await createCapableDispatch(harness, task.id, 'dispatched') const response = await updateTask(harness, task.id, 'failed', 'coordinator stopped work') @@ -107,14 +107,6 @@ describe('Task/Dispatch state invariant', () => { dispatchCompletedAt: expect.any(String), capabilityRevokedAt: expect.any(String) }) - expect( - harness.db.verifyDispatchCapability({ - dispatchId: dispatch.id, - capability, - paneKey: WORKER_PANE, - processIncarnation: WORKER_PROCESS - }) - ).toEqual({ valid: false, reason: `Dispatch ${dispatch.id} capability is revoked.` }) const laterTask = harness.db.createTask({ spec: 'later assignment', runId: harness.runId }) await expect(dispatchTask(harness, laterTask.id, WORKER_HANDLE)).resolves.toMatchObject({ @@ -132,7 +124,7 @@ describe('Task/Dispatch state invariant', () => { async (dispatchStatus, taskStatus) => { const harness = createHarness() const task = harness.db.createTask({ spec: 'supervised assignment', runId: harness.runId }) - const { dispatch, capability } = createSupervisedDispatch(harness, task.id, dispatchStatus) + const { dispatch } = createSupervisedDispatch(harness, task.id, dispatchStatus) const response = await updateTask(harness, task.id, taskStatus, 'must not persist') @@ -156,14 +148,6 @@ describe('Task/Dispatch state invariant', () => { ownershipState: 'owned', releaseState: 'not_requested' }) - expect( - harness.db.verifyDispatchCapability({ - dispatchId: dispatch.id, - capability, - paneKey: WORKER_PANE, - processIncarnation: WORKER_PROCESS - }) - ).toEqual({ valid: true }) } ) diff --git a/src/main/runtime/structured-worker-identity.ts b/src/main/runtime/structured-worker-identity.ts index 22a49e11abb2..ababbfbd86cd 100644 --- a/src/main/runtime/structured-worker-identity.ts +++ b/src/main/runtime/structured-worker-identity.ts @@ -108,7 +108,7 @@ function persistedStructuredWorkerPaneKeyIsValid( * * NOT the runtime fence: the fence is an owner-generation counter that the host bumps during its * own transparent crash recovery, so fencing identity on it would make a recovered — but same — - * worker fail `verifyDispatchCapability` forever and wedge release as `identity_unproven`. The + * worker fail its process check forever and wedge release as `identity_unproven`. The * session id is minted once per dispatch and survives that recovery, so it is the lineage. */ export function structuredWorkerProcessIncarnation(sessionId: string): string { From 46d6b76ae87abc39e03b2d5027e6a8d1604d4cdc Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Wed, 30 Sep 2026 13:04:57 -0400 Subject: [PATCH 10/79] fix(orchestration): retry worker_done while the Orca runtime is briefly unreachable (#23984) * fix(orchestration): retry worker_done while the Orca runtime is briefly unreachable A worker reports worker_done once and ends its turn, so a few-minute app outage silently stranded finished work at dispatched. The CLI now retries worker_done on runtime_unavailable for about two minutes with backoff, reusing one request id so the host's mutation ledger replays rather than double-applies it, then prints the existing recovery command. The contract probe no longer caches a failed status.get, which otherwise made every retry fail without reaching the app. Part of STA-8833. * refactor(cli): pass the worker_done retry window in the mutation options bag * Revert "refactor(cli): pass the worker_done retry window in the mutation options bag" The options bag is forwarded to client.call as-is; the retry window is not a client.call option, and folding it in needed a value scan to keep the no-options call shape. * test(cli): fold the explicit retry-request case and drop a vacuous timing assert * fix(cli): keep worker_done recovery when the last retry fails before sending Also skip the Unix-socket retry test on Windows and remove its temp profile. --- .../handlers/orchestration-migration.test.ts | 35 ++- src/cli/handlers/orchestration.test.ts | 104 ++++---- .../orchestration/message-send-handler.ts | 6 +- .../orchestration/mutation-request.test.ts | 240 ++++++++++++++++++ .../orchestration/mutation-request.ts | 59 ++++- src/cli/runtime/client.ts | 8 +- 6 files changed, 380 insertions(+), 72 deletions(-) create mode 100644 src/cli/handlers/orchestration/mutation-request.test.ts diff --git a/src/cli/handlers/orchestration-migration.test.ts b/src/cli/handlers/orchestration-migration.test.ts index 863720eaab12..84c1038e3d76 100644 --- a/src/cli/handlers/orchestration-migration.test.ts +++ b/src/cli/handlers/orchestration-migration.test.ts @@ -1,6 +1,9 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import { ORCHESTRATION_HANDLERS } from './orchestration' +// worker_done carries a CLI-minted request id so its runtime_unavailable retries replay one mutation. +const WORKER_DONE_REQUEST = { orchestrationRequestId: expect.any(String) } + const originalPaneKey = process.env.ORCA_PANE_KEY afterEach(() => { @@ -34,20 +37,24 @@ describe('orchestration CLI migration recovery', () => { json: true } as never) - expect(call).toHaveBeenCalledWith('orchestration.send', { - from: 'term_worker', - to: undefined, - run: undefined, - subject: 'Done', - body: undefined, - type: 'worker_done', - priority: undefined, - threadId: undefined, - payload: undefined, - senderPaneKey: 'tab-worker:leaf-worker', - waitForLifecycleSettlement: true, - devMode: false - }) + expect(call).toHaveBeenCalledWith( + 'orchestration.send', + { + from: 'term_worker', + to: undefined, + run: undefined, + subject: 'Done', + body: undefined, + type: 'worker_done', + priority: undefined, + threadId: undefined, + payload: undefined, + senderPaneKey: 'tab-worker:leaf-worker', + waitForLifecycleSettlement: true, + devMode: false + }, + WORKER_DONE_REQUEST + ) expect(call).toHaveBeenCalledOnce() } ) diff --git a/src/cli/handlers/orchestration.test.ts b/src/cli/handlers/orchestration.test.ts index e285f431ad40..4ca66b603eb4 100644 --- a/src/cli/handlers/orchestration.test.ts +++ b/src/cli/handlers/orchestration.test.ts @@ -1,6 +1,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' const callMock = vi.fn() +// worker_done carries a CLI-minted request id so its runtime_unavailable retries replay one mutation. +const WORKER_DONE_REQUEST = { orchestrationRequestId: expect.any(String) } const getTerminalHandleMock = vi.hoisted(() => vi.fn()) const originalTerminalHandle = process.env.ORCA_TERMINAL_HANDLE const originalPaneKey = process.env.ORCA_PANE_KEY @@ -79,24 +81,28 @@ describe('orchestration send structured payload flags', () => { ]) ) - expect(callMock).toHaveBeenCalledWith('orchestration.send', { - from: 'term_worker', - to: 'term_coord', - subject: 'done', - body: undefined, - type: 'worker_done', - priority: undefined, - threadId: undefined, - payload: JSON.stringify({ - taskId: 'task_1', - dispatchId: 'ctx_1', - outcome: 'succeeded', - filesModified: ['src/a.ts', 'src/b.ts'], - reportPath: 'reports/done.md' - }), - waitForLifecycleSettlement: true, - devMode: false - }) + expect(callMock).toHaveBeenCalledWith( + 'orchestration.send', + { + from: 'term_worker', + to: 'term_coord', + subject: 'done', + body: undefined, + type: 'worker_done', + priority: undefined, + threadId: undefined, + payload: JSON.stringify({ + taskId: 'task_1', + dispatchId: 'ctx_1', + outcome: 'succeeded', + filesModified: ['src/a.ts', 'src/b.ts'], + reportPath: 'reports/done.md' + }), + waitForLifecycleSettlement: true, + devMode: false + }, + WORKER_DONE_REQUEST + ) }) it('forwards multiline message bodies without normalization', async () => { @@ -202,18 +208,22 @@ describe('orchestration send structured payload flags', () => { ]) ) - expect(callMock).toHaveBeenCalledWith('orchestration.send', { - from: 'term_worker', - to: 'term_coord', - subject: 'done', - body: undefined, - type: 'worker_done', - priority: undefined, - threadId: undefined, - payload: JSON.stringify({ outcome: 'succeeded' }), - waitForLifecycleSettlement: true, - devMode: false - }) + expect(callMock).toHaveBeenCalledWith( + 'orchestration.send', + { + from: 'term_worker', + to: 'term_coord', + subject: 'done', + body: undefined, + type: 'worker_done', + priority: undefined, + threadId: undefined, + payload: JSON.stringify({ outcome: 'succeeded' }), + waitForLifecycleSettlement: true, + devMode: false + }, + WORKER_DONE_REQUEST + ) }) it('sends lifecycle messages from ORCA_TERMINAL_HANDLE without a liveness probe', async () => { @@ -229,18 +239,22 @@ describe('orchestration send structured payload flags', () => { ) expect(callMock).toHaveBeenCalledTimes(1) - expect(callMock).toHaveBeenCalledWith('orchestration.send', { - from: 'term_worker_env', - to: 'term_coord', - subject: 'done', - body: undefined, - type: 'worker_done', - priority: undefined, - threadId: undefined, - payload: JSON.stringify({ outcome: 'succeeded' }), - waitForLifecycleSettlement: true, - devMode: false - }) + expect(callMock).toHaveBeenCalledWith( + 'orchestration.send', + { + from: 'term_worker_env', + to: 'term_coord', + subject: 'done', + body: undefined, + type: 'worker_done', + priority: undefined, + threadId: undefined, + payload: JSON.stringify({ outcome: 'succeeded' }), + waitForLifecycleSettlement: true, + devMode: false + }, + WORKER_DONE_REQUEST + ) }) it.each(['worker_done', 'heartbeat'] as const)( @@ -265,7 +279,8 @@ describe('orchestration send structured payload flags', () => { expect(callMock).toHaveBeenCalledTimes(1) expect(callMock).toHaveBeenCalledWith( 'orchestration.send', - expect.objectContaining({ from: 'term_worker_env' }) + expect.objectContaining({ from: 'term_worker_env' }), + ...(type === 'worker_done' ? [WORKER_DONE_REQUEST] : []) ) } ) @@ -285,7 +300,8 @@ describe('orchestration send structured payload flags', () => { expect(callMock).toHaveBeenCalledWith( 'orchestration.send', - expect.objectContaining({ senderPaneKey: 'tab_worker:leaf_worker' }) + expect.objectContaining({ senderPaneKey: 'tab_worker:leaf_worker' }), + WORKER_DONE_REQUEST ) }) diff --git a/src/cli/handlers/orchestration/message-send-handler.ts b/src/cli/handlers/orchestration/message-send-handler.ts index aa9bbaff47eb..3b024de2cffa 100644 --- a/src/cli/handlers/orchestration/message-send-handler.ts +++ b/src/cli/handlers/orchestration/message-send-handler.ts @@ -12,6 +12,8 @@ import { throwNoActiveSenderTerminal } from './terminal-identity' +const WORKER_DONE_UNAVAILABLE_RETRY_MS = 120_000 + type LifecycleSendResult = | { action: 'completed' | 'failed' @@ -110,7 +112,9 @@ export const ORCHESTRATION_SEND_HANDLER: Record = { flags, 'orchestration.send', sendParams, - dispatchCapability ? { orchestrationCapability: dispatchCapability } : undefined + dispatchCapability ? { orchestrationCapability: dispatchCapability } : undefined, + // Why: a worker reports once and ends its turn, so a brief app outage must delay worker_done, not drop it. + type === 'worker_done' ? WORKER_DONE_UNAVAILABLE_RETRY_MS : 0 ) await requireWorkerDoneSettlement(client, type, sendParams.payload, result.result) if ('lifecycle' in result.result && result.result.lifecycle?.action === 'rejected') { diff --git a/src/cli/handlers/orchestration/mutation-request.test.ts b/src/cli/handlers/orchestration/mutation-request.test.ts new file mode 100644 index 000000000000..24d98e75357d --- /dev/null +++ b/src/cli/handlers/orchestration/mutation-request.test.ts @@ -0,0 +1,240 @@ +import { createServer, type Server } from 'node:net' +import { mkdtempSync, rmSync, writeFileSync } from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, describe, expect, it, vi } from 'vitest' +import { z } from 'zod' +import { ORCHESTRATION_CONTRACT_RUNTIME_CAPABILITY } from '../../../shared/protocol-version' +import { RuntimeClient, RuntimeClientError } from '../../runtime-client' +import { callOrchestrationMutation } from './mutation-request' + +const RETRY_MS = 120_000 +const WORKER_DONE = { + from: 'term_worker', + subject: 'done', + type: 'worker_done', + payload: '{"taskId":"task_1","dispatchId":"ctx_1","outcome":"succeeded"}' +} + +const RuntimeRequest = z.object({ + id: z.string(), + method: z.string(), + orchestrationRequestId: z.string().optional() +}) + +const servers = new Set() +const tempDirs = new Set() + +afterEach(async () => { + vi.useRealTimers() + await Promise.all([...servers].map((server) => new Promise((resolve) => server.close(resolve)))) + servers.clear() + for (const dir of tempDirs) { + rmSync(dir, { recursive: true, force: true }) + } + tempDirs.clear() +}) + +type CallOptions = { orchestrationRequestId?: string } + +function fakeClient(respond: (attempt: number, options?: CallOptions) => unknown): { + client: RuntimeClient + requestIds: (string | undefined)[] +} { + const requestIds: (string | undefined)[] = [] + const call = vi.fn(async (_method: string, _params: unknown, options?: CallOptions) => { + requestIds.push(options?.orchestrationRequestId) + return respond(requestIds.length, options) + }) + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: callOrchestrationMutation only uses client.call. + return { client: { call } as unknown as RuntimeClient, requestIds } +} + +function unavailable(options?: CallOptions): RuntimeClientError { + return new RuntimeClientError( + 'runtime_unavailable', + 'Could not connect to the running Orca app.', + { + orchestrationRequestId: options?.orchestrationRequestId, + originalCommand: ['orca', 'orchestration', 'send', '--type', 'worker_done'] + } + ) +} + +describe('callOrchestrationMutation runtime_unavailable retry', () => { + it.each([ + ['a fresh', new Map(), expect.stringMatching(/^[0-9a-f-]{36}$/)], + [ + 'an explicit --retry-request', + new Map([['retry-request', '11111111-2222-4333-8444-555555555555']]), + '11111111-2222-4333-8444-555555555555' + ] + ])('retries with %s request id until the runtime answers', async (_case, flags, requestId) => { + vi.useFakeTimers() + const { client, requestIds } = fakeClient((attempt, options) => { + if (attempt < 3) { + throw unavailable(options) + } + return { ok: true, result: 'sent' } + }) + const call = callOrchestrationMutation( + client, + flags, + 'orchestration.send', + WORKER_DONE, + undefined, + RETRY_MS + ) + await vi.advanceTimersByTimeAsync(3_000) + await expect(call).resolves.toEqual({ ok: true, result: 'sent' }) + expect(requestIds).toHaveLength(3) + expect(new Set(requestIds).size).toBe(1) + expect(requestIds[0]).toEqual(requestId) + }) + + it('does not retry an error other than runtime_unavailable', async () => { + const { client, requestIds } = fakeClient(() => { + throw new RuntimeClientError('runtime_timeout', 'Timed out.') + }) + await expect( + callOrchestrationMutation( + client, + new Map(), + 'orchestration.send', + WORKER_DONE, + undefined, + RETRY_MS + ) + ).rejects.toMatchObject({ code: 'runtime_timeout' }) + expect(requestIds).toHaveLength(1) + }) + + it('does not retry mutations that did not opt in', async () => { + const { client, requestIds } = fakeClient((_attempt, options) => { + throw unavailable(options) + }) + await expect( + callOrchestrationMutation(client, new Map(), 'orchestration.send', WORKER_DONE) + ).rejects.toMatchObject({ code: 'runtime_unavailable' }) + expect(requestIds).toEqual([undefined]) + }) + + it('keeps the recovery command when the last attempt fails before its request id is attached', async () => { + vi.useFakeTimers() + const { client, requestIds } = fakeClient((attempt, options) => { + throw attempt === 1 + ? unavailable(options) + : new RuntimeClientError('runtime_unavailable', 'No runtime metadata.') + }) + const settled = callOrchestrationMutation( + client, + new Map(), + 'orchestration.send', + WORKER_DONE, + undefined, + RETRY_MS + ).catch((error: unknown) => error) + await vi.advanceTimersByTimeAsync(RETRY_MS + 30_000) + expect(await settled).toMatchObject({ + data: { recovery: { orchestrationRequestId: requestIds[0] } } + }) + }) + + it('gives up after about two minutes and prints the recovery command with the same id', async () => { + vi.useFakeTimers() + const { client, requestIds } = fakeClient((_attempt, options) => { + throw unavailable(options) + }) + const call = callOrchestrationMutation( + client, + new Map(), + 'orchestration.send', + WORKER_DONE, + undefined, + RETRY_MS + ) + const settled = call.catch((error: unknown) => error) + await vi.advanceTimersByTimeAsync(RETRY_MS + 30_000) + const error = await settled + expect(requestIds.length).toBeGreaterThan(5) + expect(new Set(requestIds).size).toBe(1) + expect(error).toMatchObject({ + code: 'runtime_unavailable', + data: { + recovery: { + retryCommand: [ + 'orca', + 'orchestration', + 'send', + '--type', + 'worker_done', + '--retry-request', + requestIds[0] + ] + } + } + }) + }) + + // Why: the fake runtime listens on a Unix socket path, which Windows does not accept. + it.skipIf(process.platform === 'win32')( + 'reaches a runtime that comes back after the first attempt found it down', + async () => { + const userDataPath = mkdtempSync(join(tmpdir(), 'orca-worker-done-retry-')) + tempDirs.add(userDataPath) + const endpoint = join(userDataPath, 'runtime.sock') + const sendRequestIds: string[] = [] + const server = createServer((socket) => { + socket.setEncoding('utf8') + socket.once('data', (line: string) => { + const request = RuntimeRequest.parse(JSON.parse(line.trim())) + if (request.method === 'orchestration.send') { + sendRequestIds.push(String(request.orchestrationRequestId)) + if (sendRequestIds.length === 1) { + // A runtime that drops the connection mid-request leaves the outcome unknown. + socket.destroy() + return + } + } + socket.end( + `${JSON.stringify({ + id: request.id, + ok: true, + result: + request.method === 'status.get' + ? { capabilities: [ORCHESTRATION_CONTRACT_RUNTIME_CAPABILITY] } + : { message: { id: 'msg_1' } }, + _meta: { runtimeId: 'runtime-1' } + })}\n` + ) + }) + }) + servers.add(server) + const client = new RuntimeClient(userDataPath, 5_000, null, null, 'orca') + const call = callOrchestrationMutation( + client, + new Map(), + 'orchestration.send', + WORKER_DONE, + undefined, + RETRY_MS + ) + // The runtime is down for the first attempt: no metadata, so even the contract probe fails. + await new Promise((resolve) => server.listen(endpoint, resolve)) + writeFileSync( + join(userDataPath, 'orca-runtime.json'), + JSON.stringify({ + runtimeId: 'runtime-1', + pid: 1, + transports: [{ kind: 'unix', endpoint }], + authToken: 'token', + startedAt: 1 + }) + ) + await expect(call).resolves.toMatchObject({ result: { message: { id: 'msg_1' } } }) + expect(sendRequestIds).toHaveLength(2) + expect(sendRequestIds[1]).toBe(sendRequestIds[0]) + }, + 15_000 + ) +}) diff --git a/src/cli/handlers/orchestration/mutation-request.ts b/src/cli/handlers/orchestration/mutation-request.ts index c262a81b86c1..2e9836c20efa 100644 --- a/src/cli/handlers/orchestration/mutation-request.ts +++ b/src/cli/handlers/orchestration/mutation-request.ts @@ -1,21 +1,56 @@ -import type { RuntimeClient } from '../../runtime-client' +import { randomUUID } from 'node:crypto' +import { RuntimeClientError, type RuntimeClient } from '../../runtime-client' import { readRetryRequestFlag } from '../../retry-request-flag' import { orchestrationMutationRecoveryError } from '../../orchestration-mutation-recovery' -export function callOrchestrationMutation( +const MAX_UNAVAILABLE_RETRY_DELAY_MS = 15_000 + +export async function callOrchestrationMutation( client: RuntimeClient, flags: Map, method: string, params: unknown, - options?: { timeoutMs?: number; orchestrationCapability?: string } + options?: { timeoutMs?: number; orchestrationCapability?: string }, + unavailableRetryMs = 0 ) { - const requestId = readRetryRequestFlag(flags) - const result = requestId - ? client.call(method, params, { ...options, orchestrationRequestId: requestId }) - : options - ? client.call(method, params, options) - : client.call(method, params) - return result.catch((error) => { - throw orchestrationMutationRecoveryError(error) - }) + // Why: every retry reuses one request id, so the host replays instead of applying the mutation twice. + const requestId = + readRetryRequestFlag(flags) ?? (unavailableRetryMs > 0 ? randomUUID() : undefined) + const deadline = Date.now() + unavailableRetryMs + let sentError: RuntimeClientError | undefined + for (let delayMs = 1_000; ; delayMs = Math.min(delayMs * 2, MAX_UNAVAILABLE_RETRY_DELAY_MS)) { + try { + return requestId + ? await client.call(method, params, { + ...options, + orchestrationRequestId: requestId + }) + : options + ? await client.call(method, params, options) + : await client.call(method, params) + } catch (error) { + const unavailable = + error instanceof RuntimeClientError && error.code === 'runtime_unavailable' + if (carriesRequestId(error)) { + sentError = error + } + if (!unavailable || Date.now() + delayMs > deadline) { + // Why: a later attempt can fail before its request id is attached, though an earlier one may have landed. + throw orchestrationMutationRecoveryError( + carriesRequestId(error) ? error : (sentError ?? error) + ) + } + await new Promise((resolve) => setTimeout(resolve, delayMs)) + } + } +} + +function carriesRequestId(error: unknown): error is RuntimeClientError { + const data: unknown = error instanceof RuntimeClientError ? error.data : undefined + return ( + typeof data === 'object' && + data !== null && + 'orchestrationRequestId' in data && + typeof data.orchestrationRequestId === 'string' + ) } diff --git a/src/cli/runtime/client.ts b/src/cli/runtime/client.ts index 68a099ff2f22..5efe54c7ba35 100644 --- a/src/cli/runtime/client.ts +++ b/src/cli/runtime/client.ts @@ -245,7 +245,13 @@ export class RuntimeClient { private async ensureOrchestrationContractCompatible(timeoutMs: number): Promise { if (!this.orchestrationContractCheck) { - this.orchestrationContractCheck = this.checkOrchestrationContractCompatibility(timeoutMs) + this.orchestrationContractCheck = this.checkOrchestrationContractCompatibility( + timeoutMs + ).catch((error: unknown) => { + // Why: a failed probe must not be cached, or a retry after a brief outage never reaches the app. + this.orchestrationContractCheck = null + throw error + }) } await this.orchestrationContractCheck } From 9afd1101ffd2f1c580a2c0873474613f85f511b8 Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Wed, 30 Sep 2026 13:26:06 -0400 Subject: [PATCH 11/79] fix(orchestration): stop minting and printing the dispatch capability (#23994) * fix(orchestration): authorize worker reports without the dispatch capability Worker lifecycle reports and questions no longer depend on the per-dispatch capability token that lives only in the agent's conversation. The host now: - ignores capability_hash/capability_revoked_at for authorization on every row and checks the exact worker process instead (ask gains that check); - refuses a report whose calling terminal is provably another orchestration party (a Run coordinator or another Dispatch's worker), treating env that names no live pane here as absent; - applies one worker-state rule locally and remotely: a stop in flight refuses, while stop_unknown and start_unknown accept and settle. Minting and printing the flag are unchanged, so an older host and older preambles keep working. * fix(orchestration): stop minting the dispatch capability Dispatches no longer mint a per-Dispatch token, and preambles, the bundled skill guide and the ask resume hint stop printing --dispatch-capability. The consumer-generation bump and delivery fence that minting carried stay, now as setDispatchConsumer. Readers that inferred meaning from capability_hash read what they meant instead: worker-show's injected stage comes from the attached consumer, and a failed start copies custody identity only when no authority was ever attached. The CLI keeps accepting and forwarding the flag for older hosts. Cancelling a Task is recorded as failed with a reason; task-list now shows that reason and the guide and task-update notes document the recipe. * fix(orchestration): name the fenced party without implying which Dispatch it owns * refactor(orchestration): one worker report rule, fence only a different party - One module owns the unproven/settleable worker states and the refusal rule; local send records it, ask and remote throw it. A stale process is worker_identity_changed on every path. - The caller fence passes the worker's own terminal when its --from handle went stale. - Document the shared-tmux-server limit; drop the dead dispatch_capability_invalid rejection member; tests assert dispatch state, not the capability column. * refactor(orchestration): drop setDispatchConsumer and the dead capability retention - dispatch --inject no longer re-points the row createDispatchContext just wrote; worker-show reports every worker-less Dispatch as context_only, since Orca keeps no record of the paste. Tests re-point through a fixture. - failWorkerStart always records when the lifecycle closed; nothing authorizes on it. - Restore the ask resume hint's echo of a passed --dispatch-capability: an old host checks it before --resume. - Move the cancellation convention to #23983. * test(orchestration): drop capability-era assertions other tests already cover * refactor(orchestration): drop the host-side capability field and no-op test fixtures - RpcRequest and the SSH bridge stop carrying orchestrationCapability; the CLI's wire field stays for older hosts. - Fixtures pass identity to createRootDispatch instead of re-pointing to the same values; drop absence checks for a flag that can no longer be produced. * test(orchestration): cover a current process whose terminal moved to another pane * chore(orchestration): finish the capability cleanup in test stubs and skill wording * test(orchestration): drop needless response casts; mark the db stub cast safe --- .../orchestration-skill-guidance.test.mjs | 4 +- skill-guides/orchestration.md | 5 +- .../references/messaging-and-gates.md | 2 +- .../references/worker-contract.md | 13 ++- src/cli/bundled-skill-guides.ts | 8 +- .../orchestration-mutation-recovery.test.ts | 4 +- src/cli/orchestration-mutation-recovery.ts | 4 +- ...dinator-dispatch-unobserved-prompt.test.ts | 3 +- .../db-task-dispatch-invariant.test.ts | 4 +- .../db-task-dispatch-lifecycle-guards.test.ts | 5 +- .../db-task-dispatch-races.test.ts | 14 +-- .../db/attach-orchestration-db-methods.ts | 2 - .../db/dispatch-capability-hash.ts | 5 - .../dispatch-context/dispatch-capability.ts | 69 ----------- .../dispatch-mailbox-consumer-fencing.test.ts | 8 +- .../remote-dispatch-attachment-authority.ts | 9 +- .../db/orchestration-db-methods.ts | 2 - .../db/root-dispatch-test-fixture.ts | 16 +++ .../schema/derived-delivery-migration.test.ts | 4 +- .../worker-dispatch-authority.ts | 11 +- .../worker-dispatch-outcome.ts | 10 +- .../failed-start-dispatch-identity.ts | 4 +- .../orchestration-worker-dispatch-db.test.ts | 3 +- .../runtime/orchestration/preamble.test.ts | 23 +--- src/main/runtime/orchestration/preamble.ts | 12 +- src/main/runtime/orchestration/types.ts | 3 + ...start-unobserved-prompt-settlement.test.ts | 55 ++++++--- src/main/runtime/rpc/core.ts | 3 - src/main/runtime/rpc/dispatcher.ts | 1 - ...on-structured-worker-start-failure.test.ts | 3 +- .../federated-message-targeting.test.ts | 12 +- .../federation-lifecycle-settlement.test.ts | 19 +-- .../federation/federation-output.test.ts | 8 +- .../federation/federation.test.ts | 25 +--- .../orchestration/federation/federation.ts | 3 +- .../check-worker-consumer-fencing.test.ts | 11 +- .../run-bound-mailbox-safety.test.ts | 7 +- .../messaging/send-dispatch-authority.test.ts | 10 -- .../orchestration/runs/dispatch-methods.ts | 8 -- .../worker/composed-workers.test.ts | 2 +- .../deliver-worker-dispatch-preamble.ts | 2 - .../worker/local-worker-start.ts | 3 +- .../manual-dispatch-observation.test.ts | 12 +- .../worker/worker-interactive-wait.test.ts | 5 - .../worker/worker-observation.ts | 3 +- .../worker/worker-start-prompt-budget.test.ts | 2 - .../worker-start-prompt-contract.test.ts | 7 +- .../worker-start-readiness-settlement.test.ts | 1 - .../worker-start-readiness-settlement.ts | 4 +- .../worker/worker-start-receipt.ts | 7 +- .../worker/workers-new-worktree.test.ts | 3 +- ...ion-commit-notify-characterization.test.ts | 13 +-- ...ation-current-authority-precedence.test.ts | 110 ++++++++---------- .../rpc/orchestration-mutation-ledger.test.ts | 15 +-- ...stration-runtime-update-settlement.test.ts | 31 ++--- ...hestration-task-dispatch-invariant.test.ts | 17 +-- .../runtime/rpc/rpc-streaming-dispatcher.ts | 2 - .../structured-chat-coordinator-mail.test.ts | 27 ++--- src/main/ssh/ssh-remote-orca-cli.test.ts | 6 +- src/main/ssh/ssh-remote-orca-cli.ts | 11 +- ...completed-worker-retirement-resume.spec.ts | 39 +++---- .../completed-worker-retirement-fixture.ts | 6 +- ...tion-worker-settlement-release-cli.spec.ts | 7 +- ...ration-worker-transcript-providers.spec.ts | 59 ++++------ ...ettled-worker-tab-survives-restart.spec.ts | 39 +++---- 65 files changed, 316 insertions(+), 529 deletions(-) delete mode 100644 src/main/runtime/orchestration/db/dispatch-capability-hash.ts delete mode 100644 src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts diff --git a/config/scripts/orchestration-skill-guidance.test.mjs b/config/scripts/orchestration-skill-guidance.test.mjs index 2ed4288e2236..522249fe5a4c 100644 --- a/config/scripts/orchestration-skill-guidance.test.mjs +++ b/config/scripts/orchestration-skill-guidance.test.mjs @@ -318,7 +318,7 @@ describe('owned orchestration references', () => { expect(squash(reference)).toContain('An empty `check` never means you were replaced') }) - it('keeps heartbeat and worker_done recipes bound to the injected capability', () => { + it('keeps heartbeat and worker_done recipes bound to the injected Dispatch', () => { const reference = readReference('worker-contract.md') const recipes = [...reference.matchAll(/```text\n([\s\S]*?)```/gu)].map((match) => match[1]) const heartbeat = recipes.find((recipe) => recipe.includes('--type heartbeat')) @@ -326,7 +326,7 @@ describe('owned orchestration references', () => { for (const recipe of [heartbeat, workerDone]) { expect(recipe).toContain('--from ') - expect(recipe).toContain('--dispatch-capability ') + expect(recipe).not.toContain('--dispatch-capability') expect(recipe).toContain('--task-id --dispatch-id ') } expect(workerDone).not.toContain('--files-modified') diff --git a/skill-guides/orchestration.md b/skill-guides/orchestration.md index a7d63ff922cf..0fe7628aa926 100644 --- a/skill-guides/orchestration.md +++ b/skill-guides/orchestration.md @@ -49,8 +49,9 @@ non-Orca subagent tool when Orca orchestration provenance was requested. place workers. A Task is work. A Dispatch is one authoritative Task attempt. - Lifecycle authority comes from the active Dispatch, not a terminal title, copied ID, old database row, provider transcript, or visible pane. -- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID - in the live preamble. Never reconstruct, translate, or broaden those arguments. +- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live + preamble, plus any other flag it carries. Never reconstruct, translate, or + broaden those arguments. - After remote start, address the worker by Dispatch ID. The execution host owns process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts `live` / `unverifiable` / `exited`; contact loss is not process death. diff --git a/skill-guides/orchestration/references/messaging-and-gates.md b/skill-guides/orchestration/references/messaging-and-gates.md index 573ca90e5e79..a05ccf7786cf 100644 --- a/skill-guides/orchestration/references/messaging-and-gates.md +++ b/skill-guides/orchestration/references/messaging-and-gates.md @@ -36,7 +36,7 @@ ORCA orchestration send --to dispatch: --subject "Follow-up" --body Do not substitute a remote terminal handle. Omit `--from` for ordinary coordinator calls; a dispatched worker instead copies the exact `--from` and -capability arguments in its preamble. `check` is the exception: it identifies +other arguments in its preamble. `check` is the exception: it identifies its caller with `--terminal`, never `--from`. Group addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`, diff --git a/skill-guides/orchestration/references/worker-contract.md b/skill-guides/orchestration/references/worker-contract.md index 83a28d02ad25..a2315b7ceeee 100644 --- a/skill-guides/orchestration/references/worker-contract.md +++ b/skill-guides/orchestration/references/worker-contract.md @@ -2,7 +2,8 @@ The injected preamble is authoritative. Copy its command rather than reconstructing flags. In particular, preserve the exact executable, worker -handle, Dispatch capability, Task ID, and Dispatch ID. +handle, Task ID, and Dispatch ID, and keep any other flag it carries (an older +Orca host adds `--dispatch-capability`). ## Heartbeat @@ -10,7 +11,7 @@ Send heartbeats only at the cadence required by the live preamble. Skip them while blocked inside `ask` or `check --wait`; those calls are liveness signals. ```text -ORCA orchestration send --from --dispatch-capability --type heartbeat --subject "alive" --task-id --dispatch-id --phase "" +ORCA orchestration send --from --type heartbeat --subject "alive" --task-id --dispatch-id --phase "" ``` Use typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves @@ -22,9 +23,9 @@ Use Orca `ask` whenever the coordinator must answer. Never open a local question TUI the coordinator cannot answer. ```text -ORCA orchestration ask --from --dispatch-capability --question "" --options "," --timeout-ms 600000 +ORCA orchestration ask --from --question "" --options "," --timeout-ms 600000 -ORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000 +ORCA orchestration ask --from --resume --timeout-ms 600000 ``` A timeout or disconnect leaves the original question pending. Resume its @@ -57,7 +58,7 @@ party's terminal (a coordinator or another worker); run it from your own termina Escalate only before completion and only when the coordinator must intervene: ```text -ORCA orchestration send --from --dispatch-capability --type escalation --subject "Blocked: " --body "
" --task-id --dispatch-id +ORCA orchestration send --from --type escalation --subject "Blocked: " --body "
" --task-id --dispatch-id ``` ## Completion @@ -70,7 +71,7 @@ Append `--files-modified` or `--report-path` only when applicable, using actual paths. Do not send documentation placeholders as metadata. ```text -ORCA orchestration send --from --dispatch-capability --type worker_done --subject "" --body "" --task-id --dispatch-id --outcome succeeded +ORCA orchestration send --from --type worker_done --subject "" --body "" --task-id --dispatch-id --outcome succeeded ``` After `worker_done`, end the dispatched turn and idle. Do not poll, close your diff --git a/src/cli/bundled-skill-guides.ts b/src/cli/bundled-skill-guides.ts index 0ea0e2bf4c44..1f1af4df1db4 100644 --- a/src/cli/bundled-skill-guides.ts +++ b/src/cli/bundled-skill-guides.ts @@ -66,10 +66,10 @@ const ORCA_PER_WORKSPACE_ENV_SSH_HOST_REFERENCE_MARKDOWN = "# SSH connection mod const ORCA_PER_WORKSPACE_ENV_WINDOWS_SCRIPTS_REFERENCE_MARKDOWN = "# Windows local-side scripts\n\nLoad this when the user's desktop is Windows and you are scaffolding the local-side scripts. A bare\n`.sh` will not execute there. Either require WSL or Git Bash and point `orca.yaml` at a launcher such\nas `bash ./scripts/orca-vm/.sh` through a `.cmd` file, or scaffold PowerShell equivalents.\n\nThe remote-side commands you run inside the Linux environment stay bash regardless of the desktop OS.\n\n```powershell\n#requires -Version 5\n$ErrorActionPreference = 'Stop'\n# resolve env→state→fallback; run the provider CLI / ssh the same way;\n# capture provider output; build the result object for the chosen mode and write ONE line of JSON to stdout.\n# Orca-server mode: @{ schemaVersion=1; pairingCode=$pairingCode; projectRoot=$projectRoot; userData=@{...} }\n# SSH mode: @{ schemaVersion=1; connection=@{ type=\"ssh\"; projectRoot=$projectRoot;\n# target=@{ label=$label; host=$host; port=$port; username=$user } } }\n($result | ConvertTo-Json -Compress -Depth 6)\n# progress/errors → Write-Error / the error stream, never stdout.\n```\n\nThe doctor's executable-bit check is a POSIX concept and is skipped on Windows, so a script that is\nunusable on the user's machine for a different reason still has to be caught by the `--provision`\nself-test.\n" // oxfmt-ignore -const ORCHESTRATION_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID\n in the live preamble. Never reconstruct, translate, or broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n" +const ORCHESTRATION_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live\n preamble, plus any other flag it carries. Never reconstruct, translate, or\n broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n" // oxfmt-ignore -const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, capability, Task ID, and Dispatch ID\n in the live preamble. Never reconstruct, translate, or broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\ncapability arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" +const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live\n preamble, plus any other flag it carries. Never reconstruct, translate, or\n broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\nother arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Task ID, and Dispatch ID, and keep any other flag it carries (an older\nOrca host adds `--dispatch-capability`).\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" // oxfmt-ignore const ORCHESTRATION_COORDINATOR_LOOP_REFERENCE_MARKDOWN = "# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n" @@ -81,7 +81,7 @@ const ORCHESTRATION_LEGACY_CONTRACT_MIGRATION_REFERENCE_MARKDOWN = "# Legacy con const ORCHESTRATION_LOW_LEVEL_TOPOLOGY_REFERENCE_MARKDOWN = "# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n" // oxfmt-ignore -const ORCHESTRATION_MESSAGING_AND_GATES_REFERENCE_MARKDOWN = "# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\ncapability arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n" +const ORCHESTRATION_MESSAGING_AND_GATES_REFERENCE_MARKDOWN = "# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\nother arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n" // oxfmt-ignore const ORCHESTRATION_PLACEMENT_AND_REMOTE_REFERENCE_MARKDOWN = "# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n" @@ -90,7 +90,7 @@ const ORCHESTRATION_PLACEMENT_AND_REMOTE_REFERENCE_MARKDOWN = "# Placement and r const ORCHESTRATION_RECOVERY_AND_CLEANUP_REFERENCE_MARKDOWN = "# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n" // oxfmt-ignore -const ORCHESTRATION_WORKER_CONTRACT_REFERENCE_MARKDOWN = "# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Dispatch capability, Task ID, and Dispatch ID.\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --dispatch-capability --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --dispatch-capability --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --dispatch-capability --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --dispatch-capability --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --dispatch-capability --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" +const ORCHESTRATION_WORKER_CONTRACT_REFERENCE_MARKDOWN = "# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Task ID, and Dispatch ID, and keep any other flag it carries (an older\nOrca host adds `--dispatch-capability`).\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" // oxfmt-ignore export const BUNDLED_SKILL_GUIDES = [ diff --git a/src/cli/orchestration-mutation-recovery.test.ts b/src/cli/orchestration-mutation-recovery.test.ts index c20525e15e12..e79b72d74d21 100644 --- a/src/cli/orchestration-mutation-recovery.test.ts +++ b/src/cli/orchestration-mutation-recovery.test.ts @@ -46,7 +46,7 @@ describe('orchestration mutation recovery', () => { ) expect((result.data as { nextSteps?: string[] }).nextSteps).toEqual([ 'Run orca orchestration worker-show --dispatch dispatch_1 --json before retrying.', - 'After inspecting the Dispatch, if keyed recovery is still needed, run orca orchestration worker-start --task task_1 --retry-request request_1. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate.' + 'After inspecting the Dispatch, if keyed recovery is still needed, run orca orchestration worker-start --task task_1 --retry-request request_1 from this same terminal. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate.' ]) }) @@ -122,7 +122,7 @@ describe('orchestration mutation recovery', () => { expect((result.data as { nextSteps?: string[] }).nextSteps).toEqual([ 'Run orca-dev orchestration worker-show --dispatch dispatch_3 --json before retrying.', - "After inspecting the Dispatch, if keyed recovery is still needed, run orca-dev orchestration worker-start --task 'task 3' --comment 'literal $(do-not-run)' --retry-request request_3. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate." + "After inspecting the Dispatch, if keyed recovery is still needed, run orca-dev orchestration worker-start --task 'task 3' --comment 'literal $(do-not-run)' --retry-request request_3 from this same terminal. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate." ]) expect(result.message).toContain("'literal $(do-not-run)'") }) diff --git a/src/cli/orchestration-mutation-recovery.ts b/src/cli/orchestration-mutation-recovery.ts index 0ac89165ce03..31439fae7e4b 100644 --- a/src/cli/orchestration-mutation-recovery.ts +++ b/src/cli/orchestration-mutation-recovery.ts @@ -44,8 +44,8 @@ export function orchestrationMutationRecoveryError(error: unknown): unknown { } const retryStep = retryCommand ? dispatchId - ? `After inspecting the Dispatch, if keyed recovery is still needed, run ${renderCommand(retryCommand)}. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate.` - : `If request-show reports completed or pending, run ${renderCommand(retryCommand)}. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate. If request-show reports absent, inspect the affected state before deciding whether to retry; absence does not prove a retry is safe.` + ? `After inspecting the Dispatch, if keyed recovery is still needed, run ${renderCommand(retryCommand)} from this same terminal. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate.` + : `If request-show reports completed or pending, run ${renderCommand(retryCommand)} from this same terminal. --retry-request reuses the same operation identity so Orca can replay, join, or safely recover it without starting a separate duplicate. If request-show reports absent, inspect the affected state before deciding whether to retry; absence does not prove a retry is safe.` : 'Recovery is blocked until the exact original command is available; no retry command was emitted.' const nextSteps = [`Run ${renderCommand(queryCommand)} before retrying.`, retryStep] const message = [ diff --git a/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts b/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts index 15270b692a61..579828337a07 100644 --- a/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts +++ b/src/main/runtime/orchestration/coordinator-dispatch-unobserved-prompt.test.ts @@ -2,6 +2,7 @@ import { afterEach, describe, expect, it } from 'vitest' import { OrchestrationDb } from './db' import type { CoordinatorRuntime } from './coordinator-runtime-contract' import { dispatchTaskToWorker } from './coordinator-task-dispatch' +import { reattachDispatchConsumer } from './db/root-dispatch-test-fixture' const WORKER_PANE_KEY = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' let db: OrchestrationDb @@ -91,7 +92,7 @@ describe('coordinator dispatch with an unobserved prompt', () => { const task = db.createTask({ runId: 'run_legacy_local', spec: 'do the work' }) await dispatch(createRuntime(new Error('agent_prompt_stalled')), task.id, []) const dispatchId = db.getDispatchContext(task.id)!.id - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId, paneKey: WORKER_PANE_KEY, processIncarnation: 'incarnation-1' diff --git a/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts b/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts index a83ce65b9839..ca5f890186ad 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-invariant.test.ts @@ -4,7 +4,7 @@ import { join } from 'node:path' import { afterEach, describe, expect, it, vi } from 'vitest' import type Database from '../../sqlite/sync-database' import { OrchestrationDb } from './db' -import { createRootDispatch } from './db/root-dispatch-test-fixture' +import { createRootDispatch, reattachDispatchConsumer } from './db/root-dispatch-test-fixture' type DatabaseHarness = { db: OrchestrationDb @@ -82,7 +82,7 @@ describe('Task/Dispatch invariant transactions', () => { deps: [task.id] }) const dispatch = createRootDispatch(db, task.id, 'term_worker') - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: dispatch.id, paneKey: 'tab_worker:leaf_worker', processIncarnation: 'worker:1' diff --git a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts index 3eef47f90ffc..f58c0938ecfe 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts @@ -8,7 +8,6 @@ import { createRootDispatch } from './db/root-dispatch-test-fixture' type WorkerFixture = { dispatchId: string - capability: string handle: string paneKey: string processIncarnation: string @@ -690,7 +689,7 @@ function startWorker(database: OrchestrationDb, taskId: string, name: string): W const paneKey = `tab_${name}:aaaaaaaa-aaaa-4aaa-8aaa-${paneSuffix}` const processIncarnation = `${name}:1` const handle = `term_${name}` - const capability = database.prepareStartingWorkerAuthority({ + database.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle, paneKey, @@ -701,7 +700,7 @@ function startWorker(database: OrchestrationDb, taskId: string, name: string): W terminalOwnership: 'created' }) database.markWorkerDispatchReady(started.dispatch.id) - return { dispatchId: started.dispatch.id, capability, handle, paneKey, processIncarnation } + return { dispatchId: started.dispatch.id, handle, paneKey, processIncarnation } } function sqliteFor(database: OrchestrationDb): Database.Database { diff --git a/src/main/runtime/orchestration/db-task-dispatch-races.test.ts b/src/main/runtime/orchestration/db-task-dispatch-races.test.ts index 28a7d092ec65..9398f281e43c 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-races.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-races.test.ts @@ -221,10 +221,11 @@ describe('Task/Dispatch concurrency', () => { }) const sqlite = sqliteFor(first.db) const exec = sqlite.exec.bind(sqlite) - let winningCapability: string | undefined + let winnerClaimed = false vi.spyOn(sqlite, 'exec').mockImplementation((sql) => { - if (!winningCapability && sql === 'BEGIN IMMEDIATE') { - winningCapability = concurrent.db.prepareStartingWorkerAuthority({ + if (!winnerClaimed && sql === 'BEGIN IMMEDIATE') { + winnerClaimed = true + concurrent.db.prepareStartingWorkerAuthority({ dispatchId: winner.dispatch.id, handle: 'term_reminted', paneKey: 'tab_new:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', @@ -250,10 +251,9 @@ describe('Task/Dispatch concurrency', () => { terminalOwnership: 'created' }) ).toThrow(`already has an active dispatch (${winner.dispatch.id} for task ${winningTask.id})`) - expect(winningCapability).toBeDefined() expect(first.db.getDispatchContextById(loser.dispatch.id)).toMatchObject({ assignee_handle: null, - capability_hash: null + process_incarnation: null }) expect(first.db.getWorkerDispatch(loser.dispatch.id)).toMatchObject({ stage: 'accepted', @@ -262,7 +262,7 @@ describe('Task/Dispatch concurrency', () => { expect(first.db.getWorkerTerminalResourceByOwner(loser.dispatch.id)).toBeUndefined() expect(first.db.getDispatchContextById(winner.dispatch.id)).toMatchObject({ assignee_handle: 'term_reminted', - capability_hash: expect.any(String) + process_incarnation: 'winner:1' }) expect(first.db.getWorkerTerminalResourceByOwner(winner.dispatch.id)).toMatchObject({ terminal_handle: 'term_reminted', @@ -272,7 +272,7 @@ describe('Task/Dispatch concurrency', () => { sqlite .prepare( `SELECT COUNT(*) AS count FROM dispatch_contexts - WHERE status IN ('pending', 'dispatched') AND capability_hash IS NOT NULL` + WHERE status IN ('pending', 'dispatched') AND process_incarnation IS NOT NULL` ) .get() ).toEqual({ count: 1 }) diff --git a/src/main/runtime/orchestration/db/attach-orchestration-db-methods.ts b/src/main/runtime/orchestration/db/attach-orchestration-db-methods.ts index 135dcc5ae3fe..2c90cab8e00e 100644 --- a/src/main/runtime/orchestration/db/attach-orchestration-db-methods.ts +++ b/src/main/runtime/orchestration/db/attach-orchestration-db-methods.ts @@ -1,7 +1,6 @@ import { attachAttemptObservationStore } from './attempt-observation-store' import { attachCoordinatorRunStore } from './coordinator-runs/coordinator-run-store' import { attachDecisionGateStore } from './decision-gates/decision-gate-store' -import { attachDispatchCapability } from './dispatch-context/dispatch-capability' import { attachDispatchCompletion } from './dispatch-context/dispatch-completion' import { attachDispatchContextStore } from './dispatch-context/dispatch-context-store' import { attachDispatchLookup } from './dispatch-context/dispatch-lookup' @@ -128,7 +127,6 @@ export function attachOrchestrationDbMethods(ctor: { prototype: object }): void attachWorkerTerminalArchive(ctor) attachWorkerTerminalListing(ctor) attachDispatchContextStore(ctor) - attachDispatchCapability(ctor) attachDispatchLookup(ctor) attachDispatchDepth(ctor) attachDispatchCompletion(ctor) diff --git a/src/main/runtime/orchestration/db/dispatch-capability-hash.ts b/src/main/runtime/orchestration/db/dispatch-capability-hash.ts deleted file mode 100644 index c4e6631920cb..000000000000 --- a/src/main/runtime/orchestration/db/dispatch-capability-hash.ts +++ /dev/null @@ -1,5 +0,0 @@ -import { createHash } from 'node:crypto' - -export function hashDispatchCapability(capability: string): string { - return createHash('sha256').update(capability).digest('hex') -} diff --git a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts b/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts deleted file mode 100644 index c9c687f2b109..000000000000 --- a/src/main/runtime/orchestration/db/dispatch-context/dispatch-capability.ts +++ /dev/null @@ -1,69 +0,0 @@ -import { randomBytes } from 'node:crypto' -import { OrchestrationError } from '../../orchestration-error' -import { hashDispatchCapability } from '../dispatch-capability-hash' -import type { OrchestrationDb } from '../orchestration-db' - -export function mintDispatchCapability( - this: OrchestrationDb, - params: { - dispatchId: string - paneKey: string - processIncarnation: string - } -): string { - const dispatch = this.getDispatchContextById(params.dispatchId) - if (!dispatch || (dispatch.status !== 'pending' && dispatch.status !== 'dispatched')) { - throw new OrchestrationError( - 'dispatch_inactive', - `Dispatch ${params.dispatchId} is not active.` - ) - } - const capability = `dcap_${randomBytes(32).toString('base64url')}` - // Why: re-pointing the Dispatch at a pane/process must fence the prior consumer's Delivery in - // the same transaction, or both processes keep acking one outstanding Delivery. - this.db.exec('BEGIN IMMEDIATE') - try { - this.db - .prepare( - `UPDATE dispatch_contexts - SET capability_hash = ?, assignee_pane_key = ?, process_incarnation = ?, - capability_revoked_at = NULL, - consumer_generation = consumer_generation + 1 - WHERE id = ?` - ) - .run( - hashDispatchCapability(capability), - params.paneKey, - params.processIncarnation, - params.dispatchId - ) - this.fenceUnacknowledgedMailboxDeliveries(`dispatch:${params.dispatchId}`) - this.db.exec('COMMIT') - } catch (error) { - this.db.exec('ROLLBACK') - throw error - } - return capability -} - -export function revokeDispatchCapability(this: OrchestrationDb, dispatchId: string): void { - this.db - .prepare( - `UPDATE dispatch_contexts - SET capability_revoked_at = COALESCE(capability_revoked_at, datetime('now')) - WHERE id = ?` - ) - .run(dispatchId) -} - -export type DispatchCapabilityMethods = { - mintDispatchCapability: typeof mintDispatchCapability - revokeDispatchCapability: typeof revokeDispatchCapability -} - -export function attachDispatchCapability(ctor: { prototype: object }): void { - Object.assign(ctor.prototype, { - mintDispatchCapability, - revokeDispatchCapability - }) -} diff --git a/src/main/runtime/orchestration/db/dispatch-mailbox-consumer-fencing.test.ts b/src/main/runtime/orchestration/db/dispatch-mailbox-consumer-fencing.test.ts index 608b91a2586e..34004e6a7afb 100644 --- a/src/main/runtime/orchestration/db/dispatch-mailbox-consumer-fencing.test.ts +++ b/src/main/runtime/orchestration/db/dispatch-mailbox-consumer-fencing.test.ts @@ -1,7 +1,7 @@ import { afterEach, beforeEach, describe, expect, it } from 'vitest' import { OrchestrationDb } from '../db' import { ORCHESTRATION_CONTRACT_VERSION } from '../../../../shared/protocol-version' -import { createRootDispatch } from './root-dispatch-test-fixture' +import { createRootDispatch, reattachDispatchConsumer } from './root-dispatch-test-fixture' import type { DeliveryRow } from '../types' const PANE_A = 'tab_a:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' @@ -54,7 +54,7 @@ describe('dispatch mailbox consumer fencing', () => { it('fences worker A once worker B re-attaches, and hands B the same unread mail', () => { const dispatch = dispatchWithMail(['first', 'second']) - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: dispatch.id, paneKey: PANE_A, processIncarnation: 'runtime:pty-a:1' @@ -63,7 +63,7 @@ describe('dispatch mailbox consumer fencing', () => { const deliveryA = openDelivery(dispatch.id, dispatch.runId, generationA) expect(deliveryA?.messages.map((message) => message.subject)).toEqual(['first', 'second']) - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: dispatch.id, paneKey: PANE_B, processIncarnation: 'runtime:pty-b:1' @@ -97,7 +97,7 @@ describe('dispatch mailbox consumer fencing', () => { it("leaves A's ack able to strand mail unread only when B never took over", () => { const dispatch = dispatchWithMail(['first']) - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: dispatch.id, paneKey: PANE_A, processIncarnation: 'runtime:pty-a:1' diff --git a/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts b/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts index 4af584425a1d..04faf60a79ac 100644 --- a/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts +++ b/src/main/runtime/orchestration/db/federation/remote-dispatch-attachment-authority.ts @@ -1,7 +1,5 @@ -import { randomBytes } from 'node:crypto' import type { RemoteDispatchAttachmentRow } from '../../types' import { OrchestrationError } from '../../orchestration-error' -import { hashDispatchCapability } from '../dispatch-capability-hash' import { isEquivalentPaneKey } from '../pane-key-match' import type { OrchestrationDb } from '../orchestration-db' @@ -18,7 +16,7 @@ export function prepareRemoteAttachmentAuthority( hostScope?: string | null terminalOwnership?: 'created' | 'external' } -): string { +): void { this.db.exec('BEGIN IMMEDIATE') try { const attachment = this.getRemoteDispatchAttachment(params.dispatchId) @@ -35,18 +33,16 @@ export function prepareRemoteAttachmentAuthority( `Terminal ${params.terminalHandle} already has active remote Dispatch ${active.dispatch_id}.` ) } - const capability = `dcap_${randomBytes(32).toString('base64url')}` const result = this.db .prepare( `UPDATE remote_dispatch_attachments - SET stage = 'authority_attached', capability_hash = ?, pane_key = ?, + SET stage = 'authority_attached', pane_key = ?, process_incarnation = ?, worktree_id = ?, terminal_handle = ?, setup_state = ?, effects = ?, residual_resources = ?, updated_at = datetime('now'), consumer_generation = consumer_generation + 1 WHERE dispatch_id = ? AND state = 'starting'` ) .run( - hashDispatchCapability(capability), params.paneKey, params.processIncarnation, params.worktreeId, @@ -108,7 +104,6 @@ export function prepareRemoteAttachmentAuthority( } } this.db.exec('COMMIT') - return capability } catch (error) { this.db.exec('ROLLBACK') throw error diff --git a/src/main/runtime/orchestration/db/orchestration-db-methods.ts b/src/main/runtime/orchestration/db/orchestration-db-methods.ts index 55dcddaf44c9..03e92190a380 100644 --- a/src/main/runtime/orchestration/db/orchestration-db-methods.ts +++ b/src/main/runtime/orchestration/db/orchestration-db-methods.ts @@ -1,7 +1,6 @@ import type { AttemptObservationStoreMethods } from './attempt-observation-store' import type { CoordinatorRunStoreMethods } from './coordinator-runs/coordinator-run-store' import type { DecisionGateStoreMethods } from './decision-gates/decision-gate-store' -import type { DispatchCapabilityMethods } from './dispatch-context/dispatch-capability' import type { DispatchCompletionMethods } from './dispatch-context/dispatch-completion' import type { DispatchContextStoreMethods } from './dispatch-context/dispatch-context-store' import type { DispatchLookupMethods } from './dispatch-context/dispatch-lookup' @@ -127,7 +126,6 @@ export type OrchestrationDbMethods = AttemptObservationStoreMethods & WorkerTerminalArchiveMethods & WorkerTerminalListingMethods & DispatchContextStoreMethods & - DispatchCapabilityMethods & DispatchLookupMethods & DispatchDepthMethods & DispatchCompletionMethods & diff --git a/src/main/runtime/orchestration/db/root-dispatch-test-fixture.ts b/src/main/runtime/orchestration/db/root-dispatch-test-fixture.ts index 4b16c95e0a6a..2c46284ff5b8 100644 --- a/src/main/runtime/orchestration/db/root-dispatch-test-fixture.ts +++ b/src/main/runtime/orchestration/db/root-dispatch-test-fixture.ts @@ -26,3 +26,19 @@ export function createRootDispatch( maxDepth: Number.MAX_SAFE_INTEGER }) } + +/** Re-points a Dispatch at a new pane/process and fences the prior consumer's Delivery. */ +export function reattachDispatchConsumer( + db: OrchestrationDb, + params: { dispatchId: string; paneKey: string; processIncarnation: string } +): void { + db.db + .prepare( + `UPDATE dispatch_contexts + SET assignee_pane_key = ?, process_incarnation = ?, + consumer_generation = consumer_generation + 1 + WHERE id = ?` + ) + .run(params.paneKey, params.processIncarnation, params.dispatchId) + db.fenceUnacknowledgedMailboxDeliveries(`dispatch:${params.dispatchId}`) +} diff --git a/src/main/runtime/orchestration/db/schema/derived-delivery-migration.test.ts b/src/main/runtime/orchestration/db/schema/derived-delivery-migration.test.ts index a868a310e71e..b5f61cc09dc0 100644 --- a/src/main/runtime/orchestration/db/schema/derived-delivery-migration.test.ts +++ b/src/main/runtime/orchestration/db/schema/derived-delivery-migration.test.ts @@ -6,7 +6,7 @@ import Database from '../../../../sqlite/sync-database' import { OrchestrationDb } from '../orchestration-db' import { dropDerivedDeliverySchema } from './derived-delivery-test-fixture' import { resolveOrchestrationMigrationStartVersion } from '../../orchestration-schema-version-skew' -import { createRootDispatch } from '../root-dispatch-test-fixture' +import { createRootDispatch, reattachDispatchConsumer } from '../root-dispatch-test-fixture' import { SCHEMA_VERSION } from '../contract-constants' describe('derived delivery migration', () => { @@ -213,7 +213,7 @@ describe('derived delivery migration', () => { processIncarnation: 'worker:2' } if (consumerSource === 'dispatch') { - peer.mintDispatchCapability(authority) + reattachDispatchConsumer(peer, authority) } else { peer.prepareRemoteAttachmentAuthority({ ...authority, diff --git a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-authority.ts b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-authority.ts index 733833c9acab..fa826c7e1f69 100644 --- a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-authority.ts +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-authority.ts @@ -1,6 +1,4 @@ -import { randomBytes } from 'node:crypto' import { OrchestrationError } from '../../orchestration-error' -import { hashDispatchCapability } from '../dispatch-capability-hash' import type { OrchestrationDb } from '../orchestration-db' import { dispatchAssigneeOrcaSessionId } from '../../dispatch-assignee-orca-session-id' @@ -21,7 +19,7 @@ export function prepareStartingWorkerAuthority( // an explicit --terminal reuse; ownership transfers only from an exact owned settled resource. terminalOwnership?: 'created' | 'external' } -): string { +): void { this.db.exec('BEGIN IMMEDIATE') try { // Why: read inside the transaction so the guarded UPDATEs below cannot lose a race with a concurrent state change. @@ -49,14 +47,13 @@ export function prepareStartingWorkerAuthority( `Terminal ${params.handle} already has an active dispatch (${existing.id} for task ${existing.task_id})` ) } - const capability = `dcap_${randomBytes(32).toString('base64url')}` const endpointId = this.getWorkerDispatch(params.dispatchId)?.runtime_epoch ?? null const contextUpdate = this.db .prepare( `UPDATE dispatch_contexts SET assignee_handle = ?, assignee_pane_key = ?, assignee_orca_session_id = ?, process_incarnation = ?, host_scope = ?, - capability_hash = ?, launch_token_hash = COALESCE(launch_token_hash, ?), + launch_token_hash = COALESCE(launch_token_hash, ?), capability_revoked_at = NULL, consumer_generation = consumer_generation + 1 WHERE id = ? AND status = 'pending'` @@ -67,7 +64,6 @@ export function prepareStartingWorkerAuthority( dispatchAssigneeOrcaSessionId(params.processIncarnation), params.processIncarnation, params.hostScope ?? null, - hashDispatchCapability(capability), params.launchTokenHash ?? null, params.dispatchId ) @@ -155,7 +151,6 @@ export function prepareStartingWorkerAuthority( } } this.db.exec('COMMIT') - return capability } catch (error) { this.db.exec('ROLLBACK') throw error @@ -168,7 +163,7 @@ export function prepareStartingWorkerAuthority( * ownership to flip, so the takeover is silently dropped and a later `worker-release` closes the * pane under the user. * - * Ownership of a pane only; the Dispatch capability stays behind the boot wait, because authority + * Ownership of a pane only; lifecycle authority stays behind the boot wait, because authority * must not be handed to a process that has not come up. */ export function recordCreatedWorkerTerminalCustody( diff --git a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-outcome.ts b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-outcome.ts index 99873949f926..c687fdb60360 100644 --- a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-outcome.ts +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-outcome.ts @@ -44,11 +44,7 @@ export function failWorkerStart( this: OrchestrationDb, dispatchId: string, stage: string, - reason: string, - // Why (#16095): revocation exists to stop a worker acting on a dispatch that never landed. A - // prompt whose turn start went unobserved provably landed, so its worker keeps the authority its - // own report needs. - options: { retainCapability?: boolean } = {} + reason: string ): WorkerDispatchRow { this.db.exec('BEGIN IMMEDIATE') try { @@ -66,9 +62,7 @@ export function failWorkerStart( projection: { last_failure: reason, completed_at: now, - capability_revoked_at: options.retainCapability - ? dispatch.capability_revoked_at - : (dispatch.capability_revoked_at ?? now) + capability_revoked_at: dispatch.capability_revoked_at ?? now } }) transitionLifecycleWithDb(this.db, { diff --git a/src/main/runtime/orchestration/db/worker-terminal/failed-start-dispatch-identity.ts b/src/main/runtime/orchestration/db/worker-terminal/failed-start-dispatch-identity.ts index 3d45c9a86304..86657dfb0be4 100644 --- a/src/main/runtime/orchestration/db/worker-terminal/failed-start-dispatch-identity.ts +++ b/src/main/runtime/orchestration/db/worker-terminal/failed-start-dispatch-identity.ts @@ -7,8 +7,6 @@ import { dispatchAssigneeOrcaSessionId } from '../../dispatch-assignee-orca-sess * and release re-proves identity through it — so the custody row written at terminal creation would * name a pane no release path could match. Copy that identity across. * - * `capability_hash` stays null, so this grants nothing: it records which pane the Dispatch owns. - * * No transaction: composes inside `failWorkerStart`'s. */ export function recordFailedStartDispatchIdentity( @@ -24,7 +22,7 @@ export function recordFailedStartDispatchIdentity( `UPDATE dispatch_contexts SET assignee_handle = ?, assignee_pane_key = ?, assignee_orca_session_id = ?, process_incarnation = ?, host_scope = ? - WHERE id = ? AND status = 'failed' AND capability_hash IS NULL` + WHERE id = ? AND status = 'failed' AND process_incarnation IS NULL` ) .run( resource.terminal_handle, diff --git a/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts b/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts index d6e6038cd65b..1e1acf3fd005 100644 --- a/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts +++ b/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts @@ -28,7 +28,7 @@ describe('OrchestrationDb worker Dispatch state', () => { }) expect(d.getTask(task.id)?.status).toBe('dispatched') - const capability = d.prepareStartingWorkerAuthority({ + d.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_worker', paneKey: 'tab_worker:leaf_worker', @@ -37,7 +37,6 @@ describe('OrchestrationDb worker Dispatch state', () => { setupState: 'not_applicable', effects: [{ kind: 'terminal', action: 'created', id: 'term_worker' }] }) - expect(capability).toMatch(/^dcap_/) expect(d.markWorkerDispatchReady(started.dispatch.id)).toMatchObject({ state: 'ready', stage: 'input_accepted' diff --git a/src/main/runtime/orchestration/preamble.test.ts b/src/main/runtime/orchestration/preamble.test.ts index bb2087ea5afb..bede4e553143 100644 --- a/src/main/runtime/orchestration/preamble.test.ts +++ b/src/main/runtime/orchestration/preamble.test.ts @@ -83,7 +83,7 @@ describe('buildDispatchPreamble', () => { ) it('renders every injected lifecycle command on one cross-shell-safe line', () => { - const result = buildDispatchPreamble(baseParams({ dispatchCapability: 'dcap_secret' })) + const result = buildDispatchPreamble(baseParams()) const commandLines = result .split('\n') .filter((line) => line.trimStart().startsWith('orca orchestration')) @@ -182,27 +182,14 @@ describe('buildDispatchPreamble', () => { expect(cadence).toContain('immediately before\n # you send worker_done') }) - it('carries the minted Dispatch capability on lifecycle and question commands', () => { - const result = buildDispatchPreamble({ - ...baseParams(), - dispatchCapability: 'dcap_test_secret' - }) - - expect(result.match(/--dispatch-capability dcap_test_secret/g)).toHaveLength(4) - expect(result).not.toContain('"dispatchCapability"') - }) - - it('renders capability-bound worker_done and heartbeat recipes', () => { - const result = buildDispatchPreamble({ - ...baseParams(), - dispatchCapability: 'dcap_test_secret' - }) + it('renders worker_done and heartbeat recipes bound to the exact Dispatch', () => { + const result = buildDispatchPreamble(baseParams()) expect(result).toMatch( - /orchestration send --from term_worker --dispatch-capability dcap_test_secret --type worker_done .*?--task-id task_abc123 --dispatch-id ctx_def456/u + /orchestration send --from term_worker --type worker_done .*?--task-id task_abc123 --dispatch-id ctx_def456/u ) expect(result).toMatch( - /orchestration send --from term_worker --dispatch-capability dcap_test_secret --type heartbeat .*?--task-id task_abc123 --dispatch-id ctx_def456/u + /orchestration send --from term_worker --type heartbeat .*?--task-id task_abc123 --dispatch-id ctx_def456/u ) }) diff --git a/src/main/runtime/orchestration/preamble.ts b/src/main/runtime/orchestration/preamble.ts index a9fbb528939f..2502d288258b 100644 --- a/src/main/runtime/orchestration/preamble.ts +++ b/src/main/runtime/orchestration/preamble.ts @@ -10,7 +10,6 @@ export type PreambleParams = { // prevents stale messages from a previously-failed dispatch from completing // or refreshing the retry. dispatchId: string - dispatchCapability?: string taskSpec: string coordinatorHandle: string workerHandle: string @@ -57,9 +56,6 @@ export function buildDispatchPreamble(params: PreambleParams): string { cli, workerKind: params.workerKind ?? 'prompt-returning-agent' }) - const capabilityFlag = params.dispatchCapability - ? ` --dispatch-capability ${params.dispatchCapability}` - : '' // Why: one-line recipes paste unchanged in POSIX shells, PowerShell, and cmd.exe. // Why fenced: keeps the shell comments executable without rendering them as Chat UI headings. @@ -90,7 +86,7 @@ Don't post to Slack, GitHub, or other channels during the run; report through th # Never encode failure only in prose and never silently exit. # Include BOTH taskId and dispatchId in the payload so a late completion # from a failed retry cannot complete the current dispatch. - ${cli} orchestration send --from ${params.workerHandle}${capabilityFlag} --type worker_done --subject "" --body "<3-sentence summary: what you did, what you found, what's left>" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} --outcome succeeded + ${cli} orchestration send --from ${params.workerHandle} --type worker_done --subject "" --body "<3-sentence summary: what you did, what you found, what's left>" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} --outcome succeeded # Send a heartbeat every ${HEARTBEAT_INTERVAL_MIN} minutes # while actively working on the task. The coordinator uses this to @@ -102,7 +98,7 @@ Don't post to Slack, GitHub, or other channels during the run; report through th # attributes the heartbeat to the specific dispatch context, not just # the task, so a straggler heartbeat from a previously-failed dispatch # cannot mask a hung retry. - ${cli} orchestration send --from ${params.workerHandle}${capabilityFlag} --type heartbeat --subject "alive" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} --phase "" + ${cli} orchestration send --from ${params.workerHandle} --type heartbeat --subject "alive" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} --phase "" # Ask the coordinator a question and block until it answers. # @@ -114,11 +110,11 @@ Don't post to Slack, GitHub, or other channels during the run; report through th # blocks until the coordinator replies, then prints the reply body. If the # call times out or disconnects, resume with the returned message ID instead # of creating a duplicate question. - ${cli} orchestration ask --from ${params.workerHandle}${capabilityFlag} --question "" --options "" --timeout-ms 600000 + ${cli} orchestration ask --from ${params.workerHandle} --question "" --options "" --timeout-ms 600000 # Escalate a blocker or failure (pre-completion, when you need the # coordinator to do something before you can continue): - ${cli} orchestration send --from ${params.workerHandle}${capabilityFlag} --type escalation --subject "Blocked: " --body "
" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} + ${cli} orchestration send --from ${params.workerHandle} --type escalation --subject "Blocked: " --body "
" --task-id ${params.taskId} --dispatch-id ${params.dispatchId} # Read coordinator follow-ups. Nothing interrupts you: a durable message only # arrives when you look, so run this at each natural checkpoint — before you diff --git a/src/main/runtime/orchestration/types.ts b/src/main/runtime/orchestration/types.ts index cd2b12a7da28..d0418289e514 100644 --- a/src/main/runtime/orchestration/types.ts +++ b/src/main/runtime/orchestration/types.ts @@ -201,6 +201,7 @@ export type RemoteDispatchAttachmentRow = { home_peer_fingerprint: string protocol_version: number runtime_epoch: string + /** Written only by hosts that minted a per-Dispatch capability; never read for authority. */ capability_hash: string | null pane_key: string | null process_incarnation: string | null @@ -285,8 +286,10 @@ export type DispatchContextRow = { assignee_pane_key: string | null /** Bare Orca session id the assignee is addressed by, when it has one (today only structured sessions); a `/clear`ed chat's lineage root. */ assignee_orca_session_id: OrcaSessionId | null + /** Written only by hosts that minted a per-Dispatch capability; never read for authority. */ capability_hash: string | null process_incarnation: string | null + /** When the Dispatch's lifecycle closed (settled, stopping, abandoned); the name predates that. */ capability_revoked_at: string | null /** Dispatch ID is the Attempt identity; retries point to the prior Attempt. */ retry_of_dispatch_id: string | null diff --git a/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts b/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts index 7f7b3bd88f7a..84291ebd8d23 100644 --- a/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts +++ b/src/main/runtime/orchestration/worker-start-unobserved-prompt-settlement.test.ts @@ -1,5 +1,6 @@ import { afterEach, describe, expect, it } from 'vitest' import { OrchestrationDb } from './db' +import { reattachDispatchConsumer } from './db/root-dispatch-test-fixture' const WORKER_PANE_KEY = 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb' const INCARNATION = 'runtime_test:term_worker:1' @@ -13,7 +14,7 @@ function startWorker(spec: string): { taskId: string; dispatchId: string } { taskId: task.id, startOptions: {} }) - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: started.dispatch.id, paneKey: WORKER_PANE_KEY, processIncarnation: INCARNATION @@ -28,14 +29,11 @@ describe('worker start settled by an unobserved prompt', () => { db = new OrchestrationDb(':memory:') const { taskId, dispatchId } = startWorker('run to completion') - db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled', { - retainCapability: true - }) + db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled') expect(db.getDispatchContextById(dispatchId)).toMatchObject({ status: 'failed', - last_failure: 'agent_prompt_stalled', - capability_revoked_at: null + last_failure: 'agent_prompt_stalled' }) expect( @@ -51,7 +49,7 @@ describe('worker start settled by an unobserved prompt', () => { expect(db.getWorkerDispatch(dispatchId)).toMatchObject({ state: 'succeeded', stage: 'settled' }) }) - it('revokes and stays settled when the start failed for any other cause', () => { + it('stays settled when the start failed for any other cause', () => { db = new OrchestrationDb(':memory:') const { taskId, dispatchId } = startWorker('never became ready') @@ -68,9 +66,7 @@ describe('worker start settled by an unobserved prompt', () => { db = new OrchestrationDb(':memory:') const { taskId, dispatchId } = startWorker('reports its own failure') - db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled', { - retainCapability: true - }) + db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled') expect( db.settleWorkerReport({ taskId, dispatchId, outcome: 'failed', result: 'build broke on X' }) @@ -92,9 +88,7 @@ describe('worker start settled by an unobserved prompt', () => { it('rolls back every prompt-stall correction when the worker transition fails', () => { db = new OrchestrationDb(':memory:') const { taskId, dispatchId } = startWorker('atomic correction') - db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled', { - retainCapability: true - }) + db.failWorkerStart(dispatchId, 'dispatch_input', 'agent_prompt_stalled') // The worker correction is the last of the three, so aborting it must undo the other two. db.db.exec(` CREATE TRIGGER reject_worker_prompt_stall_correction @@ -114,12 +108,43 @@ describe('worker start settled by an unobserved prompt', () => { expect(db.getTask(taskId)).toMatchObject({ status: 'failed', result: null }) expect(db.getDispatchContextById(dispatchId)).toMatchObject({ status: 'failed', - last_failure: 'agent_prompt_stalled', - capability_revoked_at: null + last_failure: 'agent_prompt_stalled' }) expect(db.getWorkerDispatch(dispatchId)).toMatchObject({ state: 'failed', stage: 'dispatch_input' }) }) + + it('keeps the identity its authority attached when a stalled start fails', () => { + db = new OrchestrationDb(':memory:') + const task = db.createTask({ runId: 'run_legacy_local', spec: 'stalled after authority' }) + const started = db.createStartingWorkerDispatch({ + creator: { kind: 'system' }, + maxDepth: Number.MAX_SAFE_INTEGER, + taskId: task.id, + startOptions: {} + }) + db.prepareStartingWorkerAuthority({ + dispatchId: started.dispatch.id, + handle: 'term_worker', + paneKey: WORKER_PANE_KEY, + processIncarnation: INCARNATION, + worktreeId: 'repo::worker', + effects: [], + setupState: 'not_applicable', + terminalOwnership: 'created' + }) + // The custody row can name another pane of the same PTY (#17741); it must not win. + db.db + .prepare('UPDATE worker_terminal_resources SET pane_key = ? WHERE owner_dispatch_id = ?') + .run('tab_custody:cccccccc-cccc-4ccc-8ccc-cccccccccccc', started.dispatch.id) + + db.failWorkerStart(started.dispatch.id, 'dispatch_input', 'agent_prompt_stalled') + + expect(db.getDispatchContextById(started.dispatch.id)).toMatchObject({ + assignee_pane_key: WORKER_PANE_KEY, + process_incarnation: INCARNATION + }) + }) }) diff --git a/src/main/runtime/rpc/core.ts b/src/main/runtime/rpc/core.ts index 1ec159337827..1402f10b2231 100644 --- a/src/main/runtime/rpc/core.ts +++ b/src/main/runtime/rpc/core.ts @@ -47,7 +47,6 @@ export type RpcRequest = { authToken: string method: string params?: unknown - orchestrationCapability?: string orchestrationContractVersion?: number orchestrationRequestId?: string compatibilityInvocationId?: string @@ -82,8 +81,6 @@ export type RpcContext = { clientCapabilities?: readonly RuntimeCapability[] // Why: mobile v2 auth is exact-key validated; capability upgrades must mutate only the authenticated socket after auth. updateClientCapabilities?: (capabilities: readonly RuntimeCapability[]) => void - // Why: Dispatch authority rides in the authenticated RPC envelope, never in user payload fields. - orchestrationCapability?: string // Why: long-lived mutations such as ask can durably expose acceptance before their waiter settles. recordMutationReceipt?: (receipt: unknown) => void // Why: only local worker_done makes pending proof that its atomic settlement transaction never committed. diff --git a/src/main/runtime/rpc/dispatcher.ts b/src/main/runtime/rpc/dispatcher.ts index 83cd0e30f9c9..04600fcc4ecd 100644 --- a/src/main/runtime/rpc/dispatcher.ts +++ b/src/main/runtime/rpc/dispatcher.ts @@ -119,7 +119,6 @@ export class RpcDispatcher { clientKind: options?.clientKind, clientCapabilities: options?.clientCapabilities, updateClientCapabilities: options?.updateClientCapabilities, - orchestrationCapability: request.orchestrationCapability, authenticatedCallerFingerprint: options?.authenticatedCallerFingerprint, orchestrationCaller: resolved.caller }, diff --git a/src/main/runtime/rpc/methods/orchestration-structured-worker-start-failure.test.ts b/src/main/runtime/rpc/methods/orchestration-structured-worker-start-failure.test.ts index 98a843fb695b..911d620c3a82 100644 --- a/src/main/runtime/rpc/methods/orchestration-structured-worker-start-failure.test.ts +++ b/src/main/runtime/rpc/methods/orchestration-structured-worker-start-failure.test.ts @@ -104,13 +104,14 @@ function fakes() { getTerminalPaneKey: vi.fn(() => 'pane_1'), retireStructuredAgentSessionTabFromSnapshot } as unknown as OrcaRuntimeService + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: the start path calls only these three methods. const db = { createStartingWorkerDispatch: () => ({ dispatch: { id: 'd_fail', depth: 0 }, task: { id: 't1', spec: 'do the thing' } }), recordWorkerStage: () => {}, - prepareStartingWorkerAuthority: () => 'capability' + prepareStartingWorkerAuthority: () => {} } as unknown as OrchestrationDb return { runtime, db, retireStructuredAgentSessionTabFromSnapshot } } diff --git a/src/main/runtime/rpc/methods/orchestration/federation/federated-message-targeting.test.ts b/src/main/runtime/rpc/methods/orchestration/federation/federated-message-targeting.test.ts index c28ef94a4a98..6d3f00abd6a0 100644 --- a/src/main/runtime/rpc/methods/orchestration/federation/federated-message-targeting.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/federation/federated-message-targeting.test.ts @@ -38,7 +38,7 @@ describe('orchestration federated message targeting', () => { payloadHash: 'attach_payload' } }) - const capability = db.prepareRemoteAttachmentAuthority({ + db.prepareRemoteAttachmentAuthority({ dispatchId, paneKey, processIncarnation, @@ -50,22 +50,22 @@ describe('orchestration federated message targeting', () => { db.markRemoteAttachmentReady(dispatchId) const dispatcher = new RpcDispatcher({ runtime, methods: ORCHESTRATION_METHODS }) const requests: RpcRequest[] = [ - request('send_to', capability, 'orchestration.send', { + request('send_to', 'orchestration.send', { from: 'term_remote_worker', to: 'run:explicit', subject: 'Wrong explicit target' }), - request('send_run', capability, 'orchestration.send', { + request('send_run', 'orchestration.send', { from: 'term_remote_worker', run: 'run_explicit', subject: 'Wrong explicit Run' }), - request('ask_to', capability, 'orchestration.ask', { + request('ask_to', 'orchestration.ask', { from: 'term_remote_worker', to: 'run:explicit', question: 'Wrong explicit target?' }), - request('ask_run', capability, 'orchestration.ask', { + request('ask_run', 'orchestration.ask', { from: 'term_remote_worker', run: 'run_explicit', question: 'Wrong explicit Run?' @@ -89,7 +89,6 @@ describe('orchestration federated message targeting', () => { function request( id: string, - capability: string, method: 'orchestration.send' | 'orchestration.ask', params: Record ): RpcRequest { @@ -98,7 +97,6 @@ function request( authToken: 'worker-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: `request_${id}`, - orchestrationCapability: capability, method, params } diff --git a/src/main/runtime/rpc/methods/orchestration/federation/federation-lifecycle-settlement.test.ts b/src/main/runtime/rpc/methods/orchestration/federation/federation-lifecycle-settlement.test.ts index e111865d9041..94341ac9c6fe 100644 --- a/src/main/runtime/rpc/methods/orchestration/federation/federation-lifecycle-settlement.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/federation/federation-lifecycle-settlement.test.ts @@ -56,15 +56,14 @@ describe('orchestration federation lifecycle settlement', () => { failNextAckBeforeDelivery = false throw new Error('connection lost before acknowledgment') } - return (await workerDispatcher.dispatch({ + return await workerDispatcher.dispatch({ id: `remote_${method}`, authToken: 'run-home-device-token', method, params, orchestrationContractVersion: envelope?.orchestrationContractVersion, - orchestrationRequestId: envelope?.orchestrationRequestId, - orchestrationCapability: envelope?.orchestrationCapability - })) as RuntimeRpcResponse + orchestrationRequestId: envelope?.orchestrationRequestId + }) } } homeRuntime = new OrcaRuntimeService(null, undefined, { @@ -155,14 +154,11 @@ describe('orchestration federation lifecycle settlement', () => { await homeDispatcher.dispatch(startRequest(taskId)) homeRuntime.stopOrchestrationFederationRelay() const dispatch = homeDb.getDispatchContext(taskId)! - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] const sent = workerDispatcher.dispatch({ id: 'rpc_waiting_worker_done', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'waiting_worker_done_request', - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', @@ -192,15 +188,12 @@ describe('orchestration federation lifecycle settlement', () => { signal?: AbortSignal, outcome: 'succeeded' | 'failed' = 'succeeded' ) { - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] return workerDispatcher.dispatch( { id: `rpc_${requestId}`, authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: requestId, - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', @@ -458,7 +451,7 @@ describe('orchestration federation lifecycle settlement', () => { payloadHash: `persisted_protocol_${protocolVersion}_payload` } }) - const capability = workerDb.prepareRemoteAttachmentAuthority({ + workerDb.prepareRemoteAttachmentAuthority({ dispatchId, paneKey: 'tab_worker:bbbbbbbb-bbbb-4bbb-8bbb-bbbbbbbbbbbb', processIncarnation: 'windows_runtime:pty:1', @@ -475,7 +468,6 @@ describe('orchestration federation lifecycle settlement', () => { authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: `persisted_protocol_${protocolVersion}_completion_request`, - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', @@ -662,8 +654,6 @@ describe('orchestration federation lifecycle settlement', () => { await homeDispatcher.dispatch(startRequest(task.id)) homeRuntime.stopOrchestrationFederationRelay() const dispatch = homeDb.getDispatchContext(task.id)! - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] const controller = new AbortController() const sent = workerDispatcher.dispatch( { @@ -671,7 +661,6 @@ describe('orchestration federation lifecycle settlement', () => { authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'aborted_worker_done_request', - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', diff --git a/src/main/runtime/rpc/methods/orchestration/federation/federation-output.test.ts b/src/main/runtime/rpc/methods/orchestration/federation/federation-output.test.ts index 3abaa36a62f5..cb298ad873f5 100644 --- a/src/main/runtime/rpc/methods/orchestration/federation/federation-output.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/federation/federation-output.test.ts @@ -3,7 +3,6 @@ import { mkdtemp, rm, writeFile } from 'node:fs/promises' import { tmpdir } from 'node:os' import { join } from 'node:path' import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import type { RuntimeRpcResponse } from '../../../../../../shared/runtime-rpc-envelope' import { ORCHESTRATION_CONTRACT_VERSION, ORCHESTRATION_FEDERATION_FLEET_SNAPSHOT_RUNTIME_CAPABILITY, @@ -119,15 +118,14 @@ describe('orchestration federated worker output', () => { error: { code: 'relay_provider_unavailable', message: 'relay unavailable' } } } - return (await workerDispatcher.dispatch({ + return await workerDispatcher.dispatch({ id: `remote_${method}`, authToken: 'run-home-device-token', method, params, orchestrationContractVersion: envelope?.orchestrationContractVersion, - orchestrationRequestId: envelope?.orchestrationRequestId, - orchestrationCapability: envelope?.orchestrationCapability - })) as RuntimeRpcResponse + orchestrationRequestId: envelope?.orchestrationRequestId + }) } } homeRuntime = new OrcaRuntimeService(null, undefined, { diff --git a/src/main/runtime/rpc/methods/orchestration/federation/federation.test.ts b/src/main/runtime/rpc/methods/orchestration/federation/federation.test.ts index 9c71da963d45..450f9ff9a5fb 100644 --- a/src/main/runtime/rpc/methods/orchestration/federation/federation.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/federation/federation.test.ts @@ -1,5 +1,4 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import type { RuntimeRpcResponse } from '../../../../../../shared/runtime-rpc-envelope' import { ORCHESTRATION_CONTRACT_VERSION, ORCHESTRATION_FEDERATION_CONTROL_MAIL_RUNTIME_CAPABILITY @@ -55,15 +54,14 @@ describe('orchestration federation', () => { _meta: { runtimeId: workerRuntime.getRuntimeId() } } } - const response = (await workerDispatcher.dispatch({ + const response = await workerDispatcher.dispatch({ id: `remote_${method}`, authToken: 'run-home-device-token', method, params, orchestrationContractVersion: envelope?.orchestrationContractVersion, - orchestrationRequestId: envelope?.orchestrationRequestId, - orchestrationCapability: envelope?.orchestrationCapability - })) as RuntimeRpcResponse + orchestrationRequestId: envelope?.orchestrationRequestId + }) if (method === 'orchestration.federationAck' && loseNextAckResponse) { loseNextAckResponse = false throw new Error('connection lost after acknowledgment') @@ -281,16 +279,12 @@ describe('orchestration federation', () => { const started = await homeDispatcher.dispatch(startRequest(task.id)) expect(started.ok).toBe(true) const dispatch = homeDb.getDispatchContext(task.id)! - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] - expect(capability).toBeTruthy() const sent = await workerDispatcher.dispatch({ id: 'rpc_worker_done', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'worker_done_request', - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', @@ -334,14 +328,11 @@ describe('orchestration federation', () => { const task = createHomeTask() await homeDispatcher.dispatch(startRequest(task.id)) const dispatch = homeDb.getDispatchContext(task.id)! - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] const ask = workerDispatcher.dispatch({ id: 'rpc_remote_ask', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'remote_question_request', - orchestrationCapability: capability, method: 'orchestration.ask', params: { from: 'term_windows_worker', @@ -403,14 +394,11 @@ describe('orchestration federation', () => { it('keeps a timed-out remote question resumable', async () => { const task = createHomeTask() await homeDispatcher.dispatch(startRequest(task.id)) - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] const timedOut = await workerDispatcher.dispatch({ id: 'rpc_remote_ask_timeout', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'remote_question_timeout_request', - orchestrationCapability: capability, method: 'orchestration.ask', params: { from: 'term_windows_worker', @@ -441,7 +429,6 @@ describe('orchestration federation', () => { authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'remote_question_resume_request', - orchestrationCapability: capability, method: 'orchestration.ask', params: { from: 'term_windows_worker', resume: questionId, timeoutMs: 5_000 } }) @@ -457,14 +444,11 @@ describe('orchestration federation', () => { const task = createHomeTask() await homeDispatcher.dispatch(startRequest(task.id)) homeRuntime.stopOrchestrationFederationRelay() - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] await workerDispatcher.dispatch({ id: 'rpc_remote_status', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'remote_status_request', - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', @@ -575,15 +559,12 @@ describe('orchestration federation', () => { const task = createHomeTask() await homeDispatcher.dispatch(startRequest(task.id)) const dispatch = homeDb.getDispatchContext(task.id)! - const prompt = vi.mocked(workerRuntime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' - const capability = prompt.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] homeRuntime.stopOrchestrationFederationRelay() await workerDispatcher.dispatch({ id: 'rpc_restart_status', authToken: 'worker-local-token', orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'restart_status_request', - orchestrationCapability: capability, method: 'orchestration.send', params: { from: 'term_windows_worker', diff --git a/src/main/runtime/rpc/methods/orchestration/federation/federation.ts b/src/main/runtime/rpc/methods/orchestration/federation/federation.ts index 067f583e37b4..ca63c1fb46d7 100644 --- a/src/main/runtime/rpc/methods/orchestration/federation/federation.ts +++ b/src/main/runtime/rpc/methods/orchestration/federation/federation.ts @@ -239,7 +239,7 @@ export const ORCHESTRATION_FEDERATION_ATTACH_METHODS = [ if (!paneKey || !processIncarnation) { throw new Error('stable_pane_required') } - const capability = db.prepareRemoteAttachmentAuthority({ + db.prepareRemoteAttachmentAuthority({ dispatchId: params.dispatchId, paneKey, processIncarnation, @@ -259,7 +259,6 @@ export const ORCHESTRATION_FEDERATION_ATTACH_METHODS = [ taskSpec: params.taskSpec, coordinatorHandle: 'Run home (relayed by Orca)', workerHandle: terminalHandle, - dispatchCapability: capability, devMode: params.devMode, // Why the worker host's own setting: enforcement runs here, with this // host's code, against this host's cap. diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/check-worker-consumer-fencing.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/check-worker-consumer-fencing.test.ts index 350fb33de04d..54c64feed624 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/check-worker-consumer-fencing.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/check-worker-consumer-fencing.test.ts @@ -2,7 +2,10 @@ import { afterEach, describe, expect, it, vi } from 'vitest' import type { RpcContext } from '../../../core' import type { OrcaRuntimeService } from '../../../../orca-runtime' import type { OrchestrationDb } from '../../../../orchestration/db' -import { createRootDispatch } from '../../../../orchestration/db/root-dispatch-test-fixture' +import { + createRootDispatch, + reattachDispatchConsumer +} from '../../../../orchestration/db/root-dispatch-test-fixture' import { createOrchestrationRpcHarness } from '../rpc-test-harness' const PANE_A = 'tab_a:cccccccc-cccc-4ccc-8ccc-cccccccccccc' @@ -30,7 +33,7 @@ describe('orchestration.check on a re-attached Dispatch', () => { ;({ db, runtime, ctx } = h.setup()) const task = db.createTask({ spec: 'worker that gets replaced' }) const dispatch = createRootDispatch(db, task.id, 'term_worker', PANE_A) - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId: dispatch.id, paneKey: PANE_A, processIncarnation: 'runtime:pty-a:1' @@ -53,7 +56,7 @@ describe('orchestration.check on a re-attached Dispatch', () => { } function reattach(dispatchId: string): void { - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId, paneKey: PANE_B, processIncarnation: 'runtime:pty-b:1' @@ -62,7 +65,7 @@ describe('orchestration.check on a re-attached Dispatch', () => { /** Same pane, new process: bumps the generation without moving the Dispatch off PANE_A. */ function remintOnSamePane(dispatchId: string): void { - db.mintDispatchCapability({ + reattachDispatchConsumer(db, { dispatchId, paneKey: PANE_A, processIncarnation: 'runtime:pty-a:2' diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/run-bound-mailbox-safety.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/run-bound-mailbox-safety.test.ts index f5f2c57575f5..3f759cc3d15a 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/run-bound-mailbox-safety.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/run-bound-mailbox-safety.test.ts @@ -1,5 +1,8 @@ import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' -import { createRootDispatch } from '../../../../orchestration/db/root-dispatch-test-fixture' +import { + createRootDispatch, + reattachDispatchConsumer +} from '../../../../orchestration/db/root-dispatch-test-fixture' import { createOrchestrationRpcHarness } from '../rpc-test-harness' const LEAD = 'tab_lead:22222222-2222-4222-9222-222222222222' @@ -207,7 +210,7 @@ describe('Run-bound lead mailbox boundaries', () => { }) it('does not give a reused process the previous assignee mail', async () => { - state.db.mintDispatchCapability({ + reattachDispatchConsumer(state.db, { dispatchId: dispatch.id, paneKey: LEAD, processIncarnation: 'old:pty:1' diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/send-dispatch-authority.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/send-dispatch-authority.test.ts index 5a1b6ee8c277..9f57957a10a2 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/send-dispatch-authority.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/send-dispatch-authority.test.ts @@ -43,16 +43,6 @@ describe('orchestration.send Dispatch authority', () => { vi.mocked(runtime.getTerminalPaneKey).mockImplementation((handle) => handle === 'term_attacker' ? 'tab_attacker:leaf_attacker' : harness.coordinatorPaneKey ) - if (!legacyAuthority) { - ctx = { - runtime, - orchestrationCapability: db.mintDispatchCapability({ - dispatchId: attacker.id, - paneKey: 'tab_attacker:leaf_attacker', - processIncarnation: 'runtime_test:term_attacker:1' - }) - } - } const result = (await send({ from: 'term_attacker', diff --git a/src/main/runtime/rpc/methods/orchestration/runs/dispatch-methods.ts b/src/main/runtime/rpc/methods/orchestration/runs/dispatch-methods.ts index 843df33357a6..0a8b59265cd2 100644 --- a/src/main/runtime/rpc/methods/orchestration/runs/dispatch-methods.ts +++ b/src/main/runtime/rpc/methods/orchestration/runs/dispatch-methods.ts @@ -136,13 +136,6 @@ export const ORCHESTRATION_DISPATCH_METHODS = [ creator: resolveDispatchCreator(runtime, params.from, orchestrationCaller), maxDepth: runtime.getNestedWorkerMaxDepth() }) - const dispatchCapability = params.inject - ? db.mintDispatchCapability({ - dispatchId: ctx.id, - paneKey: assigneePaneKey as string, - processIncarnation: processIncarnation as string - }) - : undefined // Why: built after ctx so dispatchId is the real ctx.id, letting heartbeats attribute liveness to a specific dispatch context, not just a task. const preamble = buildDispatchPreamble({ @@ -152,7 +145,6 @@ export const ORCHESTRATION_DISPATCH_METHODS = [ taskSpec: task.spec, coordinatorHandle: params.from ?? 'coordinator', workerHandle: to, - dispatchCapability, devMode: params.devMode, cliCommand: runtime.getTerminalOrchestrationCliCommand(to) }) diff --git a/src/main/runtime/rpc/methods/orchestration/worker/composed-workers.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/composed-workers.test.ts index d265858b0243..2c95af1aa072 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/composed-workers.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/composed-workers.test.ts @@ -161,7 +161,7 @@ describe('orchestration RPC methods', () => { }) expect(runtime.sendTerminalAgentPrompt).toHaveBeenCalledWith( 'term_worker', - expect.stringContaining('--dispatch-capability dcap_'), + expect.any(String), expect.objectContaining(dispatchPreambleSendOptions(expect.any(String))) ) }) diff --git a/src/main/runtime/rpc/methods/orchestration/worker/deliver-worker-dispatch-preamble.ts b/src/main/runtime/rpc/methods/orchestration/worker/deliver-worker-dispatch-preamble.ts index bc2b9601cebc..0d80079c13d8 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/deliver-worker-dispatch-preamble.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/deliver-worker-dispatch-preamble.ts @@ -27,7 +27,6 @@ export async function deliverWorkerDispatchPreamble(args: { taskId: string taskSpec: string coordinatorHandle: string - dispatchCapability: string devMode: boolean | undefined requestId: string }): Promise<{ @@ -45,7 +44,6 @@ export async function deliverWorkerDispatchPreamble(args: { taskSpec: args.taskSpec, coordinatorHandle: args.coordinatorHandle, workerHandle: terminalHandle, - dispatchCapability: args.dispatchCapability, devMode: args.devMode, cliCommand: runtime.getTerminalOrchestrationCliCommand(terminalHandle) }) diff --git a/src/main/runtime/rpc/methods/orchestration/worker/local-worker-start.ts b/src/main/runtime/rpc/methods/orchestration/worker/local-worker-start.ts index d6b265c41a97..7e10274435b5 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/local-worker-start.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/local-worker-start.ts @@ -215,7 +215,7 @@ export async function startLocalWorker(args: { } } const terminalAuthority = requireWorkerAuthority(runtime, terminalHandle) - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: terminalHandle, ...terminalAuthority, @@ -235,7 +235,6 @@ export async function startLocalWorker(args: { structuredSession, terminalHandle, coordinatorHandle: params.from, - dispatchCapability: capability, devMode: params.devMode, requestId: orchestrationMutation?.requestId ?? started.dispatch.id, agent: agent ?? null, diff --git a/src/main/runtime/rpc/methods/orchestration/worker/manual-dispatch-observation.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/manual-dispatch-observation.test.ts index 25da0b311a38..06b16eaff1ac 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/manual-dispatch-observation.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/manual-dispatch-observation.test.ts @@ -74,8 +74,7 @@ describe('manual Dispatch observation', () => { expect(db.getDispatchContextById(result.dispatch.id)).toMatchObject({ assignee_handle: 'term_worker', assignee_pane_key: workerPaneKey, - process_incarnation: 'runtime_test:term_worker:1', - capability_hash: expect.any(String) + process_incarnation: 'runtime_test:term_worker:1' }) const workerShowMethod = eraseRpcMethods(ORCHESTRATION_METHODS).find( @@ -89,7 +88,7 @@ describe('manual Dispatch observation', () => { runtime }) ).resolves.toMatchObject({ - worker: { state: 'unsupervised', stage: 'injected' }, + worker: { state: 'unsupervised', stage: 'context_only' }, observation: { status: 'live', exactWorker: true } }) }) @@ -126,11 +125,6 @@ describe('manual Dispatch observation', () => { 'launch-hash', 'runtime_test:term_worker:1' ) - db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: 'tab_worker:leaf_worker', - processIncarnation: 'runtime_test:term_worker:1' - }) const context = { runtime } const call = async (name: string, params: Record) => { const method = eraseRpcMethods(ORCHESTRATION_METHODS).find( @@ -180,7 +174,7 @@ describe('manual Dispatch observation', () => { dispatch: dispatch.id })) as { projection: { liveness: { verdict: string } } | null } expect(workerShow).toMatchObject({ - worker: { state: 'unsupervised', stage: 'injected', agentTerminalHandle: 'term_worker' }, + worker: { state: 'unsupervised', stage: 'context_only', agentTerminalHandle: 'term_worker' }, observation: { status: 'live', exactWorker: true } }) // Why: worker-show published only PTY liveness, so it read `live` for a dispatch that diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-interactive-wait.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-interactive-wait.test.ts index 956cecc5bc46..7b1e56f8fd22 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-interactive-wait.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-interactive-wait.test.ts @@ -109,11 +109,6 @@ describe('worker-show interactive wait (STA-3714, STA-4513)', () => { 'launch-hash', // A dispatch recorded against a process that has since been replaced. opts?.breakIdentity === true ? `${incarnation}:replaced` : incarnation ) - db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey, - processIncarnation: opts?.breakIdentity === true ? `${incarnation}:replaced` : incarnation - }) const method = workerShowMethod() return method.handler(method.params?.parse({ dispatch: dispatch.id }), { runtime }) diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-observation.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-observation.ts index f1202c6e7cfe..7bef1dc934cc 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-observation.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-observation.ts @@ -177,7 +177,8 @@ function exposeContextOnlyWorker(dispatch: DispatchContextRow) { dispatchId: dispatch.id, runtimeEpoch: null, state: 'unsupervised' as const, - stage: dispatch.capability_hash ? 'injected' : 'context_only', + // Why: with no worker row Orca supervises only the context; it keeps no record of an --inject paste. + stage: 'context_only', worktreeId: null, agentTerminalHandle: dispatch.assignee_handle, setupState: 'not_applicable', diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-budget.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-budget.test.ts index d258076a6a3d..983dc9c4a957 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-budget.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-budget.test.ts @@ -15,7 +15,6 @@ describe('worker-start prompt budget', () => { const prompt = buildDispatchPreamble({ taskId: 'task_test', dispatchId: 'ctx_test', - dispatchCapability: `dcap_${'A'.repeat(43)}`, taskSpec: spec, coordinatorHandle: 'term_coordinator', workerHandle: 'term_worker' @@ -31,7 +30,6 @@ describe('worker-start prompt budget', () => { const prompt = buildDispatchPreamble({ taskId: `task_${'a'.repeat(32)}`, dispatchId: `ctx_${'b'.repeat(32)}`, - dispatchCapability: `dcap_${'C'.repeat(43)}`, taskSpec: 'x'.repeat(ORCHESTRATION_WORKER_START_TASK_SPEC_MAX_BYTES), coordinatorHandle: `term_${'d'.repeat(256)}`, workerHandle: `term_${'e'.repeat(256)}`, diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-contract.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-contract.test.ts index 526b09a6320c..cba8baad45ff 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-contract.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-prompt-contract.test.ts @@ -213,7 +213,6 @@ describe('orchestration worker-start prompt contract', () => { expect(persisted.getTask(harness.taskId)?.status).toBe('dispatched') expect(persisted.getDispatchContextById(dispatchId)).toMatchObject({ status: 'dispatched', - capability_hash: expect.any(String), capability_revoked_at: null }) expect(persisted.getWorkerDispatch(dispatchId)).toMatchObject({ @@ -276,7 +275,6 @@ describe('orchestration worker-start prompt contract', () => { expect(await pending).toMatchObject({ ok: true, result: { state: 'outcome_unknown' } }) expect(harness.db.findActiveDispatchForAssignee(harness.handle)).toMatchObject({ status: 'pending', - capability_hash: expect.any(String), capability_revoked_at: null }) expect(harness.submittedTurns()).toBe(1) @@ -305,7 +303,7 @@ describe('orchestration worker-start prompt contract', () => { expect(harness.db.getQuestion(questionId)?.status).toBe('pending') }) - it('reports a swallowed Enter as start_unknown while keeping the worker and its capability', async () => { + it('reports a swallowed Enter as start_unknown while keeping the worker reportable', async () => { vi.useFakeTimers() const harness = await createPromptContractHarness('swallowed') const pending = harness.dispatcher.dispatch(harness.request) @@ -343,9 +341,8 @@ describe('orchestration worker-start prompt contract', () => { expect(persisted.getDispatchContextById(dispatchId)).toMatchObject({ status: 'pending', last_failure: null, - // The capability survives so a worker that recovers can still report; worker-report + // Lifecycle stays open so a worker that recovers can still report; worker-report // settlement reconnects a start_unknown worker through 'ready'. - capability_hash: expect.any(String), capability_revoked_at: null }) expect(persisted.getWorkerDispatch(dispatchId)).toMatchObject({ diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.test.ts index e39a06110dcd..ceeae2710d4d 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.test.ts @@ -42,7 +42,6 @@ function settle(delivered: 'accepted' | undefined) { structuredSession: { host, identity: { sessionId: 's1' } }, terminalHandle: 'structured_worker_1', coordinatorHandle: 'term_c', - dispatchCapability: 'capability', devMode: undefined, requestId: 'r1', agent: 'claude', diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.ts index 7e9118daffb5..98ddbecd6030 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-readiness-settlement.ts @@ -31,7 +31,6 @@ export async function deliverAndSettleWorkerStartReadiness(args: { structuredSession: Awaited> | null terminalHandle: string coordinatorHandle: string - dispatchCapability: string devMode: boolean | undefined requestId: string agent: string | null @@ -56,7 +55,6 @@ export async function deliverAndSettleWorkerStartReadiness(args: { taskId: task.id, taskSpec: task.spec, coordinatorHandle: args.coordinatorHandle, - dispatchCapability: args.dispatchCapability, devMode: args.devMode, requestId: args.requestId }) @@ -89,7 +87,7 @@ export async function deliverAndSettleWorkerStartReadiness(args: { const currentWorker = db.getWorkerDispatch(args.dispatchId) const alreadySettled = currentWorker && currentWorker.state !== 'starting' if (turnStart.verdict === 'unobserved' && !alreadySettled) { - // Honest `unverifiable`: keep the dispatch capability and the terminal — the worker may + // Honest `unverifiable`: keep lifecycle authority and the terminal — the worker may // still recover and report (worker-report settlement reconnects a start_unknown worker) — // but never claim ready for a turn nobody observed. effects.push({ diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-receipt.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-receipt.ts index f91ff690922e..e7d6b0d31edd 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-start-receipt.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-start-receipt.ts @@ -1,5 +1,4 @@ import type { OrchestrationDb } from '../../../../orchestration/db' -import { isAgentPromptStalledError } from '../../../../agent-prompt-submission-verification' import { isUnknownWorkerStartOutcome, type WorkerSetupReceipt } from './worker-topology' import type { OrchestrationWorkerLaunchReceipt } from './worker-launch-preferences' import type { WorkerStartModeReceipt } from '../../orchestration-worker-start-mode' @@ -20,11 +19,7 @@ export function failWorkerStartWithReceipt(args: { const unknown = isUnknownWorkerStartOutcome(args.error, args.failedStage) const worker = unknown ? args.db.markWorkerStartUnknown(args.dispatchId, args.failedStage, reason) - : args.db.failWorkerStart(args.dispatchId, args.failedStage, reason, { - // Why (#16095): the preamble is written before submission is verified, so a stalled - // verdict never means the worker lacks its task — keep the authority its report needs. - retainCapability: isAgentPromptStalledError(args.error) - }) + : args.db.failWorkerStart(args.dispatchId, args.failedStage, reason) // Only name cleanup this start actually left behind: a terminal it created and still owns. A // structured session is discarded by the teardown, a pane the user typed into is theirs, and an // unknown outcome is not settled — none of the three has anything for `worker-release` to close. diff --git a/src/main/runtime/rpc/methods/orchestration/worker/workers-new-worktree.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/workers-new-worktree.test.ts index cda3354e8165..fc7c1f3b26cf 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/workers-new-worktree.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/workers-new-worktree.test.ts @@ -224,7 +224,7 @@ describe('orchestration new-worktree workers', () => { expect(db.getDispatchContext(task.id)).toBeUndefined() }) - it('injects the execution host CLI command and Dispatch capability together', async () => { + it('injects the execution host CLI command without a Dispatch capability', async () => { mockCreatedWorktree() vi.mocked(runtime.getTerminalOrchestrationCliCommand).mockReturnValue('orca-ide') @@ -232,7 +232,6 @@ describe('orchestration new-worktree workers', () => { const prompt = vi.mocked(runtime.sendTerminalAgentPrompt).mock.calls[0]?.[1] ?? '' expect(prompt).toContain('orca-ide orchestration send') - expect(prompt).toMatch(/--dispatch-capability dcap_[A-Za-z0-9_-]+/) expect(prompt).not.toMatch(/(^|\s)orca orchestration send/) }) diff --git a/src/main/runtime/rpc/orchestration-commit-notify-characterization.test.ts b/src/main/runtime/rpc/orchestration-commit-notify-characterization.test.ts index a990fa9905b7..4ebd49df17e6 100644 --- a/src/main/runtime/rpc/orchestration-commit-notify-characterization.test.ts +++ b/src/main/runtime/rpc/orchestration-commit-notify-characterization.test.ts @@ -49,7 +49,7 @@ describe('orchestration commit-notify recovery', () => { taskId, startOptions: {} }) - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_worker', paneKey: workerPaneKey, @@ -59,7 +59,7 @@ describe('orchestration commit-notify recovery', () => { setupState: 'not_applicable' }) db.markWorkerDispatchReady(started.dispatch.id) - return { dispatch: db.getDispatchContextById(started.dispatch.id)!, capability } + return { dispatch: db.getDispatchContextById(started.dispatch.id)! } } async function throwAfterCommitAndReplay( @@ -201,7 +201,7 @@ describe('orchestration commit-notify recovery', () => { : null ) const task = db.createTask({ spec: 'Settle once' }) - const { dispatch, capability } = createReadyLocalWorker(db, task.id, workerPaneKey) + const { dispatch } = createReadyLocalWorker(db, task.id, workerPaneKey) const dispatcher = new RpcDispatcher({ runtime, methods: ORCHESTRATION_METHODS }) const workerDone = request('rpc_worker_done', 'mutation_worker_done', 'orchestration.send', { from: 'term_worker', @@ -213,7 +213,6 @@ describe('orchestration commit-notify recovery', () => { outcome: 'succeeded' }) }) - workerDone.orchestrationCapability = capability const waiting = runtime.waitForMessage(`run:${activeRunId}`, { typeFilter: ['worker_done'], timeoutMs: 5_000 @@ -270,7 +269,7 @@ describe('orchestration commit-notify recovery', () => { handle.startsWith('term_') ? `runtime_test:${handle}:1` : null ) const task = db.createTask({ spec: 'Resume before atomic settlement', runId: run.id }) - const { dispatch, capability } = createReadyLocalWorker(db, task.id, workerPaneKey) + const { dispatch } = createReadyLocalWorker(db, task.id, workerPaneKey) const workerDone = request( 'rpc_worker_done_before_crash', 'mutation_worker_done_before_crash', @@ -286,7 +285,6 @@ describe('orchestration commit-notify recovery', () => { }) } ) - workerDone.orchestrationCapability = capability vi.spyOn(db, 'commitWorkerDoneMessageMutation').mockImplementationOnce(() => { throw new OrchestrationError( 'operation_unknown', @@ -384,7 +382,7 @@ describe('orchestration commit-notify recovery', () => { : null ) const task = db.createTask({ spec: 'Commit report and settlement together' }) - const { dispatch, capability } = createReadyLocalWorker(db, task.id, workerPaneKey) + const { dispatch } = createReadyLocalWorker(db, task.id, workerPaneKey) const dispatcher = new RpcDispatcher({ runtime, methods: ORCHESTRATION_METHODS }) const workerDone = request( 'rpc_atomic_worker_done', @@ -401,7 +399,6 @@ describe('orchestration commit-notify recovery', () => { }) } ) - workerDone.orchestrationCapability = capability const callerFingerprint = db.getOrCreateLocalMutationCallerFingerprint() inject(db) diff --git a/src/main/runtime/rpc/orchestration-current-authority-precedence.test.ts b/src/main/runtime/rpc/orchestration-current-authority-precedence.test.ts index 41e40aff7a07..d13a823ba46a 100644 --- a/src/main/runtime/rpc/orchestration-current-authority-precedence.test.ts +++ b/src/main/runtime/rpc/orchestration-current-authority-precedence.test.ts @@ -125,48 +125,44 @@ describe('current orchestration authority precedence', () => { it.each([ ['heartbeat', undefined], ['worker_done', 'succeeded'] - ] as const)( - 'routes a capability-backed current %s before legacy fallback', - async (type, outcome) => { - const harness = createHarness() - const { taskId, dispatchId, capability } = createCurrentDispatch(harness) - const payload = JSON.stringify({ - taskId, - dispatchId, - ...(outcome ? { outcome } : {}) - }) - const response = await harness.dispatcher.dispatch({ - ...request( - 'orchestration.send', - { - from: CURRENT_WORKER_HANDLE, - subject: type === 'heartbeat' ? 'alive' : 'Completed', - type, - payload - }, - currentEvidence('worker'), - `current-${type}` - ), - orchestrationCapability: capability - }) + ] as const)('routes a current %s before legacy fallback', async (type, outcome) => { + const harness = createHarness() + const { taskId, dispatchId } = createCurrentDispatch(harness) + const payload = JSON.stringify({ + taskId, + dispatchId, + ...(outcome ? { outcome } : {}) + }) + const response = await harness.dispatcher.dispatch({ + ...request( + 'orchestration.send', + { + from: CURRENT_WORKER_HANDLE, + subject: type === 'heartbeat' ? 'alive' : 'Completed', + type, + payload + }, + currentEvidence('worker'), + `current-${type}` + ) + }) - expect(response).toMatchObject({ - ok: true, - result: { - message: { - from_handle: CURRENT_WORKER_HANDLE, - to_handle: expect.stringMatching(/^run:/), - type, - delivery_contract: 'current_delivery' - } + expect(response).toMatchObject({ + ok: true, + result: { + message: { + from_handle: CURRENT_WORKER_HANDLE, + to_handle: expect.stringMatching(/^run:/), + type, + delivery_contract: 'current_delivery' } - }) - expect(response).not.toHaveProperty('result.legacyCompatibility') - expect(harness.db.getTask(taskId)?.status).toBe( - type === 'worker_done' ? 'completed' : 'dispatched' - ) - } - ) + } + }) + expect(response).not.toHaveProperty('result.legacyCompatibility') + expect(harness.db.getTask(taskId)?.status).toBe( + type === 'worker_done' ? 'completed' : 'dispatched' + ) + }) it('routes a current worker mailbox check before legacy fallback', async () => { const harness = createHarness() @@ -273,7 +269,7 @@ describe('current orchestration authority precedence', () => { '%s routes a reused legacy terminal current ask before its retained principal', async (transport) => { const harness = createHarness() - const { dispatchId, capability } = await createReusedCurrentDispatch(harness, transport) + const { dispatchId } = await createReusedCurrentDispatch(harness, transport) const response = await invoke( harness.dispatcher, @@ -287,8 +283,7 @@ describe('current orchestration authority precedence', () => { }, evidence('worker'), `reused-current-ask-${transport}` - ), - orchestrationCapability: capability + ) }, transport ) @@ -312,7 +307,6 @@ function createCurrentDispatch(harness: ReturnType): { runId: string taskId: string dispatchId: string - capability: string } { const run = harness.db.createRun({ objective: 'current work', @@ -324,14 +318,11 @@ function createCurrentDispatch(harness: ReturnType): { harness.db, task.id, CURRENT_WORKER_HANDLE, - CURRENT_WORKER_PANE + CURRENT_WORKER_PANE, + undefined, + 'process-1' ) - const capability = harness.db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: CURRENT_WORKER_PANE, - processIncarnation: 'process-1' - }) - return { runId: run.id, taskId: task.id, dispatchId: dispatch.id, capability } + return { runId: run.id, taskId: task.id, dispatchId: dispatch.id } } async function createReusedCurrentDispatch( @@ -341,7 +332,6 @@ async function createReusedCurrentDispatch( runId: string taskId: string dispatchId: string - capability: string }> { const legacyCheck = await harness.dispatcher.dispatch( request( @@ -360,11 +350,13 @@ async function createReusedCurrentDispatch( coordinatorPaneKey: CURRENT_COORDINATOR_PANE }) const task = harness.db.createTask({ spec: 'reused terminal assignment', runId: run.id }) - const dispatch = createRootDispatch(harness.db, task.id, WORKER_HANDLE, WORKER_PANE) - const capability = harness.db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: WORKER_PANE, - processIncarnation: 'process-1' - }) - return { runId: run.id, taskId: task.id, dispatchId: dispatch.id, capability } + const dispatch = createRootDispatch( + harness.db, + task.id, + WORKER_HANDLE, + WORKER_PANE, + undefined, + 'process-1' + ) + return { runId: run.id, taskId: task.id, dispatchId: dispatch.id } } diff --git a/src/main/runtime/rpc/orchestration-mutation-ledger.test.ts b/src/main/runtime/rpc/orchestration-mutation-ledger.test.ts index b87f595f4b24..6909b4f33457 100644 --- a/src/main/runtime/rpc/orchestration-mutation-ledger.test.ts +++ b/src/main/runtime/rpc/orchestration-mutation-ledger.test.ts @@ -377,18 +377,19 @@ describe('durable orchestration mutation ledger', () => { coordinatorPaneKey: 'tab_coord:leaf_coord' }) const task = db.createTask({ spec: 'ask', runId: run.id }) - const dispatch = createRootDispatch(db, task.id, 'term_worker', 'tab_worker:leaf_worker') - const capability = db.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: 'tab_worker:leaf_worker', - processIncarnation: 'runtime:pty:1' - }) + createRootDispatch( + db, + task.id, + 'term_worker', + 'tab_worker:leaf_worker', + undefined, + 'runtime:pty:1' + ) const askRequest: RpcRequest = { id: 'rpc_ask_1', authToken: 'caller-token', method: 'orchestration.ask', params: { from: 'term_worker', question: 'Proceed?', timeoutMs: 60_000 }, - orchestrationCapability: capability, orchestrationContractVersion: ORCHESTRATION_CONTRACT_VERSION, orchestrationRequestId: 'mutation_ask' } diff --git a/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts b/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts index d3cf0769e847..db1da5c93d08 100644 --- a/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts +++ b/src/main/runtime/rpc/orchestration-runtime-update-settlement.test.ts @@ -27,7 +27,6 @@ type Harness = { createDispatcher: () => RpcDispatcher taskId: string dispatchId: string - capability: string adoptedRunId: string markerPath: string } @@ -66,12 +65,18 @@ function createUpdateHarness(): Harness { spec: 'finish work across an app update', createdByTerminalHandle: COORDINATOR_HANDLE }) - const dispatch = createRootDispatch(oldRuntimeDb, task.id, WORKER_HANDLE, WORKER_PANE) - const capability = oldRuntimeDb.mintDispatchCapability({ - dispatchId: dispatch.id, - paneKey: WORKER_PANE, - processIncarnation: PROCESS_INCARNATION - }) + const dispatch = createRootDispatch( + oldRuntimeDb, + task.id, + WORKER_HANDLE, + WORKER_PANE, + undefined, + PROCESS_INCARNATION + ) + // The pre-update runtime minted a capability; its hash is what classifies the row as current. + oldRuntimeDb.db + .prepare("UPDATE dispatch_contexts SET capability_hash = 'minted-before-update' WHERE id = ?") + .run(dispatch.id) oldRuntimeDb.close() const raw = new Database(dbPath) @@ -152,7 +157,6 @@ function createUpdateHarness(): Harness { createDispatcher, taskId: task.id, dispatchId: dispatch.id, - capability, adoptedRunId: adoptedRunId as string, markerPath } @@ -233,7 +237,6 @@ describe('orchestration runtime update settlement', () => { 'worker', 'retained-worker-done' ) - completion.orchestrationCapability = harness.capability const first = await harness.createDispatcher().dispatch(completion) const replay = await harness.createDispatcher().dispatch({ ...completion, id: 'rpc_replay' }) @@ -326,7 +329,6 @@ describe('orchestration runtime update settlement', () => { 'worker', 'forged-worker-done' ) - completion.orchestrationCapability = harness.capability completion.orchestrationCompatibilityEvidence = { ...completion.orchestrationCompatibilityEvidence!, paneKey: 'tab_foreign:99999999-9999-4999-8999-999999999999' @@ -405,8 +407,8 @@ describe('orchestration runtime update settlement', () => { dispatch_id: attachment.dispatch_id } as never) - const response = await harness.createDispatcher().dispatch({ - ...request( + const response = await harness.createDispatcher().dispatch( + request( 'orchestration.send', { from: CURRENT_COORDINATOR_HANDLE, @@ -416,9 +418,8 @@ describe('orchestration runtime update settlement', () => { }, 'current-coordinator', 'retained-remote-status' - ), - orchestrationCapability: 'dcap_remote_retained' - }) + ) + ) expect(resultOf(response)).toMatchObject({ relay: { dispatchId: attachment.dispatch_id, accepted: true } diff --git a/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts b/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts index 2c8e84f15749..d09c47939f00 100644 --- a/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts +++ b/src/main/runtime/rpc/orchestration-task-dispatch-invariant.test.ts @@ -9,6 +9,7 @@ import { OrchestrationDb } from '../orchestration/db' import type { RpcRequest, RpcResponse } from './core' import { RpcDispatcher } from './dispatcher' import { ORCHESTRATION_METHODS } from './methods/orchestration' +import { reattachDispatchConsumer } from '../orchestration/db/root-dispatch-test-fixture' const COORDINATOR_HANDLE = 'term_invariant_coordinator' const COORDINATOR_PANE = 'tab_coord:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa' @@ -272,7 +273,7 @@ async function createCapableDispatch( harness: Harness, taskId: string, status: 'pending' | 'dispatched' -): Promise<{ dispatch: { id: string }; capability: string }> { +): Promise<{ dispatch: { id: string } }> { if (status === 'pending') { const dispatch = harness.db.createStartingWorkerDispatch({ creator: { kind: 'system' }, @@ -280,7 +281,7 @@ async function createCapableDispatch( taskId, startOptions: {} }).dispatch - const capability = harness.db.prepareStartingWorkerAuthority({ + harness.db.prepareStartingWorkerAuthority({ dispatchId: dispatch.id, handle: WORKER_HANDLE, paneKey: WORKER_PANE, @@ -290,29 +291,29 @@ async function createCapableDispatch( effects: [], terminalOwnership: 'external' }) - return { dispatch, capability } + return { dispatch } } const dispatch = await dispatchTask(harness, taskId, WORKER_HANDLE) - const capability = harness.db.mintDispatchCapability({ + reattachDispatchConsumer(harness.db, { dispatchId: dispatch.id, paneKey: WORKER_PANE, processIncarnation: WORKER_PROCESS }) - return { dispatch, capability } + return { dispatch } } function createSupervisedDispatch( harness: Harness, taskId: string, status: 'pending' | 'dispatched' -): { dispatch: { id: string }; capability: string } { +): { dispatch: { id: string } } { const dispatch = harness.db.createStartingWorkerDispatch({ creator: { kind: 'system' }, maxDepth: Number.MAX_SAFE_INTEGER, taskId, startOptions: {} }).dispatch - const capability = harness.db.prepareStartingWorkerAuthority({ + harness.db.prepareStartingWorkerAuthority({ dispatchId: dispatch.id, handle: WORKER_HANDLE, paneKey: WORKER_PANE, @@ -325,7 +326,7 @@ function createSupervisedDispatch( if (status === 'dispatched') { harness.db.markWorkerDispatchReady(dispatch.id) } - return { dispatch, capability } + return { dispatch } } function updateTask( diff --git a/src/main/runtime/rpc/rpc-streaming-dispatcher.ts b/src/main/runtime/rpc/rpc-streaming-dispatcher.ts index 6c2faad902e5..07706125c235 100644 --- a/src/main/runtime/rpc/rpc-streaming-dispatcher.ts +++ b/src/main/runtime/rpc/rpc-streaming-dispatcher.ts @@ -141,7 +141,6 @@ export class RpcStreamingDispatcher { clientKind: options?.clientKind, clientCapabilities: options?.clientCapabilities, updateClientCapabilities: options?.updateClientCapabilities, - orchestrationCapability: request.orchestrationCapability, authenticatedCallerFingerprint: mutation?.identity.callerFingerprint ?? legacyCoordinator?.mutationCallerFingerprint ?? @@ -196,7 +195,6 @@ export class RpcStreamingDispatcher { clientKind: options?.clientKind, clientCapabilities: options?.clientCapabilities, updateClientCapabilities: options?.updateClientCapabilities, - orchestrationCapability: request.orchestrationCapability, pairing: options?.pairing, sendBinary: options?.sendBinary, registerBinaryStreamHandler: options?.registerBinaryStreamHandler, diff --git a/src/main/runtime/structured-chat-coordinator-mail.test.ts b/src/main/runtime/structured-chat-coordinator-mail.test.ts index a36912fae955..1490f9868e3c 100644 --- a/src/main/runtime/structured-chat-coordinator-mail.test.ts +++ b/src/main/runtime/structured-chat-coordinator-mail.test.ts @@ -58,7 +58,7 @@ let requests = 0 function request( method: string, params: Record, - options: { sessionId?: string; capability?: string } = {} + options: { sessionId?: string } = {} ): RpcRequest { requests += 1 return { @@ -70,15 +70,14 @@ function request( orchestrationRequestId: `req-${requests}`, ...(options.sessionId ? { orchestrationCompatibilityEvidence: { agentSessionId: options.sessionId } } - : {}), - ...(options.capability ? { orchestrationCapability: options.capability } : {}) + : {}) } } async function call( method: string, params: Record, - options?: { sessionId?: string; capability?: string } + options?: { sessionId?: string } ): Promise> { const response = await dispatcher.dispatch(request(method, params, options)) if (!response.ok) { @@ -160,7 +159,7 @@ async function userTexts(sessionId: string): Promise { ) } -/** A capability-backed terminal worker under the coordinator's Run, and its worker_done. */ +/** A supervised terminal worker under the coordinator's Run, and its worker_done. */ async function finishWorker( taskId: string, worker: { handle: string; paneKey: string } = { handle: 'term_worker', paneKey: WORKER_PANE } @@ -171,7 +170,7 @@ async function finishWorker( taskId, startOptions: {} }) - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: worker.handle, paneKey: worker.paneKey, @@ -181,16 +180,12 @@ async function finishWorker( setupState: 'not_applicable' }) db.markWorkerDispatchReady(started.dispatch.id) - await call( - 'orchestration.send', - { - from: worker.handle, - subject: 'Done', - type: 'worker_done', - payload: JSON.stringify({ taskId, dispatchId: started.dispatch.id, outcome: 'succeeded' }) - }, - { capability } - ) + await call('orchestration.send', { + from: worker.handle, + subject: 'Done', + type: 'worker_done', + payload: JSON.stringify({ taskId, dispatchId: started.dispatch.id, outcome: 'succeeded' }) + }) } async function coordinatorRunAndTask(): Promise<{ runId: string; taskId: string }> { diff --git a/src/main/ssh/ssh-remote-orca-cli.test.ts b/src/main/ssh/ssh-remote-orca-cli.test.ts index 637a0c55ce3e..90e7830099d8 100644 --- a/src/main/ssh/ssh-remote-orca-cli.test.ts +++ b/src/main/ssh/ssh-remote-orca-cli.test.ts @@ -376,7 +376,7 @@ describe('runRemoteOrcaCli', () => { } }) - it('carries the Dispatch capability through the SSH envelope', async () => { + it("still accepts an older host's --dispatch-capability through the SSH bridge", async () => { const db = new OrchestrationDb(':memory:') const runtime = new OrcaRuntimeService() runtime.setOrchestrationDb(db) @@ -396,7 +396,7 @@ describe('runRemoteOrcaCli', () => { taskId: task.id, startOptions: {} }) - const capability = db.prepareStartingWorkerAuthority({ + db.prepareStartingWorkerAuthority({ dispatchId: started.dispatch.id, handle: 'term_ssh', paneKey: 'tab_ssh:leaf_ssh', @@ -425,7 +425,7 @@ describe('runRemoteOrcaCli', () => { '--outcome', 'succeeded', '--dispatch-capability', - capability, + 'dcap_from_an_old_host', '--json' ], cwd: '/home/alice/repo', diff --git a/src/main/ssh/ssh-remote-orca-cli.ts b/src/main/ssh/ssh-remote-orca-cli.ts index 6f476fa0d84d..3913a65cb9f5 100644 --- a/src/main/ssh/ssh-remote-orca-cli.ts +++ b/src/main/ssh/ssh-remote-orca-cli.ts @@ -220,10 +220,7 @@ async function dispatchRemoteCli( // authority as the full host CLI passthrough. senderPaneKey: env.ORCA_PANE_KEY || undefined }, - { - ...compatibilityEnvelope, - orchestrationCapability: optionalRemoteCliString(parsed.flags, 'dispatch-capability') - } + compatibilityEnvelope ) } case 'orchestration check': @@ -263,10 +260,7 @@ async function dispatchRemoteCli( run: optionalRemoteCliString(parsed.flags, 'run'), compatibilityCliCommand: 'orca' }, - { - ...compatibilityEnvelope, - orchestrationCapability: optionalRemoteCliString(parsed.flags, 'dispatch-capability') - } + compatibilityEnvelope ) case 'orchestration reply': return await call( @@ -310,7 +304,6 @@ async function call( authToken: 'remote-cli', method, params, - orchestrationCapability: envelope?.orchestrationCapability, orchestrationContractVersion: method.startsWith('orchestration.') ? ORCHESTRATION_CONTRACT_VERSION : undefined, diff --git a/tests/e2e/completed-worker-retirement-resume.spec.ts b/tests/e2e/completed-worker-retirement-resume.spec.ts index 0af7ca46f123..f626b978118e 100644 --- a/tests/e2e/completed-worker-retirement-resume.spec.ts +++ b/tests/e2e/completed-worker-retirement-resume.spec.ts @@ -12,7 +12,7 @@ import { completedWorkerFakeCodexCommand, completedWorkerLaunchEnv, listRuntimeTerminals, - readCompletedWorkerDispatchCapability, + hasCompletedWorkerReceivedPreamble, readCompletedWorkerLedger, readPersistedWorkerRecoveryRecord, runBuiltOrcaCli, @@ -176,13 +176,7 @@ for (const closeMode of ['terminal-close-cli', 'worker-release'] as const) { await expect .poll(() => readCompletedWorkerLedger().filter((event) => event.event === 'spawn')) .toHaveLength(1) - let dispatchCapability: string | null = null - await expect - .poll(() => { - dispatchCapability = readCompletedWorkerDispatchCapability() - return dispatchCapability - }) - .not.toBeNull() + await expect.poll(() => hasCompletedWorkerReceivedPreamble()).toBe(true) await expect .poll(() => readCompletedWorkerLedger() @@ -190,9 +184,6 @@ for (const closeMode of ['terminal-close-cli', 'worker-release'] as const) { .map((event) => event.mode) ) .toEqual(['bracketed']) - if (!dispatchCapability) { - throw new Error('Background worker did not receive its dispatch capability') - } seedCurrentCodexTranscript(isolatedHome, PROVIDER_SESSION_ID, targetWorktreePath) @@ -237,21 +228,17 @@ for (const closeMode of ['terminal-close-cli', 'worker-release'] as const) { ) .toEqual(expectedRecovery) - const completed = await client.call<{ message: { type: string } }>( - 'orchestration.send', - { - from: workerHandle, - subject: 'Completed', - body: 'The fixture completed. It found no work. Nothing remains.', - type: 'worker_done', - payload: JSON.stringify({ - taskId: task.result.task.id, - dispatchId: started.result.dispatchId, - outcome: 'succeeded' - }) - }, - { orchestrationCapability: dispatchCapability } - ) + const completed = await client.call<{ message: { type: string } }>('orchestration.send', { + from: workerHandle, + subject: 'Completed', + body: 'The fixture completed. It found no work. Nothing remains.', + type: 'worker_done', + payload: JSON.stringify({ + taskId: task.result.task.id, + dispatchId: started.result.dispatchId, + outcome: 'succeeded' + }) + }) expect(completed.result.message.type).toBe('worker_done') await expect .poll( diff --git a/tests/e2e/helpers/completed-worker-retirement-fixture.ts b/tests/e2e/helpers/completed-worker-retirement-fixture.ts index 7b217a6e69a6..9135a84fb8a0 100644 --- a/tests/e2e/helpers/completed-worker-retirement-fixture.ts +++ b/tests/e2e/helpers/completed-worker-retirement-fixture.ts @@ -150,12 +150,12 @@ export function readCompletedWorkerLedger(): LifecycleEvent[] { .map((line) => JSON.parse(line) as LifecycleEvent) } -export function readCompletedWorkerDispatchCapability(): string | null { - const input = readCompletedWorkerLedger() +export function hasCompletedWorkerReceivedPreamble(): boolean { + return readCompletedWorkerLedger() .filter((event) => event.event === 'input') .map((event) => event.input ?? '') .join('') - return input.match(/--dispatch-capability\s+(\S+)/)?.[1] ?? null + .includes('--type worker_done') } export function runBuiltOrcaCli( diff --git a/tests/e2e/orchestration-worker-settlement-release-cli.spec.ts b/tests/e2e/orchestration-worker-settlement-release-cli.spec.ts index 43b1f6a196fe..ce2278aed457 100644 --- a/tests/e2e/orchestration-worker-settlement-release-cli.spec.ts +++ b/tests/e2e/orchestration-worker-settlement-release-cli.spec.ts @@ -27,7 +27,6 @@ if (process.argv.slice(2).includes('app-server')) { process.stderr.write("error: unrecognized subcommand 'app-server'\\n") process.exit(2) } -let capability = null let acknowledged = false ${FAKE_AGENT_PASTE_END_SCANNER_SOURCE} process.stdout.write('\\u001b]0;Codex Ready\\u0007OpenAI Codex\\nmodel: e2e\\ndirectory: e2e\\n') @@ -38,7 +37,6 @@ process.stdin.on('data', (chunk) => { if (pasteEndScan.pasteEndOffset !== null) { process.stdout.write('\\x1b[?25h') } - capability ||= input.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] || null if (!acknowledged) { fakeAgentMaybeAck(pasteEndScan, input, (mode) => { acknowledged = true @@ -48,12 +46,11 @@ process.stdin.on('data', (chunk) => { }) } const encoded = input.match(/ORCA_E2E_WORKER_DONE:([A-Za-z0-9+/=]+)/)?.[1] - if (!encoded || !capability) return + if (!encoded) return const request = JSON.parse(Buffer.from(encoded, 'base64').toString('utf8')) const args = [ 'orchestration', 'send', '--from', request.mismatch ? 'term_foreign' : process.env.ORCA_TERMINAL_HANDLE, - '--dispatch-capability', capability, '--to', request.coordinator, '--type', 'worker_done', '--subject', request.mismatch ? 'wrong sender' : 'completed', @@ -235,7 +232,7 @@ test('compiled CLI rejects false completion then reconciles the dead retained wo expect.soft(rejected.status).not.toBe(0) expect.soft(JSON.parse(rejected.stdout)).toMatchObject({ ok: false, - error: { code: 'dispatch_capability_invalid' } + error: { code: 'consumer_fenced' } }) const stillDispatched = await client.call<{ dispatch: { status: string } | null }>( 'orchestration.dispatchShow', diff --git a/tests/e2e/orchestration-worker-transcript-providers.spec.ts b/tests/e2e/orchestration-worker-transcript-providers.spec.ts index 78314f06d716..903c55ca24a8 100644 --- a/tests/e2e/orchestration-worker-transcript-providers.spec.ts +++ b/tests/e2e/orchestration-worker-transcript-providers.spec.ts @@ -105,7 +105,7 @@ const PROVIDERS: readonly { ] const fakeCliDir = mkdtempSync(path.join(os.tmpdir(), 'orca-e2e-worker-transcript-providers-')) -const capabilityLedgerPath = path.join(fakeCliDir, 'capabilities.jsonl') +const preambleLedgerPath = path.join(fakeCliDir, 'preambles.jsonl') const fakeGrokHome = path.join(fakeCliDir, 'grok-home') const fakeOmpHome = path.join(fakeCliDir, 'omp-home') @@ -114,7 +114,7 @@ function writeFakeProvider(agent: TranscriptProvider, title: string): string { const hookPath = `/hook/${agent}` const source = ` const { appendFileSync, readFileSync } = require('node:fs') -const ledger = ${JSON.stringify(capabilityLedgerPath)} +const ledger = ${JSON.stringify(preambleLedgerPath)} const configPath = ${JSON.stringify(configPath)} let hookSent = false async function sendProviderHook() { @@ -142,9 +142,8 @@ async function sendProviderHook() { process.stdout.write('\\u001b]0;${title.replaceAll("'", "\\'")}\\u0007') process.stdin.on('data', (chunk) => { const input = chunk.toString() - const capability = input.match(/--dispatch-capability (dcap_[A-Za-z0-9_-]+)/)?.[1] - if (capability) { - appendFileSync(ledger, JSON.stringify({ agent: '${agent}', capability }) + '\\n') + if (input.includes('--type worker_done')) { + appendFileSync(ledger, '${agent}\\n') void sendProviderHook() } }) @@ -191,14 +190,10 @@ test.afterAll(() => { rmSync(fakeCliDir, { recursive: true, force: true }) }) -function readCapabilities(): { agent: TranscriptProvider; capability: string }[] { - if (!existsSync(capabilityLedgerPath)) { - return [] - } - return readFileSync(capabilityLedgerPath, 'utf8') - .split(/\r?\n/) - .filter(Boolean) - .map((line) => JSON.parse(line) as { agent: TranscriptProvider; capability: string }) +function readPreambleRecipients(): string[] { + return existsSync(preambleLedgerPath) + ? readFileSync(preambleLedgerPath, 'utf8').split(/\r?\n/).filter(Boolean) + : [] } async function listWorker(client: RuntimeClient, handle: string): Promise { @@ -215,7 +210,7 @@ test('worker-read uses provider transcripts across supported orchestration agent electronApp }) => { test.setTimeout(240_000) - rmSync(capabilityLedgerPath, { force: true }) + rmSync(preambleLedgerPath, { force: true }) await waitForSessionReady(orcaPage) await orcaPage.evaluate( async ({ commands, terminalWindowsShell }) => { @@ -366,30 +361,18 @@ test('worker-read uses provider transcripts across supported orchestration agent ) ).toEqual([[provider.third]]) - await expect - .poll(() => readCapabilities().find((entry) => entry.agent === provider.agent)) - .toBeTruthy() - const capability = readCapabilities().find( - (entry) => entry.agent === provider.agent - )?.capability - if (!capability) { - throw new Error(`${provider.agent} worker did not receive a dispatch capability`) - } - await client.call( - 'orchestration.send', - { - from: worker.handle, - subject: 'Completed', - body: `The ${provider.agent} transcript read passed. Nothing remains.`, - type: 'worker_done', - payload: JSON.stringify({ - taskId: task.result.task.id, - dispatchId: started.result.dispatchId, - outcome: 'succeeded' - }) - }, - { orchestrationCapability: capability } - ) + await expect.poll(() => readPreambleRecipients()).toContain(provider.agent) + await client.call('orchestration.send', { + from: worker.handle, + subject: 'Completed', + body: `The ${provider.agent} transcript read passed. Nothing remains.`, + type: 'worker_done', + payload: JSON.stringify({ + taskId: task.result.task.id, + dispatchId: started.result.dispatchId, + outcome: 'succeeded' + }) + }) await expect .poll(async () => { const dispatch = await client.call<{ dispatch: { status: string } | null }>( diff --git a/tests/e2e/settled-worker-tab-survives-restart.spec.ts b/tests/e2e/settled-worker-tab-survives-restart.spec.ts index 695abfa0325a..10de089ec5ea 100644 --- a/tests/e2e/settled-worker-tab-survives-restart.spec.ts +++ b/tests/e2e/settled-worker-tab-survives-restart.spec.ts @@ -20,7 +20,7 @@ import { completedWorkerFakeCodexCommand, completedWorkerLaunchEnv, listRuntimeTerminals, - readCompletedWorkerDispatchCapability, + hasCompletedWorkerReceivedPreamble, readCompletedWorkerLedger, seedCurrentCodexTranscript } from './helpers/completed-worker-retirement-fixture' @@ -208,16 +208,7 @@ for (const daemonSessionGone of [false, true]) { const workerTabId = worker.tabId const workerPaneKey = `${worker.tabId}:${worker.leafId}` await backgroundMountTab(first.page, targetWorktreeId, workerTabId) - let dispatchCapability: string | null = null - await expect - .poll(() => { - dispatchCapability = readCompletedWorkerDispatchCapability() - return dispatchCapability - }) - .not.toBeNull() - if (!dispatchCapability) { - throw new Error('Background worker did not receive its dispatch capability') - } + await expect.poll(() => hasCompletedWorkerReceivedPreamble()).toBe(true) const transcriptPath = seedCurrentCodexTranscript( isolatedHome, PROVIDER_SESSION_ID, @@ -267,21 +258,17 @@ for (const daemonSessionGone of [false, true]) { worktreeId: targetWorktreeId } ) - const completed = await client.call<{ message: { type: string } }>( - 'orchestration.send', - { - from: workerHandle, - subject: 'Completed', - body: 'The fixture completed and stays open for inspection.', - type: 'worker_done', - payload: JSON.stringify({ - taskId: task.result.task.id, - dispatchId: started.result.dispatchId, - outcome: 'succeeded' - }) - }, - { orchestrationCapability: dispatchCapability } - ) + const completed = await client.call<{ message: { type: string } }>('orchestration.send', { + from: workerHandle, + subject: 'Completed', + body: 'The fixture completed and stays open for inspection.', + type: 'worker_done', + payload: JSON.stringify({ + taskId: task.result.task.id, + dispatchId: started.result.dispatchId, + outcome: 'succeeded' + }) + }) expect(completed.result.message.type).toBe('worker_done') const taskBeforeRestart = ( await client.call('orchestration.taskList', { run: run.result.run.id }) From 304735301797ccc94878930b4b86863f978a24ba Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Wed, 30 Sep 2026 13:41:34 -0400 Subject: [PATCH 12/79] fix(orchestration): worker-abandon settles a stuck worker and records who did it (#23983) * fix(orchestration): worker-abandon settles a stuck worker and records who abandoned it worker-abandon refused or no-oped in the states it exists to escape: a stop stranded by a dead runtime, an active attempt that was no longer the Task's latest, and settled workers whose terminal release was stuck at requested or unknown. It now settles every non-terminal worker except this runtime's own in-flight stop, records who abandoned it, and retains (never closes) an owned terminal whose release is not already in flight. Part of STA-8833. * refactor(orchestration): abandon retains through worker-retain's rule; record cancellations - One retain helper serves worker-retain and worker-abandon. A committed release (releasing, unknown) keeps its state and archive, since the tab may already be closed; a retained terminal drops its stale archive. - worker-abandon keeps the published stale field (this attempt was not the Task's current one) on the worker path. - task-list shows a failed Task's reason, whitespace-collapsed; task-update help and the recovery guide document cancel = failed + --result cancelled. * test(orchestration): drop duplicate abandon asserts; leave task-list output unchanged Cancellation stays documented in task-update help and the recovery guide. * fix(orchestration): abandoning an already-settled worker changes nothing Only the settling path retains an owned terminal. * fix(orchestration): attribute abandon only to a verified caller and keep the prior diagnostic * fix(orchestration): report an already-settled abandon as stale, as main did --- .../references/recovery-and-cleanup.md | 13 +- src/cli/bundled-skill-guides.ts | 4 +- src/cli/specs/orchestration.ts | 7 +- .../db-stopping-worker-task-guard.test.ts | 4 +- .../db-task-dispatch-lifecycle-guards.test.ts | 18 +- .../worker-dispatch-abandon.test.ts | 191 ++++++++++++++++++ .../worker-dispatch-abandon.ts | 89 ++++---- .../worker-dispatch/worker-dispatch-stop.ts | 15 +- .../worker-terminal-archive.ts | 37 +++- .../orchestration-worker-dispatch-db.test.ts | 4 +- .../check-superseded-terminal.test.ts | 2 +- .../worker/worker-abandon-caller.test.ts | 90 +++++++++ .../orchestration/worker/worker-control.ts | 18 +- .../worker-release-ownership-guard.test.ts | 4 +- .../worker/worker-release.test.ts | 4 +- .../worker-stop-liveness-verdict.test.ts | 2 +- .../runtime/structured-worker-at-rest.test.ts | 2 +- 17 files changed, 426 insertions(+), 78 deletions(-) create mode 100644 src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.test.ts create mode 100644 src/main/runtime/rpc/methods/orchestration/worker/worker-abandon-caller.test.ts diff --git a/skill-guides/orchestration/references/recovery-and-cleanup.md b/skill-guides/orchestration/references/recovery-and-cleanup.md index 07cf9fbc7b73..5664f2c88ae8 100644 --- a/skill-guides/orchestration/references/recovery-and-cleanup.md +++ b/skill-guides/orchestration/references/recovery-and-cleanup.md @@ -131,7 +131,18 @@ ORCA orchestration worker-abandon --dispatch --json `worker-stop` closes only the exact proven supervised agent terminal. It never deletes the worktree, setup terminal, configured tabs, or unrelated processes. `worker-abandon` fences orchestration while accepting that resources may remain -live; it performs no remote, process, or filesystem action. +live; it performs no remote, process, or filesystem action. When it settles a +worker, it retains an owned terminal by the same rule as `worker-retain`, so Orca +stops owing its release; a release already committed (`releasing`, +`release_unknown`) is left as it is. An already-settled worker is left untouched. + +To cancel a Task, settle its worker with `worker-stop` or `worker-abandon`, then +record the reason. A cancelled Task is `failed`, so its dependents stay blocked +until a `--retry-of` replacement completes it: + +```text +ORCA orchestration task-update --id --status failed --result cancelled --json +``` ## Retain and release diff --git a/src/cli/bundled-skill-guides.ts b/src/cli/bundled-skill-guides.ts index 1f1af4df1db4..53146931176d 100644 --- a/src/cli/bundled-skill-guides.ts +++ b/src/cli/bundled-skill-guides.ts @@ -69,7 +69,7 @@ const ORCA_PER_WORKSPACE_ENV_WINDOWS_SCRIPTS_REFERENCE_MARKDOWN = "# Windows loc const ORCHESTRATION_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live\n preamble, plus any other flag it carries. Never reconstruct, translate, or\n broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n" // oxfmt-ignore -const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live\n preamble, plus any other flag it carries. Never reconstruct, translate, or\n broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\nother arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Task ID, and Dispatch ID, and keep any other flag it carries (an older\nOrca host adds `--dispatch-capability`).\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" +const ORCHESTRATION_FULL_MARKDOWN = "---\nname: orchestration\ndescription: >-\n Coordinate supervised Orca workers: threaded messages, blocking ask/reply,\n task dispatch, worker_done/escalation waits, task DAGs, decision gates,\n coordinator loops, and decomposing work across agents. Use `orca-cli` for full\n ownership handoffs — \"hand off\", \"handoff\", \"handover\", \"give this to another\n agent\", \"another worktree\" — unless asked to supervise, monitor, or coordinate\n a DAG, and for terminal control, lightweight terminal prompts, shell commands,\n Orca worktree management, and reading or waiting on terminals.\n---\n\n# Orca orchestration\n\nOrchestration is Orca's structured coordination layer. It records who owns work,\nwhich attempt is authoritative, and when supervised work has settled.\n\n## Outcome\n\n**Result:** every in-scope Task has one explicit outcome and every settled worker\nterminal has a next owner or cleanup decision. **Next consumer:** the user who\nrequested supervision. **Done:** all expected Dispatches have settled, every\ndelivered message was processed before acknowledgment, each settled worker was\nreused, explicitly retained, or released, and the turn ends only when the report\nto that user names, per Task, its outcome, the evidence behind it, and any\nunresolved blocker.\n\n**Safe failure:** preserve work and authority and report the state as unknown or\n`unverifiable`. Only positive proof of exit authorizes stop, abandon, or retry,\nand only an accepted settlement authorizes release. Every other observation,\nabsence included, is a checkpoint.\n\n## Classify the role\n\n| Current context | Role | Route |\n| ---------------------------------------------------------------------------------------------------------------------------------------------- | ----------------------- | ------------------------------------------------------------------------------ |\n| The user explicitly asks to supervise, monitor, wait for results, track completion, coordinate a DAG, use a decision gate, or manage ask/reply | Coordinator | Use the supervised loop below |\n| The current prompt contains a live injected preamble with Task and Dispatch IDs | Dispatched worker | Follow the preamble and the worker obligations below |\n| The user asks to hand off ownership or start another agent/worktree without supervision | Handoff owner | Use `orca-cli`; create no Run, Task, or Dispatch and do not monitor completion |\n| A message carries a legacy authority label | Compatibility operator | Load the legacy contract reference before any lifecycle mutation |\n| No live preamble and no explicit supervision | Ordinary terminal agent | Do not emit lifecycle messages; use `orca-cli` for terminal/worktree work |\n\nModel or effort selection does not make a handoff supervised. Never substitute a\nnon-Orca subagent tool when Orca orchestration provenance was requested.\n\n## Authority and safety floor\n\n- A Run is a durable namespace and coordinator inbox; it does not schedule or\n place workers. A Task is work. A Dispatch is one authoritative Task attempt.\n- Lifecycle authority comes from the active Dispatch, not a terminal title,\n copied ID, old database row, provider transcript, or visible pane.\n- Workers use the exact executable, handle, Task ID, and Dispatch ID in the live\n preamble, plus any other flag it carries. Never reconstruct, translate, or\n broaden those arguments.\n- After remote start, address the worker by Dispatch ID. The execution host owns\n process, filesystem, transcript, stop, and cleanup facts. Preserve the verdicts\n `live` / `unverifiable` / `exited`; contact loss is not process death.\n- Liveness is layered: `worker-list`'s `projection.liveness` is the fleet verdict\n for the agent; `worker-show`'s `observation.status` is PTY liveness only. A live\n terminal can still hold a dead or stuck agent.\n- Folder workspaces are valid; never require Git or assume a worktree.\n- Clients and remote servers update independently. Treat unknown optional fields\n as absent. A new stream operation requires advertised capability because old\n decoders may silently drop unknown opcodes. Never fall back to local execution\n when remote authority or capability is unproven.\n- Use the executable you used to run `skills get` for the entire run. In the\n examples below, replace `ORCA` with it; do not create a shell variable or run\n `ORCA` literally. If it fails, report that exact error instead of switching.\n- A successful `orchestration send` proves durable enqueue; its wake or nudge is\n best-effort attention only and does not prove the recipient read or accepted it.\n\n## Worker obligations\n\nThe injected preamble is authoritative. A dispatched worker must:\n\n1. Do only the current Task and use the preamble's `ask` command for a blocking\n coordinator question. Never open a local question TUI the coordinator cannot\n answer. Resume the same message ID after an ask timeout.\n2. Send heartbeats only at the cadence in the preamble. A heartbeat proves\n liveness, not completion.\n3. Read coordinator follow-ups at each natural checkpoint — before starting a\n new file, after a test run — and once more immediately before `worker_done`:\n `ORCA orchestration check --terminal --json`.\n4. Send `worker_done` exactly once, from the dispatched terminal, with a\n three-sentence executive summary, both lifecycle IDs, and explicit\n `--outcome succeeded` or `--outcome failed`. Never encode failure only in prose.\n5. Append `--files-modified` and `--report-path` only with real values when\n applicable. After `worker_done`, end the dispatched turn and idle; do not poll\n or start new work.\n\nA direct user instruction after completion starts new user-owned work and takes\nprecedence over the idle rule. Do not reuse the settled lifecycle IDs.\n\n## Canonical supervised loop\n\nConfirm the runtime, bind one Run, and start the full independent wave before\nwaiting. `worker-start --spec` creates the Task and its attempt in one call:\n\n```text\nORCA status --json\nORCA orchestration run-create --objective \"\" --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent codex --json\nORCA orchestration worker-start --spec \"\" --worktree current --agent claude --json\nORCA orchestration check --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nIf `worker-start` exits non-zero, do not relaunch. Read the receipt's\n`failedStage` and `residualResources`, then load\n`references/recovery-and-cleanup.md`.\n\nUse `task-create` plus `worker-start --task ` for planned fan-out with\ndependencies or a retry of a known Task. Use dependencies only for real ordering\nand prefer parallel waves over chains deeper than three or four steps; nested\nworkers obey the depth limit, and a new Run does not reset the caller's depth.\n\nA consuming `check` names its caller with `--terminal `, never `--from`;\nomit it inside the coordinator's own Orca terminal. It returns the bound Run's\noldest FIFO Delivery and replays that batch until acknowledged. Process every\nmessage: reply to questions, validate each `worker_done` against the expected\nactive Dispatch, and decide each settled terminal's next owner before the ack:\n\n```text\nORCA orchestration reply --id --body \"\" --json\nORCA orchestration worker-release --dispatch --json\nORCA orchestration check --ack --wait --types \"worker_done,escalation,question\" --timeout-ms 900000 --json\n```\n\nKeep waiting until every expected Dispatch settles. A timeout or empty result is\na checkpoint, not a failure. Do not stop, retry, release, or launch a duplicate\neditor without the positive proof `## Outcome` requires.\n\nAfter three consecutive empty waits, stop waiting blindly and enumerate with\n`ORCA orchestration worker-list --include-remote --json` (defaults to the bound\nRun; `--run ` overrides; the receipt's `scope` names which), acting on\neach row's `projection.attention` categories, `projection.attention.requiresAction`, and literal `projection.nextAction` argv.\nRows come newest first and page at 100: while `page.hasMore`, follow `page.nextCursor` with `--cursor `.\nA `none` `nextAction` has no argv to run: read `liveness.reason` and keep waiting\nwith `check --wait`. Absence never earns an argv; settlement and pending work still do.\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Then load `references/recovery-and-cleanup.md` and choose\n`worker-stop` or `worker-abandon` explicitly. `unverifiable` is absence,\nincluding when `worker-show` reports `agentWait` null. Absence never authorizes\nstop, abandon, retry, or release; keep waiting or inspect.\n\n`worker-start` is the normal path, composing placement, terminal readiness,\nprompt injection, and supervised resource ownership. `dispatch --inject` leaves\nan operator-created process unsupervised and is only for an expressiveness gap.\n\n## Task-spec contract\n\nEvery Task spec must be self-contained and name:\n\n- **Target:** the files, component, or environment in scope.\n- **Change:** the concrete result to produce.\n- **Constraints:** invariants, compatibility rules, and do-not-touch boundaries.\n- **Ownership:** what this worker may edit and any coordination boundary.\n- **Observable acceptance:** the test, output, or evidence that proves completion.\n\n## Completion accounting\n\nAfter an accepted success or failure report, immediately do exactly one:\n\n1. Reuse the same proven agent terminal for an immediate follow-up Dispatch.\n2. Record user-requested retention with `worker-retain`.\n3. Run `worker-release`.\n\nRelease is post-settlement cleanup, not cancellation. Only an accepted\nsettlement authorizes it; no other observation does. If release is uncertain,\nfollow its exact recovery receipt and never substitute `terminal close`.\n\nA valid `worker_done` settles the Task and Dispatch automatically; do not follow\nit with `task-update --status completed`. Enumerate the terminals still owing a\ndecision with `worker-list --run --terminal-state reclaimable --json`,\nand do not end the coordinator turn until it returns none.\n\n## Conditional references\n\nThis compact guide is sufficient for the normal local loop. At an action gate\nbelow, run `ORCA skills get orchestration --reference references/.md` and\nread only that document; `--references` lists the names. If the CLI rejects\n`--reference`, run `ORCA skills get orchestration --full` once instead: it\nreturns this exact kernel and every reference, so read only the named one. If an\nolder CLI rejects `--full`, keep this kernel's safety floor, use that command's\n`--help`, and never guess newer flags.\n\n| Action gate | Bundled reference |\n| ------------------------------------------------------------------------------------------------------------- | ----------------------------------------- |\n| Expanded DAG waves, launch model/effort, same-terminal reuse, or review ownership | `references/coordinator-loop.md` |\n| You are a dispatched worker and the live preamble does not answer your question, or `check` returned an error | `references/worker-contract.md` |\n| New worktree, exact workspace, SSH, WSL, or connected-server placement | `references/placement-and-remote.md` |\n| Inbox replay, follow-up messages, group addresses, or decision gates | `references/messaging-and-gates.md` |\n| Failed/stopped/unknown attempts, retry, stop, abandon, retain, or uncertain release | `references/recovery-and-cleanup.md` |\n| Custom argv or terminal topology that `worker-start` cannot express | `references/low-level-topology.md` |\n| Any legacy label, adopted Run, compatibility receipt, or takeover | `references/legacy-contract-migration.md` |\n\nRetired scheduler commands are not aliases for Run creation. Recovery commands\nmust provide their exact next action; follow it with the same selected executable.\n\n---\n\n# Bundled references\n\nThese references belong to the version-matched guide above. Read only the documents named by its action gates.\n\n\n\n# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n\n\n\n# Legacy contract migration\n\nLoad this reference only for an authority label, adopted Run, compatibility or\nrecovery receipt, or explicit legacy takeover. A newly created attempt always\nuses the current grammar.\n\n## Authority labels\n\n- `[LEGACY COMPATIBILITY]` is live and attested. Run only the exact supported\n command printed with the message, using the same selected executable and\n arguments supplied by the original prompt.\n- `[LEGACY RECOVERY REPLAY — MAY HAVE BEEN SEEN]` is one bounded,\n at-least-once cutover replay. Process it idempotently and acknowledge only\n through the exact displayed guidance.\n- `[LEGACY READ-ONLY]` is inspection-only. It has no reply, acknowledgment, or\n lifecycle mutation.\n- An unlabeled current message uses the current guide and grammar.\n\nAn explicitly selected current Run, attested current binding, current Dispatch,\nor federated attachment takes precedence over legacy fallback. A retained\nadoption record alone does not grant mutation authority. If liveness, principal\nownership, capability, or the exact legacy contract is unproven, degrade to\nread-only inspection and never fall back to local execution.\n\nAdoption preserves the live agent process, PTY/session, terminal handle,\ntab/pane, worktree or folder workspace, Task, and Dispatch. It never restarts or\nreplaces the worker and never revives the retired scheduler. Loss of lifecycle\nauthority does not invalidate the existing process, assignment, or filesystem\nwork. Exact recovery may restore the same PTY once in its original inactive\nbackground tab; it must not spawn, write, signal, stop, switch, focus, split, or\ninject a terminal.\n\n## Compatibility recovery\n\nWhen a compatibility response returns structured next-step arguments, execute\nthose exact arguments with the same selected CLI executable. Do not translate\nfrom memory, broaden the recipient, or retry as a current mutation unless the\nreceipt explicitly authorizes it.\n\nA pending ask, reply, final Dispatch settlement, and consuming check have\ndurable recovery identities. Heartbeat and escalation remain at-least-once\nacross a manual contract-boundary retry. If an ask may already have been\nanswered, run the exact non-consuming recovery check printed by Orca before\ncreating any new question. Never guess among identical question threads.\n\nOn packaged Windows, a legacy ask uses a two-step commit/resume protocol. The\ninitial command commits the question, prints its exact\n`ask --resume ` command, and exits with launcher status `75`. Run\nthat exact resume after the launcher or update boundary. For an attested WSL\nlaunch, preserve the printed `orca-ide` executable and distro route. Older WSL\nworkers without launch proof remain lifecycle read-only even while their\nterminal and filesystem work continue.\n\n## Read-only inspection and takeover\n\nRead-only inspection does not consume mail:\n\n```text\nORCA orchestration run-list --json\nORCA orchestration run-show --id run_legacy_local --json\nORCA orchestration run-show --id --json\nORCA orchestration task-list --run --json\nORCA orchestration inbox --full --json\nORCA orchestration check --terminal --peek --format --json\nORCA terminal read --terminal --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\n```\n\n`run_legacy_local` is an empty audit tombstone after adoption. Find the ordinary\nRun whose objective is `Recovered orchestration work from a contract update`.\n\nOnly when the original coordinator is unavailable or cannot prove retained\nauthority may a new live coordinator take over from its own terminal:\n\n```text\nORCA orchestration run-use --id --takeover-legacy --json\nORCA orchestration check --run --json\n```\n\nTakeover binds the authenticated invoking terminal; `--from` cannot nominate\nanother coordinator. It fences only the old coordinator and moves pending mail\ninto current Run delivery. It preserves live workers, Tasks, Dispatches, processes, and files.\nNever take over while the original coordinator is actively coordinating.\n\nDo not launch a replacement editor merely because Orca updated or authority is\nunclear. Keep the original worker as the only editor until a stable handoff\npoint, then use a fresh current Dispatch in a conflict-free placement.\n\n\n\n# Low-level topology\n\nLoad this reference only when `worker-start` cannot express required custom argv\nor terminal topology. It is not the normal supervised loop and is never a full\nhandoff recipe.\n\n```text\nORCA terminal create --worktree active --title --command \"\" --json\nORCA terminal wait --terminal --for tui-idle --timeout-ms 60000 --json\nORCA orchestration dispatch --task --to --inject --json\n```\n\nWait for readiness only when startup could lose injected input. Prefer\nagent-first `worker-start` whenever its argv and topology are sufficient.\n\n`dispatch --inject` creates authoritative Task/Dispatch context but deliberately\nkeeps an operator-created process unsupervised: it creates no supervised worker\nresource row. `worker-show`, `worker-read`, and `worker-list` report the lane as\n`unsupervised`; `worker-stop` and `worker-abandon` do not close that process, and\nsettled retain/release take no process action.\n\nUse `worker-start --terminal ` when lifecycle ownership of an existing\nagent terminal is required. Never imply that low-level dispatch retroactively\nowns a process, never use it to route around the nested-depth limit, and never\nuse it for an ownership handoff.\n\n\n\n# Messaging and gates\n\nLoad this reference for inbox replay, attempt-specific guidance, group\naddresses, blocking questions, or coordinator-managed DAG decisions.\n\nA successful `send` proves durable enqueue. Wake and nudge are best-effort\nattention only: neither proves the recipient read the message, began a turn, or\naccepted steering.\n\n## Coordinator delivery loop\n\n`check` names its caller with `--terminal ` and is the only verb that\nrejects `--from`. Omit `--terminal` inside an Orca terminal, where Orca resolves\nthe caller; pass it explicitly from anywhere else, including a dispatched\nworker reading coordinator follow-ups.\n\nA consuming coordinator `check` returns the bound Run's oldest FIFO Delivery,\nup to 50 messages, and replays that exact batch until acknowledged. Process\nevery row and required terminal ownership decision before `--ack`. Type filters\ndecide when a waiter wakes; they do not authorize skipping older actionable\nmail. A Delivery therefore always carries the whole FIFO batch whatever its\ntypes, and a `check` without `--wait` hands that batch over unfiltered.\n`--peek` and `--all` are read-only inspection, not progress through the\ncoordinator inbox.\n\nAn empty wait or timeout is a checkpoint. Continue rolling waits until every\nexpected Dispatch settles. Heartbeat or visible activity means alive, not done.\n\n## Addresses\n\nUse a stable Dispatch address for attempt-specific coordinator guidance:\n\n```text\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\n```\n\nDo not substitute a remote terminal handle. Omit `--from` for ordinary\ncoordinator calls; a dispatched worker instead copies the exact `--from` and\nother arguments in its preamble. `check` is the exception: it identifies\nits caller with `--terminal`, never `--from`.\n\nGroup addresses include `@all`, `@idle`, `@claude`, `@codex`, `@opencode`,\n`@gemini`, `@droid`, `@grok`, `@cursor`, and `@worktree:`. Every group but\n`@worktree:` means the live Dispatches of the sender's own Run. Mail goes\nto each `dispatch:` mailbox, except a worker coordinating a child Run\nreceives it in that `run:` mailbox. A sender bound to no Run is refused;\n`--run` must match the group audience and never grants membership.\nA Run group excludes its owning coordinator; a worker raising a blocker sends\nto `run:`. A worker that created its own Run addresses that Run's workers,\nnot its siblings. `@worktree:` reaches matching workspace terminals,\nincluding coordinators. Use groups only for intentional fan-out status or\nquestions. `worker_done`, heartbeat, and other\nDispatch lifecycle messages never target groups.\n\n## Questions and gates\n\nA worker uses `ask`; its timeout leaves one durable question pending, which the\nworker resumes by message ID. The coordinator answers that message with `reply`.\n\nUse a gate only for a coordinator-owned Task-DAG decision:\n\n```text\nORCA orchestration gate-create --task --question \"\" --options --json\nORCA orchestration gate-resolve --id --resolution \"\" --json\nORCA orchestration gate-list --task --json\n```\n\nPass `json_array` using the quoting rules of the active shell; do not copy POSIX\nsingle-quote syntax into PowerShell or `cmd.exe`.\n\nDo not create a gate merely to answer a worker's `ask`.\n\n\n\n# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n\n\n\n# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action. When it settles a\nworker, it retains an owned terminal by the same rule as `worker-retain`, so Orca\nstops owing its release; a release already committed (`releasing`,\n`release_unknown`) is left as it is. An already-settled worker is left untouched.\n\nTo cancel a Task, settle its worker with `worker-stop` or `worker-abandon`, then\nrecord the reason. A cancelled Task is `failed`, so its dependents stay blocked\nuntil a `--retry-of` replacement completes it:\n\n```text\nORCA orchestration task-update --id --status failed --result cancelled --json\n```\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n\n\n\n# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Task ID, and Dispatch ID, and keep any other flag it carries (an older\nOrca host adds `--dispatch-capability`).\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" // oxfmt-ignore const ORCHESTRATION_COORDINATOR_LOOP_REFERENCE_MARKDOWN = "# Coordinator loop\n\nLoad this reference for expanded DAG waves, per-invocation launch preferences,\nsame-terminal reuse, or review ownership. The compact guide remains the source\nof truth for the loop order and completion boundary.\n\n## Ready waves\n\nCreate independent Tasks before the first wait. Encode only real dependencies,\nthen use the ready view as external memory:\n\n```text\nORCA orchestration task-create --spec \"\" --deps --json\nORCA orchestration task-list --ready --brief --json\n```\n\n`--brief` collapses whitespace and caps echoed specs at 160 characters;\n`spec_truncated` identifies shortened rows. Omit it when full specs are needed or\nwhen an older CLI rejects the flag. A nested worker must respect\n`nested_worker_depth_exceeded`; creating another Run does not reset depth.\n\n## Launch preferences\n\nFor a fresh Claude, Codex, Cursor, Antigravity, or Muse terminal, `--model`\naccepts an opaque provider model ID. Pass it only when the user named a model;\notherwise omit it so the worker inherits the user's configured agent default.\nAdd `--effort` only when that model supports it:\n\n```text\nORCA orchestration worker-start --task --worktree current --agent claude --model opus --effort high --json\nORCA orchestration worker-start --task --worktree current --agent muse --model muse-spark-1.3 --json\n```\n\nOther agents, including `opencode`, reject `--model`; they run the model set in\ntheir own config, so a coordinator wanting a same-model opencode worker relies\non that config.\n\n`--effort` requires `--model`; neither option combines with `--terminal`. A\nconnected worker server must advertise launch-preference support before Orca\nforwards either field. Compare `launch.requested` with `launch.effective`; never\nclaim a model or effort from requested arguments alone.\n\n## Reuse after settlement\n\nChoose the terminal's next owner before acknowledging the Delivery. When the\nsame exact agent has immediate follow-up work, recover the proven handle and\ntransfer cleanup ownership to the new Dispatch:\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-start --task --terminal --json\n```\n\nOtherwise explicitly retain or release the settled worker. Do not leave it live\nonly to inspect output; archived output remains available through `worker-read`.\n\n## Review ownership\n\nA review-only `worker_done` authorizes synthesis of findings, not coordinator\nfile edits. Dispatch or hand off fixes unless the user explicitly assigned them\nto the coordinator. If the user's plan names a next owner, post-review fixes and\nPR preparation remain with that owner; the coordinator routes and synthesizes.\n" @@ -87,7 +87,7 @@ const ORCHESTRATION_MESSAGING_AND_GATES_REFERENCE_MARKDOWN = "# Messaging and ga const ORCHESTRATION_PLACEMENT_AND_REMOTE_REFERENCE_MARKDOWN = "# Placement and remote execution\n\nLoad this reference before creating a new worktree or placing work through SSH,\nWSL, or another connected Orca server.\n\n## Placement choices\n\nA fresh worker means a fresh agent terminal, not a new Git worktree. Use the\ncurrent or an exact existing workspace by default. Create a worktree only when\nthe user requested one or a concrete checkout or filesystem conflict makes\nsharing unsafe.\n\n```text\n# Current workspace; setup is not rerun.\nORCA orchestration worker-start --task --worktree current --agent codex --json\n\n# Stacked child worktree.\nORCA orchestration worker-start --task --worktree new-child --name --agent codex --setup run --json\n\n# Independent top-level worktree.\nORCA orchestration worker-start --task --worktree new-top-level --name --agent codex --setup run --json\n```\n\nCurrent and exact existing workspaces create a fresh terminal unless\n`--terminal` is explicit. Folder workspaces are first-class; do not invoke Git\nor require worktree lineage when the selected workspace is a folder.\n\nRegister a folder workspace through project setup. `repo add --path `\nrequires a valid Git repository and rejects a plain directory:\n\n```text\nORCA project setup-existing-folder --project --host --path --kind folder --json\n```\n\nThen place work on the returned workspace with an exact selector. A worktree\nselector needs the full `::` value Orca returned, passed as\n`id:`; a bare repo id is not a worktree id. `new-child` and\n`new-top-level` are worktree creation and do not apply to a folder.\n\nNew worktrees use agent-first creation and run setup by default. Preserve the\nrepository's startup policy: `start-immediately` can report setup as `running`,\nwhile `wait-for-setup` gates prompt delivery on success. Orca lineage, Git base,\nfilesystem isolation, coordination parentage, UI grouping, and execution host\nare separate decisions.\n\n## Connected servers\n\nThe Run and Tasks remain authoritative on the current server. `--on` selects\nonly the worker's execution server and appears only on `worker-start`:\n\n```text\nORCA orchestration worker-start --task --on --worktree new-top-level --repo --name --agent codex --setup run --json\n```\n\nRemote `current` and `new-child` are invalid because they are ambiguous across\nservers. Use an exact discovered remote workspace, or `new-top-level` with an\nexact remote repository selector. After start, route every follow-up, read,\nstop, and cleanup by Dispatch ID; never repeat `--on` or substitute a remote\nterminal handle.\n\n```text\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\nORCA orchestration send --to dispatch: --subject \"Follow-up\" --body \"\" --json\nORCA orchestration worker-list --run --include-remote --json\n```\n\n`worker-list` reads local fleet state only; enumerate remote workers with\n`--include-remote` or every one of them reads `unverifiable`. Scope every list\nwith `--run `: unscoped, it reports every Dispatch this runtime has\nrecorded, and the workers you are waiting on are lost in that history.\n\n## Execution-host and mixed-version floor\n\nThe execution host owns process, filesystem, transcript, stop, and cleanup\nfacts. Render only `live`, `unverifiable`, or `exited`. Connection loss, relay\nabsence, missing client inventory, or timeout yields `unverifiable`, never\nsynthetic exit and never a client-local substitute action.\n\nClients and servers update independently. Optional response fields may be\nabsent. Forward model/effort, transcript reads, cleanup, or another new remote\noperation only when the peer advertises the relevant capability; unknown stream\nopcodes can be silently dropped. A narrow unsupported response may degrade to a\ndocumented older path, but must not broaden the target or cross the execution\nboundary. Changing host-published content reaches old clients even without a\nwire-shape change, so preserve established semantics or negotiate the behavior.\n\nFor WSL, use the exact executable and arguments returned by Orca so the distro\nand packaged launcher remain bound. Do not translate a printed `orca-ide`\nrecovery command into a PATH-resolved local command.\n" // oxfmt-ignore -const ORCHESTRATION_RECOVERY_AND_CLEANUP_REFERENCE_MARKDOWN = "# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action.\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n" +const ORCHESTRATION_RECOVERY_AND_CLEANUP_REFERENCE_MARKDOWN = "# Recovery and cleanup\n\nLoad this reference only after a failed/stopped/unknown attempt, explicit retry\ndecision, stop/abandon request, retention request, or uncertain release.\n\n| Proven state | Safe action |\n| ----------------------- | ------------------------------------------------------------------ |\n| `ready` or active | Keep waiting; optionally read bounded output |\n| `failed` or `stopped` | Start a replacement with `--retry-of`; repeat placement explicitly |\n| `outcome_unknown` | Inspect, then choose `worker-stop` or explicit `worker-abandon` |\n| Accepted `worker_done` | Reuse, retain, or release |\n| Remote contact lost | Preserve `unverifiable`; do not stop or retry from absence alone |\n| `unverifiable` liveness | Keep waiting or inspect; never stop, abandon, retry, or release |\n| Proven `exited` agent | Enumerate with `worker-list`; follow its `nextAction` |\n\n## Inspect before acting\n\n```text\nORCA orchestration worker-list --run --json\nORCA orchestration worker-list --run --include-remote --json\nORCA orchestration worker-show --dispatch --json\nORCA orchestration worker-read --dispatch --limit 50 --json\n```\n\n`worker-list` is the enumerating command and the authority on agent liveness:\neach row carries `projection.liveness`, `projection.attention.categories`,\n`projection.attention.requiresAction`, and a literal `projection.nextAction`\nargv to run. Always scope it with `--run `; an unscoped list reports\nevery Dispatch this runtime has ever recorded and buries the live ones.\n`worker-show`'s `observation.status` is PTY liveness only, so a `live` terminal\nwhose agent died at a trust prompt still reads `live` there.\n\nWhen the two disagree, the fleet verdict decides — unless the fleet row is\n`unverifiable` for a reason that names a gap on this client rather than a fact\nabout the worker. `missing_status`, `host_unavailable`, and\n`capability_unsupported` are such gaps: the first means this runtime holds no\nstatus row, the second that it could not ask the execution host at all, and the\nthird that a stale peer answered but lacks the fleet-snapshot capability.\nAgainst any of them, a `worker-show` verdict sourced from the execution host is\nthe better evidence and outranks the row. Only `host_unavailable` is contact\nloss; the other two mean the host was never asked or answered without the\ncapability.\n\nThis never promotes absence. `unverifiable` from either command still authorizes\nnothing — only a positive `live` or `exited` verdict does.\n\nA worker started with `--on ` reads `unverifiable` until you\nenumerate with `--include-remote`, which asks its execution host for the\nverdict. Rows come newest first and past 100 the response pages, so follow\n`page.nextCursor` with `--cursor ` until `page.hasMore` is false.\n\n## Stall needs positive evidence\n\nLeave the wait only on positive proof the agent stopped: `exited` liveness, the\nworker's own observation of process exit, or a transcript whose final agent turn\nsent no `worker_done`. Only then choose `worker-stop` or `worker-abandon`.\n\n`unverifiable` is always absence — `missing_status`, `stale_status`,\n`restored_unconfirmed`, or a remote worker with no connection — and a null\n`agentWait` or an unchanged `worker-read` tail is that same absence seen again.\nAbsence never authorizes stop, abandon, retry, or release: keep waiting, or\ninspect until you hold one of the positive signals above. A `nextAction` that\nnames an inspecting command is asking for evidence, not for cleanup.\n\n`worker-read --source auto` uses a proven provider transcript when available and\notherwise returns bounded terminal output with a typed `fallbackReason`.\nContinue with its top-level cursor, which is pinned to that source. If Orca\nreports `source_changed`, restart without the old cursor. A bounded initial\ntranscript tail can return an EOF cursor that follows only newly appended records;\nread `contentComplete`, `clipping`, and `warnings` before assuming omitted older\nrecords are pageable. Never guess a provider session ID, transcript path, or\nremote terminal handle.\n\n## Was the mutation applied?\n\nWhen a mutation's response was lost and named no Dispatch, do not replay blind.\nEvery orchestration mutation accepts `--retry-request `, which reuses one\noperation identity so Orca can replay, join, or recover it instead of starting a\nduplicate. Ask what happened first:\n\n```text\nORCA orchestration request-show --request --json\n```\n\n`completed` means the mutation already took effect; read its recorded receipt\ninstead of rerunning. `pending` means the original mutation is still running or\nOrca restarted before recording its outcome; replay the original command with\n`--retry-request `. `absent` means this runtime holds no receipt\nunder your caller identity — that is not proof nothing happened, so inspect the\naffected Task, Dispatch, and terminal before deciding whether to retry.\n\nWhen a worker's terminal accepted input but the submit is unconfirmed, use\n`terminal send --wait-submit `: it observes the accepted prompt for that\nlong and, on timeout, returns the input-accepted receipt without resending.\n\n## Refused starts\n\n`dispatch` and `worker-start` refuse the following preflight cases with a stable\n`error.code`; read it before choosing a recovery, and treat `error.data.nextSteps`\nas the exact recovery text. Older hosts may omit `data`, so treat every field as\noptional.\n\n| Code | Meaning | Recovery |\n| -------------------- | --------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------ |\n| `task_not_found` | No Task with that id, or not in the bound Run (`data.taskId`, `data.runId`) | Check `task-list --json`; create the Task with `task-create` if it does not exist |\n| `task_not_startable` | Task cannot start now: not `ready`, or invalid `--retry-of` (`data.status`, `data.unmetDependencies`, `data.retryOf`) | Wait for running dependencies with `check --wait`; retry or unblock failed ones; inspect `dispatch-show` if already dispatched |\n| `inject_rejected` | `--inject` refused because no recognized agent runs in the target (`data.terminal`, `data.reason`) | Start a recognized agent there or pick another terminal; or dispatch without `--inject` and use `terminal send` |\n| `runtime_error` | Any other failure, including a target terminal that already owns an active Dispatch | Read the message, inspect state, and do not retry unchanged |\n\n## Retry, stop, and abandon\n\nRetry only a positively proven failed or stopped attempt. Name the failed Task\nwith `--task`, since `--spec` creates a new one. Placement is never silently\ninherited:\n\n```text\nORCA orchestration worker-start --task --retry-of --worktree --agent --json\n```\n\nAfter three consecutive failures for one Task, its dispatch context\ncircuit-breaks and the Task is failed. Do not route around that boundary with a\nnew Run or an unrelated Dispatch.\n\nFor `outcome_unknown`, inspect first, then make an explicit choice:\n\n```text\nORCA orchestration worker-stop --dispatch --json\nORCA orchestration worker-abandon --dispatch --json\n```\n\n`worker-stop` closes only the exact proven supervised agent terminal. It never\ndeletes the worktree, setup terminal, configured tabs, or unrelated processes.\n`worker-abandon` fences orchestration while accepting that resources may remain\nlive; it performs no remote, process, or filesystem action. When it settles a\nworker, it retains an owned terminal by the same rule as `worker-retain`, so Orca\nstops owing its release; a release already committed (`releasing`,\n`release_unknown`) is left as it is. An already-settled worker is left untouched.\n\nTo cancel a Task, settle its worker with `worker-stop` or `worker-abandon`, then\nrecord the reason. A cancelled Task is `failed`, so its dependents stay blocked\nuntil a `--retry-of` replacement completes it:\n\n```text\nORCA orchestration task-update --id --status failed --result cancelled --json\n```\n\n## Retain and release\n\n```text\nORCA orchestration worker-retain --dispatch --json\nORCA orchestration worker-release --dispatch --json\n```\n\nRetain only when the user explicitly wants the settled terminal kept live.\nRelease works after succeeded and failed reports, archives readable output, and\ncloses only the exact terminal owned by that settled Dispatch. Replays may call\nrelease again safely. Reused, pre-existing, setup, coordinator, active,\nuser-taken-over, and unproven terminals are retained.\n\nA `worker-start` that failed before its agent was ready still owns the terminal\nit created. Its receipt names `worker-release`, and `worker-list` reports that\nrow as `reclaimable`; release it there rather than closing the terminal by hand.\n\nNever release because of timeout, TUI idle, heartbeat, status, question,\nescalation, or stale/rejected completion. If the receipt says `release_pending`\nor `release_unknown`, follow its exact recovery action. Never substitute\n`terminal close`.\n\n`orchestration reset` is destructive recovery. Do not run it during active\ncoordination unless the user explicitly abandons that state.\n" // oxfmt-ignore const ORCHESTRATION_WORKER_CONTRACT_REFERENCE_MARKDOWN = "# Worker contract\n\nThe injected preamble is authoritative. Copy its command rather than\nreconstructing flags. In particular, preserve the exact executable, worker\nhandle, Task ID, and Dispatch ID, and keep any other flag it carries (an older\nOrca host adds `--dispatch-capability`).\n\n## Heartbeat\n\nSend heartbeats only at the cadence required by the live preamble. Skip them\nwhile blocked inside `ask` or `check --wait`; those calls are liveness signals.\n\n```text\nORCA orchestration send --from --type heartbeat --subject \"alive\" --task-id --dispatch-id --phase \"\"\n```\n\nUse typed lifecycle flags, not a hand-written JSON payload. A heartbeat proves\nliveness, never completion.\n\n## Ask and resume\n\nUse Orca `ask` whenever the coordinator must answer. Never open a local question\nTUI the coordinator cannot answer.\n\n```text\nORCA orchestration ask --from --question \"\" --options \",\" --timeout-ms 600000\n\nORCA orchestration ask --from --resume --timeout-ms 600000\n```\n\nA timeout or disconnect leaves the original question pending. Resume its\nmessage ID; do not create a duplicate question.\n\n## Reading coordinator follow-ups\n\nThe coordinator steers a running worker with `send --to dispatch:`. That\nenqueue is durable but does not interrupt you, so nothing arrives unless you\nlook:\n\n```text\nORCA orchestration check --terminal --json\n```\n\nRun it at each natural checkpoint — before starting a new file, after a test\nrun — and once more immediately before `worker_done`, so a redirect or a\ncancellation lands before the Task settles. `check` names its caller with\n`--terminal`, never `--from`. Stop checking after `worker_done`.\n\nIf `check` returns `consumer_fenced`, this process no longer owns its Dispatch:\nthe Attempt was re-attached to another worker or settled without you. Stop, do\nnot send `worker_done`, and do not retry the check. An empty `check` never means\nyou were replaced; `consumer_fenced` is the only way you learn that.\nIf `send` or `ask` returns `consumer_fenced`, the command ran from another\nparty's terminal (a coordinator or another worker); run it from your own terminal.\n\n## Escalation\n\nEscalate only before completion and only when the coordinator must intervene:\n\n```text\nORCA orchestration send --from --type escalation --subject \"Blocked: \" --body \"
\" --task-id --dispatch-id \n```\n\n## Completion\n\nSend exactly one terminal report. `--body` is three sentences: what changed,\nwhat was found, and what remains. Use `--outcome failed` when the requested work\nis not complete; never hide failure in prose or silently exit.\n\nAppend `--files-modified` or `--report-path` only when applicable, using actual\npaths. Do not send documentation placeholders as metadata.\n\n```text\nORCA orchestration send --from --type worker_done --subject \"\" --body \"\" --task-id --dispatch-id --outcome succeeded\n```\n\nAfter `worker_done`, end the dispatched turn and idle. Do not poll, close your\nown terminal, or begin unrelated work. A later direct user instruction is new\nuser-owned work and must not reuse settled lifecycle IDs; a supervised follow-up\narrives with a fresh preamble and Task block.\n" diff --git a/src/cli/specs/orchestration.ts b/src/cli/specs/orchestration.ts index 8c4418f38ba5..90cb4971c4ce 100644 --- a/src/cli/specs/orchestration.ts +++ b/src/cli/specs/orchestration.ts @@ -167,10 +167,13 @@ export const ORCHESTRATION_COMMAND_SPECS: CommandSpec[] = [ path: ['orchestration', 'task-update'], summary: 'Update a task status', usage: - 'orca orchestration task-update --id --status [--result ] [--run ] [--from ] [--retry-request ] [--json]', + 'orca orchestration task-update --id --status [--result ] [--run ] [--from ] [--retry-request ] [--json]', allowedFlags: [...GLOBAL_FLAGS, 'id', 'status', 'result', 'run', 'from', 'retry-request'], identityFlagRoles: { from: 'caller' }, - notes: ['Valid --status values: pending, ready, dispatched, completed, failed, blocked.'] + notes: [ + 'Valid --status values: pending, ready, dispatched, completed, failed, blocked.', + 'To cancel a Task, stop or abandon its worker, then set --status failed --result cancelled; a later worker-start --retry-of reopens it.' + ] }, ...ORCHESTRATION_WORKER_COMMAND_SPECS, { diff --git a/src/main/runtime/orchestration/db-stopping-worker-task-guard.test.ts b/src/main/runtime/orchestration/db-stopping-worker-task-guard.test.ts index a747ae07a510..01e238e51b51 100644 --- a/src/main/runtime/orchestration/db-stopping-worker-task-guard.test.ts +++ b/src/main/runtime/orchestration/db-stopping-worker-task-guard.test.ts @@ -93,7 +93,9 @@ describe('a Task whose supervised worker is stopping', () => { expect(db.getWorkerDispatch(dispatch.id)?.runtime_epoch).toBe('epoch_new_runtime') db.markWorkerStopUnknown(dispatch.id, 'the execution host did not answer') - expect(db.abandonWorkerDispatch(dispatch.id)).toMatchObject({ disposition: 'abandoned' }) + expect(db.abandonWorkerDispatch(dispatch.id, 'epoch_test')).toMatchObject({ + disposition: 'abandoned' + }) expect(db.getTask(task.id)?.status).toBe('blocked') }) diff --git a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts index f58c0938ecfe..88445b95bac2 100644 --- a/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts +++ b/src/main/runtime/orchestration/db-task-dispatch-lifecycle-guards.test.ts @@ -112,7 +112,7 @@ describe('Task/Dispatch lifecycle guards', () => { }) it.each(['failed', 'stopped'] as const)( - 'treats abandon of an already %s worker as stale without a lifecycle conflict', + 'treats abandon of an already %s worker as settled without a lifecycle conflict', (state) => { const database = createDatabase() const task = database.createTask({ @@ -127,8 +127,8 @@ describe('Task/Dispatch lifecycle guards', () => { database.settleWorkerStop(worker.dispatchId) } - expect(database.abandonWorkerDispatch(worker.dispatchId)).toMatchObject({ - disposition: 'stale', + expect(database.abandonWorkerDispatch(worker.dispatchId, 'epoch_test')).toMatchObject({ + disposition: 'already_settled', worker: { state } }) } @@ -420,7 +420,7 @@ describe('Task/Dispatch lifecycle guards', () => { const released = operation === 'stop' ? database.beginWorkerStop(contextOnly.id, 'runtime_test') - : database.abandonWorkerDispatch(contextOnly.id) + : database.abandonWorkerDispatch(contextOnly.id, 'epoch_test') expect(released).toMatchObject({ disposition: 'context_only', alreadySettled: false, @@ -455,7 +455,7 @@ describe('Task/Dispatch lifecycle guards', () => { const released = operation === 'stop' ? database.beginWorkerStop(contextOnly.id, 'runtime_test') - : database.abandonWorkerDispatch(contextOnly.id) + : database.abandonWorkerDispatch(contextOnly.id, 'epoch_test') expect(released).toMatchObject({ disposition: 'context_only', @@ -514,7 +514,9 @@ describe('Task/Dispatch lifecycle guards', () => { ) expect(database.settleWorkerStop(released.dispatchId).state).toBe('stopped') } else { - expect(database.abandonWorkerDispatch(released.dispatchId).disposition).toBe('abandoned') + expect(database.abandonWorkerDispatch(released.dispatchId, 'epoch_test').disposition).toBe( + 'abandoned' + ) } expect(database.getTask(task.id)?.status).toBe('dispatched') @@ -545,7 +547,9 @@ describe('Task/Dispatch lifecycle guards', () => { expect(database.beginWorkerStop(stopping.dispatchId, 'runtime_test').disposition).toBe( 'stopping' ) - expect(database.abandonWorkerDispatch(abandoned.dispatchId).disposition).toBe('abandoned') + expect(database.abandonWorkerDispatch(abandoned.dispatchId, 'epoch_test').disposition).toBe( + 'abandoned' + ) expect(database.getTask(task.id)?.status).toBe('dispatched') expect(database.settleWorkerStop(stopping.dispatchId).state).toBe('stopped') diff --git a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.test.ts b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.test.ts new file mode 100644 index 000000000000..1878c72a72bd --- /dev/null +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.test.ts @@ -0,0 +1,191 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { OrchestrationDb } from '../orchestration-db' + +const THIS_RUNTIME = 'epoch_this_runtime' + +describe('worker-abandon settles a stuck worker', () => { + let db: OrchestrationDb + beforeEach(() => { + db = new OrchestrationDb(':memory:') + }) + afterEach(() => db.close()) + + function startWorker(taskId?: string, name = 'w') { + const task = taskId + ? db.getTask(taskId)! + : db.createTask({ runId: 'run_legacy_local', spec: 'abandon work' }) + const { dispatch } = db.createStartingWorkerDispatch({ + taskId: task.id, + startOptions: {}, + creator: { kind: 'system' }, + maxDepth: 9 + }) + db.prepareStartingWorkerAuthority({ + dispatchId: dispatch.id, + handle: `term_${name}`, + paneKey: `tab_${name}:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaa${name.length}`, + processIncarnation: `inc_${name}`, + worktreeId: 'wt', + effects: [], + setupState: 'not_configured', + terminalOwnership: 'created' + }) + return { task, dispatch } + } + + function readyWorker(taskId?: string, name?: string) { + const started = startWorker(taskId, name) + db.markWorkerDispatchReady(started.dispatch.id) + return started + } + + function setReleaseState(dispatchId: string, releaseState: string) { + db.db + .prepare('UPDATE worker_terminal_resources SET release_state = ? WHERE owner_dispatch_id = ?') + .run(releaseState, dispatchId) + } + + function storeArchive(dispatchId: string) { + const resource = db.getWorkerTerminalResourceByOwner(dispatchId)! + db.db + .prepare( + "INSERT INTO worker_terminal_archives (dispatch_id, resource_id, kind, content) VALUES (?, ?, 'terminal_tail', '{}')" + ) + .run(dispatchId, resource.id) + } + + function expectAbandoned(dispatchId: string, taskId: string, priorError?: string) { + expect(db.getWorkerDispatch(dispatchId)).toMatchObject({ + state: 'abandoned', + stage: 'abandoned', + last_error: [priorError, 'Abandoned by term_coordinator.'].filter(Boolean).join(' ') + }) + expect(db.getDispatchContextById(dispatchId)).toMatchObject({ + status: 'failed', + last_failure: 'abandoned', + capability_revoked_at: expect.any(String) + }) + expect(db.getTask(taskId)?.status).toBe('blocked') + } + + it('settles a stop another runtime left stranded in stopping', () => { + const { task, dispatch } = readyWorker() + db.beginWorkerStop(dispatch.id, 'epoch_dead_runtime') + + expect(db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME, 'term_coordinator')).toMatchObject({ + disposition: 'abandoned' + }) + expectAbandoned(dispatch.id, task.id) + }) + + it.each([ + [ + 'stop_unknown', + 'the terminal is external', + (id: string) => { + db.beginWorkerStop(id, THIS_RUNTIME) + db.markWorkerStopUnknown(id, 'the terminal is external') + } + ], + [ + 'start_unknown', + 'lost contact', + (id: string) => db.markWorkerStartUnknown(id, 'prompt', 'lost contact') + ] + ] as const)('settles a %s worker and records who abandoned it', (state, priorError, reach) => { + const { task, dispatch } = startWorker() + if (state === 'stop_unknown') { + db.markWorkerDispatchReady(dispatch.id) + } + reach(dispatch.id) + + expect(db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME, 'term_coordinator')).toMatchObject({ + disposition: 'abandoned' + }) + expectAbandoned(dispatch.id, task.id, priorError) + }) + + it('says so when it cannot tell who abandoned the worker', () => { + const { dispatch } = readyWorker() + db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME) + + expect(db.getWorkerDispatch(dispatch.id)?.last_error).toBe( + 'Abandoned by an unidentified caller.' + ) + }) + + it('settles an active attempt that is no longer the Task latest without touching the newer one', () => { + const first = readyWorker(undefined, 'first') + // An interleaved re-dispatch leaves two active attempts; the older one used to be a no-op. + db.db.prepare("UPDATE tasks SET status = 'ready' WHERE id = ?").run(first.task.id) + const second = readyWorker(first.task.id, 'second') + + expect(db.abandonWorkerDispatch(first.dispatch.id, THIS_RUNTIME).disposition).toBe('abandoned') + expect(db.getWorkerDispatch(first.dispatch.id)?.state).toBe('abandoned') + expect(db.getWorkerDispatch(second.dispatch.id)?.state).toBe('ready') + expect(db.getTask(first.task.id)?.status).toBe('dispatched') + }) + + it.each(['requested', 'not_requested'] as const)( + 'retains the terminal of a worker it settles whose release is %s', + (releaseState) => { + const { dispatch } = readyWorker() + setReleaseState(dispatch.id, releaseState) + storeArchive(dispatch.id) + + expect(db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME).disposition).toBe('abandoned') + expect(db.getWorkerTerminalResourceByOwner(dispatch.id)).toMatchObject({ + ownership_state: 'owned', + release_state: 'retained', + retained_reason: 'user_requested' + }) + expect(db.getWorkerTerminalArchive(dispatch.id)).toBeUndefined() + } + ) + + it('leaves a settled worker and its terminal release untouched', () => { + const { dispatch } = readyWorker() + db.failDispatch(dispatch.id, 'operator closed the tab', { workerProcessExited: true }) + setReleaseState(dispatch.id, 'requested') + + expect(db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME).disposition).toBe('already_settled') + expect(db.getWorkerTerminalResourceByOwner(dispatch.id)?.release_state).toBe('requested') + }) + + it('keeps a committed release and a terminal Orca does not own as they are', () => { + const releasing = readyWorker(undefined, 'releasing') + setReleaseState(releasing.dispatch.id, 'releasing') + const unknown = readyWorker(undefined, 'unknown') + setReleaseState(unknown.dispatch.id, 'unknown') + storeArchive(unknown.dispatch.id) + const external = readyWorker(undefined, 'external') + db.db + .prepare( + "UPDATE worker_terminal_resources SET ownership_state = 'external', release_state = 'unknown' WHERE owner_dispatch_id = ?" + ) + .run(external.dispatch.id) + + db.abandonWorkerDispatch(releasing.dispatch.id, THIS_RUNTIME) + db.abandonWorkerDispatch(unknown.dispatch.id, THIS_RUNTIME) + db.abandonWorkerDispatch(external.dispatch.id, THIS_RUNTIME) + + expect(db.getWorkerTerminalResourceByOwner(releasing.dispatch.id)?.release_state).toBe( + 'releasing' + ) + expect(db.getWorkerTerminalResourceByOwner(unknown.dispatch.id)?.release_state).toBe('unknown') + expect(db.getWorkerTerminalArchive(unknown.dispatch.id)).toBeDefined() + expect(db.getWorkerTerminalResourceByOwner(external.dispatch.id)).toMatchObject({ + ownership_state: 'external', + release_state: 'unknown' + }) + }) + + it('reports an abandon repeated on an abandoned worker as already abandoned', () => { + const { dispatch } = readyWorker() + db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME) + + expect(db.abandonWorkerDispatch(dispatch.id, THIS_RUNTIME).disposition).toBe( + 'already_abandoned' + ) + }) +}) diff --git a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.ts b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.ts index 7549cb4eb26e..a20e483e7d19 100644 --- a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.ts +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-abandon.ts @@ -7,14 +7,20 @@ import { import type { OrchestrationDb } from '../orchestration-db' import { reconcileTaskAfterDispatchInterruption } from '../dispatch-context/task-dispatch-reconciliation' import { transitionLifecycleWithDb } from '../lifecycle-transition' +import { WORKER_SETTLED_STATES } from '../../worker-terminal-ownership' +import { isStopStrandedByAnotherRuntime } from './worker-dispatch-stop' +import { retainTerminalResourceInTransaction } from '../worker-terminal/worker-terminal-archive' export function abandonWorkerDispatch( this: OrchestrationDb, - dispatchId: string + dispatchId: string, + runtimeEpoch: string, + abandonedBy?: string ): | { - disposition: 'abandoned' | 'already_abandoned' | 'stale' + disposition: 'abandoned' | 'already_abandoned' | 'already_settled' worker: WorkerDispatchRow + superseded: boolean } | ({ disposition: 'context_only' } & ContextOnlyDispatchReleaseResult) { this.db.exec('BEGIN IMMEDIATE') @@ -32,54 +38,63 @@ export function abandonWorkerDispatch( this.db.exec('COMMIT') return { disposition: 'context_only', ...released } } - if (worker.state === 'abandoned') { - this.db.exec('COMMIT') - return { disposition: 'already_abandoned', worker } - } - if (this.getDispatchContext(dispatch.task_id)?.id !== dispatchId) { - this.db.exec('COMMIT') - return { disposition: 'stale', worker } - } - if (worker.state === 'stopping') { + // Only this runtime's own in-flight stop is refused: it always ends in stopped or stop_unknown. + if (worker.state === 'stopping' && !isStopStrandedByAnotherRuntime(worker, runtimeEpoch)) { throw new OrchestrationError( 'dispatch_inactive', `Dispatch ${dispatchId} is stopping; wait for worker-stop to settle before abandoning.` ) } - if (worker.state === 'failed' || worker.state === 'stopped') { - this.db.exec('COMMIT') - return { disposition: 'stale', worker } - } - if (worker.state === 'succeeded') { - throw new OrchestrationError( - 'dispatch_inactive', - `Dispatch ${dispatchId} already succeeded and cannot be abandoned.` - ) - } - transitionLifecycleWithDb(this.db, { - entity: 'worker', - id: dispatchId, - from: worker.state, - to: 'abandoned', - projection: { stage: 'abandoned', updated_at: new Date().toISOString() } - }) - if (['pending', 'dispatched'].includes(dispatch.status)) { + const settles = !WORKER_SETTLED_STATES.includes(worker.state) + const superseded = this.getDispatchContext(dispatch.task_id)?.id !== dispatchId + if (settles) { + const now = new Date().toISOString() transitionLifecycleWithDb(this.db, { - entity: 'dispatch', + entity: 'worker', id: dispatchId, - from: dispatch.status, - to: 'failed', + from: worker.state, + to: 'abandoned', projection: { - capability_revoked_at: dispatch.capability_revoked_at ?? new Date().toISOString(), - completed_at: dispatch.completed_at ?? new Date().toISOString() + stage: 'abandoned', + // Why: an unknown start or stop's diagnostic stays readable beside the attribution. + last_error: [ + worker.last_error, + `Abandoned by ${abandonedBy ?? 'an unidentified caller'}.` + ] + .filter(Boolean) + .join(' '), + updated_at: now } }) + if (['pending', 'dispatched'].includes(dispatch.status)) { + transitionLifecycleWithDb(this.db, { + entity: 'dispatch', + id: dispatchId, + from: dispatch.status, + to: 'failed', + projection: { + last_failure: 'abandoned', + capability_revoked_at: dispatch.capability_revoked_at ?? now, + completed_at: dispatch.completed_at ?? now + } + }) + } + reconcileTaskAfterDispatchInterruption(this, dispatch.task_id, dispatchId) + this.closeQuestionsForDispatch(dispatchId) + const terminal = this.getWorkerTerminalResourceByOwner(dispatchId) + // Abandon hands an owned terminal back instead of closing it, by the same rule as worker-retain. + if (terminal?.ownership_state === 'owned') { + retainTerminalResourceInTransaction(this, terminal.id, dispatchId) + } } - reconcileTaskAfterDispatchInterruption(this, dispatch.task_id, dispatchId) - this.closeQuestionsForDispatch(dispatchId) this.db.exec('COMMIT') return { - disposition: 'abandoned', + disposition: settles + ? 'abandoned' + : worker.state === 'abandoned' + ? 'already_abandoned' + : 'already_settled', + superseded, worker: this.getWorkerDispatch(dispatchId) as WorkerDispatchRow } } catch (error) { diff --git a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-stop.ts b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-stop.ts index fd4ee773bb4e..62ff9dba3c56 100644 --- a/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-stop.ts +++ b/src/main/runtime/orchestration/db/worker-dispatch/worker-dispatch-stop.ts @@ -32,6 +32,14 @@ export function isDispatchProcessCurrent( ) } +/** A `stopping` row whose stop belongs to a runtime that can no longer report its outcome. */ +export function isStopStrandedByAnotherRuntime( + worker: WorkerDispatchRow, + runtimeEpoch: string +): boolean { + return worker.state === 'stopping' && worker.runtime_epoch !== runtimeEpoch +} + export function beginWorkerStop( this: OrchestrationDb, dispatchId: string, @@ -69,9 +77,10 @@ export function beginWorkerStop( // follows claims a clean stop only from `stopping` under its own epoch // (failActiveDispatchOnExit), so it would then read the operator's stop as a crash and // escalate it. Same predicate as that reader, so both agree on whose stop this is. - const stopStrandedByAnotherRuntime = - worker.state === 'stopping' && worker.runtime_epoch !== runtimeEpoch - if (!['ready', 'start_unknown'].includes(worker.state) && !stopStrandedByAnotherRuntime) { + if ( + !['ready', 'start_unknown'].includes(worker.state) && + !isStopStrandedByAnotherRuntime(worker, runtimeEpoch) + ) { throw new OrchestrationError( 'dispatch_inactive', `Dispatch ${dispatchId} cannot stop from ${worker.state}.` diff --git a/src/main/runtime/orchestration/db/worker-terminal/worker-terminal-archive.ts b/src/main/runtime/orchestration/db/worker-terminal/worker-terminal-archive.ts index 4bb4fcff8234..4f5d401b6f3b 100644 --- a/src/main/runtime/orchestration/db/worker-terminal/worker-terminal-archive.ts +++ b/src/main/runtime/orchestration/db/worker-terminal/worker-terminal-archive.ts @@ -141,6 +141,31 @@ export function revertWorkerTerminalReleaseToRetained( return this.getWorkerTerminalResource(resourceId) as WorkerTerminalResourceRow } +/** + * The one retain rule; the caller holds the transaction. A committed release (releasing, unknown) + * may already have closed the tab, so it keeps its state and the archive that preserved the output. + */ +export function retainTerminalResourceInTransaction( + database: OrchestrationDb, + resourceId: string, + dispatchId: string +): boolean { + const { changes } = database.db + .prepare( + `UPDATE worker_terminal_resources + SET release_state = 'retained', retained_reason = 'user_requested', + updated_at = datetime('now') + WHERE id = ? AND release_state IN ('not_requested', 'retained', 'requested')` + ) + .run(resourceId) + if (changes > 0) { + database.db + .prepare('DELETE FROM worker_terminal_archives WHERE dispatch_id = ?') + .run(dispatchId) + } + return changes > 0 +} + export function retainWorkerTerminalResource( this: OrchestrationDb, dispatchId: string @@ -171,20 +196,12 @@ export function retainWorkerTerminalResource( this.db.exec('COMMIT') return { disposition: 'already_released', resource } } - this.db - .prepare( - `UPDATE worker_terminal_resources - SET release_state = 'retained', retained_reason = 'user_requested', - updated_at = datetime('now') - WHERE id = ? AND release_state IN ('not_requested', 'retained', 'requested')` - ) - .run(resource.id) + const retained = retainTerminalResourceInTransaction(this, resource.id, dispatchId) const updated = this.getWorkerTerminalResource(resource.id) as WorkerTerminalResourceRow - if (updated.release_state !== 'retained') { + if (!retained) { this.db.exec('COMMIT') return { disposition: 'release_committed', resource: updated } } - this.db.prepare('DELETE FROM worker_terminal_archives WHERE dispatch_id = ?').run(dispatchId) this.db.exec('COMMIT') return { disposition: 'retained', resource: updated } } catch (error) { diff --git a/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts b/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts index 1e1acf3fd005..c8f6963e7c18 100644 --- a/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts +++ b/src/main/runtime/orchestration/orchestration-worker-dispatch-db.test.ts @@ -297,8 +297,8 @@ describe('OrchestrationDb worker Dispatch state', () => { }) d.markWorkerDispatchReady(second.dispatch.id) - expect(d.abandonWorkerDispatch(first.dispatch.id)).toMatchObject({ - disposition: 'stale', + expect(d.abandonWorkerDispatch(first.dispatch.id, 'epoch_test')).toMatchObject({ + disposition: 'already_settled', worker: { state: 'failed' } }) expect(d.getTask(task.id)?.status).toBe('dispatched') diff --git a/src/main/runtime/rpc/methods/orchestration/messaging/check-superseded-terminal.test.ts b/src/main/runtime/rpc/methods/orchestration/messaging/check-superseded-terminal.test.ts index 65cc3192b1d5..9287cc0c1dc4 100644 --- a/src/main/runtime/rpc/methods/orchestration/messaging/check-superseded-terminal.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/messaging/check-superseded-terminal.test.ts @@ -55,7 +55,7 @@ describe('orchestration.check from a terminal whose Attempt was superseded', () ;({ db, ctx } = h.setup()) const task = db.createTask({ spec: 'work that moves terminals' }) const abandoned = startWorker(task.id, 'term_old', PANE_OLD) - db.abandonWorkerDispatch(abandoned) + db.abandonWorkerDispatch(abandoned, 'epoch_test') startWorker(task.id, 'term_new', PANE_NEW, abandoned) return abandoned } diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-abandon-caller.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-abandon-caller.test.ts new file mode 100644 index 000000000000..ab7588bf7d3b --- /dev/null +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-abandon-caller.test.ts @@ -0,0 +1,90 @@ +import { afterEach, beforeEach, describe, expect, it } from 'vitest' +import { OrcaRuntimeService } from '../../../../orca-runtime' +import { OrchestrationDb } from '../../../../orchestration/db' +import type { OrchestrationSessionCaller } from '../../../../orchestration/orchestration-caller-identity' +import { ORCHESTRATION_METHODS } from '../../orchestration' +import { eraseRpcMethods, type RpcContext } from '../../../core' +import { parseOrcaSessionAddress } from '../../../../../../shared/orca-session-address' + +describe('orchestration.workerAbandon', () => { + let db: OrchestrationDb + let runtime: OrcaRuntimeService + + beforeEach(() => { + db = new OrchestrationDb(':memory:') + runtime = new OrcaRuntimeService() + runtime.setOrchestrationDb(db) + }) + afterEach(() => db.close()) + + function readyWorker(): string { + const task = db.createTask({ runId: 'run_legacy_local', spec: 'abandon caller' }) + const { dispatch } = db.createStartingWorkerDispatch({ + taskId: task.id, + startOptions: {}, + creator: { kind: 'system' }, + maxDepth: 9 + }) + db.prepareStartingWorkerAuthority({ + dispatchId: dispatch.id, + handle: 'term_worker', + paneKey: 'tab_worker:aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa', + processIncarnation: 'inc_worker', + worktreeId: 'wt', + effects: [], + setupState: 'not_configured' + }) + db.markWorkerDispatchReady(dispatch.id) + return dispatch.id + } + + async function abandon(dispatchId: string, ctx: Partial) { + const method = eraseRpcMethods(ORCHESTRATION_METHODS).find( + (m) => m.name === 'orchestration.workerAbandon' + )! + return method.handler(method.params!.parse({ dispatch: dispatchId }), { runtime, ...ctx }) + } + + it('never records an unverified terminal handle as the one who abandoned the worker', async () => { + const dispatchId = readyWorker() + + await expect( + abandon(dispatchId, { orchestrationCompatibilityEvidence: { terminalHandle: 'term_coord' } }) + ).resolves.toMatchObject({ state: 'abandoned', alreadySettled: false }) + expect(db.getWorkerDispatch(dispatchId)?.last_error).toBe( + 'Abandoned by an unidentified caller.' + ) + }) + + it('records the resolved Orca session as the one who abandoned the worker', async () => { + const dispatchId = readyWorker() + const orcaSessionId = parseOrcaSessionAddress('session:chat_1')! + const session: OrchestrationSessionCaller = { + address: 'session:chat_1', + terminalHandle: null, + paneKey: null, + orcaSessionId, + sessionId: orcaSessionId, + workspaceId: 'wt' + } + + await abandon(dispatchId, { + orchestrationCaller: session, + orchestrationCompatibilityEvidence: { terminalHandle: 'term_coord' } + }) + expect(db.getWorkerDispatch(dispatchId)?.last_error).toBe('Abandoned by session:chat_1.') + }) + + it('reports an already-settled worker as stale and changes nothing', async () => { + const dispatchId = readyWorker() + db.failDispatch(dispatchId, 'tab closed', { workerProcessExited: true }) + const before = db.getWorkerDispatch(dispatchId) + + await expect(abandon(dispatchId, {})).resolves.toMatchObject({ + state: 'failed', + alreadySettled: true, + stale: true + }) + expect(db.getWorkerDispatch(dispatchId)).toEqual(before) + }) +}) diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-control.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-control.ts index a9211532626a..9fe3b3979252 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-control.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-control.ts @@ -199,8 +199,13 @@ export const ORCHESTRATION_WORKER_CONTROL_METHODS = [ defineMethod({ name: 'orchestration.workerAbandon', params: WorkerDispatchParams, - handler: (params, { runtime }) => { - const abandoned = runtime.getOrchestrationDb().abandonWorkerDispatch(params.dispatch) + handler: (params, { runtime, orchestrationCaller }) => { + const abandoned = runtime.getOrchestrationDb().abandonWorkerDispatch( + params.dispatch, + runtime.getRuntimeId(), + // Why: only a session caller is verified; terminal env could name anyone. + orchestrationCaller?.address + ) if (abandoned.disposition === 'context_only') { if (!abandoned.alreadySettled) { // Abandon settles the Dispatch, so it owes the same binding release stop and release do: @@ -227,12 +232,13 @@ export const ORCHESTRATION_WORKER_CONTROL_METHODS = [ dispatchId: params.dispatch, state: worker.state, alreadySettled: abandoned.disposition !== 'abandoned', - stale: abandoned.disposition === 'stale', + // Kept for --json readers: this attempt was no longer current, as main reported it. + stale: abandoned.superseded || abandoned.disposition === 'already_settled', processAction: 'none', warning: - abandoned.disposition === 'stale' - ? 'The Dispatch is no longer current; no state or process changed.' - : 'Possibly-live resources were retained; no process was stopped or deleted.', + abandoned.disposition === 'abandoned' + ? 'Possibly-live resources were retained; no process was stopped or deleted.' + : `The worker was already ${worker.state}; its terminal was left open and no process changed.`, residualResources: JSON.parse(worker.residual_resources) as unknown[] } } diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-release-ownership-guard.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-release-ownership-guard.test.ts index d7f6e9502bff..9a6a399cfb16 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-release-ownership-guard.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-release-ownership-guard.test.ts @@ -53,7 +53,7 @@ describe('workerRelease on a retained resource whose process exited', () => { it('records the archive as unavailable rather than retaining the pane forever', async () => { const { dispatchId } = await harness.startWorker() // Abandoned workers never reach `requested`, the only state that writes an archive. - expect(harness.db.abandonWorkerDispatch(dispatchId).disposition).toBe('abandoned') + expect(harness.db.abandonWorkerDispatch(dispatchId, 'epoch_test').disposition).toBe('abandoned') expect(harness.db.getWorkerTerminalArchive(dispatchId)).toBeFalsy() harness.inspectProcessLiveness.mockResolvedValue('exited') @@ -83,7 +83,7 @@ describe('workerRelease on a retained resource whose process exited', () => { it('still refuses when an archive names a different resource', async () => { const { dispatchId } = await harness.startWorker() const resource = harness.db.getWorkerTerminalResourceByOwner(dispatchId)! - expect(harness.db.abandonWorkerDispatch(dispatchId).disposition).toBe('abandoned') + expect(harness.db.abandonWorkerDispatch(dispatchId, 'epoch_test').disposition).toBe('abandoned') harness.db.storeWorkerTerminalArchive({ dispatchId, resourceId: `${resource.id}-other`, diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-release.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-release.test.ts index e6466ed48c47..f6f84c4ee4ac 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-release.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-release.test.ts @@ -192,7 +192,7 @@ describe('orchestration worker release', () => { h.db.beginWorkerStop(dispatchId, h.runtime.getRuntimeId()) h.db.settleWorkerStop(dispatchId) } else { - h.db.abandonWorkerDispatch(dispatchId) + h.db.abandonWorkerDispatch(dispatchId, 'epoch_test') } h.inspectProcessLiveness.mockResolvedValue('exited') @@ -221,7 +221,7 @@ describe('orchestration worker release', () => { h.db.beginWorkerStop(dispatchId, h.runtime.getRuntimeId()) h.db.settleWorkerStop(dispatchId) } else { - h.db.abandonWorkerDispatch(dispatchId) + h.db.abandonWorkerDispatch(dispatchId, 'epoch_test') } h.inspectProcessLiveness.mockResolvedValue('unverifiable') diff --git a/src/main/runtime/rpc/methods/orchestration/worker/worker-stop-liveness-verdict.test.ts b/src/main/runtime/rpc/methods/orchestration/worker/worker-stop-liveness-verdict.test.ts index c8dec854e4c0..fa35717998d0 100644 --- a/src/main/runtime/rpc/methods/orchestration/worker/worker-stop-liveness-verdict.test.ts +++ b/src/main/runtime/rpc/methods/orchestration/worker/worker-stop-liveness-verdict.test.ts @@ -229,7 +229,7 @@ describe('worker-stop against a terminal we lost contact with', () => { const dispatch = createWorker() expect(db.beginWorkerStop(dispatch.id, runtime.getRuntimeId()).disposition).toBe('stopping') - expect(() => db.abandonWorkerDispatch(dispatch.id)).toThrow( + expect(() => db.abandonWorkerDispatch(dispatch.id, runtime.getRuntimeId())).toThrow( 'is stopping; wait for worker-stop to settle before abandoning' ) expect(db.getWorkerDispatch(dispatch.id)?.state).toBe('stopping') diff --git a/src/main/runtime/structured-worker-at-rest.test.ts b/src/main/runtime/structured-worker-at-rest.test.ts index 28317e4da884..adfb181524d1 100644 --- a/src/main/runtime/structured-worker-at-rest.test.ts +++ b/src/main/runtime/structured-worker-at-rest.test.ts @@ -189,7 +189,7 @@ describe('an open dispatch keeps its worker running (P2-19 i)', () => { const handle = resource.terminal_handle const recipients = () => listAddressableStructuredWorkers(db) // At rest, its dispatch abandoned: still a recipient, as a terminal worker left running is. - db.abandonWorkerDispatch(dispatch.id) + db.abandonWorkerDispatch(dispatch.id, 'epoch_test') const worktreeGroup = () => resolveGroupAddress('@worktree:wt_1', 'term_sender', recipients(), () => 'idle') expect(worktreeGroup()).toEqual([handle]) From cb363444f3f39993b7e657a20fb5338a6233c49b Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 11:36:57 -0700 Subject: [PATCH 13/79] feat(native-chat): register Codex default-mode helpers as subagents (#22619) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * feat(native-chat): register Codex default-mode helpers as subagents Codex's default multi-agent mode announces a helper only as the collabAgentToolCall that spawned it; it sends no subAgentActivity. The roster and the background-task tracker registered children only from subAgentActivity, so such a helper had no record, no strip row and no roster row, and its commands read as the session's own. One announcement reader now yields a child from either wire shape, and both the journal roster and the tracker register through it into the same executions, so a session sending both keeps one child per thread. A finished closeAgent ends the helper's running turn as stopped through the executions, beside the child's own turn and thread frames. Each collab call renders as a tool row (spawn_agent, wait_agent, close_agent, ...) naming the helper the way the roster does, with what the helper said back as its output, instead of the raw provider row. * fix(native-chat): name a spawn row by its prompt until the roster holds its helper Also pin that the roster row appears when the helper's first turn arrives before the spawn call finishes. * test(native-chat): a spawn call keeps its own row beside the roster row it starts * fix(native-chat): pair a structured tool row's output with its own call A run paired results to calls by position alone, so once one call finished with no output (a Codex spawn_agent row) every later output drew under the call before its own. A structured row carries its call and output together, so the projected result now names its call id and pairing honors it, falling back to position for results that name none. * fix(native-chat): the Codex roster row follows the executions, so every child ending settles it The subagent-group row was rewritten only on a child's turn/started and turn/completed. A child turn ended any other way — a fatal error, its thread closing, its caller closing it — settled the strip and the host record through the executions but left the transcript row reading working until the session ended. The executions now say when a child's execution changes, and the roster revises its row from that, whichever frame changed it. handleTurn still re-derives to hand back its write admission; the revision is idempotent. * fix(native-chat): a restored Codex call row names its helper, not its thread id History replay never runs the live item router, so the roster never learned the helpers a restored thread had spawned, and a restored wait_agent, close_agent or send_input row labelled its helper with the raw thread id. Replay now registers each announced helper for its name and membership only: register starts no execution, so no strip entry, record or roster row claims the helper runs until a live turn of its own says so. The replayed-item handling moves into the restore module beside the replay that calls it. Also name the roster's render contract in handleItem, and say why the collab tool-name map is not a spelling fix. * fix(native-chat): register a Codex helper whose spawn call failed but created its thread Codex reports a spawn as failed when the helper it created errored at birth, yet the call still names the thread it created, and that thread can run. The spawn was registered only when the call completed, so such a helper existed nowhere and its shell read as the session's own bare command: the original default-mode bug. A spawn now registers whenever it names a receiver; one in progress, or one that created nothing, names none. A failed closeAgent still ends nothing, because the helper was not closed. * refactor(native-chat): each Codex thread's latest token total gets its own home Every thread's running total, member or not, with the rule that the newest frame replaces the last and the recency-ordered cap, moves out of the roster into codex-thread-token-totals.ts. The roster still selects its children's totals at write time. With the producer linkage the roster now builds, it was past the size limit. * refactor(native-chat): the messages a session list quotes get their own module The readers of a session's newest own prompt and own assistant prose move from the status projection into structured-agent-session-latest-messages.ts, and are re-exported so their consumers keep one import site. With the status clock the projection now carries, a tool result naming its call put it past the size limit. * test(native-chat): a Codex helper's command is a live record until its process reports its exit A helper's in-turn shell is a live command record owned by the helper and is also its open Bash operation; its exit removes the record. A caller's closeAgent kills the helper's processes, and each still reports its exit on the helper's thread, so the close leaves the command to that exit rather than ending it itself. * fix(native-chat): every reader of a tool run pairs a result with the call it names The folded desktop run now pairs a result with the call it names, but mobile's tool run, the desktop edit cards and the task lists still paired by position through `pairToolBlocks`. In a Codex default-mode session the new `spawn_agent` row finishes with no output, so on mobile the helper's reply drew under `spawn_agent` and `wait_agent` showed none, and a desktop edit card could take the next command's output as its own. `pairToolBlocks` now follows the same rule as `pairNativeChatToolResults`: a result that names its call answers that call, and one that names none answers the oldest unanswered call as before. * fix(native-chat): a Codex helper's turn ends on its own frames, not on its caller's closeAgent A finished `closeAgent` call ended the helper's running turn as `stopped`. But the call's status is not the close's outcome: Codex sets it from the helper's own agent status, so a close that errors on a running helper still reports `completed`. Orca then marked a helper that was still running as cancelled in the strip, the host record and the roster row, and because the first ending a turn gets stands, its real ending could never correct it. A close that works does not need the edge either. Codex's shutdown of the helper aborts its running turn, and the app-server reports that as the helper's own `turn/completed` with status `interrupted`, which Orca already maps to `stopped`. So a helper's turn now ends only on its own frames (its `turn/completed`, a fatal `error`, `thread/closed`) or the session ending, and the close is only its call row. This also drops the child-work evidence re-keying that existed only for the close: every remaining frame names the thread that sent it. * fix(native-chat): a tool result that names a call is never given to a different call A result that named a call id with no unanswered match fell back to the oldest unanswered call. On mobile, a run shows at most six calls, so a command past that window whose output named its own call gave that output to an earlier call that finished with none, such as `spawn_agent`. A result that names its call now answers only that call and otherwise stays unpaired. A result that names no call still answers the oldest unanswered one. Both pairing readers share the one rule through `answeredToolCallIndex`. * test(native-chat): name the Codex default-mode test file after the subagents it covers * fix(native-chat): a Codex helper is known from any call that names it, not only its spawn A helper whose spawn Orca never saw (history replay after compaction dropped the spawn, or a resume or message to a helper from earlier) stayed unregistered, so its shell read as the session's own command: the bug this PR fixes, in another shape. Every collab call now announces each helper it names, skipping a receiver the finished call reports notFound. The spawn's prompt still labels a helper when it was seen; otherwise the helper has no label and reads as any unnamed subagent does. A send_input, like the spawn, names the parent turn the helper's next run belongs to. The session's own thread is excluded once, in the reader, instead of at each of its three callers. * fix(native-chat): every finished Codex collab call row carries what the call reports A client that predates result call ids (an older mobile app, or an older desktop reading a newer host) projects the journal itself and pairs each tool result with the oldest unanswered call. This PR publishes collab calls as tool rows, and a finished spawn_agent, send_input, close_agent or resume_agent on a running helper had no output, so such a client drew each later output in the run under the call before its own (the parent's shell output under spawn_agent, the helper's reply under the shell). Every finished call now has an output taken from the item: a wait's reply from each helper that finished (an errored helper's error), and for every other call the helper's reported status in Codex's own words (Pending init, Running, Completed, ...). A close or resume no longer shows the helper's last reply as if the call returned it. A wait whose end names no helper (it timed out, or v2) reads Finished waiting; any other call with no state reads its own status. A call also keeps naming the helpers its started item named: Codex ends a timed-out wait with no receivers, so its row lost the helper's name when it finished. * refactor(native-chat): the desktop run's tool pairing is a view of pairToolBlocks Desktop runs and every other reader (mobile, edit cards, task lists, the ask row) each had their own pairing loop sharing one index rule. The desktop's pairNativeChatToolResults now reads the pairs pairToolBlocks makes, so a run is paired by one loop and a new reader cannot add a third. No behaviour change. * test(native-chat): type the positional-client pairs without assertions * fix(native-chat): a finished Codex collab call row says what the call did, in Codex's own words A close_agent row read `Running` and a spawn_agent row `Pending init`: each showed the helper's status snapshot from before the call took effect, so a close that stopped its helper read as though it had not worked. Each finished call's output now follows Codex's own client: spawn reads `Spawned` (or `Agent spawn failed` when no helper was created), send_input `Sent input`, close `Closed`, and resume the helper's status summary. A wait keeps each finished helper's reply, with other statuses in Codex's summary wording (`Error - `). The row label already names the helper, so the output does not repeat it. * docs(native-chat): the collab call reader says which calls show the helper's snapshot as output Since spawn, send_input and close rows say what the call did, the reader's header was wrong to claim every call row shows the reported snapshot as its output: only a wait or resume does. * fix(native-chat): a Codex spawn call no longer claims it ran an agent Codex's spawn_agent call ends as soon as the helper starts, so counting it as running an agent drew "Ran 1 agent" directly above "Kicked off 1 subagent · working" while the helper was still working, and "Ran 1 agent · 1 failed" when Stop cancelled a spawn before any helper existed. Claude's Task call lasts as long as its subagent, so it keeps the agent category; the Codex spawn row is now a plain tool call and the roster row alone stands for the helper. * test(native-chat): a Codex helper's row settles on the failed completion that follows a fatal error Codex follows every turn-ending `error` with the turn's failed `turn/completed`, on a helper's thread as on the primary, and only that completion ends the turn. The roster test now sends both and checks the row, strip and record stay working through the error and settle on the completion. * fix(native-chat): a Codex helper's section opens while the parent spawns, waits on or messages it A running chat holds a subagent's section open while the parent's newest row delegates to that subagent. For Codex that rule only knew the raw collab status row; this PR writes each collab call as a tool row (spawn_agent, wait_agent, send_input, close_agent, ...) naming its helpers in `input.agents`, so a default-mode helper's section stayed shut for its whole run while Claude's opened. The delegation reader now reads a Codex collab tool row as a delegation to the first helper it names, the same rule the raw row keeps for journals written before this change; a call naming no helper stays ordinary output. The row names and the `agents` key live in one shared module the host writes from and the reader reads, instead of a second list. The new test drives the real adapter's rows through the transcript projection to the delegation; the collab frame harness moves to a fixture shared with the positional-clients test. * fix(native-chat): a Codex collab call with a long prompt still names its helper A collab call row bounded its whole input as one value, so a spawn or send_input whose prompt passed the 16 KB journal limit was stored as a clipped wrapper: the row lost the helper's name and thread ids, and with them the label and the delegation that opens the helper's section. The prompt is now clipped on its own with the journal's inline-text bound, marker included; the helper's name, ids, model and effort are bounded as before and always survive. --- config/tsconfig.node.json | 1 + .../codex/codex-background-task-frames.ts | 52 +- .../codex-background-task-tracker.test.ts | 6 +- .../codex/codex-background-task-tracker.ts | 16 +- src/main/codex/codex-child-work-evidence.ts | 30 +- .../codex-collab-agent-item-translation.ts | 154 ++++ .../codex/codex-collab-agent-tool-call.ts | 102 +++ .../codex-collab-call-delegation.test.ts | 114 +++ .../codex/codex-collab-call-frames-fixture.ts | 127 ++++ ...ab-call-rows-on-positional-clients.test.ts | 247 ++++++ .../codex-default-mode-subagents.test.ts | 704 ++++++++++++++++++ src/main/codex/codex-item-run-state.ts | 16 + .../codex-structured-item-translation.ts | 38 +- .../codex/codex-structured-journal-items.ts | 17 +- .../codex-structured-journal-settlement.ts | 7 +- ...-structured-journal-translation-restore.ts | 42 +- ...-structured-journal-translation-writers.ts | 8 +- .../codex-structured-journal-translation.ts | 22 +- src/main/codex/codex-subagent-activity.ts | 73 +- src/main/codex/codex-subagent-executions.ts | 18 + src/main/codex/codex-subagent-roster.ts | 96 +-- .../codex/codex-thread-token-totals.test.ts | 24 + src/main/codex/codex-thread-token-totals.ts | 32 + .../provider-frame-disposition.ts | 8 +- .../native-chat-subagent-delegation.ts | 18 +- src/shared/codex-collab-agent-tools.ts | 41 + src/shared/native-chat-tool-fold.ts | 23 +- src/shared/native-chat-tool-icon.test.ts | 7 + src/shared/native-chat-tool-icon.ts | 2 + .../native-chat-tool-pair-limit.test.ts | 37 + src/shared/native-chat-tool-pairing.test.ts | 26 + src/shared/native-chat-tool-pairing.ts | 43 +- src/shared/native-chat-types.ts | 2 + ...tructured-agent-session-projection.test.ts | 17 + .../structured-agent-session-projection.ts | 4 +- 35 files changed, 1981 insertions(+), 193 deletions(-) create mode 100644 src/main/codex/codex-collab-agent-item-translation.ts create mode 100644 src/main/codex/codex-collab-agent-tool-call.ts create mode 100644 src/main/codex/codex-collab-call-delegation.test.ts create mode 100644 src/main/codex/codex-collab-call-frames-fixture.ts create mode 100644 src/main/codex/codex-collab-call-rows-on-positional-clients.test.ts create mode 100644 src/main/codex/codex-default-mode-subagents.test.ts create mode 100644 src/main/codex/codex-item-run-state.ts create mode 100644 src/main/codex/codex-thread-token-totals.test.ts create mode 100644 src/main/codex/codex-thread-token-totals.ts create mode 100644 src/shared/codex-collab-agent-tools.ts diff --git a/config/tsconfig.node.json b/config/tsconfig.node.json index ff4e9a762764..7d4d951d2d62 100644 --- a/config/tsconfig.node.json +++ b/config/tsconfig.node.json @@ -10,6 +10,7 @@ "../src/renderer/src/lib/skill-freshness-display-status.ts", "../src/renderer/src/components/native-chat/native-chat-resolution-receipt.ts", "../src/renderer/src/components/native-chat/structured-agent-question-projection.ts", + "../src/renderer/src/components/native-chat/native-chat-subagent-delegation.ts", "../src/preload/**/*", "../src/shared/**/*", "../src/relay/**/*", diff --git a/src/main/codex/codex-background-task-frames.ts b/src/main/codex/codex-background-task-frames.ts index f01863956b6c..06536a857b8c 100644 --- a/src/main/codex/codex-background-task-frames.ts +++ b/src/main/codex/codex-background-task-frames.ts @@ -1,22 +1,23 @@ import type { NativeChatSubagentState } from '../../shared/native-chat-types' -import { - codexSubagentLabel, - isCodexRootAgentActivity, - readCodexSubagentActivity -} from './codex-subagent-activity' +import { readCodexSubagentAnnouncements } from './codex-subagent-activity' import { codexChildTurnState } from './codex-subagent-executions' import { readRecord } from './codex-item-field-readers' import { readCodexThreadItem } from './codex-structured-item-translation' import { readCodexTurnId } from './codex-structured-thread-facts' +export type CodexAnnouncedChild = { + agentThreadId: string + label: string | null + parentTurnId: string | null | undefined + /** The reporting thread, for a spawn: the agent that spawned the child. */ + spawnerThreadId: string | undefined +} + export type CodexBackgroundTaskFrame = | { - kind: 'subagent' - agentThreadId: string - label: string | null - parentTurnId: string | null | undefined - /** The reporting thread, for a `started` activity: the agent that spawned the child. */ - spawnerThreadId: string | undefined + /** An item naming children, several for a call on more than one helper. */ + kind: 'subagents' + children: CodexAnnouncedChild[] } | { kind: 'turn' @@ -78,23 +79,14 @@ export function readCodexBackgroundTaskFrame( return null } const item = readCodexThreadItem(readRecord(event.params).item) - const activity = item && readCodexSubagentActivity(item) - if ( - !activity || - activity.agentThreadId === primaryThreadId || - isCodexRootAgentActivity(activity) - ) { - return null - } - return { - kind: 'subagent', - agentThreadId: activity.agentThreadId, - label: codexSubagentLabel(activity), - parentTurnId: - activity.kind === 'started' || activity.kind === 'interacted' - ? readCodexTurnId(event.params) - : undefined, - // Only `started` names the spawner: other kinds ride whichever agent acted. - spawnerThreadId: activity.kind === 'started' ? event.threadId : undefined - } + const children = (item ? readCodexSubagentAnnouncements(item, primaryThreadId) : []).map( + (announcement) => ({ + agentThreadId: announcement.agentThreadId, + label: announcement.label, + parentTurnId: announcement.namesParentTurn ? readCodexTurnId(event.params) : undefined, + // Only a spawn names the spawner: other announcements ride whichever agent acted. + spawnerThreadId: announcement.spawned ? event.threadId : undefined + }) + ) + return children.length > 0 ? { kind: 'subagents', children } : null } diff --git a/src/main/codex/codex-background-task-tracker.test.ts b/src/main/codex/codex-background-task-tracker.test.ts index 0a70442a0b37..6c32896140b3 100644 --- a/src/main/codex/codex-background-task-tracker.test.ts +++ b/src/main/codex/codex-background-task-tracker.test.ts @@ -76,10 +76,8 @@ function command( describe('readCodexBackgroundTaskFrame', () => { it('reads activity as child metadata without inferring execution state', () => { expect(readCodexBackgroundTaskFrame(activity('interacted'), PRIMARY)).toEqual({ - kind: 'subagent', - agentThreadId: CHILD, - label: 'count_a', - parentTurnId: PARENT_TURN + kind: 'subagents', + children: [{ agentThreadId: CHILD, label: 'count_a', parentTurnId: PARENT_TURN }] }) }) diff --git a/src/main/codex/codex-background-task-tracker.ts b/src/main/codex/codex-background-task-tracker.ts index 6429a84d4fda..9fd2880dc75e 100644 --- a/src/main/codex/codex-background-task-tracker.ts +++ b/src/main/codex/codex-background-task-tracker.ts @@ -73,13 +73,15 @@ export class CodexBackgroundTaskTracker { : []) ] const frame = readCodexBackgroundTaskFrame(event, this.primaryThreadId) - if (frame?.kind === 'subagent') { - this.executions.register( - frame.agentThreadId, - frame.label, - frame.parentTurnId, - frame.spawnerThreadId - ) + if (frame?.kind === 'subagents') { + for (const child of frame.children) { + this.executions.register( + child.agentThreadId, + child.label, + child.parentTurnId, + child.spawnerThreadId + ) + } } else if (frame?.kind === 'thread-closed') { this.executions.closeThread(frame.threadId) } else if (frame && frame.threadId !== this.primaryThreadId) { diff --git a/src/main/codex/codex-child-work-evidence.ts b/src/main/codex/codex-child-work-evidence.ts index 93bda7a54a1f..af946afeedc0 100644 --- a/src/main/codex/codex-child-work-evidence.ts +++ b/src/main/codex/codex-child-work-evidence.ts @@ -100,15 +100,13 @@ export class CodexChildWorkEvidence { commands: readonly CodexBackgroundCommandChange[] ): void { this.queueCommands(commands) - const threadId = this.childThread(event, frame) - if (threadId === null) { - return - } - const facts = this.factsFor(threadId) - if (facts && event.threadId === threadId) { - this.record(facts, event) + for (const threadId of this.childThreads(event, frame)) { + const facts = this.factsFor(threadId) + if (facts && event.threadId === threadId) { + this.record(facts, event) + } + this.queueChild(threadId) } - this.queueChild(threadId) } /** The provider session is gone, with the commands it ended: no child it still ran can report @@ -125,17 +123,17 @@ export class CodexChildWorkEvidence { return pending.map((edge) => edge(observedAt)) } - private childThread( + private childThreads( event: CodexBackgroundTaskEvent, frame: CodexBackgroundTaskFrame | null - ): string | null { - const threadId = - frame?.kind === 'subagent' - ? frame.agentThreadId + ): string[] { + const threadIds = + frame?.kind === 'subagents' + ? frame.children.map((child) => child.agentThreadId) : frame || CHILD_FRAME_METHODS.has(event.method) - ? event.threadId - : null - return threadId === this.primaryThreadId ? null : threadId + ? [event.threadId] + : [] + return threadIds.filter((threadId) => threadId !== this.primaryThreadId) } /** A command belongs to the child thread that launched it; the session's own agent is no owner. diff --git a/src/main/codex/codex-collab-agent-item-translation.ts b/src/main/codex/codex-collab-agent-item-translation.ts new file mode 100644 index 000000000000..f12d22b7339d --- /dev/null +++ b/src/main/codex/codex-collab-agent-item-translation.ts @@ -0,0 +1,154 @@ +// A Codex `collabAgentToolCall` item → a tool row that names the helper it acted on. +// +// The row is the call the agent made (`spawn_agent`, `wait_agent`, `close_agent`, …); the helper +// itself is the subagent roster's row. A helper is named the way the roster names it, so the two +// rows read as the same child. + +import type { AgentJournalItemBody } from '../../shared/agent-session-journal-types' +import { CODEX_COLLAB_ROW_AGENTS_KEY } from '../../shared/codex-collab-agent-tools' +import { + boundInlineText, + boundToolInput, + DEFAULT_JOURNAL_PAYLOAD_LIMITS +} from '../native-chat/agent-session-journal/journal-payload-bounds' +import { + codexCollabHelperLabel, + codexCollabToolName, + readCodexCollabAgentToolCall, + type CodexCollabAgentToolCall +} from './codex-collab-agent-tool-call' +import { readString } from './codex-item-field-readers' +import { codexItemRunState } from './codex-item-run-state' +import type { CodexThreadItem } from './codex-thread-item-identity' + +/** The roster's name for a helper thread, or null for one it holds no name for. */ +export type CodexHelperName = (threadId: string) => string | null + +/** Who the call acted on. A spawn names its helper by its prompt until the roster holds the + * thread it became; a helper with no name (its spawn was never seen) is named by its thread id. */ +function helperNames(call: CodexCollabAgentToolCall, helperName?: CodexHelperName): string { + if (call.tool === 'spawnAgent') { + const spawned = call.receiverThreadIds[0] + return (spawned && helperName?.(spawned)) || codexCollabHelperLabel(call.prompt) || '' + } + return call.receiverThreadIds.map((threadId) => helperName?.(threadId) ?? threadId).join(', ') +} + +/** Codex's own words for a helper's `CollabAgentStatus`, as its client shows them. */ +const HELPER_STATUS_TEXT = new Map([ + ['pendingInit', 'Pending init'], + ['running', 'Running'], + ['interrupted', 'Interrupted'], + ['completed', 'Completed'], + ['shutdown', 'Shutdown'], + ['notFound', 'Not found'] +]) + +const CALL_STATUS_TEXT = new Map([ + ['completed', 'Completed'], + ['failed', 'Failed'], + ['interrupted', 'Interrupted'] +]) + +type HelperState = CodexCollabAgentToolCall['states'][number] + +/** A helper's status summarised as Codex's own client does: a completed helper with its last + * reply, an errored one with its error. */ +function statusSummary(state: HelperState): string | null { + if (state.status === 'errored') { + return `Error - ${state.message ?? 'Agent errored'}` + } + if (state.status === 'completed' && state.message) { + return `Completed - ${state.message}` + } + return state.status === null ? null : (HELPER_STATUS_TEXT.get(state.status) ?? state.status) +} + +/** A finished call's output, worded as Codex's own client words the call. Every finished call has + * one: a client that pairs results by position (one predating result call ids) would otherwise + * draw each later output in the run under the call before its own. The row label already names + * the helper, so the output does not. */ +function outputText(call: CodexCollabAgentToolCall, helperName?: CodexHelperName): string | null { + if (call.status === null || call.status === 'inProgress') { + return null + } + // The status a spawn or close reports predates what it did (a closed helper reads `running`), + // so these say what the call did. + switch (call.tool) { + case 'spawnAgent': + return call.receiverThreadIds.length > 0 ? 'Spawned' : 'Agent spawn failed' + case 'sendInput': + return 'Sent input' + case 'closeAgent': + return 'Closed' + case 'resumeAgent': + return (call.states[0] && statusSummary(call.states[0])) ?? 'Error - Agent resume failed' + } + const reports = call.states.flatMap((state) => { + // A wait returns what each finished helper said. + const text = + call.tool === 'wait' && state.status === 'completed' && state.message + ? state.message + : statusSummary(state) + return text === null ? [] : [{ threadId: state.threadId, text }] + }) + if (reports.length === 0) { + // A wait's end names only helpers that finished (none when it timed out; v2's never names any). + return call.tool === 'wait' + ? 'Finished waiting' + : (CALL_STATUS_TEXT.get(call.status) ?? call.status) + } + if (reports.length === 1 && call.receiverThreadIds.length === 1) { + return reports[0].text + } + return reports + .map(({ threadId, text }) => `${helperName?.(threadId) ?? threadId}: ${text}`) + .join('\n') +} + +/** The prompt is clipped on its own: bounded with the rest, a long one would clip away the + * helper's name and ids the row is read by. */ +function collabRowInput(fields: Record, prompt: string | null): unknown { + const bounded = boundToolInput(fields, DEFAULT_JOURNAL_PAYLOAD_LIMITS) + if (bounded !== fields) { + return bounded + } + const input = prompt + ? { ...fields, prompt: boundInlineText(prompt, DEFAULT_JOURNAL_PAYLOAD_LIMITS).text } + : fields + return Object.keys(input).length > 0 ? input : null +} + +/** `started` is the call's started item, when the caller still holds it. */ +export function codexCollabAgentToolCallBody( + item: CodexThreadItem, + helperName?: CodexHelperName, + started?: CodexThreadItem +): AgentJournalItemBody | null { + const call = readCodexCollabAgentToolCall(item, started) + if (!call) { + return null + } + const description = helperNames(call, helperName) + const model = readString(item, 'model') + const reasoningEffort = readString(item, 'reasoningEffort') + const fields = { + // `description` is the key the row label reads, so the helper's name leads the row. + ...(description ? { description } : {}), + ...(model ? { model } : {}), + ...(reasoningEffort ? { reasoningEffort } : {}), + ...(call.receiverThreadIds.length > 0 + ? { [CODEX_COLLAB_ROW_AGENTS_KEY]: call.receiverThreadIds } + : {}) + } + const text = outputText(call, helperName) + const output = text === null ? null : boundInlineText(text, DEFAULT_JOURNAL_PAYLOAD_LIMITS) + return { + kind: 'tool-call', + name: codexCollabToolName(call), + callId: call.id, + input: collabRowInput(fields, call.prompt), + state: codexItemRunState(item), + ...(output === null ? {} : { output: output.bounded }) + } +} diff --git a/src/main/codex/codex-collab-agent-tool-call.ts b/src/main/codex/codex-collab-agent-tool-call.ts new file mode 100644 index 000000000000..01809a7fb51d --- /dev/null +++ b/src/main/codex/codex-collab-agent-tool-call.ts @@ -0,0 +1,102 @@ +// Reading Codex's `collabAgentToolCall` items: the calls an agent makes to spawn, message, wait on +// and close its helpers. +// +// Codex's default multi-agent mode reports a helper ONLY this way; it emits no `subAgentActivity`. +// Shapes are from the app-server's generated schema (0.155) and a live default-mode session: +// * `spawnAgent` starts with `receiverThreadIds: []`. The helper's thread id first appears when +// the call ends, beside the `prompt` it was given — even when the call ends `failed`. +// * Every other call names its helpers from its start. A finished call reports a thread id it +// does not know as `notFound`; any other state says the helper exists. +// * Codex core also knows each receiver's nickname and role, but the app-server item does not +// carry them yet; `readCodexSubagentAnnouncements` is where they would be adopted. +// * The item names no nickname or task path, so the prompt is the only text that tells one +// helper from another. +// * A wait that times out ends naming no receiver and no state. +// * `agentsStates` is the caller's last-known snapshot of each receiver. The helper's own turn +// frames own its execution, so nothing here reads it as execution state: only `notFound`, +// which says the receiver is no helper, and the snapshot a wait or resume reports, which its +// row shows as output. A spawn's or close's predates what the call did, so its row does not. + +import { codexCollabToolRowName } from '../../shared/codex-collab-agent-tools' +import { collapsedToolInputPrefix } from '../../shared/native-chat-tool-preview-prefix' +import { readRecord, readString } from './codex-item-field-readers' +import type { CodexThreadItem } from './codex-thread-item-identity' + +export const CODEX_COLLAB_AGENT_TOOL_CALL_ITEM_TYPE = 'collabAgentToolCall' + +/** Row length for a helper named by its prompt: long enough to tell two helpers apart. */ +const MAX_HELPER_LABEL_CHARS = 80 + +export type CodexCollabAgentToolCall = { + id: string + /** The schema's `CollabAgentTool`: `spawnAgent`, `sendInput`, `wait`, `closeAgent`, … */ + tool: string + /** `inProgress`, `completed`, `failed` or `interrupted`. */ + status: string | null + receiverThreadIds: string[] + /** The receivers that are helpers: every one but those the call reports `notFound`. */ + helperThreadIds: string[] + prompt: string | null + /** Each receiver's reported snapshot, in `receiverThreadIds` order: its `CollabAgentStatus` + * (`pendingInit`, `running`, `completed`, `errored`, `shutdown`, …) and its message. */ + states: { threadId: string; status: string | null; message: string | null }[] +} + +function readReceiverThreadIds(item: CodexThreadItem | undefined): string[] { + return Array.isArray(item?.receiverThreadIds) + ? item.receiverThreadIds.filter((id): id is string => typeof id === 'string' && id.length > 0) + : [] +} + +/** `started` is the same call's started item, when it was seen: a call keeps naming the helpers + * it started on even when it ends naming fewer, as a timed-out wait ends naming none. */ +export function readCodexCollabAgentToolCall( + item: CodexThreadItem, + started?: CodexThreadItem +): CodexCollabAgentToolCall | null { + const tool = readString(item, 'tool') + if (item.type !== CODEX_COLLAB_AGENT_TOOL_CALL_ITEM_TYPE || tool === null) { + return null + } + const receiverThreadIds = [ + ...new Set([...readReceiverThreadIds(started), ...readReceiverThreadIds(item)]) + ] + const snapshots = readRecord(item.agentsStates) + const states = receiverThreadIds.flatMap((threadId) => { + if (!Object.hasOwn(snapshots, threadId)) { + return [] + } + const snapshot = readRecord(snapshots[threadId]) + return [ + { threadId, status: readString(snapshot, 'status'), message: readString(snapshot, 'message') } + ] + }) + return { + id: item.id, + tool, + status: readString(item, 'status'), + receiverThreadIds, + helperThreadIds: receiverThreadIds.filter( + (threadId) => readString(readRecord(snapshots[threadId]), 'status') !== 'notFound' + ), + prompt: readString(item, 'prompt'), + states + } +} + +/** A helper's row label: the head of the prompt it was spawned with, on one line. */ +export function codexCollabHelperLabel(prompt: string | null): string | null { + const collapsed = prompt === null ? '' : collapsedToolInputPrefix(prompt) + if (collapsed.length <= MAX_HELPER_LABEL_CHARS) { + return collapsed.length > 0 ? collapsed : null + } + const keep = MAX_HELPER_LABEL_CHARS - 1 + // Never end on half a surrogate pair: the label lands in a durable row. + const last = collapsed.charCodeAt(keep - 1) + const end = last >= 0xd800 && last <= 0xdbff ? keep - 1 : keep + return `${collapsed.slice(0, end)}…` +} + +export function codexCollabToolName(call: CodexCollabAgentToolCall): string { + return codexCollabToolRowName(call.tool) +} diff --git a/src/main/codex/codex-collab-call-delegation.test.ts b/src/main/codex/codex-collab-call-delegation.test.ts new file mode 100644 index 000000000000..96131248ddc6 --- /dev/null +++ b/src/main/codex/codex-collab-call-delegation.test.ts @@ -0,0 +1,114 @@ +// A running chat holds a subagent's section open while the parent's newest row delegates to it. +// In Codex's default multi-agent mode that row is the collab call the host writes as a tool row, +// so the rows the real adapter publishes must read as a delegation to the helper they name. + +import { describe, expect, it } from 'vitest' +import { codexCollabRowAgentIds } from '../../shared/codex-collab-agent-tools' +import { projectNativeChatTranscript } from '../../shared/native-chat-transcript-projection' +import type { NativeChatMessage } from '../../shared/native-chat-types' +import { projectStructuredAgentSessionMessages } from '../../shared/structured-agent-session-message-projection' +import { + nativeChatSubagentDelegation, + type NativeChatSubagentDelegation +} from '../../renderer/src/components/native-chat/native-chat-subagent-delegation' +import { + collab, + HELPER, + HELPER_TURN, + PARENT_TURN, + publishedRows, + shell, + spawn, + turn, + waitStarted, + type Frame +} from './codex-collab-call-frames-fixture' +import { THREAD_ID } from './codex-structured-session-adapter-fixture' + +/** The delegation the parent's newest tool run reads as, the row a running chat's frontier judges. */ +async function newestRunDelegation(frames: Frame[]): Promise { + const { conversation } = projectNativeChatTranscript( + projectStructuredAgentSessionMessages(await publishedRows(frames), [], []) + ) + const runs = conversation.filter((message: NativeChatMessage) => + message.blocks.some((block) => block.type === 'tool-call') + ) + const newest = runs.at(-1) + return newest ? nativeChatSubagentDelegation(newest) : null +} + +const spawned = [ + turn('turn/started', THREAD_ID, PARENT_TURN), + spawn('item/started'), + spawn('item/completed'), + turn('turn/started', HELPER, HELPER_TURN), + shell('item/started', HELPER, HELPER_TURN, 'call-helper-shell', 'CHILD_DONE') +] + +describe("a Codex default-mode collab call row as the parent's delegation", () => { + it('delegates to the helper a finished spawn names', async () => { + expect(await newestRunDelegation(spawned)).toEqual({ kind: 'call', agentId: HELPER }) + }) + + it('delegates to the helper a running wait names', async () => { + expect(await newestRunDelegation([...spawned, waitStarted])).toEqual({ + kind: 'call', + agentId: HELPER + }) + }) + + it('delegates to the helper a close names, as a raw collab row did', async () => { + const close = collab('item/started', { + id: 'call-close', + tool: 'closeAgent', + status: 'inProgress', + receiverThreadIds: [HELPER] + }) + expect(await newestRunDelegation([...spawned, close])).toEqual({ + kind: 'call', + agentId: HELPER + }) + }) + + it('reads a spawn still starting, which names no helper, as ordinary output', async () => { + expect( + await newestRunDelegation([ + turn('turn/started', THREAD_ID, PARENT_TURN), + spawn('item/started') + ]) + ).toBeNull() + }) + + it("reads the parent's next call as superseding the delegation", async () => { + expect( + await newestRunDelegation([ + ...spawned, + shell('item/started', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE') + ]) + ).toBeNull() + }) + + it('keeps naming the helper when its prompt is past the journal limit', async () => { + const prompt = `Review lane a. ${'x'.repeat(20_000)}` + const longSpawn = [ + turn('turn/started', THREAD_ID, PARENT_TURN), + collab('item/completed', { + id: 'call-spawn', + tool: 'spawnAgent', + status: 'completed', + receiverThreadIds: [HELPER], + prompt + }) + ] + const row = (await publishedRows(longSpawn)) + .map(({ body }) => body) + .find((body) => body.kind === 'tool-call' && body.name === 'spawn_agent') + const input = row?.kind === 'tool-call' ? row.input : undefined + expect(input).toMatchObject({ + description: expect.stringMatching(/^Review lane a\. x+…$/), + prompt: expect.stringContaining('[Orca: output truncated') + }) + expect(codexCollabRowAgentIds(input)).toEqual([HELPER]) + expect(await newestRunDelegation(longSpawn)).toEqual({ kind: 'call', agentId: HELPER }) + }) +}) diff --git a/src/main/codex/codex-collab-call-frames-fixture.ts b/src/main/codex/codex-collab-call-frames-fixture.ts new file mode 100644 index 000000000000..8926e46dcca9 --- /dev/null +++ b/src/main/codex/codex-collab-call-frames-fixture.ts @@ -0,0 +1,127 @@ +// Default-mode collab frames as a live session sent them, and the journal rows the real adapter +// publishes for them: what a client reads. + +import type { AgentJournalRenderItem } from '../../shared/agent-session-journal-types' +import type { StructuredAgentSessionEventSink } from '../native-chat/agent-session-wire/structured-agent-session-event-sink' +import { fakeCodex, identityFor, THREAD_ID } from './codex-structured-session-adapter-fixture' +import { CodexStructuredSessionAdapter } from './codex-structured-session-adapter' + +// Shapes as a live default-mode session sent them (codex-cli 0.157); only the sender is remapped. +export const HELPER = '01a0ea72-80bc-7632-a97f-4a9a0d50d08f' +export const PARENT_TURN = 'parent-turn-1' +export const HELPER_TURN = 'helper-turn-1' +export const PROMPT = 'Run exactly one foreground shell command: `sleep 150; echo LATE`.' + +export type Frame = { method: string; params: Record } + +export const turn = ( + method: 'turn/started' | 'turn/completed', + threadId: string, + id: string +): Frame => ({ + method, + params: { threadId, turn: { id, status: 'completed' } } +}) +export const item = ( + method: 'item/started' | 'item/completed', + threadId: string, + turnId: string, + fields: Record +): Frame => ({ method, params: { threadId, turnId, item: fields } }) +export const collab = ( + method: 'item/started' | 'item/completed', + fields: Record +): Frame => + item(method, THREAD_ID, PARENT_TURN, { + type: 'collabAgentToolCall', + senderThreadId: THREAD_ID, + prompt: null, + model: null, + reasoningEffort: null, + agentsStates: {}, + ...fields + }) +export const shell = ( + method: 'item/started' | 'item/completed', + threadId: string, + turnId: string, + id: string, + output: string +): Frame => + item(method, threadId, turnId, { + type: 'commandExecution', + id, + command: `/bin/zsh -lc 'echo ${output}'`, + cwd: '/work/repo', + status: method === 'item/started' ? 'inProgress' : 'completed', + ...(method === 'item/completed' ? { exitCode: 0, aggregatedOutput: `${output}\n` } : {}) + }) + +export const spawn = (method: 'item/started' | 'item/completed'): Frame => + collab(method, { + id: 'call-spawn', + tool: 'spawnAgent', + status: method === 'item/started' ? 'inProgress' : 'completed', + receiverThreadIds: method === 'item/started' ? [] : [HELPER], + prompt: PROMPT, + model: 'gpt-5.5', + reasoningEffort: 'medium', + ...(method === 'item/completed' + ? { agentsStates: { [HELPER]: { status: 'pendingInit', message: null } } } + : {}) + }) +export const waitStarted = collab('item/started', { + id: 'call-wait', + tool: 'wait', + status: 'inProgress', + receiverThreadIds: [HELPER] +}) + +export async function publishedRows(frames: Frame[]): Promise { + const codex = fakeCodex() + const adapter = new CodexStructuredSessionAdapter({ + resolveLaunch: async () => ({ + command: 'codex', + args: ['app-server'], + cwd: '/work/repo', + codexHome: null, + resumeThreadId: null + }), + openConnection: codex.openConnection, + readProcessStartTime: async () => 1_700_000_000_000, + now: () => 1_700_000_000_500 + }) + // The latest revision of each row, in first-written order, as a client's journal holds it. + const rows = new Map() + const journal: StructuredAgentSessionEventSink = { + appendItem: (identity, body, options) => { + const itemId = JSON.stringify(identity) + const previous = rows.get(itemId) + const sequence = previous?.sequence ?? rows.size + 1 + const { agentId, parentAgentId, producerKind } = options + rows.set(itemId, { + ...previous, + ...(agentId === undefined ? {} : { agentId }), + ...(parentAgentId === undefined ? {} : { parentAgentId }), + ...(producerKind === undefined ? {} : { producerKind }), + itemId, + sequence, + revision: (previous?.revision ?? 0) + 1, + observedAt: sequence, + body + }) + }, + appendTombstone: () => {}, + publish: () => {} + } + await adapter.acquire({ + identity: identityFor('session-1'), + fence: 7, + spawnToken: 'spawn-9', + events: journal + }) + for (const frame of frames) { + codex.connections[0]!.handlers.onNotification?.(frame.method, frame.params) + } + return [...rows.values()] +} diff --git a/src/main/codex/codex-collab-call-rows-on-positional-clients.test.ts b/src/main/codex/codex-collab-call-rows-on-positional-clients.test.ts new file mode 100644 index 000000000000..795716ed553c --- /dev/null +++ b/src/main/codex/codex-collab-call-rows-on-positional-clients.test.ts @@ -0,0 +1,247 @@ +// A client that predates result call ids (an older mobile app, or an older desktop reading a newer +// host) projects the journal itself and pairs each tool result with the oldest unanswered call. +// So every finished collab call row must carry an output, or each later output in its run is drawn +// under the call before its own. + +import { describe, expect, it } from 'vitest' +import type { + AgentJournalItemBody, + AgentJournalRenderItem +} from '../../shared/agent-session-journal-types' +import { pairToolBlocks } from '../../shared/native-chat-tool-fold' +import { pairNativeChatToolResults } from '../../shared/native-chat-tool-pairing' +import { projectNativeChatTranscript } from '../../shared/native-chat-transcript-projection' +import type { NativeChatBlock, NativeChatMessage } from '../../shared/native-chat-types' +import { projectStructuredAgentSessionMessages } from '../../shared/structured-agent-session-message-projection' +import { + collab, + HELPER, + HELPER_TURN, + item, + PARENT_TURN, + PROMPT, + publishedRows, + shell, + spawn, + turn, + waitStarted, + type Frame +} from './codex-collab-call-frames-fixture' +import { THREAD_ID } from './codex-structured-session-adapter-fixture' + +type PairedOutput = [call: string, output: string | undefined] + +/** What a client without result call ids draws: each call with the output its pairing gives it. */ +function positionalRuns(rows: AgentJournalRenderItem[]): { + mobile: PairedOutput[][] + desktop: PairedOutput[][] +} { + const withoutCallIds = (message: NativeChatMessage): NativeChatMessage => ({ + ...message, + blocks: message.blocks.map((block): NativeChatBlock => { + if (block.type !== 'tool-result') { + return block + } + const { callId: _callId, ...legacy } = block + return legacy + }) + }) + const transcript = projectNativeChatTranscript( + projectStructuredAgentSessionMessages(rows, [], []).map(withoutCallIds) + ) + // The conversation's runs, then each helper's section's. + const runs = [ + ...transcript.conversation, + ...[...transcript.subagentRows.values()].flat().map((row) => row.message) + ].filter((message) => message.blocks.some((block) => block.type === 'tool-call')) + return { + mobile: runs.map((message) => + pairToolBlocks(message.blocks).map((pair): PairedOutput => [ + pair.call?.name ?? '(none)', + pair.result?.output.trim() + ]) + ), + desktop: runs.map((message) => { + const pairing = pairNativeChatToolResults(message.blocks) + return message.blocks.flatMap((block): PairedOutput[] => + block.type === 'tool-call' + ? [[block.name, pairing.resultByCall.get(block)?.output.trim()]] + : [] + ) + }) + } +} + +const outputOf = (rows: AgentJournalRenderItem[], name: string): AgentJournalItemBody | undefined => + rows.map((row) => row.body).find((body) => body.kind === 'tool-call' && body.name === name) + +describe('Codex collab call rows on a client that pairs results by position', () => { + it('draws each output under its own call through spawn, shell, wait and close', async () => { + const rows = await publishedRows([ + turn('turn/started', THREAD_ID, PARENT_TURN), + spawn('item/started'), + spawn('item/completed'), + turn('turn/started', HELPER, HELPER_TURN), + shell('item/started', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE'), + shell('item/completed', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE'), + shell('item/started', HELPER, HELPER_TURN, 'call-helper-shell', 'CHILD_DONE'), + shell('item/completed', HELPER, HELPER_TURN, 'call-helper-shell', 'CHILD_DONE'), + item('item/completed', HELPER, HELPER_TURN, { + type: 'agentMessage', + id: 'msg-child', + text: 'CHILD_REPLY' + }), + turn('turn/completed', HELPER, HELPER_TURN), + waitStarted, + collab('item/completed', { + id: 'call-wait', + tool: 'wait', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'completed', message: 'CHILD_REPLY' } } + }), + collab('item/started', { + id: 'call-close', + tool: 'closeAgent', + status: 'inProgress', + receiverThreadIds: [HELPER] + }), + collab('item/completed', { + id: 'call-close', + tool: 'closeAgent', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'completed', message: 'CHILD_REPLY' } } + }), + shell('item/started', THREAD_ID, PARENT_TURN, 'call-parent-shell-2', 'AFTER_CLOSE'), + shell('item/completed', THREAD_ID, PARENT_TURN, 'call-parent-shell-2', 'AFTER_CLOSE') + ]) + // The parent's calls are one run; the helper's own shell is in the helper's section. + const runs = [ + [ + ['spawn_agent', 'Spawned'], + ['shell', 'PARENT_DONE'], + ['wait_agent', 'CHILD_REPLY'], + ['close_agent', 'Closed'], + ['shell', 'AFTER_CLOSE'] + ], + [['shell', 'CHILD_DONE']] + ] + const { mobile, desktop } = positionalRuns(rows) + expect(mobile).toEqual(runs) + expect(desktop).toEqual(runs) + }) + + it('gives a wait that timed out an output, and keeps naming the helpers it waited on', async () => { + const rows = await publishedRows([ + turn('turn/started', THREAD_ID, PARENT_TURN), + spawn('item/completed'), + turn('turn/started', HELPER, HELPER_TURN), + waitStarted, + // Codex ends a timed-out wait naming no receiver and no state. + collab('item/completed', { + id: 'call-wait', + tool: 'wait', + status: 'completed', + receiverThreadIds: [], + agentsStates: {} + }), + collab('item/started', { + id: 'call-close', + tool: 'closeAgent', + status: 'inProgress', + receiverThreadIds: [HELPER] + }), + collab('item/completed', { + id: 'call-close', + tool: 'closeAgent', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'running', message: null } } + }), + shell('item/started', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE'), + shell('item/completed', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE') + ]) + expect(outputOf(rows, 'wait_agent')).toMatchObject({ + state: 'completed', + input: { description: PROMPT, agents: [HELPER] } + }) + const runs = [ + [ + ['spawn_agent', 'Spawned'], + ['wait_agent', 'Finished waiting'], + // Its snapshot says `running`: the status from before the close. + ['close_agent', 'Closed'], + ['shell', 'PARENT_DONE'] + ] + ] + const { mobile, desktop } = positionalRuns(rows) + expect(mobile).toEqual(runs) + expect(desktop).toEqual(runs) + }) + + it('gives every other finished call what it did or what it reports', async () => { + const OTHER = '01a0ea72-0000-7000-8000-000000000002' + const finished = (id: string, tool: string, fields: Record): Frame[] => [ + collab('item/started', { id, tool, status: 'inProgress', receiverThreadIds: [HELPER] }), + collab('item/completed', { + id, + tool, + status: 'completed', + receiverThreadIds: [HELPER], + ...fields + }) + ] + const rows = await publishedRows([ + turn('turn/started', THREAD_ID, PARENT_TURN), + spawn('item/completed'), + turn('turn/started', HELPER, HELPER_TURN), + // A spawn that created no thread names nothing. + collab('item/completed', { + id: 'call-spawn-failed', + tool: 'spawnAgent', + status: 'failed', + receiverThreadIds: [], + prompt: 'second helper' + }), + ...finished('call-send', 'sendInput', { + prompt: 'keep going', + agentsStates: { [HELPER]: { status: 'running', message: null } } + }), + ...finished('call-resume', 'resumeAgent', { + agentsStates: { [HELPER]: { status: 'completed', message: 'OLD_REPLY' } } + }), + collab('item/started', { + id: 'call-wait', + tool: 'wait', + status: 'inProgress', + receiverThreadIds: [HELPER, OTHER] + }), + // A wait's end names only the helpers that finished; an errored one's message is its error. + collab('item/completed', { + id: 'call-wait', + tool: 'wait', + status: 'failed', + receiverThreadIds: [OTHER], + agentsStates: { [OTHER]: { status: 'errored', message: 'boom' } } + }), + shell('item/started', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE'), + shell('item/completed', THREAD_ID, PARENT_TURN, 'call-parent-shell', 'PARENT_DONE') + ]) + const runs = [ + [ + ['spawn_agent', 'Spawned'], + ['spawn_agent', 'Agent spawn failed'], + ['send_input', 'Sent input'], + ['resume_agent', 'Completed - OLD_REPLY'], + ['wait_agent', `${OTHER}: Error - boom`], + ['shell', 'PARENT_DONE'] + ] + ] + const { mobile, desktop } = positionalRuns(rows) + expect(mobile).toEqual(runs) + expect(desktop).toEqual(runs) + // The wait keeps naming both helpers it waited on. + expect(outputOf(rows, 'wait_agent')).toMatchObject({ input: { agents: [HELPER, OTHER] } }) + }) +}) diff --git a/src/main/codex/codex-default-mode-subagents.test.ts b/src/main/codex/codex-default-mode-subagents.test.ts new file mode 100644 index 000000000000..335f903efbbf --- /dev/null +++ b/src/main/codex/codex-default-mode-subagents.test.ts @@ -0,0 +1,704 @@ +// Codex's default multi-agent mode announces a helper only by the `collabAgentToolCall` items that +// spawn, message, wait on or close it. These frames, through the real adapter, must register that +// helper the same way a `subAgentActivity` does: one child in the strip, the host's records and +// the roster row. + +import { describe, expect, it } from 'vitest' +import type { AgentJournalItemBody } from '../../shared/agent-session-journal-types' +import { createAgentChildWorkAdmission } from '../../shared/agent-status-child-work-admission' +import type { AgentChildWorkRecord } from '../../shared/agent-status-child-work' +import { reconcileAgentChildWorkEvidence } from '../../shared/agent-status-child-work-reconciliation' +import { createAgentStatusStore } from '../../shared/agent-status-store' +import { makeStructuredAgentStatusSubject } from '../../shared/agent-status-subject' +import type { StructuredAgentSessionEventSink } from '../native-chat/agent-session-wire/structured-agent-session-event-sink' +import { fakeCodex, identityFor, THREAD_ID } from './codex-structured-session-adapter-fixture' +import { CodexStructuredSessionAdapter } from './codex-structured-session-adapter' +import { CodexBackgroundTaskTracker } from './codex-background-task-tracker' +import { createCodexJournalTranslator } from './codex-structured-journal-translation' +import { CodexSubagentExecutions } from './codex-subagent-executions' +import { readCodexSubagentAnnouncements } from './codex-subagent-activity' +import type { CodexThreadItem } from './codex-thread-item-identity' + +const parent = makeStructuredAgentStatusSubject( + { + executionHostId: 'local', + wslDistro: null, + workspaceId: 'workspace-1', + workspaceKind: 'folder' + }, + 'session-1' +) + +// The spawn and wait items below are verbatim from a live default-mode session (codex-cli +// 0.155.0-alpha.9.2, `multi_agent` on, `multi_agent_v2` off); only the sender is the fixture's +// thread. That session sent no `subAgentActivity` at all. +const HELPER = '01a0d114-2b8e-73e2-a69b-8e86df067ae9' +const PARENT_TURN = '01a0d112-6f86-7092-8c87-b8618f259efa' +const HELPER_TURN = 'helper-turn-1' +const PROMPT = + 'Run the shell command exactly: `sleep 45; echo CHILD_DONE`. After it completes, reply with the single word `CHILD_REPLY` and nothing else.' +/** The prompt's head, as the helper's row names it. */ +const LABEL = 'Run the shell command exactly: `sleep 45; echo CHILD_DONE`. After it completes,…' +const COMMAND = "/bin/zsh -lc 'sleep 45; echo CHILD_DONE'" + +type Frame = { method: string; params: Record } + +const turn = ( + method: 'turn/started' | 'turn/completed', + threadId: string, + id: string, + status = 'completed' +): Frame => ({ method, params: { threadId, turn: { id, status } } }) +const item = ( + method: 'item/started' | 'item/completed', + threadId: string, + turnId: string, + fields: Record +): Frame => ({ method, params: { threadId, turnId, item: fields } }) +const collab = ( + method: 'item/started' | 'item/completed', + fields: Record +): Frame => + item(method, THREAD_ID, PARENT_TURN, { + type: 'collabAgentToolCall', + senderThreadId: THREAD_ID, + prompt: null, + model: null, + reasoningEffort: null, + agentsStates: {}, + ...fields + }) + +const spawnStarted = collab('item/started', { + id: 'call_dxoSnQY1cHVswlN8MGFJBxHt', + tool: 'spawnAgent', + status: 'inProgress', + receiverThreadIds: [], + prompt: PROMPT, + model: '', + reasoningEffort: 'medium' +}) +const spawnCompleted = collab('item/completed', { + id: 'call_dxoSnQY1cHVswlN8MGFJBxHt', + tool: 'spawnAgent', + status: 'completed', + receiverThreadIds: [HELPER], + prompt: PROMPT, + model: 'gpt-5.5', + reasoningEffort: 'medium', + agentsStates: { [HELPER]: { status: 'pendingInit', message: null } } +}) +const waitStarted = collab('item/started', { + id: 'call_rMn9MIAPavyhOD6ifjH38A8E', + tool: 'wait', + status: 'inProgress', + receiverThreadIds: [HELPER] +}) +const waitCompleted = collab('item/completed', { + id: 'call_rMn9MIAPavyhOD6ifjH38A8E', + tool: 'wait', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'completed', message: 'CHILD_REPLY' } } +}) +const closeCompleted = collab('item/completed', { + id: 'call-close', + tool: 'closeAgent', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'shutdown', message: null } } +}) +const helperShell = (method: 'item/started' | 'item/completed'): Frame => + item(method, HELPER, HELPER_TURN, { + type: 'commandExecution', + id: 'call_3iUPkSEtwkUYstngZm5CNJ52', + command: COMMAND, + cwd: '/work/repo', + // As the live session reported it; a command is tracked the same whatever its source. + source: 'unifiedExecStartup', + status: method === 'item/started' ? 'inProgress' : 'completed', + ...(method === 'item/completed' ? { exitCode: 0, aggregatedOutput: 'CHILD_DONE\n' } : {}) + }) +const helperReply = item('item/completed', HELPER, HELPER_TURN, { + type: 'agentMessage', + id: 'msg-child', + text: 'CHILD_REPLY' +}) +const activityStarted: Frame = item('item/started', THREAD_ID, PARENT_TURN, { + type: 'subAgentActivity', + id: 'activity-helper', + kind: 'started', + agentThreadId: HELPER, + agentPath: '/root/sleeper' +}) + +async function session() { + const codex = fakeCodex() + const store = createAgentStatusStore({ epoch: 'epoch-1', mode: 'authority' }) + expect(store.applyMutation({ parent: { subject: parent } })).not.toBeNull() + let minted = 0 + const admission = createAgentChildWorkAdmission(store, { + mintChildWorkId: () => `child-${++minted}` + }) + const adapter = new CodexStructuredSessionAdapter({ + resolveLaunch: async () => ({ + command: 'codex', + args: ['app-server'], + cwd: '/work/repo', + codexHome: null, + resumeThreadId: null + }), + openConnection: codex.openConnection, + readProcessStartTime: async () => 1_700_000_000_000, + now: () => 1_700_000_000_500, + onChildWorkEvidence: (_sessionId, evidence) => + reconcileAgentChildWorkEvidence({ store, admission, parent, provider: 'codex', evidence }) + }) + // The latest revision of each journal row, in first-written order. + const rows = new Map() + const journal: StructuredAgentSessionEventSink = { + appendItem: (identity, body) => rows.set(JSON.stringify(identity), body), + appendTombstone: () => {}, + publish: () => {} + } + await adapter.acquire({ + identity: identityFor('session-1'), + fence: 7, + spawnToken: 'spawn-9', + events: journal + }) + const send = (...frames: Frame[]): void => { + for (const frame of frames) { + codex.connections[0]!.handlers.onNotification?.(frame.method, frame.params) + } + } + const agents = (): AgentChildWorkRecord[] => + store.getChildren(parent).filter((record) => record.kind === 'agent') + const commands = (): AgentChildWorkRecord[] => + store.getChildren(parent).filter((record) => record.kind === 'command') + const strip = () => adapter.backgroundTaskState('session-1')?.tasks ?? [] + const toolRow = (name: string) => + [...rows.values()].find((body) => body.kind === 'tool-call' && body.name === name) + const rosterRows = () => + [...rows.values()].flatMap((body) => + body.kind === 'message' + ? body.blocks.flatMap((block) => (block.type === 'subagent-group' ? [block] : [])) + : [] + ) + const rawCollabRows = () => + [...rows.values()].filter( + (body) => body.kind === 'status' && body.providerFrame?.kind === 'item:collabAgentToolCall' + ) + return { send, agents, commands, strip, toolRow, rosterRows, rawCollabRows } +} + +describe('Codex default-mode helpers', () => { + it('registers a helper announced only by its spawn call, in the strip, the records and the roster row', async () => { + const run = await session() + run.send(turn('turn/started', THREAD_ID, PARENT_TURN), spawnStarted) + // The call has not said which thread the helper is yet. + expect(run.agents()).toEqual([]) + expect(run.strip()).toEqual([]) + + run.send(spawnCompleted, turn('turn/started', HELPER, HELPER_TURN), waitStarted) + expect(run.agents()).toEqual([ + expect.objectContaining({ + membership: 'live', + state: 'working', + description: LABEL, + invocation: { invocationId: HELPER_TURN, generation: 1 } + }) + ]) + expect(run.strip()).toEqual([ + { id: `codex-agent:${HELPER}`, kind: 'agent', description: LABEL } + ]) + expect(run.rosterRows()).toEqual([ + expect.objectContaining({ + agents: [expect.objectContaining({ id: HELPER, label: LABEL, state: 'working' })] + }) + ]) + + // While the helper runs, its command is the helper's work, not the session's own. + run.send(helperShell('item/started')) + expect(run.strip()).toEqual([ + { id: `codex-agent:${HELPER}`, kind: 'agent', description: LABEL } + ]) + expect(run.commands()).toEqual([ + expect.objectContaining({ + membership: 'live', + description: COMMAND, + parentChildWorkId: run.agents()[0]?.childWorkId + }) + ]) + expect(run.agents()[0]?.operation).toMatchObject({ toolName: 'Bash', input: COMMAND }) + + run.send(helperShell('item/completed')) + // A finished command leaves no record behind. + expect(run.commands()).toEqual([]) + + run.send(helperReply, turn('turn/completed', HELPER, HELPER_TURN)) + expect(run.agents()).toEqual([ + expect.objectContaining({ + membership: 'settled', + outcome: 'succeeded', + lastMessage: 'CHILD_REPLY' + }) + ]) + expect(run.strip()).toEqual([]) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, state: 'completed' }) + ]) + }) + + it('labels a command the helper leaves running with the helper, once its turn is over', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + helperShell('item/started'), + turn('turn/completed', HELPER, HELPER_TURN) + ) + expect(run.strip()).toEqual([ + expect.objectContaining({ kind: 'command', description: `${LABEL} — ${COMMAND}` }) + ]) + }) + + it('renders each collab call as a tool row naming its helper, never as a raw provider row', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnStarted, + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + waitStarted + ) + expect(run.toolRow('wait_agent')).toMatchObject({ + state: 'running', + input: { description: LABEL, agents: [HELPER] } + }) + run.send(turn('turn/completed', HELPER, HELPER_TURN), waitCompleted) + expect(run.toolRow('spawn_agent')).toMatchObject({ + state: 'completed', + input: { description: LABEL, prompt: PROMPT, model: 'gpt-5.5', agents: [HELPER] } + }) + expect(run.toolRow('wait_agent')).toMatchObject({ + state: 'completed', + input: { description: LABEL }, + output: expect.objectContaining({ head: 'CHILD_REPLY' }) + }) + expect(run.rawCollabRows()).toEqual([]) + }) + + it('keeps naming the helper when the turn ends with its wait still open', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + waitStarted, + turn('turn/completed', THREAD_ID, PARENT_TURN, 'interrupted') + ) + expect(run.toolRow('wait_agent')).toMatchObject({ + state: 'failed', + input: { description: LABEL } + }) + }) + + it('registers the helper whichever order its first turn and its spawn arrive in', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + turn('turn/started', HELPER, HELPER_TURN), + spawnCompleted + ) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'live', state: 'working', description: LABEL }) + ]) + expect(run.strip()).toHaveLength(1) + expect(run.rosterRows()).toEqual([ + expect.objectContaining({ + agents: [expect.objectContaining({ id: HELPER, label: LABEL, state: 'working' })] + }) + ]) + // The roster row does not stand in for the call's own row. + expect(run.toolRow('spawn_agent')).toMatchObject({ + state: 'completed', + input: { description: LABEL } + }) + }) + + it('registers a helper whose spawn call failed but created its thread, as Codex reports an errored one', async () => { + const run = await session() + // The shape Codex builds for a helper that errored at birth: the call fails, yet names the + // thread it created, and that thread can still run. + const spawnFailedWithThread = collab('item/completed', { + id: 'call-failed-spawn', + tool: 'spawnAgent', + status: 'failed', + receiverThreadIds: [HELPER], + prompt: PROMPT, + model: 'gpt-5.5', + reasoningEffort: 'medium', + agentsStates: { [HELPER]: { status: 'errored', message: null } } + }) + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnFailedWithThread, + turn('turn/started', HELPER, HELPER_TURN), + helperShell('item/started') + ) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'live', state: 'working', description: LABEL }) + ]) + // Its command is the helper's, not a bare command of the session's own. + expect(run.strip()).toEqual([ + { id: `codex-agent:${HELPER}`, kind: 'agent', description: LABEL } + ]) + expect(run.rosterRows()).toEqual([ + expect.objectContaining({ + agents: [expect.objectContaining({ id: HELPER, state: 'working' })] + }) + ]) + }) + + it('registers nothing for a spawn that created no thread', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + collab('item/completed', { + id: 'call-refused-spawn', + tool: 'spawnAgent', + status: 'failed', + receiverThreadIds: [], + prompt: PROMPT + }) + ) + expect(run.agents()).toEqual([]) + expect(run.strip()).toEqual([]) + expect(run.rosterRows()).toEqual([]) + }) + + it('leaves a helper running when its caller failed to close it', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + collab('item/completed', { + id: 'call-close-failed', + tool: 'closeAgent', + status: 'failed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'notFound', message: null } } + }) + ) + expect(run.agents()).toEqual([expect.objectContaining({ membership: 'live' })]) + expect(run.strip()).toHaveLength(1) + }) + + it("ends a closed helper only on its own interrupted turn, never on the caller's close item", async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + // What Codex reports for a close that failed on a running helper: the status is the + // helper's, not the close's, so `completed` says nothing about whether it stopped. + collab('item/completed', { + id: 'call-close-errored', + tool: 'closeAgent', + status: 'completed', + receiverThreadIds: [HELPER], + agentsStates: { [HELPER]: { status: 'running', message: null } } + }) + ) + expect(run.agents()).toEqual([expect.objectContaining({ membership: 'live' })]) + expect(run.strip()).toHaveLength(1) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, state: 'working' }) + ]) + + // A close that works aborts the helper's turn, which Codex reports on the helper's thread. + run.send(closeCompleted, turn('turn/completed', HELPER, HELPER_TURN, 'interrupted')) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'settled', outcome: 'cancelled' }) + ]) + expect(run.strip()).toEqual([]) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, state: 'stopped' }) + ]) + expect(run.toolRow('close_agent')).toMatchObject({ + state: 'completed', + input: { description: LABEL } + }) + }) + + it("keeps a closed helper's running command until Codex reports the killed process's exit", async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + helperShell('item/started'), + turn('turn/completed', HELPER, HELPER_TURN), + closeCompleted + ) + // The close shuts the helper's thread down and kills its processes; each still reports its + // exit on the helper's thread, and that exit is what ends the command. + expect(run.commands()).toEqual([expect.objectContaining({ membership: 'live' })]) + run.send(helperShell('item/completed')) + expect(run.commands()).toEqual([]) + expect(run.strip()).toEqual([]) + }) + + it('leaves a finished helper finished when its caller closes it', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + turn('turn/completed', HELPER, HELPER_TURN), + closeCompleted + ) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'settled', outcome: 'succeeded' }) + ]) + }) + + it('keeps one child for a session that announces the helper both ways', async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + activityStarted, + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + { ...activityStarted, method: 'item/completed' } + ) + // The activity landed first, so its task name labels the one child. + expect(run.agents()).toEqual([expect.objectContaining({ description: 'sleeper' })]) + expect(run.strip()).toEqual([ + { id: `codex-agent:${HELPER}`, kind: 'agent', description: 'sleeper' } + ]) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, label: 'sleeper' }) + ]) + }) +}) + +describe('a helper whose spawn was never seen', () => { + const sendInput = collab('item/completed', { + id: 'call-send-input', + tool: 'sendInput', + status: 'completed', + receiverThreadIds: [HELPER], + prompt: 'Now run it again.', + agentsStates: { [HELPER]: { status: 'running', message: null } } + }) + + it("registers from any call that names it, so its shell is no longer the session's own", async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + turn('turn/started', HELPER, HELPER_TURN), + helperShell('item/started') + ) + expect(run.strip()).toEqual([expect.objectContaining({ kind: 'command' })]) + + run.send(sendInput) + // No spawn named it, so it reads as any unnamed subagent does. + expect(run.strip()).toEqual([{ id: `codex-agent:${HELPER}`, kind: 'agent' }]) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'live', state: 'working' }) + ]) + expect(run.commands()).toEqual([ + expect.objectContaining({ + membership: 'live', + parentChildWorkId: run.agents()[0]?.childWorkId + }) + ]) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, label: 'subagent', state: 'working' }) + ]) + expect(run.toolRow('send_input')).toMatchObject({ input: { description: HELPER } }) + }) + + it('registers each helper a call on several names', async () => { + const second = 'helper-two' + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + turn('turn/started', HELPER, HELPER_TURN), + turn('turn/started', second, 'helper-two-turn'), + collab('item/started', { + id: 'call-wait-both', + tool: 'wait', + status: 'inProgress', + receiverThreadIds: [HELPER, second] + }) + ) + expect(run.strip()).toEqual([ + { id: `codex-agent:${HELPER}`, kind: 'agent' }, + { id: `codex-agent:${second}`, kind: 'agent' } + ]) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'live' }), + expect.objectContaining({ membership: 'live' }) + ]) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, label: 'subagent' }), + expect.objectContaining({ id: second, label: 'subagent 2' }) + ]) + }) + + it("puts a helper's run in the turn that sent it work, even after that turn ended", async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + spawnCompleted, + turn('turn/started', HELPER, HELPER_TURN), + turn('turn/completed', HELPER, HELPER_TURN), + turn('turn/completed', THREAD_ID, PARENT_TURN), + turn('turn/started', THREAD_ID, 'parent-turn-2'), + { ...sendInput, params: { ...sendInput.params, turnId: 'parent-turn-2' } }, + turn('turn/completed', THREAD_ID, 'parent-turn-2'), + turn('turn/started', HELPER, 'helper-turn-2') + ) + const groups = new Set(run.rosterRows().map((row) => row.groupId)) + expect(groups.size).toBe(2) + expect(run.rosterRows().at(-1)?.agents).toEqual([ + expect.objectContaining({ id: HELPER, state: 'working' }) + ]) + }) + + it('registers every helper a call names, and none it reports not found', () => { + const [second, missing] = ['helper-two', 'helper-missing'] + const waited: CodexThreadItem = { + type: 'collabAgentToolCall', + id: 'call-wait-many', + tool: 'wait', + status: 'completed', + receiverThreadIds: [HELPER, second, missing], + agentsStates: { + [HELPER]: { status: 'completed', message: 'done' }, + [second]: { status: 'running', message: null }, + [missing]: { status: 'notFound', message: null } + } + } + expect(readCodexSubagentAnnouncements(waited, THREAD_ID)).toEqual([ + { agentThreadId: HELPER, label: null, namesParentTurn: false, spawned: false }, + { agentThreadId: second, label: null, namesParentTurn: false, spawned: false } + ]) + }) +}) + +describe('the roster row follows a helper whose turn ends', () => { + const lastRow = (run: Awaited>) => run.rosterRows().at(-1)?.agents + const running = async () => { + const run = await session() + run.send( + turn('turn/started', THREAD_ID, PARENT_TURN), + activityStarted, + turn('turn/started', HELPER, HELPER_TURN) + ) + expect(lastRow(run)).toEqual([expect.objectContaining({ id: HELPER, state: 'working' })]) + return run + } + + it('settles the row with the strip and the record on the failed completion that follows a fatal error', async () => { + const run = await running() + // Codex sends the fatal error, then the helper's failed `turn/completed` 0-32 ms later. + run.send({ + method: 'error', + params: { + threadId: HELPER, + turnId: HELPER_TURN, + willRetry: false, + error: { message: 'Selected model is at capacity.', codexErrorInfo: 'serverOverloaded' } + } + }) + expect(run.strip()).toEqual([expect.objectContaining({ id: `codex-agent:${HELPER}` })]) + expect(run.agents()).toEqual([expect.objectContaining({ membership: 'live' })]) + expect(lastRow(run)).toEqual([expect.objectContaining({ id: HELPER, state: 'working' })]) + + run.send(turn('turn/completed', HELPER, HELPER_TURN, 'failed')) + expect(run.strip()).toEqual([]) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'settled', outcome: 'failed' }) + ]) + expect(lastRow(run)).toEqual([expect.objectContaining({ id: HELPER, state: 'failed' })]) + }) + + it('settles the row with the strip and the record when its thread closes with no turn/completed', async () => { + const run = await running() + run.send({ method: 'thread/closed', params: { threadId: HELPER } }) + expect(run.strip()).toEqual([]) + expect(run.agents()).toEqual([ + expect.objectContaining({ membership: 'settled', outcome: 'unknown' }) + ]) + expect(lastRow(run)).toEqual([expect.objectContaining({ id: HELPER, state: 'unverifiable' })]) + }) +}) + +describe('a restored thread', () => { + function restored(replayed = [spawnCompleted, waitCompleted]) { + const executions = new CodexSubagentExecutions() + const rows = new Map() + const translator = createCodexJournalTranslator({ + sink: { + appendItem: (identity, body) => rows.set(JSON.stringify(identity), body), + appendTombstone: () => {}, + publish: () => {} + }, + primaryThreadId: () => THREAD_ID, + subagentExecutions: executions, + schedule: (run: () => void) => { + run() + return () => {} + } + }) + const tracker = new CodexBackgroundTaskTracker(THREAD_ID, executions) + const admission = translator.restoreThread(THREAD_ID, { + turns: [{ id: PARENT_TURN, items: replayed.map((frame) => frame.params.item) }] + }) + expect(admission).toEqual({ accepted: true }) + const bodies = () => [...rows.values()] + return { executions, tracker, bodies } + } + + it('names the helper on a replayed call row, as the live row did', () => { + const { bodies } = restored() + expect(bodies()).toContainEqual( + expect.objectContaining({ + kind: 'tool-call', + name: 'wait_agent', + input: expect.objectContaining({ description: LABEL }) + }) + ) + }) + + it('claims no running helper for a name it learned from history', () => { + const { executions, tracker, bodies } = restored() + expect(executions.workingChildren()).toEqual([]) + // A live frame republishes the strip and drains evidence: neither holds the helper. + tracker.observe({ + method: 'turn/started', + threadId: THREAD_ID, + params: { threadId: THREAD_ID, turn: { id: 'next-turn', status: 'inProgress' } } + }) + expect(tracker.state).toBeNull() + expect(tracker.drainChildWorkEvidence(1)).toEqual([]) + expect( + bodies().filter( + (body) => + body.kind === 'message' && body.blocks.some((block) => block.type === 'subagent-group') + ) + ).toEqual([]) + }) + + it('knows a helper whose spawn history compacted away, from a call on it', () => { + const { tracker } = restored([waitCompleted]) + tracker.observe({ ...turn('turn/started', HELPER, HELPER_TURN), threadId: HELPER }) + expect(tracker.state?.tasks).toEqual([{ id: `codex-agent:${HELPER}`, kind: 'agent' }]) + }) +}) diff --git a/src/main/codex/codex-item-run-state.ts b/src/main/codex/codex-item-run-state.ts new file mode 100644 index 000000000000..a3b22eff8aed --- /dev/null +++ b/src/main/codex/codex-item-run-state.ts @@ -0,0 +1,16 @@ +import { readString } from './codex-item-field-readers' +import type { CodexThreadItem } from './codex-thread-item-identity' + +/** Codex reports `inProgress` then a terminal status; a zero exit code is the + * only thing that makes a finished command a success. */ +export function codexItemRunState(item: CodexThreadItem): 'running' | 'completed' | 'failed' { + const status = readString(item, 'status') + if (status === null || status === 'inProgress') { + return 'running' + } + if (status !== 'completed') { + return 'failed' + } + const exitCode = item.exitCode + return typeof exitCode === 'number' && exitCode !== 0 ? 'failed' : 'completed' +} diff --git a/src/main/codex/codex-structured-item-translation.ts b/src/main/codex/codex-structured-item-translation.ts index 34f07eefda7e..25b821d90d9f 100644 --- a/src/main/codex/codex-structured-item-translation.ts +++ b/src/main/codex/codex-structured-item-translation.ts @@ -9,6 +9,11 @@ import { import { unhandledProviderFrameJournalItem } from '../native-chat/agent-session-wire/unhandled-provider-frame' import { codexImageItemBody } from './codex-image-item-translation' import { commandActionFacts } from './codex-command-action-class' +import { + codexCollabAgentToolCallBody, + type CodexHelperName +} from './codex-collab-agent-item-translation' +import { codexItemRunState } from './codex-item-run-state' import { readFirstString, readRecord, @@ -66,20 +71,6 @@ export function codexMessageBlocks(item: CodexThreadItem): NativeChatBlock[] { return blocks } -/** Codex reports `inProgress` then a terminal status; a zero exit code is the - * only thing that makes a finished command a success. */ -function commandState(item: CodexThreadItem): 'running' | 'completed' | 'failed' { - const status = readString(item, 'status') - if (status === null || status === 'inProgress') { - return 'running' - } - if (status !== 'completed') { - return 'failed' - } - const exitCode = item.exitCode - return typeof exitCode === 'number' && exitCode !== 0 ? 'failed' : 'completed' -} - export type CodexJournalItem = { body: AgentJournalItemBody | null handled: boolean @@ -103,7 +94,7 @@ function commandItem(item: CodexThreadItem): CodexJournalItem { { command: item.command ?? null, cwd: item.cwd ?? null, ...parsed?.fields }, DEFAULT_JOURNAL_PAYLOAD_LIMITS ), - state: commandState(item), + state: codexItemRunState(item), ...toolExecutionMetadata(item), ...(bounded === null ? {} : { output: bounded.bounded }) }, @@ -127,7 +118,7 @@ function fileChangeItem(item: CodexThreadItem): CodexJournalItem { name: 'apply_patch', callId: item.id, input: boundToolInput({ changes: item.changes ?? null }, DEFAULT_JOURNAL_PAYLOAD_LIMITS), - state: commandState(item) + state: codexItemRunState(item) }, handled: true } @@ -180,7 +171,7 @@ function mcpToolCallItem(item: CodexThreadItem): CodexJournalItem { callId: item.id, ...(server && tool ? { mcpIdentity: { server, tool } } : {}), input: boundToolInput(mcpToolArguments(item.arguments), DEFAULT_JOURNAL_PAYLOAD_LIMITS), - state: failure === null ? commandState(item) : 'failed', + state: failure === null ? codexItemRunState(item) : 'failed', ...(bounded === null ? {} : { output: bounded.bounded }) }, handled: true @@ -234,9 +225,14 @@ function webSearchItem(item: CodexThreadItem): CodexJournalItem { * Journal body for a Codex item, or null for one with nothing to render. * * Known empty items wait for later deltas. Unknown types become bounded status - * rows so a provider release cannot make new activity invisible. + * rows so a provider release cannot make new activity invisible. `started` is a + * finished item's own started revision, when the caller still holds it. */ -export function codexJournalItem(item: CodexThreadItem): CodexJournalItem { +export function codexJournalItem( + item: CodexThreadItem, + helperName?: CodexHelperName, + started?: CodexThreadItem +): CodexJournalItem { if (item.type === 'userMessage' || item.type === 'agentMessage') { const blocks = codexMessageBlocks(item) return { @@ -262,6 +258,10 @@ export function codexJournalItem(item: CodexThreadItem): CodexJournalItem { if (item.type === 'imageView' || item.type === 'imageGeneration') { return { body: codexImageItemBody(item), handled: true } } + const collab = codexCollabAgentToolCallBody(item, helperName, started) + if (collab) { + return { body: collab, handled: true } + } if (item.type === 'plan') { const text = readTextContent(item, 'text') return { diff --git a/src/main/codex/codex-structured-journal-items.ts b/src/main/codex/codex-structured-journal-items.ts index 4e6ebe8e5e08..ec6ac67df5d4 100644 --- a/src/main/codex/codex-structured-journal-items.ts +++ b/src/main/codex/codex-structured-journal-items.ts @@ -12,6 +12,7 @@ import { type CodexThreadItem } from './codex-structured-item-translation' import { createCodexStructuredItemStreams } from './codex-structured-item-streams' +import type { CodexHelperName } from './codex-collab-agent-item-translation' import { boundStreamItem, codexStructuredItemKey } from './codex-structured-item-stream-bounds' import { codexCommandOutlivesTurn } from './codex-command-lifecycle' import type { @@ -46,7 +47,8 @@ export class CodexJournalItems { 'sink' | 'coalesceMs' | 'maxRetainedBytes' | 'schedule' > & { maxMetadataBytes?: number; attributionFor: CodexRowAttribution }, private readonly activeTurn: (threadId: string) => string | null, - private readonly suppress: (threadId: string, turnId: string) => void + private readonly suppress: (threadId: string, turnId: string) => void, + private readonly helperName?: CodexHelperName ) { this.streams = createCodexStructuredItemStreams({ sink: deps.sink, @@ -96,15 +98,17 @@ export class CodexJournalItems { ) { return { handled: true, admission: { accepted: false, reason: 'failed' } } } - const translated = codexJournalItem(item) + const itemKey = codexStructuredItemKey(event.threadId, item.id) + const started = + event.method === 'item/completed' ? this.activeItems.get(itemKey)?.item : undefined + const translated = codexJournalItem(item, this.helperName, started) const command = readCodexJournalString(item, 'command') if (command) { const boundedCommand = Buffer.from(command, 'utf8') .subarray(0, MAX_CODEX_DETAIL_BYTES) .toString('utf8') - this.details.set(codexStructuredItemKey(event.threadId, item.id), boundedCommand) + this.details.set(itemKey, boundedCommand) } - const itemKey = codexStructuredItemKey(event.threadId, item.id) if (!translated.body) { if (event.method === 'item/completed') { this.streams.forget(event.threadId, item.id) @@ -193,7 +197,8 @@ export class CodexJournalItems { threadId, turnId, identity, - item: retainedItem + item: retainedItem, + ...(this.helperName ? { helperName: this.helperName } : {}) }) } @@ -234,7 +239,7 @@ export class CodexJournalItems { } const evicted = this.activeItems.get(oldest) if (evicted) { - const translated = codexJournalItem(evicted.item).body + const translated = codexJournalItem(evicted.item, this.helperName).body if (translated) { const admission = appendCodexLifecycleItem( this.deps.sink, diff --git a/src/main/codex/codex-structured-journal-settlement.ts b/src/main/codex/codex-structured-journal-settlement.ts index a3f4c7f5599c..131de9824aae 100644 --- a/src/main/codex/codex-structured-journal-settlement.ts +++ b/src/main/codex/codex-structured-journal-settlement.ts @@ -20,6 +20,7 @@ import { type CodexTurnOrdinals } from './codex-structured-item-translation' import type { CodexStructuredItemStreams } from './codex-structured-item-streams' +import type { CodexHelperName } from './codex-collab-agent-item-translation' import type { CodexStructuredSessionEvent } from './codex-structured-session-adapter' import { codexCommandOutlivesTurn } from './codex-command-lifecycle' import { @@ -34,6 +35,8 @@ export type CodexActiveJournalItem = { turnId: string | null identity: AgentJournalItemIdentity item: CodexThreadItem + /** Names the helpers a collab call acted on, so a settled revision keeps naming them. */ + helperName?: CodexHelperName } export type CodexPendingJournalPrompt = { @@ -66,7 +69,7 @@ export function settleCodexJournalSession(input: { const streamed = input.streams.snapshot(active.threadId, active.item.id) const translated = streamed ? codexStreamingJournalItem(active.item, streamed.text) - : codexJournalItem(active.item) + : codexJournalItem(active.item, active.helperName) const body = interruptedBody(translated.body) if (body) { mutations.push(settledRow(input.attributionFor, active, body)) @@ -138,7 +141,7 @@ export function settleCodexJournalTurn(input: { const streamed = input.streams.snapshot(active.threadId, active.item.id) const translated = streamed ? codexStreamingJournalItem(active.item, streamed.text) - : codexJournalItem(active.item) + : codexJournalItem(active.item, active.helperName) const body = interruptedBody(translated.body) if (body) { mutations.push(settledRow(input.attributionFor, active, body)) diff --git a/src/main/codex/codex-structured-journal-translation-restore.ts b/src/main/codex/codex-structured-journal-translation-restore.ts index 5dc68d6f764d..9f7efc15ae4c 100644 --- a/src/main/codex/codex-structured-journal-translation-restore.ts +++ b/src/main/codex/codex-structured-journal-translation-restore.ts @@ -1,5 +1,12 @@ import type { AgentJournalTurnLifecycle } from '../../shared/agent-session-journal-types' -import type { CodexTurnOrdinals } from './codex-structured-item-translation' +import { readCodexThreadItem, type CodexTurnOrdinals } from './codex-structured-item-translation' +import type { CodexJournalCompactions } from './codex-structured-journal-compactions' +import type { CodexJournalItems } from './codex-structured-journal-items' +import { + readCodexNotificationThreadItem, + readCodexSubagentAnnouncements +} from './codex-subagent-activity' +import type { CodexSubagentExecutions } from './codex-subagent-executions' import { readCodexJournalRecord, readCodexJournalString @@ -12,6 +19,33 @@ import { } from './codex-structured-journal-translation-turns' import { readCodexTurnDurationMs, readCodexTurnStatus } from './codex-structured-thread-facts' +type CodexHistoryItemEvent = { threadId: string; method: string; params: unknown } + +/** One replayed item. History never runs the live item router, so a helper the replay announces + * is registered here for its name and membership only: `register` starts no execution, so no + * strip entry, record or roster row claims it runs until a live turn of its own says so. */ +export function restoreCodexHistoryItem( + event: CodexHistoryItemEvent, + input: { + primaryThreadId: string | null + compactions: Pick + items: Pick + executions: Pick + } +): CodexJournalTranslationAdmission { + const item = readCodexNotificationThreadItem(event.params, readCodexThreadItem) + const announcements = item ? readCodexSubagentAnnouncements(item, input.primaryThreadId) : [] + for (const announcement of announcements) { + input.executions.register(announcement.agentThreadId, announcement.label, undefined) + } + const compaction = input.compactions.handle(event) + if (compaction) { + return compaction + } + const translated = input.items.handle(event, 'history') + return translated.handled ? translated.admission : { accepted: false, reason: 'untranslated' } +} + /** Old providers may return the complete thread from resume. Keep that fallback * bounded before admitting any rows to the asynchronous sink. */ export const CODEX_RESTORE_MAX_OPERATIONS = 1_024 @@ -22,11 +56,7 @@ export function restoreCodexJournalThread(input: { thread: Record currentTurnIds: Map> ordinals: CodexTurnOrdinals - handleItem: (event: { - threadId: string - method: string - params: unknown - }) => CodexJournalTranslationAdmission + handleItem: (event: CodexHistoryItemEvent) => CodexJournalTranslationAdmission /** Absent when the caller has no session identity to key lifecycle rows by. */ restoreTurnLifecycle?: ( turnLifecycle: AgentJournalTurnLifecycle diff --git a/src/main/codex/codex-structured-journal-translation-writers.ts b/src/main/codex/codex-structured-journal-translation-writers.ts index b13a7212142d..9e577074c5e4 100644 --- a/src/main/codex/codex-structured-journal-translation-writers.ts +++ b/src/main/codex/codex-structured-journal-translation-writers.ts @@ -39,8 +39,12 @@ export function createCodexJournalTranslatorWriters(deps: CodexJournalTranslator }) const producerDeps = { ...deps, attributionFor } const genericFrames = new CodexJournalGenericFrames(producerDeps, activeTurn) - const items = new CodexJournalItems(producerDeps, activeTurn, (threadId, turnId) => - genericFrames.suppress(threadId, turnId) + const items = new CodexJournalItems( + producerDeps, + activeTurn, + (threadId, turnId) => genericFrames.suppress(threadId, turnId), + // A collab call's row names its helpers the way the roster does. + (threadId) => subagents.executions.label(threadId) ) return { activeTurns, diff --git a/src/main/codex/codex-structured-journal-translation.ts b/src/main/codex/codex-structured-journal-translation.ts index 8e05506509a3..abc094bc3064 100644 --- a/src/main/codex/codex-structured-journal-translation.ts +++ b/src/main/codex/codex-structured-journal-translation.ts @@ -7,7 +7,10 @@ import { type CodexJournalTranslatorDeps } from './codex-structured-journal-contracts' import { settleCodexJournalSession } from './codex-structured-journal-settlement' -import { restoreCodexJournalThread } from './codex-structured-journal-translation-restore' +import { + restoreCodexHistoryItem, + restoreCodexJournalThread +} from './codex-structured-journal-translation-restore' import { CodexJournalTurnBoundaries } from './codex-structured-journal-translation-turn-boundaries' import { createCodexJournalTranslatorWriters } from './codex-structured-journal-translation-writers' import { publishCodexTurnLifecycle } from './codex-structured-journal-translation-turns' @@ -113,16 +116,13 @@ export function createCodexJournalTranslator( thread, currentTurnIds: activeTurns.byThread, ordinals: items.ordinals, - handleItem: (event) => { - const compaction = compactions.handle(event) - if (compaction) { - return compaction - } - const translated = items.handle(event, 'history') - return translated.handled - ? translated.admission - : { accepted: false, reason: 'untranslated' } - }, + handleItem: (event) => + restoreCodexHistoryItem(event, { + primaryThreadId: deps.primaryThreadId?.() ?? null, + compactions, + items, + executions: subagents.executions + }), ...(deps.sessionId !== undefined ? { restoreTurnLifecycle: (turnLifecycle) => diff --git a/src/main/codex/codex-subagent-activity.ts b/src/main/codex/codex-subagent-activity.ts index 9761f8634f8c..0a00297dff8a 100644 --- a/src/main/codex/codex-subagent-activity.ts +++ b/src/main/codex/codex-subagent-activity.ts @@ -6,12 +6,19 @@ // * `agentPath` is a tree path (`/root`, `/root/list_directory`); the trailing // segment is a semantic task name and the only label available. There is no // `thread/started` for a child, so nickname/role/depth do not exist. -// * `agentsStates` on `collabAgentToolCall` arrived empty (`{}`) throughout the -// probe, so nothing here reads it; child turn events own execution state. +// * Codex's DEFAULT multi-agent mode sends no `subAgentActivity` at all; a +// helper appears only in the `collabAgentToolCall` items that spawn, message, +// wait on or close it (read in `codex-collab-agent-tool-call.ts`). Either item +// announces the same child, keyed by its thread id, and child turn events own +// its execution state. // * `thread/tokenUsage/updated` reports a per-thread RUNNING TOTAL, so the // latest frame replaces the previous one — it is never accumulated. -import type { CodexThreadItem } from './codex-structured-item-translation' +import { + codexCollabHelperLabel, + readCodexCollabAgentToolCall +} from './codex-collab-agent-tool-call' +import type { CodexThreadItem } from './codex-thread-item-identity' export const CODEX_SUBAGENT_ITEM_TYPE = 'subAgentActivity' export const CODEX_TOKEN_USAGE_METHOD = 'thread/tokenUsage/updated' @@ -85,6 +92,66 @@ export function codexSubagentLabel(activity: CodexSubagentActivity): string | nu return trailing !== undefined && trailing.length > 0 ? trailing : null } +/** A child the item says exists, from either wire shape Codex announces one with. */ +export type CodexSubagentAnnouncement = { + agentThreadId: string + label: string | null + /** The item's turn is the one the child was spawned or messaged from. */ + namesParentTurn: boolean + /** The item is the spawn itself, so the thread that carried it spawned the child. */ + spawned: boolean +} + +/** Collab calls that hand a helper work from the caller's turn, as an `interacted` activity does. */ +const CODEX_COLLAB_TOOLS_THAT_TASK_A_HELPER: ReadonlySet = new Set([ + 'spawnAgent', + 'sendInput' +]) + +/** The children an item announces: a `subAgentActivity`'s, or every helper a collab call names, + * keyed by thread id so repeats announce one child. Neither the tree root nor the session's own + * thread is a child. Any call counts, since a helper whose spawn was never seen (compacted out of + * history) is known only from the calls on it. A spawn counts whatever its status: Codex reports + * `failed` for a helper that errored at birth, yet names the thread it created, and that thread + * can still run. */ +export function readCodexSubagentAnnouncements( + item: CodexThreadItem, + primaryThreadId: string | null +): CodexSubagentAnnouncement[] { + return readAnnouncements(item).filter(({ agentThreadId }) => agentThreadId !== primaryThreadId) +} + +function readAnnouncements(item: CodexThreadItem): CodexSubagentAnnouncement[] { + const activity = readCodexSubagentActivity(item) + if (activity) { + return isCodexRootAgentActivity(activity) + ? [] + : [ + { + agentThreadId: activity.agentThreadId, + label: codexSubagentLabel(activity), + namesParentTurn: activity.kind === 'started' || activity.kind === 'interacted', + spawned: activity.kind === 'started' + } + ] + } + const call = readCodexCollabAgentToolCall(item) + if (!call) { + return [] + } + const spawned = call.tool === 'spawnAgent' + // Only a spawn's prompt describes its helper; another call's prompt is a later message to it, + // so a helper no spawn named reads as any unnamed subagent does. + const label = spawned ? codexCollabHelperLabel(call.prompt) : null + const namesParentTurn = CODEX_COLLAB_TOOLS_THAT_TASK_A_HELPER.has(call.tool) + return call.helperThreadIds.map((agentThreadId) => ({ + agentThreadId, + label, + namesParentTurn, + spawned + })) +} + export type CodexThreadTokenTotal = { threadId: string; totalTokens: number } /** `{threadId, tokenUsage: {total: {totalTokens}}}`. Older builds put the total diff --git a/src/main/codex/codex-subagent-executions.ts b/src/main/codex/codex-subagent-executions.ts index 87d9327c70c0..0d0fe0c5bf4e 100644 --- a/src/main/codex/codex-subagent-executions.ts +++ b/src/main/codex/codex-subagent-executions.ts @@ -25,10 +25,20 @@ export type CodexExecutionChild = { turnCount: number } +/** Told when a child's current execution changes, whichever frame changed it. */ +export type CodexExecutionListener = (child: Readonly) => void + /** Child turn events own execution; activity items only identify the child. */ export class CodexSubagentExecutions { private readonly children = new Map() private readonly settledTurns = new Map() + private readonly listeners = new Set() + + /** Every reader of a child's execution follows it here, so none needs the frames that end it. */ + onExecutionChanged(listener: CodexExecutionListener): () => void { + this.listeners.add(listener) + return () => this.listeners.delete(listener) + } register( agentThreadId: string, @@ -90,6 +100,7 @@ export class CodexSubagentExecutions { } if (state === 'working' || !child.execution || child.execution.turnId === turnId) { child.execution = execution + this.changed(child) } return { child, execution } } @@ -133,6 +144,7 @@ export class CodexSubagentExecutions { for (const child of this.children.values()) { if (child.execution?.state === 'working') { child.execution = { ...child.execution, state: 'unverifiable' } + this.changed(child) } } } @@ -147,6 +159,12 @@ export class CodexSubagentExecutions { return { children: this.children.size, settledTurns: this.settledTurns.size } } + private changed(child: CodexExecutionChild): void { + for (const listener of this.listeners) { + listener(child) + } + } + private child(agentThreadId: string): CodexExecutionChild | undefined { const existing = this.children.get(agentThreadId) if (existing) { diff --git a/src/main/codex/codex-subagent-roster.ts b/src/main/codex/codex-subagent-roster.ts index db02ac7f4547..042183a6e6f8 100644 --- a/src/main/codex/codex-subagent-roster.ts +++ b/src/main/codex/codex-subagent-roster.ts @@ -1,6 +1,8 @@ // The Codex subagent roster: one journal row per spawn group, revised in place. // -// Activity supplies membership; child turn events supply execution state. +// An announcement supplies membership — a `subAgentActivity` item, or in +// Codex's default multi-agent mode any collab call naming the helper — and +// child turn events supply execution state. // // KNOWN LIMITATION: `groups` is process-local and is never seeded from the // journal, while the row's identity is keyed on the group id alone. So once a @@ -21,9 +23,9 @@ import type { StructuredAgentSessionSinkAdmission } from '../native-chat/agent-session-wire/structured-agent-session-event-sink' import { - codexSubagentLabel, - isCodexRootAgentActivity, readCodexSubagentActivity, + readCodexSubagentAnnouncements, + type CodexSubagentAnnouncement, readCodexThreadTokenTotal } from './codex-subagent-activity' import { @@ -46,9 +48,9 @@ import { import type { CodexThreadItem } from './codex-structured-item-translation' import { MAX_CODEX_SUBAGENT_GROUPS, - MAX_CODEX_SUBAGENTS_PER_GROUP, - MAX_CODEX_TOKEN_USAGE_THREADS + MAX_CODEX_SUBAGENTS_PER_GROUP } from './codex-structured-journal-limits' +import { CodexThreadTokenTotals } from './codex-thread-token-totals' const ADMITTED: StructuredAgentSessionSinkAdmission = { accepted: true } @@ -64,48 +66,54 @@ export type CodexSubagentRosterDeps = { export class CodexSubagentRoster { private readonly groups = new Map() - /** Latest reported total per thread, kept regardless of roster membership: a - * usage frame can arrive before the child's first activity item, and filtering - * at receipt would lose it permanently. Children are selected at write time; - * the map itself is LRU-capped in `handleTokenUsage`. */ - private readonly tokensByThread = new Map() + /** Every thread's total, members or not; children are selected at write time. */ + private readonly tokensByThread = new CodexThreadTokenTotals() private readonly now: () => number - private readonly executions: CodexSubagentExecutions + /** The one owner of child membership and turn state; the rows of calls on a helper read it too. */ + readonly executions: CodexSubagentExecutions + private readonly unfollow: () => void /** Who produced a row, from what this roster learned about each child thread. */ readonly linkage: CodexSubagentLinkage constructor(private readonly deps: CodexSubagentRosterDeps) { this.now = deps.now ?? (() => Date.now()) this.executions = deps.executions ?? new CodexSubagentExecutions() + // The row follows the executions, so every frame that ends a child's turn — its own + // `turn/completed` (a failed one included) or its thread closing — settles it. + // A refused write clears `lastSerialized`, so the next write of the group retries it. + this.unfollow = this.executions.onExecutionChanged( + (child) => child.execution && this.follow(child, child.execution) + ) this.linkage = new CodexSubagentLinkage({ primaryThreadId: deps.primaryThreadId, executions: this.executions }) } - /** Consume a `subAgentActivity` item. Returns null when the item is not one. */ + /** Consume an item that announces children. Null means the item is not this roster's to render: + * a `subAgentActivity` item renders as the roster row alone, while a collab call keeps its own + * row, so it is claimed only to hand back a refused write. */ handleItem(input: { threadId: string turnId: string | null item: CodexThreadItem }): StructuredAgentSessionSinkAdmission | null { - const activity = readCodexSubagentActivity(input.item) - if (!activity) { - return null - } - // The root node is the parent turn itself, not a child it spawned. - if ( - activity.agentThreadId === this.deps.primaryThreadId() || - isCodexRootAgentActivity(activity) - ) { - return ADMITTED - } + const refused = readCodexSubagentAnnouncements(input.item, this.deps.primaryThreadId()) + .map((announcement) => this.announce(input, announcement)) + .find((admission) => !admission.accepted) + return refused ?? (readCodexSubagentActivity(input.item) !== null ? ADMITTED : null) + } + + private announce( + input: { threadId: string; turnId: string | null }, + announcement: CodexSubagentAnnouncement + ): StructuredAgentSessionSinkAdmission { const child = this.executions.register( - activity.agentThreadId, - codexSubagentLabel(activity), - activity.kind === 'started' || activity.kind === 'interacted' ? input.turnId : undefined, - // Only `started` names the spawner: other kinds ride whichever agent acted. - activity.kind === 'started' ? input.threadId : undefined + announcement.agentThreadId, + announcement.label, + announcement.namesParentTurn ? input.turnId : undefined, + // Only a spawn names the spawner: other announcements ride whichever agent acted. + announcement.spawned ? input.threadId : undefined ) if (!child?.execution) { return ADMITTED @@ -146,12 +154,19 @@ export class CodexSubagentRoster { return ADMITTED } const observed = this.executions.observeTurn(input.threadId, input.turnId, input.state) - if (!observed || !observed.child.registered) { + // Followed already if the execution changed; re-derived (idempotently) for its admission. + return observed ? this.follow(observed.child, observed.execution) : ADMITTED + } + + private follow( + child: Readonly, + execution: CodexChildExecution + ): StructuredAgentSessionSinkAdmission { + if (!child.registered) { return ADMITTED } - const { child, execution } = observed - if (input.state === 'working') { - const parent = this.deps.primaryThreadId() ?? input.threadId + if (execution.state === 'working') { + const parent = this.deps.primaryThreadId() ?? child.agentThreadId const group = this.executionGroup(child.agentThreadId, execution.turnId) ?? this.groupFor(parent, this.deps.activeTurn(parent) ?? child.parentTurnId) @@ -159,7 +174,7 @@ export class CodexSubagentRoster { return this.write(group) } for (const group of this.groups.values()) { - if (group.executionTurns.get(input.threadId) !== input.turnId) { + if (group.executionTurns.get(child.agentThreadId) !== execution.turnId) { continue } this.recordExecution(group, child, execution) @@ -177,19 +192,7 @@ export class CodexSubagentRoster { if (!usage) { return null } - // A running total: the newest frame REPLACES the previous one. Summing - // updates would multiply a single child's usage by its frame count. - // Re-insert so the eviction scan below sees recency: `set` on an existing - // key keeps its original position, which would age out an active thread. - this.tokensByThread.delete(usage.threadId) - this.tokensByThread.set(usage.threadId, usage.totalTokens) - while (this.tokensByThread.size > MAX_CODEX_TOKEN_USAGE_THREADS) { - const oldest = this.tokensByThread.keys().next().value - if (typeof oldest !== 'string') { - break - } - this.tokensByThread.delete(oldest) - } + this.tokensByThread.record(usage.threadId, usage.totalTokens) for (const group of this.groups.values()) { if (!group.entries.has(usage.threadId)) { continue @@ -222,6 +225,7 @@ export class CodexSubagentRoster { } dispose(): void { + this.unfollow() this.groups.clear() this.tokensByThread.clear() } diff --git a/src/main/codex/codex-thread-token-totals.test.ts b/src/main/codex/codex-thread-token-totals.test.ts new file mode 100644 index 000000000000..1dab7dc6f6e7 --- /dev/null +++ b/src/main/codex/codex-thread-token-totals.test.ts @@ -0,0 +1,24 @@ +import { describe, expect, it } from 'vitest' +import { MAX_CODEX_TOKEN_USAGE_THREADS } from './codex-structured-journal-limits' +import { CodexThreadTokenTotals } from './codex-thread-token-totals' + +describe('CodexThreadTokenTotals', () => { + it("replaces a thread's total with its newest report instead of summing reports", () => { + const totals = new CodexThreadTokenTotals() + totals.record('child-1', 100) + totals.record('child-1', 250) + expect(totals.get('child-1')).toBe(250) + }) + + it('evicts by recency of report, so a thread still reporting outlives quieter ones', () => { + const totals = new CodexThreadTokenTotals() + totals.record('child-1', 1) + for (let index = 0; index < MAX_CODEX_TOKEN_USAGE_THREADS - 1; index++) { + totals.record(`other-${index}`, index) + } + totals.record('child-1', 2) + totals.record('newest', 3) + expect(totals.get('child-1')).toBe(2) + expect(totals.get('other-0')).toBeUndefined() + }) +}) diff --git a/src/main/codex/codex-thread-token-totals.ts b/src/main/codex/codex-thread-token-totals.ts new file mode 100644 index 000000000000..23bc6a9d1995 --- /dev/null +++ b/src/main/codex/codex-thread-token-totals.ts @@ -0,0 +1,32 @@ +import { MAX_CODEX_TOKEN_USAGE_THREADS } from './codex-structured-journal-limits' + +/** Latest reported token total per thread, kept regardless of roster membership: a + * usage frame can arrive before the child's first activity item, and filtering at + * receipt would lose it permanently. LRU-capped at MAX_CODEX_TOKEN_USAGE_THREADS. */ +export class CodexThreadTokenTotals { + private readonly totals = new Map() + + /** A running total: the newest frame REPLACES the previous one. Summing updates + * would multiply a single child's usage by its frame count. */ + record(threadId: string, totalTokens: number): void { + // Re-insert so eviction sees recency: `set` on an existing key keeps its + // original position, which would age out an active thread. + this.totals.delete(threadId) + this.totals.set(threadId, totalTokens) + while (this.totals.size > MAX_CODEX_TOKEN_USAGE_THREADS) { + const oldest = this.totals.keys().next().value + if (typeof oldest !== 'string') { + break + } + this.totals.delete(oldest) + } + } + + get(threadId: string): number | undefined { + return this.totals.get(threadId) + } + + clear(): void { + this.totals.clear() + } +} diff --git a/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts b/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts index 22b3b26a453c..b4d2286145b0 100644 --- a/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts +++ b/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts @@ -238,10 +238,10 @@ const CODEX_ITEM_CLASSIFICATIONS: Record = // `restoreThread` replays them straight through `items.handle`, which is where // the classification earns its keep. // - // `collabAgentToolCall` is deliberately NOT suppressed with it. Nothing - // guarantees a session reports subagent work as `subAgentActivity` at all; one - // that only ever emits the collab tool call gets no roster row, and suppressing - // that too would leave its fan-out showing nothing. + // `collabAgentToolCall` is deliberately NOT suppressed with it. Codex's default + // multi-agent mode reports subagent work ONLY as that call, which renders as its + // own tool row and never reaches this catalog; were it ever to, suppressing it + // would leave that session's fan-out showing nothing. [CODEX_SUBAGENT_ITEM_TYPE]: 'status-chrome', // `{id, durationMs}` and nothing else — Codex's own transcript renders it as // nothing at all. Every other item type this build does not model carries text diff --git a/src/renderer/src/components/native-chat/native-chat-subagent-delegation.ts b/src/renderer/src/components/native-chat/native-chat-subagent-delegation.ts index 6d3012c271a1..392e858aaa85 100644 --- a/src/renderer/src/components/native-chat/native-chat-subagent-delegation.ts +++ b/src/renderer/src/components/native-chat/native-chat-subagent-delegation.ts @@ -3,10 +3,15 @@ // A roster row names the agents it announces, in the order they were added. A Codex collab // call (spawn, wait, resume, message, close) names the agents it acts on by thread id, which // is the agent id, and delegates to the first; one naming none (a spawn still starting, a -// wait on any agent) is ordinary output. A Claude spawn call names no agent — only its call id, which is re-minted on resume +// wait on any agent) is ordinary output. The host writes it as a tool call; journals from +// before that hold it as a raw status row. A Claude spawn call names no agent — only its call id, which is re-minted on resume // and never a join key — so it belongs to the roster that announces its agent. A drawn row is // a spawn when its newest part is one. +import { + codexCollabRowAgentIds, + isCodexCollabToolRowName +} from '../../../../shared/codex-collab-agent-tools' import { isSubagentGroupBlock, type NativeChatMessage } from '../../../../shared/native-chat-types' export type NativeChatSubagentDelegation = @@ -49,9 +54,14 @@ function derive(message: NativeChatMessage): NativeChatSubagentDelegation | null } // A tool run folds into the message before it, so its last call is the newest thing in it. const newest = message.blocks.findLast((block) => block.type !== 'tool-result') - return newest?.type === 'tool-call' && CLAUDE_SPAWN_TOOLS.has(newest.name) - ? { kind: 'spawn' } - : null + if (newest?.type !== 'tool-call') { + return null + } + if (isCodexCollabToolRowName(newest.name)) { + const [agentId] = codexCollabRowAgentIds(newest.input) + return agentId === undefined ? null : { kind: 'call', agentId } + } + return CLAUDE_SPAWN_TOOLS.has(newest.name) ? { kind: 'spawn' } : null } function receiverThreadIds(head: string): readonly string[] { diff --git a/src/shared/codex-collab-agent-tools.ts b/src/shared/codex-collab-agent-tools.ts new file mode 100644 index 000000000000..a19de5aff2a2 --- /dev/null +++ b/src/shared/codex-collab-agent-tools.ts @@ -0,0 +1,41 @@ +// The tool rows a Codex `collabAgentToolCall` item becomes: the calls an agent makes to spawn, +// message, wait on and close its helpers. The host names each row; readers recognise it here. + +/** The name the model called the tool by, which is what its row shows. The wire item spells the + * tool in camelCase; the model-facing name is snake_case, so this map is not a spelling fix. */ +const CODEX_COLLAB_TOOL_NAMES: ReadonlyMap = new Map([ + ['spawnAgent', 'spawn_agent'], + ['sendInput', 'send_input'], + ['resumeAgent', 'resume_agent'], + ['wait', 'wait_agent'], + ['closeAgent', 'close_agent'], + ['sendMessage', 'send_message'], + ['followupTask', 'followup_task'], + ['interruptAgent', 'interrupt_agent'], + ['listAgents', 'list_agents'] +]) + +const CODEX_COLLAB_ROW_NAMES: ReadonlySet = new Set(CODEX_COLLAB_TOOL_NAMES.values()) + +/** A tool Codex adds later keeps its wire spelling. */ +export function codexCollabToolRowName(wireTool: string): string { + return CODEX_COLLAB_TOOL_NAMES.get(wireTool) ?? wireTool +} + +export function isCodexCollabToolRowName(name: string): boolean { + return CODEX_COLLAB_ROW_NAMES.has(name) +} + +/** The row input's key for the helper thread ids a call acts on, in the order the call names them. */ +export const CODEX_COLLAB_ROW_AGENTS_KEY = 'agents' + +/** The helper thread ids a collab call row names; none when its input was clipped. */ +export function codexCollabRowAgentIds(input: unknown): string[] { + if (typeof input !== 'object' || input === null || !(CODEX_COLLAB_ROW_AGENTS_KEY in input)) { + return [] + } + const ids = input[CODEX_COLLAB_ROW_AGENTS_KEY] + return Array.isArray(ids) + ? ids.filter((id): id is string => typeof id === 'string' && id.length > 0) + : [] +} diff --git a/src/shared/native-chat-tool-fold.ts b/src/shared/native-chat-tool-fold.ts index 7b5509abe8f5..d48d48120ca6 100644 --- a/src/shared/native-chat-tool-fold.ts +++ b/src/shared/native-chat-tool-fold.ts @@ -152,16 +152,29 @@ export type NativeChatToolPair = { result?: NativeChatToolResultBlock } -/** Pair calls and results by FIFO ordinal because transcript blocks carry no tool ids. */ +/** Where in `unanswered` (oldest first) the call `result` answers is, or -1 for none. */ +function answeredToolCallIndex( + unanswered: readonly T[], + result: NativeChatToolResultBlock, + callIdOf: (entry: T) => string | undefined +): number { + if (result.callId === undefined) { + return unanswered.length > 0 ? 0 : -1 + } + return unanswered.findIndex((entry) => callIdOf(entry) === result.callId) +} + +/** Pair results to calls by `answeredToolCallIndex`. Every reader of a run pairs through this + * (`pairNativeChatToolResults` included), so they all agree on who owns an output. */ export function pairToolBlocks( blocks: readonly NativeChatBlock[], limit = Infinity ): NativeChatToolPair[] { const pairs: NativeChatToolPair[] = [] + /** Slots of retained calls not yet answered, oldest first. */ const callSlots: number[] = [] - let resultOrdinal = 0 for (const block of blocks) { - if (pairs.length >= limit && resultOrdinal >= callSlots.length) { + if (pairs.length >= limit && callSlots.length === 0) { break } if (block.type === 'tool-call') { @@ -174,13 +187,13 @@ export function pairToolBlocks( if (block.type !== 'tool-result') { continue } - const slot = callSlots[resultOrdinal] + const answered = answeredToolCallIndex(callSlots, block, (slot) => pairs[slot]?.call?.callId) + const [slot] = answered === -1 ? [] : callSlots.splice(answered, 1) if (slot === undefined) { if (pairs.length < limit) { pairs.push({ result: block }) } } else { - resultOrdinal += 1 pairs[slot]!.result = block } } diff --git a/src/shared/native-chat-tool-icon.test.ts b/src/shared/native-chat-tool-icon.test.ts index f8cabed950cb..f84522443d2d 100644 --- a/src/shared/native-chat-tool-icon.test.ts +++ b/src/shared/native-chat-tool-icon.test.ts @@ -56,6 +56,13 @@ describe('native chat tool icons', () => { expect(nativeChatToolCategory('web search')).toBe('webSearch') }) + it('counts no Codex helper call as running an agent, since a spawn ends once its helper starts', () => { + expect(nativeChatToolCategory('task')).toBe('subAgentActivity') + expect(nativeChatToolCategory('spawn_agent')).toBeNull() + expect(nativeChatToolCategory('wait_agent')).toBeNull() + expect(nativeChatToolCategory('close_agent')).toBeNull() + }) + it('maps the tool names the Claude lane renders verbatim', () => { expect(nativeChatToolIconName('Read')).toBe('eye') expect(nativeChatToolIconName('Bash')).toBe('square-terminal') diff --git a/src/shared/native-chat-tool-icon.ts b/src/shared/native-chat-tool-icon.ts index 1bdc5736c62f..9da8672374a2 100644 --- a/src/shared/native-chat-tool-icon.ts +++ b/src/shared/native-chat-tool-icon.ts @@ -82,6 +82,8 @@ const CATEGORY_BY_ROW_WORD = new Map([ // Claude's tool names, which its lane renders verbatim. ['grep', 'search'], ['glob', 'search'], + // Claude's Task call lasts as long as its subagent. Codex's `spawn_agent` ends once the helper + // starts, so it is deliberately absent: its roster row, not the call, stands for the helper. ['task', 'subAgentActivity'], ['webfetch', 'webSearch'], ['todowrite', 'todoList'], diff --git a/src/shared/native-chat-tool-pair-limit.test.ts b/src/shared/native-chat-tool-pair-limit.test.ts index f279b70c996f..d54bfc9b89ba 100644 --- a/src/shared/native-chat-tool-pair-limit.test.ts +++ b/src/shared/native-chat-tool-pair-limit.test.ts @@ -77,6 +77,43 @@ describe('tool pair limits', () => { expect(pairToolBlocks(blocks, 2)).toEqual(original(blocks, 2)) }) + it('gives a result that names its call to that call, past one that finished with no output', () => { + const named = (name: string, callId: string): NativeChatBlock => ({ + type: 'tool-call', + name, + input: {}, + callId + }) + const [spawn, wait, shell] = [ + named('spawn_agent', 's'), + named('wait_agent', 'w'), + named('Bash', 'x') + ] + const waited: NativeChatBlock = { type: 'tool-result', output: 'CHILD_REPLY', callId: 'w' } + const ran: NativeChatBlock = { type: 'tool-result', output: 'CHILD_DONE', callId: 'x' } + expect(pairToolBlocks([spawn, wait, waited, shell, ran])).toEqual([ + { call: spawn }, + { call: wait, result: waited }, + { call: shell, result: ran } + ]) + }) + + it('gives no retained call the output of a call past the limit', () => { + const named = (callId: string): NativeChatBlock => ({ + type: 'tool-call', + name: 'Bash', + input: {}, + callId + }) + const [spawn, shell, later] = [named('s'), named('x'), named('y')] + const ran: NativeChatBlock = { type: 'tool-result', output: 'x', callId: 'x' } + const laterRan: NativeChatBlock = { type: 'tool-result', output: 'y', callId: 'y' } + expect(pairToolBlocks([spawn, shell, ran, later, laterRan], 2)).toEqual([ + { call: spawn }, + { call: shell, result: ran } + ]) + }) + it('keeps a leading stray result and then stops at the limit', () => { const blocks = [result, call, result, call, result] expect(pairToolBlocks(blocks, 1)).toEqual(original(blocks, 1)) diff --git a/src/shared/native-chat-tool-pairing.test.ts b/src/shared/native-chat-tool-pairing.test.ts index 3d2b5f37a974..72d7c46d656a 100644 --- a/src/shared/native-chat-tool-pairing.test.ts +++ b/src/shared/native-chat-tool-pairing.test.ts @@ -43,6 +43,32 @@ describe('pairNativeChatToolResults', () => { expect(pairedResults.has(orphan)).toBe(false) }) + it('gives a result that names its call to that call, past one that finished with no output', () => { + const named = (name: string, callId: string): NativeChatToolCallBlock => ({ + ...call(name), + callId + }) + const [spawn, wait, shell] = [named('spawn', 's'), named('wait', 'w'), named('shell', 'x')] + const [waited, ran]: NativeChatToolResultBlock[] = [ + { ...result('CHILD_REPLY'), callId: 'w' }, + { ...result('CHILD_DONE'), callId: 'x' } + ] + const { resultByCall } = pairNativeChatToolResults([spawn, wait, waited, shell, ran]) + + expect(resultByCall.has(spawn)).toBe(false) + expect(resultByCall.get(wait)).toBe(waited) + expect(resultByCall.get(shell)).toBe(ran) + }) + + it('leaves a result that names a call not in the run unpaired, rather than give it to another', () => { + const waiting: NativeChatToolCallBlock = { ...call('spawn'), callId: 's' } + const stray: NativeChatToolResultBlock = { ...result('elsewhere'), callId: 'missing' } + const { resultByCall, pairedResults } = pairNativeChatToolResults([waiting, stray]) + + expect(resultByCall.has(waiting)).toBe(false) + expect(pairedResults.has(stray)).toBe(false) + }) + it('ignores blocks that are neither a call nor a result', () => { const text: NativeChatBlock = { type: 'text', text: 'hi' } const [a, ra] = [call('a'), result('r')] diff --git a/src/shared/native-chat-tool-pairing.ts b/src/shared/native-chat-tool-pairing.ts index f4dd3db9d399..490008927c13 100644 --- a/src/shared/native-chat-tool-pairing.ts +++ b/src/shared/native-chat-tool-pairing.ts @@ -7,16 +7,18 @@ // as the command that produced it. Pairing lets the call own its output, so the // run reads as the work it did. // -// Pairing is positional, the same FIFO rule `dropUnattributableToolResults` -// already uses to decide a result is attributable at all: a result answers the -// oldest call that has not been answered yet. +// A result that names its call answers that call, and no other: one naming a call +// that is not waiting (outside a bounded window, say) stays unpaired. One that names +// none is paired positionally, the same FIFO rule `dropUnattributableToolResults` +// already uses to decide a result is attributable at all: it answers the oldest call +// that has not been answered yet. Position alone misattributes every later result once +// one call finishes with no output, which is why a producer that knows the call names it. -import { - isToolCallBlock, - isToolResultBlock, - type NativeChatBlock, - type NativeChatToolCallBlock, - type NativeChatToolResultBlock +import { pairToolBlocks } from './native-chat-tool-fold' +import type { + NativeChatBlock, + NativeChatToolCallBlock, + NativeChatToolResultBlock } from './native-chat-types' export type NativeChatToolPairing = { @@ -31,30 +33,17 @@ export const NO_NATIVE_CHAT_TOOL_PAIRING: NativeChatToolPairing = { pairedResults: new Set() } +/** The desktop run's view of `pairToolBlocks`, so every reader of a run pairs with one loop. */ export function pairNativeChatToolResults( blocks: readonly NativeChatBlock[] ): NativeChatToolPairing { const resultByCall = new Map() const pairedResults = new Set() - const unanswered: NativeChatToolCallBlock[] = [] - let answered = 0 - for (const block of blocks) { - if (isToolCallBlock(block)) { - unanswered.push(block) - continue + for (const { call, result } of pairToolBlocks(blocks)) { + if (call && result) { + resultByCall.set(call, result) + pairedResults.add(result) } - if (!isToolResultBlock(block)) { - continue - } - // Results carry no call id, so the journal's FIFO order is the only stable - // attribution available when calls are interleaved. - const call = unanswered[answered] - if (call === undefined) { - continue - } - answered += 1 - resultByCall.set(call, block) - pairedResults.add(block) } return { resultByCall, pairedResults } } diff --git a/src/shared/native-chat-types.ts b/src/shared/native-chat-types.ts index a1a8b3a51430..6d9d344f5069 100644 --- a/src/shared/native-chat-types.ts +++ b/src/shared/native-chat-types.ts @@ -96,6 +96,8 @@ export type NativeChatToolResultBlock = { type: 'tool-result' output: string isError?: boolean + /** The call this result answers, when the producer knows it; otherwise pairing is positional. */ + callId?: string /** Present only for edit tools whose result reported resolved hunks. */ editPatch?: NativeChatEditPatch } diff --git a/src/shared/structured-agent-session-projection.test.ts b/src/shared/structured-agent-session-projection.test.ts index 8570b0c3374d..c91bd7102b82 100644 --- a/src/shared/structured-agent-session-projection.test.ts +++ b/src/shared/structured-agent-session-projection.test.ts @@ -124,6 +124,23 @@ describe('structured agent session status projection', () => { }) }) + it('names the call a tool row output answers, so a run pairs it by id', () => { + const projected = projectStructuredItemToNativeChat( + item('wait', 1, { + kind: 'tool-call', + name: 'wait_agent', + callId: 'call-wait', + input: null, + state: 'completed', + output: { head: 'CHILD_REPLY', digest: 'd', byteLength: 11, truncated: false } + }) + ) + expect(projected?.blocks).toEqual([ + expect.objectContaining({ type: 'tool-call', callId: 'call-wait' }), + { type: 'tool-result', output: 'CHILD_REPLY', isError: false, callId: 'call-wait' } + ]) + }) + it('projects running, attention, and completed lifecycle states', () => { const running = item('running', 1, { kind: 'status', diff --git a/src/shared/structured-agent-session-projection.ts b/src/shared/structured-agent-session-projection.ts index 39a3d0a67e4b..0502a32798f0 100644 --- a/src/shared/structured-agent-session-projection.ts +++ b/src/shared/structured-agent-session-projection.ts @@ -89,7 +89,9 @@ function itemBlocks(item: AgentJournalRenderItem): { { type: 'tool-result' as const, output: boundedText(body.output), - isError: body.state === 'failed' + isError: body.state === 'failed', + // The call and its output are one journal row, so the result names its call. + ...(body.callId !== undefined ? { callId: body.callId } : {}) } ] : []) From ba39c6d5fd6bf37c4abc66dd071dc0ec0c2bdbc5 Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 11:45:05 -0700 Subject: [PATCH 14/79] fix(native-chat): a failed startup chat-lease save no longer puts the app into "Session restore failed" (#23964) * fix(native-chat): report a failed startup chat reconcile instead of failing app startup At startup the chat host re-checks every saved chat's lease and writes the result to agent-sessions.json. If that write failed (the file lock gave up, the file could not be written, or the file was written by a newer Orca and is read-only here), reconcileRestartLeases rejected, the startup IPC call rejected, and the renderer fell into its degraded "Session restore failed. Changes won't be saved until restart" mode. The reconcile is bookkeeping: a lease left unreconciled grants no writer, and every attach, send and read of a chat reconciles its own lease again. So the startup reconcile now reports its failure through a new optional host dependency, onStartupReconcileFailure, and resolves. The runtime routes it to its onError sink under the scope structured-agent-session-startup-reconcile, or logs it when no sink is installed (the desktop installs none). * fix(native-chat): read restored chats without waiting on lease bookkeeping With native chat on and a chat tab open at quit, the renderer's startup also awaits the chat tab restore (session.tabs.listAll). That restore re-ran the lease reconcile before reading each chat and rethrew its store failure, then recorded each restored tab as visible through a store transaction that throws on a held lock or a read-only store. Either one failed the restore, so startup still fell into "Session restore failed". Reading a chat grants no writer, so the reconcile startup and the restore run is now a reader's: createReaderReconcile never throws, answers whether every lease is settled (recovery is resolved only then; the journal opens either way), and reports each distinct failure once until a reconcile settles. Attach and agent start keep the strict reconcile. The restore's tab republish logs a failed visibility write and still publishes the tab, since a client drops every unpublished chat tab; user-driven publishes still refuse. The host dependency is renamed onLeaseReconcileFailure (scope structured-agent-session-lease-reconcile), since it now also reports for reads. * fix(native-chat): keep every record-store write off the startup chat read path Round-2 review found two more writes on the startup chat restore that could still fail it and put the app into "Session restore failed": republishing a /clear replacement recorded its tab visibility strictly, and resolving a chat's recovery rethrew its store error. The restore also paid one lock wait per tab and per batch of chats while the lock stayed held. The restore now derives tabs from state it already holds: - publishStructuredAgentSessionTab splits into the strict write and projectStructuredAgentSessionTab, which only updates the runtime's snapshot. The restore and /clear replacements only project: a saved tab index already lists every restored chat, and a /clear moves the tab in the same write that commits it. visibilityWriteMayFail is gone. - Chats a legacy profile restores that the index does not list are recorded in one best-effort transaction (store.showSessionTabs), so a failure leaves the index absent to seed again rather than partial. - The read restore's recovery resolution is caught and reported through onLeaseReconcileFailure, deduplicated with the reconcile's reports. - Once lease bookkeeping fails in a restore pass, the rest of that pass skips it, so a held lock costs one wait for the startup reconcile and one for the restore, however many chats are open. User actions (create, reveal, attach, send, the /clear commit) keep their strict writes. * fix(native-chat): start each restore pass from one lease check and stop its bookkeeping at the first failure The restore now runs one reader lease check for the pass and lets each chat re-check and resolve recovery only while the pass is still settled. The first refusal or failed write clears it for the rest of the pass, and every chat is still opened for reading. With another process holding the lock, startup waits on it once in prepare and once in the restore, however many chats are open; a legacy profile waits once more for its tab-index seed. * docs(native-chat): correct restore comments and a test name to match the final design * fix(native-chat): keep a throwing failure sink from failing the startup chat read The lease bookkeeping failure reporter called the host's failure sink directly, so a sink that threw turned a reported, recoverable store failure back into a rejected startup reconcile or read restore. The reporter now catches a sink throw and logs both the original failure and the sink error with console.warn. --- .../structured-agent-session-host-types.ts | 4 + .../structured-agent-session-host.ts | 10 +- ...ed-agent-session-readable-restorer.test.ts | 4 +- ...uctured-agent-session-restart-reconcile.ts | 70 ++++ ...ured-agent-session-restart-restore.test.ts | 67 +++- ...tructured-agent-session-restart-restore.ts | 36 +- ...ent-session-restore-without-import.test.ts | 4 +- .../structured-agent-session-reveal.ts | 34 +- ...-session-startup-reconcile-failure.test.ts | 220 ++++++++++ .../runtime/agent-session-record-store.ts | 8 +- src/main/runtime/agent-session-tab-table.ts | 11 + ...tore-structured-agent-session-tabs-once.ts | 51 ++- ...runtime-structured-session-restore.test.ts | 28 +- .../rpc/methods/structured-agent-session.ts | 2 +- .../structured-agent-session-runtime.ts | 4 + ...ed-agent-session-startup-reconcile.test.ts | 110 +++++ ...-agent-session-startup-tab-restore.test.ts | 378 ++++++++++++++++++ ...structured-agent-session-tab-index-seed.ts | 35 ++ 18 files changed, 1023 insertions(+), 53 deletions(-) create mode 100644 src/main/native-chat/agent-session-wire/structured-agent-session-startup-reconcile-failure.test.ts create mode 100644 src/main/runtime/structured-agent-session-startup-reconcile.test.ts create mode 100644 src/main/runtime/structured-agent-session-startup-tab-restore.test.ts create mode 100644 src/main/runtime/structured-agent-session-tab-index-seed.ts diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-host-types.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-host-types.ts index 80ea85e86835..f69bc3fb06aa 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-host-types.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-host-types.ts @@ -120,6 +120,10 @@ export type StructuredAgentSessionHostDeps = { /** Whether an orchestration dispatch still owns this session's worker; absent answers no. */ hasOpenDispatch?: (record: AgentSessionRecord) => boolean onEventSinkError?: (input: { sessionId: string; error: unknown }) => void + /** Lease bookkeeping run for startup or a read (the reconcile, or resolving a chat's recovery) + * that refused or threw, once per distinct failure. Startup and the read carry on: the next + * attach or send reconciles and resolves recovery again before it acts. */ + onLeaseReconcileFailure?: (failure: unknown) => void /** Every status projection this host publishes. `replay` marks a re-projection of state the host * already knew (restore, an arriving subscriber) rather than a fresh journal edge. */ onSessionStatusChanged?: ( diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts index 79a129118ee0..bcda340ee79e 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts @@ -135,7 +135,7 @@ export class StructuredAgentSessionHost { flushStreamedEvents: (sessionId) => this.flushStreamedEvents(sessionId) }) this.restore = createStructuredAgentSessionHostRestore(deps, { - reconcile: this.reconcileLeases, + reconcileLeases: this.reconcileLeases, resolveRecovery: (sessionId) => this.runtimeState.resolveRecovery(sessionId), serialize: (sessionId, task) => this.serialize(sessionId, task), hasSession: this.hasSession, @@ -215,6 +215,7 @@ export class StructuredAgentSessionHost { listSessionTabs = () => sessionTabs.listStructuredAgentSessionTabs(this.sessions) getPersistedVisibleSessionTabIndex = () => this.deps.store.getVisibleSessionTabIndex() getSessionTabId = (sessionId: string): string | null => this.deps.store.getSessionTabId(sessionId) + showSessionTabs = (sessionIds: readonly string[]) => this.deps.store.showSessionTabs(sessionIds) setSessionTabVisibility = async ( sessionId: string, @@ -228,12 +229,7 @@ export class StructuredAgentSessionHost { } } - reconcileRestartLeases = async (): Promise => { - const refusal = await this.reconcileLeases('startup') - if (refusal) { - throw new Error(refusal.code) - } - } + reconcileRestartLeases = (): Promise => this.restore.reconcileRestartLeases() restoreReadableSessions = (sessionIds?: readonly string[]): Promise => this.restore.restoreReadableSessions(sessionIds) diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-readable-restorer.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-readable-restorer.test.ts index 5eaedfffb384..09cab618ab9a 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-readable-restorer.test.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-readable-restorer.test.ts @@ -39,8 +39,8 @@ describe('StructuredAgentSessionReadableRestorer', () => { adapter: {} }, supportsRecord: () => true, - reconcile: async () => null, - resolveRecovery: async () => undefined, + reconcile: async () => true, + resolveRecovery: async () => true, serialize: async (_sessionId, task) => task(), hasSession: () => false, onReadable: () => undefined diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-reconcile.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-reconcile.ts index 3b6cc30271e5..f349f252ebed 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-reconcile.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-reconcile.ts @@ -45,6 +45,76 @@ export function createRestartReconciler(deps: { } } +/** Reports each distinct failure once, until `clear` says the bookkeeping settled again: the + * startup check, the restore pass and a restore retried after a failed journal open would each + * log the same store failure. */ +export type ReaderBookkeepingFailures = { + report: (failure: unknown) => void + clear: () => void +} + +export function reportEachFailureOnce( + onFailure: ((failure: unknown) => void) | undefined +): ReaderBookkeepingFailures { + let reported: string | null = null + return { + report: (failure) => { + const key = failureKey(failure) + if (key !== reported) { + reported = key + try { + onFailure?.(failure) + } catch (sinkError) { + // A throwing sink must not turn the reported failure back into a failed read. + console.warn('[structured-agent-session] reporting a lease bookkeeping failure failed', { + failure, + sinkError + }) + } + } + }, + clear: () => { + reported = null + } + } +} + +/** The reconcile a reader runs, at startup and before each restored read: it never throws, since + * an unreconciled lease grants no writer and the next send reconciles again before it acts. + * Answers whether every lease is settled. */ +export function createReaderReconcile( + reconcile: (sessionId: string) => Promise, + failures: ReaderBookkeepingFailures +): (sessionId: string) => Promise { + return async (sessionId) => { + let failure: unknown + try { + const refusal = await reconcile(sessionId) + if (!refusal) { + failures.clear() + return true + } + failure = refusal + } catch (error) { + failure = error + } + failures.report(failure) + return false + } +} + +function failureKey(failure: unknown): string { + if (typeof failure === 'object' && failure !== null) { + if ('code' in failure && failure.code) { + return String(failure.code) + } + if ('message' in failure) { + return String(failure.message) + } + } + return String(failure) +} + async function reconcileCurrentLeases(deps: { store: AgentSessionRecordStore probe: (record: AgentSessionRecord) => Promise diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.test.ts index c7d04ea64cc4..0cffa1e32c75 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.test.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.test.ts @@ -62,8 +62,8 @@ describe('restart journal restoration', () => { const restoration = restoreStructuredAgentSessionsOnRestart({ openDeps: NO_OPEN_DEPS, records, - reconcile: async () => null, - resolveRecovery: async () => undefined, + reconcile: async () => true, + resolveRecovery: async () => true, serialize: async (_sessionId, task) => task(), hasSession: () => false, onReadable: () => undefined @@ -103,8 +103,8 @@ describe('restart journal restoration', () => { await restoreStructuredAgentSessionsOnRestart({ openDeps: NO_OPEN_DEPS, records, - reconcile: async () => null, - resolveRecovery: async () => undefined, + reconcile: async () => true, + resolveRecovery: async () => true, serialize: async (_sessionId, task) => task(), hasSession: () => false, onReadable: () => undefined @@ -150,9 +150,10 @@ describe('restart journal restoration', () => { // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: the restore reads only the record's session id here. records: [{ sessionId: 'session-1' } as AgentSessionRecord], openDeps: NO_OPEN_DEPS, - reconcile: async () => null, + reconcile: async () => true, resolveRecovery: async () => { calls.push('resolveRecovery') + return true }, serialize: async (_sessionId, task) => task(), hasSession: () => false, @@ -164,6 +165,58 @@ describe('restart journal restoration', () => { expect(calls).toEqual(['resolveRecovery', 'open', 'onReadable:restored']) }) + // Each failed bookkeeping call stands for one wait on a held store lock. + describe('once lease bookkeeping fails in a pass', () => { + const records = Array.from( + { length: 8 }, + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: the restore reads only the record's session id here. + (_, index) => ({ sessionId: `session-${index}` }) as AgentSessionRecord + ) + const restore = ( + bookkeeping: Pick< + Parameters[0], + 'reconcile' | 'resolveRecovery' + > + ) => + restoreStructuredAgentSessionsOnRestart({ + openDeps: NO_OPEN_DEPS, + records, + ...bookkeeping, + serialize: async (_sessionId, task) => task(), + hasSession: () => false, + onReadable: () => undefined + }) + + /** A failure that takes a while, as a lock wait does, so the chats open at once overlap it. */ + const slowFailure = async (): Promise => { + await new Promise((resolve) => setTimeout(resolve, 5)) + return false + } + + beforeEach(() => restoreRead.mockResolvedValue(null)) + + it('skips it for every chat when the pass check fails, and still opens them all', async () => { + const reconcile = vi.fn(slowFailure) + const resolveRecovery = vi.fn(async () => true) + + await restore({ reconcile, resolveRecovery }) + + expect(reconcile).toHaveBeenCalledOnce() + expect(resolveRecovery).not.toHaveBeenCalled() + expect(restoreRead).toHaveBeenCalledTimes(records.length) + }) + + it('starts no more after the first failed recovery, and still opens every chat', async () => { + const resolveRecovery = vi.fn(slowFailure) + + await restore({ reconcile: async () => true, resolveRecovery }) + + // Only those already started when the first failed: at most one per chat open at once. + expect(resolveRecovery.mock.calls.length).toBeLessThanOrEqual(4) + expect(restoreRead).toHaveBeenCalledTimes(records.length) + }) + }) + it('does not settle again when a second restore finds the session already open', async () => { restoreRead.mockResolvedValue({ session: { journal: {}, params: {}, child: null }, @@ -174,8 +227,8 @@ describe('restart journal restoration', () => { // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: the restore reads only the record's session id here. records: [{ sessionId: 'session-1' } as AgentSessionRecord], openDeps: NO_OPEN_DEPS, - reconcile: async () => null, - resolveRecovery: async () => undefined, + reconcile: async () => true, + resolveRecovery: async () => true, serialize: async (_sessionId, task) => task(), hasSession: () => true, onReadable: () => undefined diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.ts index 4e4b7f1e5ddd..29ea17845d8a 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-restart-restore.ts @@ -13,7 +13,6 @@ import { setImmediate as yieldToEventLoop } from 'node:timers/promises' import type { AgentSessionRecord } from '../../../shared/agent-session-record' -import type { AgentSessionWireRefusal } from '../../../shared/agent-session-wire' import { mapWithConcurrency } from '../../../shared/map-with-concurrency' import type { AgentSessionRecordStore } from '../../runtime/agent-session-record-store' import type { @@ -28,8 +27,11 @@ export type StructuredAgentSessionReadRestoreDeps = { openDeps: StructuredAgentSessionConversationOpenDeps & { store: Pick } - reconcile: (sessionId: string) => Promise - resolveRecovery: (sessionId: string) => Promise + // Lease bookkeeping. Neither throws: a read grants no writer, so bookkeeping must not block it. + /** Whether every lease is settled. */ + reconcile: (sessionId: string) => Promise + /** False when its store write failed; the next attach or send resolves it again. */ + resolveRecovery: (sessionId: string) => Promise serialize: (sessionId: string, task: () => Promise) => Promise hasSession: (sessionId: string) => boolean onReadable: ( @@ -41,13 +43,10 @@ export type StructuredAgentSessionReadRestoreDeps = { /** One session's share of the restart restore. Startup maps this over every supported record. */ async function restoreOneStructuredAgentSessionRead( input: StructuredAgentSessionReadRestoreDeps, - sessionId: string + sessionId: string, + settleLeases: (sessionId: string) => Promise ): Promise { - const unreconciled = await input.reconcile(sessionId) - if (!unreconciled) { - // A session latched in recovery exits here at startup, without waiting for a client. - await input.resolveRecovery(sessionId) - } + await settleLeases(sessionId) await input.serialize(sessionId, () => restoreOneStructuredAgentSessionReadUnderSerialize(input, sessionId) ) @@ -73,9 +72,26 @@ async function restoreOneStructuredAgentSessionReadUnderSerialize( export async function restoreStructuredAgentSessionsOnRestart( input: StructuredAgentSessionReadRestoreDeps & { records: AgentSessionRecord[] } ): Promise { + const [first] = input.records + if (!first) { + return + } + // One check for the pass. Each chat checks again while it holds, since another writer can mark + // leases unreconciled mid-pass; after the first failure, retrying per chat only waits on the + // same store again, and the next attach or send settles those chats instead. + let settled = await input.reconcile(first.sessionId) + const settleLeases = async (sessionId: string): Promise => { + // A session latched in recovery exits here at startup, without waiting for a client. + if ( + settled && + !((await input.reconcile(sessionId)) && (await input.resolveRecovery(sessionId))) + ) { + settled = false + } + } await mapWithConcurrency(input.records, JOURNAL_RESTORE_CONCURRENCY, async ({ sessionId }) => { // A journal open is synchronous SQLite: without a macrotask per chat the restore is one long task. await yieldToEventLoop() - await restoreOneStructuredAgentSessionRead(input, sessionId) + await restoreOneStructuredAgentSessionRead(input, sessionId, settleLeases) }) } diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-restore-without-import.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-restore-without-import.test.ts index 226e6536c9c1..3290ba05f0de 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-restore-without-import.test.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-restore-without-import.test.ts @@ -232,8 +232,8 @@ async function restore(sessionIds: readonly string[]) { await restoreStructuredAgentSessionsOnRestart({ openDeps: deps, records: sessionIds.map(recordFor), - reconcile: async () => null, - resolveRecovery: async () => undefined, + reconcile: async () => true, + resolveRecovery: async () => true, serialize: async (_sessionId, task) => task(), hasSession: (sessionId) => sessions.has(sessionId), onReadable: (sessionId, opened) => { diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-reveal.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-reveal.ts index 2a1899ab1ccd..262c9e18d3c6 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-reveal.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-reveal.ts @@ -10,10 +10,15 @@ // a send does. And a journal it cannot open is not a refusal — the chat shows that failure with a // Retry, so the tab is worth publishing either way. +import type { AgentSessionWireRefusal } from '../../../shared/agent-session-wire' import { agentSessionRefusalError } from '../../../shared/agent-session-wire-refusals' import { adapterSupportsRecord } from './structured-agent-session-provider-support' import { StructuredAgentSessionReadableRestorer } from './structured-agent-session-readable-restorer' import { StructuredAgentSessionRestartRestoreGate } from './structured-agent-session-restart-restore-gate' +import { + createReaderReconcile, + reportEachFailureOnce +} from './structured-agent-session-restart-reconcile' import type { StructuredAgentSessionHostDeps, StructuredAgentSessionReveal @@ -51,23 +56,44 @@ export async function revealStructuredAgentSession( } } -/** The host's startup readable-restore sweep: reconcile, resolve, then open each chat's journal. */ +/** The host's startup readable-restore sweep: reconcile, resolve, then open each chat's journal. + * Its lease bookkeeping is a reader's, which never fails a read or startup; startup shares it. */ export function createStructuredAgentSessionHostRestore( deps: StructuredAgentSessionHostDeps, wiring: Omit< ConstructorParameters[0], - 'openDeps' | 'supportsRecord' - > + 'openDeps' | 'supportsRecord' | 'reconcile' | 'resolveRecovery' + > & { + reconcileLeases: (sessionId: string) => Promise + resolveRecovery: (sessionId: string) => Promise + } ): { + reconcileRestartLeases: () => Promise restoreReadableSessions: (sessionIds?: readonly string[]) => Promise } { + const { reconcileLeases, resolveRecovery, ...rest } = wiring + const failures = reportEachFailureOnce(deps.onLeaseReconcileFailure) + const reconcile = createReaderReconcile(reconcileLeases, failures) const restorer = new StructuredAgentSessionReadableRestorer({ openDeps: deps, supportsRecord: (record) => adapterSupportsRecord(deps.adapter, record), - ...wiring + reconcile, + // The next attach or send resolves recovery again, strictly, before it acts. + resolveRecovery: (sessionId) => + resolveRecovery(sessionId).then( + () => true, + (error: unknown) => { + failures.report(error) + return false + } + ), + ...rest }) const gate = new StructuredAgentSessionRestartRestoreGate() return { + reconcileRestartLeases: async () => { + await reconcile('startup') + }, restoreReadableSessions: (sessionIds) => gate.run(() => restorer.restore(sessionIds)) } } diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-startup-reconcile-failure.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-startup-reconcile-failure.test.ts new file mode 100644 index 000000000000..1d7b6621b3ef --- /dev/null +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-startup-reconcile-failure.test.ts @@ -0,0 +1,220 @@ +// The lease reconcile is bookkeeping: a lock that gives up or a store this build may not write is +// reported, and startup and every read carry on. Nothing is owed after it, because an unreconciled +// lease grants no writer and the next send reconciles every lease again before it acts. + +import { cp, readdir, readFile, rm } from 'node:fs/promises' +import { dirname } from 'node:path' +import { afterEach, expect, it, vi } from 'vitest' +import type * as FileTransactionLock from '../../file-transaction-lock' +import { AGENT_SESSION_STORE_SCHEMA_VERSION } from '../../runtime/agent-session-record-store-file' +import { + editPersistedTestAgentSessionStore, + openTestAgentSessionRecordStore, + type PersistedTestAgentSessionStore, + testAgentSessionStoreFilePath +} from '../../runtime/agent-session-record-store-test-harness' +import { + StructuredAgentSessionHost, + type StructuredAgentSessionHostDeps +} from './structured-agent-session-host' +import { + adapter, + attach, + CALLER, + envelope, + hostTestState, + replaceHostTestState +} from './structured-agent-session-host-test-harness' +import { + HOST_TEST_NOW as NOW, + HOST_TEST_SESSION as SESSION, + hostTestMessage +} from './structured-agent-session-host-test-data' +import { openTestJournalHostDatabase } from '../agent-session-journal/journal-host-database-test-support' + +const lock = vi.hoisted(() => ({ failing: false })) + +vi.mock('../../file-transaction-lock', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + withFileTransactionLock: (...args: Parameters) => + lock.failing + ? // What proper-lockfile throws once its retries give up. + Promise.reject( + Object.assign(new Error('Lock file is already being held'), { code: 'ELOCKED' }) + ) + : actual.withFileTransactionLock(...args) + } +}) + +const relaunchedRoots: string[] = [] + +afterEach(async () => { + lock.failing = false + await Promise.all( + relaunchedRoots.splice(0).map((dir) => rm(dir, { recursive: true, force: true })) + ) +}) + +/** A host with one chat, relaunched over a copy of its files; `rewrite` edits the copied store. */ +async function relaunch( + rewrite?: (persisted: PersistedTestAgentSessionStore) => void, + probeOwner: StructuredAgentSessionHostDeps['probeOwner'] = async () => ({ + outcome: 'pid-absent' + }) +) { + const dying = hostTestState() + await attach() + // An empty renewal queues behind every record write, so they are on disk. + await dying.store.renewLeases([]) + const relaunched = `${dying.root}-relaunched` + relaunchedRoots.push(relaunched) + // A dead process holds no lock. + await cp(dying.root, relaunched, { + recursive: true, + filter: (source) => !source.includes('.lock') + }) + const storePath = testAgentSessionStoreFilePath(relaunched) + if (rewrite) { + await editPersistedTestAgentSessionStore(relaunched, rewrite) + } + const store = await openTestAgentSessionRecordStore(relaunched) + const onLeaseReconcileFailure = vi.fn() + const host = new StructuredAgentSessionHost({ + store, + adapter: adapter(), + journalDatabase: openTestJournalHostDatabase(relaunched), + claimKeyId: 'key-1', + mintSpawnToken: () => 'spawn-next', + probeOwner, + stopOwnerProcess: () => { + throw new Error('an owner not proven alive must not be stopped') + }, + now: () => NOW, + onLeaseReconcileFailure + }) + replaceHostTestState({ store, host }) + return { host, store, storePath, onLeaseReconcileFailure } +} + +it('reports a startup reconcile whose store write fails, and does not reject', async () => { + const { host, store, onLeaseReconcileFailure } = await relaunch() + + lock.failing = true + await expect(host.reconcileRestartLeases()).resolves.toBeUndefined() + + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(onLeaseReconcileFailure).toHaveBeenCalledWith(expect.objectContaining({ code: 'ELOCKED' })) + // Nothing was adjudicated, so the lease still grants no writer. + expect(store.getRecord(SESSION)?.lease.unreconciled).toBe(true) +}) + +it('reconciles the chat on its next send once the store can be written again', async () => { + const { host, store, onLeaseReconcileFailure } = await relaunch() + lock.failing = true + await host.reconcileRestartLeases() + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + lock.failing = false + + const body = hostTestMessage('sent after a startup reconcile failed') + await expect( + host.send(CALLER, { envelope: envelope('agentSession.send', { body }), body }) + ).resolves.toMatchObject({ ok: true }) + + // The send is queued; delivering it starts the agent, and that start reconciles the lease. + await vi.waitFor(() => expect(hostTestState().dispatch).toHaveBeenCalledOnce(), { + timeout: 10_000 + }) + expect(store.getRecord(SESSION)?.lease.unreconciled).toBe(false) + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + // Before the relaunched directory is removed, so the child's wind-down can write its lease. + await host.flushAllStreamedEvents() +}) + +it('reports a store a newer Orca wrote without writing it, and does not reject', async () => { + const { host, store, storePath, onLeaseReconcileFailure } = await relaunch((persisted) => { + persisted.schemaVersion = AGENT_SESSION_STORE_SCHEMA_VERSION + 1 + }) + expect(store.readOnly).toBe(true) + const bytes = await readFile(storePath) + const files = await readdir(dirname(storePath)) + + await expect(host.reconcileRestartLeases()).resolves.toBeUndefined() + + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(onLeaseReconcileFailure).toHaveBeenCalledWith( + expect.objectContaining({ message: 'agent_session_legacy_required' }) + ) + expect(store.listRecords().map((record) => record.sessionId)).toEqual([SESSION]) + expect(await readFile(storePath)).toEqual(bytes) + expect(await readdir(dirname(storePath))).toEqual(files) +}) + +it('restores a chat for reading while the reconcile keeps failing, and reports it once', async () => { + const { host, store, onLeaseReconcileFailure } = await relaunch() + lock.failing = true + await host.reconcileRestartLeases() + + await expect(host.restoreReadableSessions([SESSION])).resolves.toBeUndefined() + + expect(host.hasSession(SESSION)).toBe(true) + expect(store.getRecord(SESSION)?.lease.unreconciled).toBe(true) + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(onLeaseReconcileFailure).toHaveBeenCalledWith(expect.objectContaining({ code: 'ELOCKED' })) +}) + +it('restores a chat for reading from a store a newer Orca wrote', async () => { + const { host, storePath, onLeaseReconcileFailure } = await relaunch((persisted) => { + persisted.schemaVersion = AGENT_SESSION_STORE_SCHEMA_VERSION + 1 + }) + const bytes = await readFile(storePath) + + await expect(host.restoreReadableSessions([SESSION])).resolves.toBeUndefined() + + expect(host.hasSession(SESSION)).toBe(true) + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(await readFile(storePath)).toEqual(bytes) +}) + +// A chat whose owner could not be proven gone is left recovering; the next attach or send retries it. +it('restores a chat for reading when resolving its recovery cannot write the store', async () => { + const { host, store, onLeaseReconcileFailure } = await relaunch(undefined, async () => ({ + outcome: 'indeterminate', + reason: 'probe' + })) + await host.reconcileRestartLeases() + expect(store.getRecord(SESSION)?.lease.handoffStage).toBe('recovering') + lock.failing = true + + await expect(host.restoreReadableSessions([SESSION])).resolves.toBeUndefined() + + expect(host.hasSession(SESSION)).toBe(true) + expect(store.getRecord(SESSION)?.lease.handoffStage).toBe('recovering') + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(onLeaseReconcileFailure).toHaveBeenCalledWith(expect.objectContaining({ code: 'ELOCKED' })) +}) + +it.each([ + ['startup reconcile', (host: StructuredAgentSessionHost) => host.reconcileRestartLeases()], + ['read restore', (host: StructuredAgentSessionHost) => host.restoreReadableSessions([SESSION])] +])('keeps the %s resolving when the failure sink throws', async (_step, read) => { + const { host, onLeaseReconcileFailure } = await relaunch() + const sinkError = new Error('error sink failed') + onLeaseReconcileFailure.mockImplementation(() => { + throw sinkError + }) + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + lock.failing = true + try { + await expect(read(host)).resolves.toBeUndefined() + + expect(onLeaseReconcileFailure).toHaveBeenCalledOnce() + expect(warn).toHaveBeenCalledWith( + expect.stringContaining('reporting a lease bookkeeping failure failed'), + expect.objectContaining({ failure: expect.objectContaining({ code: 'ELOCKED' }), sinkError }) + ) + } finally { + warn.mockRestore() + } +}) diff --git a/src/main/runtime/agent-session-record-store.ts b/src/main/runtime/agent-session-record-store.ts index 925fa38d8d18..603b5ed02faf 100644 --- a/src/main/runtime/agent-session-record-store.ts +++ b/src/main/runtime/agent-session-record-store.ts @@ -66,7 +66,7 @@ import { agentSessionStorePath, type AgentSessionStoreState } from './agent-session-record-store-file' -import { setAgentSessionTabVisibility } from './agent-session-tab-table' +import { setAgentSessionTabVisibility, showAgentSessionTabs } from './agent-session-tab-table' import { loadProtectedAgentSessionStore } from './agent-session-record-store-security' import { AgentSessionStoreTransactionQueue, @@ -146,6 +146,12 @@ export class AgentSessionRecordStore { return this.transact(() => setAgentSessionTabVisibility(this.state, sessionId, visible, tabId)) } + /** Shows each session that still has a record, in one write: an index written part way would + * read as complete at the next launch and drop the rest. */ + showSessionTabs(sessionIds: readonly string[]): Promise { + return this.transact(() => showAgentSessionTabs(this.state, sessionIds)) + } + listByScope(location: AgentSessionExecutionLocation): AgentSessionRecord[] { const scope = agentSessionScopeKey(location) return this.listRecords().filter((record) => agentSessionScopeKey(record.location) === scope) diff --git a/src/main/runtime/agent-session-tab-table.ts b/src/main/runtime/agent-session-tab-table.ts index 6022aac8bf2b..72551d563cab 100644 --- a/src/main/runtime/agent-session-tab-table.ts +++ b/src/main/runtime/agent-session-tab-table.ts @@ -131,6 +131,17 @@ export function setAgentSessionTabVisibility( } } +export function showAgentSessionTabs( + state: AgentSessionStoreState, + sessionIds: readonly string[] +): void { + for (const sessionId of sessionIds) { + if (state.records.has(sessionId)) { + setAgentSessionTabVisibility(state, sessionId, true) + } + } +} + export type PersistedAgentSessionTab = { tabId: string; sessionId: string } export function serializeAgentSessionTabTable(table: AgentSessionTabTable): { diff --git a/src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts b/src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts index 8d29a0d360b9..10dd920b253b 100644 --- a/src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts +++ b/src/main/runtime/orca-runtime-restore-structured-agent-session-tabs-once.ts @@ -5,6 +5,7 @@ import { getStructuredAgentSessionHost } from '../native-chat/agent-session-wire import { replaceConversationInSnapshot } from './structured-conversation-tab-replacement' import type { ConversationReplacement } from '../native-chat/agent-session-wire/structured-conversation-command' import { collectSavedStructuredAgentSessionIds } from './saved-structured-agent-session-restoration' +import { seedStructuredAgentSessionTabIndex } from './structured-agent-session-tab-index-seed' import { LOCAL_EXECUTION_HOST_ID } from '../../shared/execution-host' import type { RuntimeMobileSessionAgentTab, @@ -28,7 +29,9 @@ import { parseAppSshPtyId } from '../../shared/ssh-pty-id' import type { PtyProcessInspection } from '../providers/pty-process-inspection' export class OrcaRuntimeWithRestoreStructuredAgentSessionTabsOnce extends OrcaRuntimeWithGetStructuredAgentSessionCreateSupport { - async replaceStructuredAgentSessionTab(replacement: ConversationReplacement): Promise { + /** Projects only: a replacement's chat already has its tab in the store, which the /clear commit + * moved in the same write. */ + replaceStructuredAgentSessionTab(replacement: ConversationReplacement): void { const prior = this.mobileSessionTabsByWorktree.get(replacement.workspaceId) const next = prior ? replaceConversationInSnapshot(prior, replacement) : null if (next && next !== prior) { @@ -39,7 +42,7 @@ export class OrcaRuntimeWithRestoreStructuredAgentSessionTabsOnce extends OrcaRu (tab) => tab.type === 'agent-session' && tab.sessionId === replacement.sessionId ) ) { - await this.publishStructuredAgentSessionTab({ + this.projectStructuredAgentSessionTab({ ...replacement, replacesSessionId: replacement.sourceSessionId, activate: false @@ -57,9 +60,8 @@ export class OrcaRuntimeWithRestoreStructuredAgentSessionTabsOnce extends OrcaRu const profileIds = collectSavedStructuredAgentSessionIds( this.store?.getWorkspaceSession?.(LOCAL_EXECUTION_HOST_ID) ?? null ) - await host?.restoreReadableSessions( - persistedVisibleIndex.present ? persistedVisibleIndex.sessionIds : profileIds - ) + const targets = persistedVisibleIndex.present ? persistedVisibleIndex.sessionIds : profileIds + await host?.restoreReadableSessions(targets) for (const worktreeId of this.getKnownWorkspaceSessionWorktreeIds()) { this.hydrateHeadlessMobileSessionTabsFromWorkspaceSession(worktreeId, { allowAttachedWindow: true, @@ -67,24 +69,27 @@ export class OrcaRuntimeWithRestoreStructuredAgentSessionTabsOnce extends OrcaRu }) } this.hydrateHeadlessMobileSessionTabsFromWorkspaceSession() - for (const replacement of host?.conversationReplacements?.() ?? []) { - await this.replaceStructuredAgentSessionTab(replacement) - } - for (const session of host?.listSessionTabs() ?? []) { + const restored = (host?.listSessionTabs() ?? []).flatMap((session) => { if (session.agent !== 'codex' && session.agent !== 'claude') { - continue + return [] } let sessionId = session.sessionId while (sessionId.startsWith('agent-session:')) { sessionId = sessionId.slice('agent-session:'.length) } - await this.publishStructuredAgentSessionTab({ - ...session, - agent: session.agent, - sessionId, - activate: false, - notify: false - }) + return [{ ...session, agent: session.agent, sessionId }] + }) + await seedStructuredAgentSessionTabIndex( + host, + targets, + restored.map((session) => session.sessionId) + ) + // Past the seed, projecting records nothing. + for (const replacement of host?.conversationReplacements?.() ?? []) { + this.replaceStructuredAgentSessionTab(replacement) + } + for (const session of restored) { + this.projectStructuredAgentSessionTab({ ...session, activate: false, notify: false }) } const wasUnverifiable = this.structuredAgentSessionInventoryUnverifiable // No host means no one can say which chats exist; with none on disk, empty is the answer. @@ -114,6 +119,18 @@ export class OrcaRuntimeWithRestoreStructuredAgentSessionTabsOnce extends OrcaRu ...(input.tabId ? [input.tabId] : []) ) } + this.projectStructuredAgentSessionTab(input) + } + + /** The runtime's own snapshot of a chat tab. Records nothing: the caller owns the store write. */ + projectStructuredAgentSessionTab(input: { + workspaceId: string + sessionId: string + agent: 'claude' | 'codex' + activate: boolean + notify?: boolean + replacesSessionId?: string + }): void { const existing = this.mobileSessionTabsByWorktree.get(input.workspaceId) const id = `agent-session:${input.sessionId}` if (existing?.tabs.some((tab) => tab.id === id)) { diff --git a/src/main/runtime/orca-runtime-structured-session-restore.test.ts b/src/main/runtime/orca-runtime-structured-session-restore.test.ts index 1d8105d07f81..929ee82b6c89 100644 --- a/src/main/runtime/orca-runtime-structured-session-restore.test.ts +++ b/src/main/runtime/orca-runtime-structured-session-restore.test.ts @@ -324,7 +324,7 @@ describe('structured session cold restoration', () => { it('publishes restored Claude tabs with the Claude title', async () => { const runtime = new OrcaRuntimeService() - const publish = vi.spyOn(runtime, 'publishStructuredAgentSessionTab') + const project = vi.spyOn(runtime, 'projectStructuredAgentSessionTab') const internal = runtime as unknown as { hasPersistedStructuredAgentSessionStore(): boolean getKnownWorkspaceSessionWorktreeIds(): Set @@ -351,7 +351,7 @@ describe('structured session cold restoration', () => { await runtime.restoreStructuredAgentSessionTabs() - expect(publish).toHaveBeenCalledWith({ + expect(project).toHaveBeenCalledWith({ workspaceId: 'workspace-1', sessionId: 'restored-claude', agent: 'claude', @@ -372,6 +372,30 @@ describe('structured session cold restoration', () => { ) }) + // The /clear commit moved the tab in the store already, so replacing it only projects. + it('replaces a cleared conversation tab without a store write', async () => { + const runtime = new OrcaRuntimeService() + const setSessionTabVisibility = vi.fn(async () => undefined) + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: replacing a tab reaches the host only through setSessionTabVisibility, which must stay uncalled. + setStructuredAgentSessionHost({ setSessionTabVisibility } as never) + + runtime.replaceStructuredAgentSessionTab({ + sourceSessionId: 'cleared-session', + sessionId: 'replacement-session', + workspaceId: 'workspace-1', + agent: 'codex' + }) + + const snapshot = await runtime.listMobileSessionTabs('id:workspace-1') + expect(snapshot.tabs).toEqual([ + expect.objectContaining({ + id: 'agent-session:replacement-session', + replacesSessionId: 'cleared-session' + }) + ]) + expect(setSessionTabVisibility).not.toHaveBeenCalled() + }) + it('commits the host close when the renderer already removed the structured tab', async () => { const runtime = new OrcaRuntimeService() runtime.setNotifier({ diff --git a/src/main/runtime/rpc/methods/structured-agent-session.ts b/src/main/runtime/rpc/methods/structured-agent-session.ts index 534d49677793..463772d33375 100644 --- a/src/main/runtime/rpc/methods/structured-agent-session.ts +++ b/src/main/runtime/rpc/methods/structured-agent-session.ts @@ -123,7 +123,7 @@ export const STRUCTURED_AGENT_SESSION_METHODS = [ .conversationReplacements() .find((entry) => entry.sourceSessionId === params.envelope.sessionId) if (replacement) { - await ctx.runtime.replaceStructuredAgentSessionTab(replacement) + ctx.runtime.replaceStructuredAgentSessionTab(replacement) } await host.close(params.envelope.sessionId) } diff --git a/src/main/runtime/structured-agent-session-runtime.ts b/src/main/runtime/structured-agent-session-runtime.ts index 82c9803e2a76..bb82e070e125 100644 --- a/src/main/runtime/structured-agent-session-runtime.ts +++ b/src/main/runtime/structured-agent-session-runtime.ts @@ -317,6 +317,10 @@ async function installOnJournal( : {}), onEventSinkError: ({ sessionId, error }) => deps.onError?.({ scope: `structured-agent-session-journal:${sessionId}`, error }), + onLeaseReconcileFailure: (error) => + deps.onError + ? deps.onError({ scope: 'structured-agent-session-lease-reconcile', error }) + : console.warn('[structured-agent-session] reconciling chat leases failed', error), ...(deps.onSessionStatusChanged ? { onSessionStatusChanged: deps.onSessionStatusChanged } : {}), ...(deps.statusSink ? { statusSink: deps.statusSink } : {}), ...(deps.hasOpenDispatch ? { hasOpenDispatch: deps.hasOpenDispatch } : {}), diff --git a/src/main/runtime/structured-agent-session-startup-reconcile.test.ts b/src/main/runtime/structured-agent-session-startup-reconcile.test.ts new file mode 100644 index 000000000000..33b26f7569bb --- /dev/null +++ b/src/main/runtime/structured-agent-session-startup-reconcile.test.ts @@ -0,0 +1,110 @@ +// A profile whose chat records a newer Orca wrote opens read-only. The startup reconcile cannot +// write there, which is bookkeeping: startup must still finish rather than put the whole app into +// its degraded "Session restore failed" mode, and the file must be left exactly as it was. + +import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, expect, it, vi } from 'vitest' +import { agentSessionRecordFixture } from '../../shared/agent-session-record.test-fixture' +import { getStructuredAgentSessionHost } from '../native-chat/agent-session-wire/structured-agent-session-registry' +import { + AGENT_SESSION_STORE_SCHEMA_VERSION, + agentSessionStorePath +} from './agent-session-record-store-file' +import { OrcaRuntimeService } from './orca-runtime' +import { + ensureStructuredAgentSessionHost, + stopStructuredAgentSessionRuntime +} from './structured-agent-session-runtime' + +let root: string + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'orca-startup-reconcile-')) +}) + +afterEach(async () => { + await stopStructuredAgentSessionRuntime().catch(() => undefined) + vi.restoreAllMocks() + await rm(root, { recursive: true, force: true }) +}) + +/** A profile whose record store a newer Orca wrote, with one chat in it. */ +async function seedNewerStore() { + const storeDirectory = join(root, 'agent-sessions') + const path = agentSessionStorePath(storeDirectory) + const record = agentSessionRecordFixture() + await mkdir(storeDirectory, { recursive: true }) + await writeFile( + path, + JSON.stringify({ + schemaVersion: AGENT_SESSION_STORE_SCHEMA_VERSION + 1, + hostId: 'local', + records: { [record.sessionId]: record }, + operations: {}, + retiredClaimKeys: [], + unusableRecords: {} + }) + ) + return { storeDirectory, path, sessionId: record.sessionId } +} + +function startupRuntime(onError?: (input: { scope: string; error: unknown }) => void) { + const runtime = new OrcaRuntimeService() + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: these are the runtime's own protected members; the test roots the host at `root` and stubs the PTY daemon. + const internal = runtime as unknown as { + hasPersistedStructuredAgentSessionStore(): boolean + ensureStructuredAgentSessionHost(): Promise + refreshMobileSessionPtyRecords(): Promise | null> + } + internal.hasPersistedStructuredAgentSessionStore = () => true + internal.ensureStructuredAgentSessionHost = () => + ensureStructuredAgentSessionHost({ + stateDirectory: root, + hostId: 'local', + claimKeyId: 'key-1', + resolveWorkspacePath: async () => root, + resolveEnvironment: async () => ({}), + resolveClaudeAuthPolicy: () => ({ stripAuthEnv: true }), + ...(onError ? { onError } : {}) + }) + internal.refreshMobileSessionPtyRecords = async () => new Set() + return runtime +} + +it('finishes startup over records a newer Orca wrote, reports it, and writes nothing', async () => { + const { storeDirectory, path, sessionId } = await seedNewerStore() + const bytes = await readFile(path) + const files = await readdir(storeDirectory) + const onError = vi.fn() + const runtime = startupRuntime(onError) + + // What the renderer's startup awaits through `app:prepareTerminalStartupRestoration`. + await expect(runtime.prepareStructuredAgentSessionStartupRestoration()).resolves.toBeUndefined() + + expect(onError).toHaveBeenCalledOnce() + expect(onError).toHaveBeenCalledWith({ + scope: 'structured-agent-session-lease-reconcile', + error: expect.objectContaining({ message: 'agent_session_legacy_required' }) + }) + expect(getStructuredAgentSessionHost()?.sessionAgent(sessionId)).toBe('claude') + await stopStructuredAgentSessionRuntime() + expect(await readFile(path)).toEqual(bytes) + expect(await readdir(storeDirectory)).toEqual(files) +}) + +// The desktop installs its host with no error sink, so the failure is logged rather than dropped. +it('logs the failure when the host has no error sink', async () => { + await seedNewerStore() + const warn = vi.spyOn(console, 'warn').mockImplementation(() => undefined) + + await expect( + startupRuntime().prepareStructuredAgentSessionStartupRestoration() + ).resolves.toBeUndefined() + + expect(warn).toHaveBeenCalledWith( + '[structured-agent-session] reconciling chat leases failed', + expect.objectContaining({ message: 'agent_session_legacy_required' }) + ) +}) diff --git a/src/main/runtime/structured-agent-session-startup-tab-restore.test.ts b/src/main/runtime/structured-agent-session-startup-tab-restore.test.ts new file mode 100644 index 000000000000..61fa020e4276 --- /dev/null +++ b/src/main/runtime/structured-agent-session-startup-tab-restore.test.ts @@ -0,0 +1,378 @@ +// With native chat on, the renderer's startup also awaits the chat tab restore (`session.tabs.listAll`), +// which reads every chat whose tab was open at quit. That read must not wait on record-store +// bookkeeping: with a saved tab index it writes nothing, and a store that cannot be written costs +// a bounded number of lock waits, not one per chat. + +import { mkdir, mkdtemp, readdir, readFile, rm, writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { AgentSessionRecord } from '../../shared/agent-session-record' +import type { RuntimeMobileSessionTabsSnapshot } from '../../shared/runtime-types' +import { + agentSessionLeaseFixture, + agentSessionRecordFixture +} from '../../shared/agent-session-record.test-fixture' +import type * as FileTransactionLock from '../file-transaction-lock' +import { JournalHostDatabase } from '../native-chat/agent-session-journal/journal-host-database' +import { openAgentSessionJournal } from '../native-chat/agent-session-journal/journal-store-factory' +import { journalIdentityFor } from '../native-chat/agent-session-wire/structured-agent-session-attach' +import { attachParamsForRecord } from '../native-chat/agent-session-wire/structured-agent-session-conversation-open' +import { getStructuredAgentSessionHost } from '../native-chat/agent-session-wire/structured-agent-session-registry' +import { AgentSessionRecordStore } from './agent-session-record-store' +import { + AGENT_SESSION_STORE_SCHEMA_VERSION, + agentSessionStorePath +} from './agent-session-record-store-file' +import { OrcaRuntimeService } from './orca-runtime' +import { + ensureStructuredAgentSessionHost, + stopStructuredAgentSessionRuntime +} from './structured-agent-session-runtime' + +// `failing` refuses every take; `grants` lets that many more through, then refuses. +const lock = vi.hoisted(() => ({ failing: false, grants: Infinity, refused: 0 })) + +vi.mock('../file-transaction-lock', async (importOriginal) => { + const actual = await importOriginal() + return { + ...actual, + withFileTransactionLock: (...args: Parameters) => { + if (lock.failing || lock.grants <= 0) { + lock.refused += 1 + // What proper-lockfile throws once its retries give up, about 3 s later. + return Promise.reject( + Object.assign(new Error('Lock file is already being held'), { code: 'ELOCKED' }) + ) + } + lock.grants -= 1 + return actual.withFileTransactionLock(...args) + } + } +}) + +const PROMPT = 'add a retry' +const CHAT_A = 'chat-a-0001' +const CHAT_B = 'chat-b-0002' +const CLEARED = 'chat-s-0003' + +let root: string + +beforeEach(async () => { + root = await mkdtemp(join(tmpdir(), 'orca-startup-tab-restore-')) +}) + +afterEach(async () => { + Object.assign(lock, { failing: false, grants: Infinity, refused: 0 }) + await stopStructuredAgentSessionRuntime().catch(() => undefined) + vi.restoreAllMocks() + await rm(root, { recursive: true, force: true }) +}) + +/** A released chat, so no startup probe looks for a live owner. */ +function chatRecord( + sessionId: string, + options: { codex?: boolean; clearedInto?: string } = {} +): AgentSessionRecord { + const record = agentSessionRecordFixture( + agentSessionLeaseFixture({ + sessionId, + ownerProcess: null, + reservedSpawnToken: null, + claimStatus: 'released' + }) + ) + const codex = options.codex + ? { + provider: 'codex' as const, + providerHandleChain: record.providerHandleChain.map((link) => ({ + ...link, + handle: { provider: 'codex' as const, threadId: `thread-${sessionId}` } + })), + accountHome: { variable: 'CODEX_HOME' as const, path: join(root, 'codex-home') } + } + : {} + const clear = options.clearedInto + ? { + conversationCommand: { + command: 'clear' as const, + state: 'completed' as const, + phase: 'committed' as const, + operationId: `clear-${sessionId}`, + callerKey: 'client-1', + replacementSessionId: options.clearedInto + } + } + : {} + return { ...record, ...codex, ...clear } +} + +/** Writes the record store; `visible` is the saved tab index, absent on a legacy profile. */ +async function seedStore( + records: AgentSessionRecord[], + options: { newer?: boolean; visible?: string[] } = {} +) { + const storeDirectory = join(root, 'agent-sessions') + const path = agentSessionStorePath(storeDirectory) + await mkdir(storeDirectory, { recursive: true }) + await writeFile( + path, + JSON.stringify({ + schemaVersion: AGENT_SESSION_STORE_SCHEMA_VERSION + (options.newer ? 1 : 0), + hostId: 'local', + records: Object.fromEntries(records.map((record) => [record.sessionId, record])), + operations: {}, + retiredClaimKeys: [], + unusableRecords: {}, + ...(options.visible ? { visibleSessionIds: options.visible } : {}) + }) + ) + return { storeDirectory, path } +} + +/** Each chat's history as the last run left it: one prompt, accepted, so nothing is left to send. */ +async function seedHistory(records: AgentSessionRecord[]): Promise { + const database = JournalHostDatabase.open(root) + for (const record of records) { + const fence = record.lease.runtimeFence + const journal = await openAgentSessionJournal({ + identity: journalIdentityFor( + record, + attachParamsForRecord(record, { clientOperationId: 'seed', expectedRuntimeFence: fence }) + ), + database + }) + await journal.appendSubmission({ + clientMessageId: 'client-1', + payloadFingerprint: 'fp-1', + body: { kind: 'message', role: 'user', blocks: [{ type: 'text', text: PROMPT }] }, + fence, + handoverRecorded: true + }) + await journal.resolveDispatch({ + clientMessageId: 'client-1', + fence, + state: 'accepted', + providerIdentity: null + }) + await journal.close() + } + database.close() +} + +function startupRuntime(options: { afterInstall?: () => void; profileChats?: string[] } = {}) { + const onError = vi.fn() + const runtime = new OrcaRuntimeService() + // oxlint-disable-next-line typescript/consistent-type-assertions -- SAFETY: these are the runtime's own protected members; the test roots the host at `root`, stubs the PTY daemon and gives it a profile. + const internal = runtime as unknown as { + hasPersistedStructuredAgentSessionStore(): boolean + ensureStructuredAgentSessionHost(): Promise + refreshMobileSessionPtyRecords(): Promise | null> + mobileSessionTabsByWorktree: Map + store: { getWorkspaceSession: () => unknown } + } + internal.hasPersistedStructuredAgentSessionStore = () => true + internal.ensureStructuredAgentSessionHost = async () => { + const installed = await ensureStructuredAgentSessionHost({ + stateDirectory: root, + hostId: 'local', + claimKeyId: 'key-1', + resolveWorkspacePath: async () => root, + resolveEnvironment: async () => ({}), + resolveClaudeAuthPolicy: () => ({ stripAuthEnv: true }), + onError + }) + options.afterInstall?.() + return installed + } + internal.refreshMobileSessionPtyRecords = async () => new Set() + // A legacy profile's saved chat tabs, which a store with no tab index restores from. + internal.store = { + getWorkspaceSession: () => ({ + activeRepoId: null, + activeWorktreeId: 'workspace-1', + activeTabId: null, + tabsByWorktree: {}, + terminalLayoutsByTabId: {}, + unifiedTabs: { + 'workspace-1': (options.profileChats ?? []).map((sessionId, index) => ({ + id: `agent-session:${sessionId}`, + entityId: sessionId, + groupId: 'group-1', + worktreeId: 'workspace-1', + contentType: 'agent-session', + label: 'Codex Chat', + customLabel: null, + color: null, + sortOrder: index, + createdAt: 1 + })) + } + }) + } + return { + runtime, + onError, + published: () => internal.mobileSessionTabsByWorktree.get('workspace-1')?.tabs ?? [] + } +} + +async function expectHistory(sessionId: string): Promise { + const host = getStructuredAgentSessionHost() + expect(JSON.stringify((await host!.journalSnapshot(sessionId)).items)).toContain(PROMPT) +} + +function spyOnTabWrites() { + return { + visibility: vi.spyOn(AgentSessionRecordStore.prototype, 'setSessionTabVisibility'), + seed: vi.spyOn(AgentSessionRecordStore.prototype, 'showSessionTabs') + } +} + +async function tabIndexOnDisk(path: string): Promise { + return JSON.parse(await readFile(path, 'utf-8')).visibleSessionIds +} + +describe('restoring the chat tabs open at quit', () => { + it.each([ + { store: 'records a newer Orca wrote', newer: true, lockFails: false }, + { store: 'a lock that keeps failing', newer: false, lockFails: true } + ])('lists and reads every chat from $store, and writes nothing', async ({ newer, lockFails }) => { + const records = [ + chatRecord(CHAT_A), + chatRecord(CHAT_B), + chatRecord(CLEARED, { clearedInto: CHAT_A }) + ] + const { storeDirectory, path } = await seedStore(records, { newer, visible: [CHAT_A, CHAT_B] }) + await seedHistory(records.slice(0, 2)) + const bytes = await readFile(path) + const files = await readdir(storeDirectory) + const writes = spyOnTabWrites() + vi.spyOn(console, 'warn').mockImplementation(() => undefined) + // Free when the store opens, then held for good. + const { runtime, onError, published } = startupRuntime({ + afterInstall: () => { + lock.failing = lockFails + } + }) + + await expect(runtime.restoreStructuredAgentSessionTabs()).resolves.toBeUndefined() + + expect(published().map((tab) => tab.id)).toEqual([ + `agent-session:${CHAT_A}`, + `agent-session:${CHAT_B}` + ]) + expect(published()[0]).toMatchObject({ replacesSessionId: CLEARED }) + await expectHistory(CHAT_A) + await expectHistory(CHAT_B) + expect(onError).toHaveBeenCalledOnce() + expect(onError).toHaveBeenCalledWith({ + scope: 'structured-agent-session-lease-reconcile', + error: expect.objectContaining( + newer ? { message: 'agent_session_legacy_required' } : { code: 'ELOCKED' } + ) + }) + expect(writes.visibility).not.toHaveBeenCalled() + expect(writes.seed).not.toHaveBeenCalled() + await stopStructuredAgentSessionRuntime() + if (newer) { + expect(await readFile(path)).toEqual(bytes) + expect(await readdir(storeDirectory)).toEqual(files) + } + }) + + // Each refused take stands for one lock wait of about 3 s. + it.each([1, 4, 8])( + 'waits on a held lock once per startup step with %i chats open', + async (count) => { + const chats = Array.from({ length: count }, (_, index) => `chat-${index}-000${index}`) + const records = chats.map((sessionId) => chatRecord(sessionId)) + await seedStore(records, { visible: chats }) + await seedHistory(records) + vi.spyOn(console, 'warn').mockImplementation(() => undefined) + const { runtime, published } = startupRuntime({ + afterInstall: () => { + lock.failing = true + } + }) + + await runtime.prepareStructuredAgentSessionStartupRestoration() + const prepared = lock.refused + await runtime.restoreStructuredAgentSessionTabs() + + expect(published()).toHaveLength(count) + expect(prepared).toBe(1) + expect(lock.refused - prepared).toBe(1) + } + ) + + describe('on a legacy profile, with no tab index yet', () => { + const legacyChats = () => [ + chatRecord(CHAT_A, { codex: true }), + chatRecord(CHAT_B, { codex: true }) + ] + + it('records every restored chat in one write', async () => { + const records = legacyChats() + const { path } = await seedStore(records) + await seedHistory(records) + const writes = spyOnTabWrites() + const { runtime, published } = startupRuntime({ profileChats: [CHAT_A, CHAT_B] }) + await runtime.prepareStructuredAgentSessionStartupRestoration() + // One more take, then held: a seed written chat by chat would stop part way. + lock.grants = 1 + + await runtime.restoreStructuredAgentSessionTabs() + + expect(published().map((tab) => tab.id)).toEqual([ + `agent-session:${CHAT_A}`, + `agent-session:${CHAT_B}` + ]) + expect(await tabIndexOnDisk(path)).toEqual([CHAT_A, CHAT_B]) + expect(writes.seed).toHaveBeenCalledOnce() + expect(writes.visibility).not.toHaveBeenCalled() + }) + + it('waits on a held lock once more, for that one write', async () => { + const records = legacyChats() + await seedStore(records) + await seedHistory(records) + vi.spyOn(console, 'warn').mockImplementation(() => undefined) + const { runtime, published } = startupRuntime({ + profileChats: [CHAT_A, CHAT_B], + afterInstall: () => { + lock.failing = true + } + }) + + await runtime.prepareStructuredAgentSessionStartupRestoration() + const prepared = lock.refused + await runtime.restoreStructuredAgentSessionTabs() + + expect(published()).toHaveLength(2) + expect(prepared).toBe(1) + // The restore's lease check and the seed. + expect(lock.refused - prepared).toBe(2) + }) + + it('still lists the chats when that write fails, and leaves the index absent', async () => { + const records = legacyChats() + const { path } = await seedStore(records) + await seedHistory(records) + const warn = vi.spyOn(console, 'warn').mockImplementation(() => undefined) + const { runtime, published } = startupRuntime({ profileChats: [CHAT_A, CHAT_B] }) + await runtime.prepareStructuredAgentSessionStartupRestoration() + lock.failing = true + + await expect(runtime.restoreStructuredAgentSessionTabs()).resolves.toBeUndefined() + + expect(published()).toHaveLength(2) + await expectHistory(CHAT_A) + expect(warn).toHaveBeenCalledWith( + '[structured-agent-session] recording restored chat tabs failed', + { sessionIds: [CHAT_A, CHAT_B], error: expect.objectContaining({ code: 'ELOCKED' }) } + ) + expect(await tabIndexOnDisk(path)).toBeUndefined() + }) + }) +}) diff --git a/src/main/runtime/structured-agent-session-tab-index-seed.ts b/src/main/runtime/structured-agent-session-tab-index-seed.ts new file mode 100644 index 000000000000..181d67128a1f --- /dev/null +++ b/src/main/runtime/structured-agent-session-tab-index-seed.ts @@ -0,0 +1,35 @@ +// A profile saved before the store kept a tab index restores its chats from the profile's own tabs. +// Those chats are recorded once, all together: an index written part way would read as complete at +// the next launch and drop the rest. With an index present, every restored chat is already listed. + +import type { StructuredAgentSessionHost } from '../native-chat/agent-session-wire/structured-agent-session-host' + +type TabIndexHost = Pick< + StructuredAgentSessionHost, + 'getPersistedVisibleSessionTabIndex' | 'showSessionTabs' +> + +/** Best-effort: the tabs publish either way, and an index still absent is seeded again next launch. */ +export async function seedStructuredAgentSessionTabIndex( + host: Partial | null | undefined, + targets: readonly string[], + restored: readonly string[] +): Promise { + if (!host?.getPersistedVisibleSessionTabIndex || !host.showSessionTabs) { + return + } + const listed = new Set(host.getPersistedVisibleSessionTabIndex().sessionIds) + const opened = new Set(restored) + // In the restore's own order, so the seeded tabs keep the order the profile gave them. + const unlisted = [...new Set([...targets, ...restored])].filter( + (sessionId) => opened.has(sessionId) && !listed.has(sessionId) + ) + if (unlisted.length > 0) { + await host.showSessionTabs(unlisted).catch((error: unknown) => + console.warn('[structured-agent-session] recording restored chat tabs failed', { + sessionIds: unlisted, + error + }) + ) + } +} From 373d951bf072698871c9e738ec2e1665ffd56830 Mon Sep 17 00:00:00 2001 From: Dunz Koi <258568505+dunzkoi@users.noreply.github.com> Date: Thu, 1 Oct 2026 04:01:44 +0900 Subject: [PATCH 15/79] fix(editor): highlight shell startup dotfiles (#24066) * fix(editor): highlight shell startup dotfiles Opening ~/.zshrc, ~/.bashrc or ~/.profile rendered as plaintext because extname() treats a leading-dot name as having no extension, and Monaco's shell association only lists .sh/.bash. Map the common bash/zsh/POSIX startup filenames to the shell language in FILENAME_TO_LANGUAGE so both the editor and diffs highlight them. Co-Authored-By: Claude Opus 5.5 (1M context) * test(editor): cover every mapped shell dotfile Co-Authored-By: Claude Opus 5.5 (1M context) * fix(editor): match exact filenames case-insensitively Shell dotfiles (.ZSHRC, .BASHRC) on case-insensitive filesystems still fell to plaintext. Exact match wins; lowercase fallback covers the rest without touching extension/Monaco order. --------- Co-authored-by: djlee Co-authored-by: Claude Opus 5.5 (1M context) Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com> --- src/renderer/src/lib/language-detect.test.ts | 31 ++++++++++++++++++++ src/renderer/src/lib/language-detect.ts | 21 ++++++++++++- 2 files changed, 51 insertions(+), 1 deletion(-) diff --git a/src/renderer/src/lib/language-detect.test.ts b/src/renderer/src/lib/language-detect.test.ts index 9493bae94853..a323692e4736 100644 --- a/src/renderer/src/lib/language-detect.test.ts +++ b/src/renderer/src/lib/language-detect.test.ts @@ -177,4 +177,35 @@ describe('detectLanguage', () => { ])('detects dotenv names without overriding specific mappings: %s', (filePath, expected) => { expect(detectLanguage(filePath)).toBe(expected) }) + + it.each([ + ['/Users/me/.zshrc', 'shell'], + ['/home/me/.bashrc', 'shell'], + ['C:\\Users\\me\\.bash_profile', 'shell'], + ['/home/me/.bash_login', 'shell'], + ['/home/me/.bash_logout', 'shell'], + ['/home/me/.profile', 'shell'], + ['/home/me/.zshenv', 'shell'], + ['/home/me/.zprofile', 'shell'], + ['/home/me/.zlogin', 'shell'], + ['/home/me/.zlogout', 'shell'] + ])('maps shell startup dotfiles to shell: %s', (filePath, expected) => { + expect(detectLanguage(filePath)).toBe(expected) + }) + + it.each([ + ['/Users/me/.ZSHRC', 'shell'], + ['C:\\Users\\me\\.BASHRC', 'shell'], + ['/home/me/.Bash_Profile', 'shell'], + ['/home/me/.PROFILE', 'shell'], + ['/home/me/.ZPROFILE', 'shell'], + ['C:\\repo\\DOCKERFILE', 'dockerfile'], + ['C:\\repo\\dockerfile', 'dockerfile'], + ['C:\\repo\\MAKEFILE', 'makefile'], + ['C:\\repo\\makefile', 'makefile'], + ['C:\\repo\\CMAKELISTS.TXT', 'cmake'], + ['C:\\repo\\.GITIGNORE', 'ini'] + ])('maps exact filenames case-insensitively: %s', (filePath, expected) => { + expect(detectLanguage(filePath)).toBe(expected) + }) }) diff --git a/src/renderer/src/lib/language-detect.ts b/src/renderer/src/lib/language-detect.ts index c3d766d37100..b90b5a36ae20 100644 --- a/src/renderer/src/lib/language-detect.ts +++ b/src/renderer/src/lib/language-detect.ts @@ -126,9 +126,24 @@ const FILENAME_TO_LANGUAGE: Record = { '.env': 'ini', '.env.local': 'ini', '.env.development': 'ini', - '.env.production': 'ini' + '.env.production': 'ini', + '.bashrc': 'shell', + '.bash_profile': 'shell', + '.bash_login': 'shell', + '.bash_logout': 'shell', + '.profile': 'shell', + '.zshrc': 'shell', + '.zshenv': 'shell', + '.zprofile': 'shell', + '.zlogin': 'shell', + '.zlogout': 'shell' } +// Exact match wins; lowercase map covers case-insensitive filesystems. +const FILENAME_LOWER_TO_LANGUAGE: Record = Object.fromEntries( + Object.entries(FILENAME_TO_LANGUAGE).map(([name, language]) => [name.toLowerCase(), language]) +) + export function detectLanguage(filePath: string): string { // Check exact filename first const parts = filePath.split(/[\\/]/) @@ -136,6 +151,10 @@ export function detectLanguage(filePath: string): string { if (Object.hasOwn(FILENAME_TO_LANGUAGE, filename)) { return FILENAME_TO_LANGUAGE[filename] } + const lowerFilename = filename.toLowerCase() + if (Object.hasOwn(FILENAME_LOWER_TO_LANGUAGE, lowerFilename)) { + return FILENAME_LOWER_TO_LANGUAGE[lowerFilename] + } // Check extension const ext = extname(filename).toLowerCase() From 2caa79e097825c2e1c96906b251241384e10a1a7 Mon Sep 17 00:00:00 2001 From: FenjuFu <92919259+FenjuFu@users.noreply.github.com> Date: Thu, 1 Oct 2026 03:12:21 +0800 Subject: [PATCH 16/79] fix(source-control): drop reasoning model think blocks from generated messages (#24005) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(source-control): drop reasoning model think blocks from generated messages Custom commit-message commands that run a reasoning model print the reasoning before the answer, either as a ... block or, when the chat template prefills , as text ending in a lone . The cleaner kept all of it, so the first reasoning line became the commit subject. Strip everything through the first when the output starts with or has no before the close tag. Output that quotes both tags is left unchanged. Fixes #24004 Signed-off-by: FenjuFu * fix(source-control): scope lone stripping to custom commands and add Kimi-VL tags A lone closing tag is only stripped for custom commands, so a built-in agent's message that mentions is kept. PR fields parse the raw JSON first and strip reasoning only when that fails, so a body quoting the tag still parses. Adds Kimi-VL-Thinking's ◁think▷ tags. --------- Signed-off-by: FenjuFu Co-authored-by: Jinjing <6427696+AmethystLiang@users.noreply.github.com> --- ...e-text-generation-generated-output.test.ts | 80 +++++++++++++++++++ ...source-control-text-generation-requests.ts | 49 +++++++++++- src/shared/commit-message-agent-output.ts | 38 ++++++++- src/shared/commit-message-prompt.test.ts | 39 +++++++++ src/shared/commit-message-prompt.ts | 3 +- 5 files changed, 202 insertions(+), 7 deletions(-) diff --git a/src/main/text-generation/commit-message-text-generation-generated-output.test.ts b/src/main/text-generation/commit-message-text-generation-generated-output.test.ts index 2d9167338f58..21928f675ef4 100644 --- a/src/main/text-generation/commit-message-text-generation-generated-output.test.ts +++ b/src/main/text-generation/commit-message-text-generation-generated-output.test.ts @@ -54,6 +54,52 @@ describe('generateCommitMessageFromContext', () => { }) }) + it('drops prefilled reasoning from a custom command but not from a built-in agent', async () => { + const stdout = + "We need the message only.\r\nLet's final.Fix typo in README.md\r\n\r\n- Correct spelling\r\n" + const generate = (params: { agentId: 'custom' | 'claude'; model: string }) => + generateCommitMessageFromContext( + { branch: 'main', stagedSummary: 'M\tREADME.md', stagedPatch: '+hello' }, + { ...params, customAgentCommand: 'llama-completion' }, + { + kind: 'remote', + cwd: '/repo', + missingBinaryLocation: 'remote PATH', + execute: async () => ({ stdout, stderr: '', exitCode: 0, timedOut: false }) + } + ) + + await expect(generate({ agentId: 'custom', model: '' })).resolves.toEqual({ + success: true, + message: 'Fix typo in README.md\n\n- Correct spelling', + agentLabel: 'llama-completion' + }) + await expect(generate({ agentId: 'claude', model: 'sonnet' })).resolves.toMatchObject({ + success: true, + message: expect.stringContaining("Let's final.Fix typo in README.md") + }) + }) + + it('reports a custom command that printed only reasoning as an empty message', async () => { + const result = await generateCommitMessageFromContext( + { branch: 'main', stagedSummary: 'M\tREADME.md', stagedPatch: '+hello' }, + { agentId: 'custom', model: '', customAgentCommand: 'agent' }, + { + kind: 'remote', + cwd: '/repo', + missingBinaryLocation: 'remote PATH', + execute: async () => ({ + stdout: 'Still reasoning.\n', + stderr: '', + exitCode: 0, + timedOut: false + }) + } + ) + + expect(result).toEqual({ success: false, error: 'agent returned an empty message.' }) + }) + it('reports empty remote commit-message output as an empty message', async () => { let operation = '' const result = await generateCommitMessageFromContext( @@ -90,6 +136,40 @@ describe('generateCommitMessageFromContext', () => { }) }) + it('keeps custom-command PR JSON that quotes a lone closing tag, and strips reasoning otherwise', async () => { + const generate = (stdout: string) => + generatePullRequestFieldsFromContext( + { + branch: 'feature/pr-fields', + base: 'main', + branchChangedByPreparation: false, + currentTitle: '', + currentBody: '', + currentDraft: false, + commitSummary: '- feat: update README', + changeSummary: 'M\tREADME.md', + patch: '+hello' + }, + { agentId: 'custom', model: '', customAgentCommand: 'agent' }, + { + kind: 'remote', + cwd: '/repo', + missingBinaryLocation: 'remote PATH', + execute: async () => ({ stdout, stderr: '', exitCode: 0, timedOut: false }) + } + ) + const json = '{"title":"Strip from output","body":"Drops reasoning.","draft":false}' + + await expect(generate(json)).resolves.toMatchObject({ + success: true, + fields: { title: 'Strip from output', body: 'Drops reasoning.' } + }) + await expect(generate(`Need JSON only.\n${json}`)).resolves.toMatchObject({ + success: true, + fields: { title: 'Strip from output' } + }) + }) + it('reports empty remote pull-request field output as empty details', async () => { let operation = '' const result = await generatePullRequestFieldsFromContext( diff --git a/src/main/text-generation/source-control-text-generation-requests.ts b/src/main/text-generation/source-control-text-generation-requests.ts index 596ebc891cf7..a061201ac8c6 100644 --- a/src/main/text-generation/source-control-text-generation-requests.ts +++ b/src/main/text-generation/source-control-text-generation-requests.ts @@ -14,7 +14,12 @@ import { sanitizeBranchSlug, type BranchNameWorkContext } from '../../shared/branch-name-from-work' -import type { CommandTemplateBackslash } from '../../shared/commit-message-prompt' +import { + cleanGeneratedCommitMessage, + stripPrefilledReasoningPreamble, + type CommandTemplateBackslash +} from '../../shared/commit-message-prompt' +import { isCustomAgentId } from '../../shared/commit-message-agent-spec' import { planCommitMessageGeneration, type CommitMessagePlan @@ -56,7 +61,7 @@ async function executeGenerationPlan(input: { operation: TextGenerationOperation spawnAgent: SpawnSourceControlAgent }): Promise { - return input.target.kind === 'remote' + const result = await (input.target.kind === 'remote' ? runRemoteSourceControlPlan({ plan: input.plan, target: input.target, @@ -70,7 +75,25 @@ async function executeGenerationPlan(input: { emptyResultName: input.emptyResultName, operation: input.operation, spawnAgent: input.spawnAgent - }) + })) + // Why: only a custom command runs a raw model whose chat template can swallow + // the opening think tag; a built-in agent's message may just mention the tag. + // PR fields are JSON, so they strip only when parsing fails instead. + if ( + !result.success || + !isCustomAgentId(input.params.agentId) || + input.operation === 'pull-request-fields' + ) { + return result + } + const answer = stripPrefilledReasoningPreamble(result.rawOutput) + if (answer === result.rawOutput) { + return result + } + const rawOutput = cleanGeneratedCommitMessage(answer) + return rawOutput + ? { ...result, rawOutput } + : { success: false, error: `${input.plan.label} returned an empty ${input.emptyResultName}.` } } export async function generateCommitMessage(input: { @@ -168,7 +191,7 @@ export async function generatePullRequestFields(input: { try { return { success: true, - fields: parseGeneratedPullRequestFields(result.rawOutput, context), + fields: parsePullRequestFieldsOutput(result.rawOutput, context, params.agentId), agentLabel: result.agentLabel, branchChangedByPreparation: context.branchChangedByPreparation } @@ -181,6 +204,24 @@ export async function generatePullRequestFields(input: { } } +// Why: a body may quote a lone closing tag inside valid JSON, so reasoning is +// stripped only when the untouched output does not parse. +function parsePullRequestFieldsOutput( + raw: string, + context: PullRequestDraftContext, + agentId: GenerateParams['agentId'] +): GeneratedPullRequestFields { + try { + return parseGeneratedPullRequestFields(raw, context) + } catch (error) { + const answer = isCustomAgentId(agentId) ? stripPrefilledReasoningPreamble(raw) : raw + if (answer === raw) { + throw error + } + return parseGeneratedPullRequestFields(cleanGeneratedCommitMessage(answer), context) + } +} + export async function generateBranchName(input: { context: BranchNameWorkContext params: GenerateParams diff --git a/src/shared/commit-message-agent-output.ts b/src/shared/commit-message-agent-output.ts index 27c0d201a7df..3d17e14fb919 100644 --- a/src/shared/commit-message-agent-output.ts +++ b/src/shared/commit-message-agent-output.ts @@ -1,6 +1,6 @@ /** Strips noise around the agent's output: surrounding whitespace, a single - * enclosing fenced code block, and lone "Generating…" preamble lines some - * CLIs print before the real answer. */ + * enclosing fenced code block, lone "Generating…" preamble lines some CLIs + * print before the real answer, and a reasoning block the output opens with. */ export function cleanGeneratedCommitMessage(raw: string): string { // Why: agent output can include very large generated bodies; normalize and // unwrap by scanning boundaries instead of building newline-sized arrays. @@ -17,6 +17,8 @@ export function cleanGeneratedCommitMessage(raw: string): string { } } + text = stripLeadingReasoningBlock(text) + const fenced = findEnclosingCommitMessageFenceBody(text) if (fenced !== null) { text = fenced.trim() @@ -29,6 +31,38 @@ export function cleanGeneratedCommitMessage(raw: string): string { return text } +// Why: reasoning models print their chain of thought before the answer. +// DeepSeek-R1, Qwen3 and Kimi K2 use ; Kimi-VL-Thinking uses ◁think▷. +const REASONING_TAGS = [ + { open: '', close: '' }, + { open: '◁think▷', close: '◁/think▷' } +] as const + +function stripLeadingReasoningBlock(text: string): string { + for (const { open, close } of REASONING_TAGS) { + if (!text.startsWith(open)) { + continue + } + const closeIndex = text.indexOf(close, open.length) + return closeIndex === -1 ? text : text.slice(closeIndex + close.length).trim() + } + return text +} + +/** Drops reasoning that ends in a closing tag with no opening tag before it. + * Chat templates that prefill the opening tag in the prompt (Qwen3-Thinking, + * DeepSeek-R1-0528, Kimi K2.5) keep it out of stdout. Only custom commands + * should use this: any other message may legitimately mention the closing tag. */ +export function stripPrefilledReasoningPreamble(text: string): string { + for (const { open, close } of REASONING_TAGS) { + const closeIndex = text.indexOf(close) + if (closeIndex !== -1 && text.lastIndexOf(open, closeIndex) === -1) { + return text.slice(closeIndex + close.length).trim() + } + } + return text +} + function normalizeGeneratedCommitMessageLineFeeds(value: string): string { let crlfStart = value.indexOf('\r\n') if (crlfStart === -1) { diff --git a/src/shared/commit-message-prompt.test.ts b/src/shared/commit-message-prompt.test.ts index f080aabc5a21..7776bde7833c 100644 --- a/src/shared/commit-message-prompt.test.ts +++ b/src/shared/commit-message-prompt.test.ts @@ -5,6 +5,7 @@ import { excerptAgentFailureOutput, planCustomCommand, STAGED_DIFF_BYTE_BUDGET, + stripPrefilledReasoningPreamble, tokenizeCustomCommandTemplate, truncateDiffForPrompt } from './commit-message-prompt' @@ -134,6 +135,44 @@ describe('cleanGeneratedCommitMessage', () => { it('returns empty string when input is whitespace', () => { expect(cleanGeneratedCommitMessage(' \n\t')).toBe('') }) + + it('drops a leading reasoning block and unwraps the answer', () => { + expect( + cleanGeneratedCommitMessage('\nThe diff fixes a typo.\n\n\nFix typo\n\n- Why') + ).toBe('Fix typo\n\n- Why') + expect(cleanGeneratedCommitMessage('short\n```\nfeat: add parser\n```')).toBe( + 'feat: add parser' + ) + }) + + it('drops a leading Kimi-VL ◁think▷ reasoning block', () => { + expect(cleanGeneratedCommitMessage('◁think▷Typo in README.◁/think▷Fix typo in README')).toBe( + 'Fix typo in README' + ) + }) + + it('keeps think tags that are not a leading reasoning block', () => { + const quoted = 'Strip … blocks from generated messages' + expect(cleanGeneratedCommitMessage(quoted)).toBe(quoted) + expect(cleanGeneratedCommitMessage('fix: handle stray in stream')).toBe( + 'fix: handle stray in stream' + ) + expect(cleanGeneratedCommitMessage('still reasoning')).toBe('still reasoning') + }) +}) + +describe('stripPrefilledReasoningPreamble', () => { + it('drops reasoning that ends in a closing tag whose opening tag was prefilled', () => { + expect( + stripPrefilledReasoningPreamble("We need the message only.\nLet's final.Fix typo") + ).toBe('Fix typo') + expect(stripPrefilledReasoningPreamble('Reasoning.◁/think▷\nFix typo')).toBe('Fix typo') + }) + + it('keeps a closing tag that follows its own opening tag', () => { + const quoted = 'Strip … blocks' + expect(stripPrefilledReasoningPreamble(quoted)).toBe(quoted) + }) }) describe('excerptAgentFailureOutput', () => { diff --git a/src/shared/commit-message-prompt.ts b/src/shared/commit-message-prompt.ts index ecf4cb390694..908f25644d2c 100644 --- a/src/shared/commit-message-prompt.ts +++ b/src/shared/commit-message-prompt.ts @@ -21,7 +21,8 @@ Staged diff: export { cleanGeneratedCommitMessage, excerptAgentFailureOutput, - sanitizeAgentFailureDetail + sanitizeAgentFailureDetail, + stripPrefilledReasoningPreamble } from './commit-message-agent-output' /** Builds the final prompt sent to the agent. The custom suffix is appended verbatim From cfe4c633ebc12719b5eed9d7f6c7e6a39976d2bc Mon Sep 17 00:00:00 2001 From: Neil <4138956+nwparker@users.noreply.github.com> Date: Wed, 30 Sep 2026 12:14:13 -0700 Subject: [PATCH 17/79] fix(mobile): publish the Android APK's size and checksum with the release (#24037) An APK that fails to install with a missing certificate or a package-parse error is usually a download that died near the end: the signature block sits in the last ~100 KB of a 133 MB file, so a truncated APK looks complete and carries no signature at all. The release published neither a size nor a digest, so there was no way to tell that apart from a bad build without deriving both from the asset by hand. The release now uploads app-release.apk.sha256 next to the APK in `sha256sum -c` format (binary marker, so Git Bash cannot translate line endings while hashing) and puts the exact byte size and digest in the release body, naming `shasum -a 256 -c` for readers on macOS. The upload path rewrites the body too: --clobber replaces the APK, so a digest left over from the previous build would describe a file nobody can download, and a reader comparing against it would reject a good APK. Both paths reserve the section's own length out of the release-body cap before truncating, so the section always survives and the body always fits; MAX_RELEASE_BODY_LENGTH is exported from the desktop release script rather than restated. Refs #24011, #12248, #11444. --- .github/workflows/mobile-android-release.yml | 98 +++++++++++++++++--- config/scripts/create-draft-release.mjs | 2 +- 2 files changed, 84 insertions(+), 16 deletions(-) diff --git a/.github/workflows/mobile-android-release.yml b/.github/workflows/mobile-android-release.yml index cdb5a59b1549..b9e6e1b1b021 100644 --- a/.github/workflows/mobile-android-release.yml +++ b/.github/workflows/mobile-android-release.yml @@ -85,11 +85,44 @@ jobs: echo "Mobile shell: $EXPO_PUBLIC_MOBILE_SHELL" cd android && ./gradlew assembleRelease + # Why: an install that fails with a missing certificate or a package-parse error is + # usually a download that died near the end, and the signature block sits in the last + # ~100 KB. Publishing the size and digest is what lets a reporter tell a truncated + # download apart from a bad build without anyone re-deriving them from the asset. + - name: Checksum the APK + id: apk + run: | + set -euo pipefail + shopt -s nullglob + apks=(android/app/build/outputs/apk/release/*.apk) + if [ "${#apks[@]}" -ne 1 ]; then + echo "Expected exactly one release APK, found ${#apks[@]}" >&2 + exit 1 + fi + + apk="${apks[0]}" + bytes="$(wc -c < "$apk" | tr -d ' ')" + sha256="$(sha256sum "$apk" | cut -d ' ' -f 1)" + # A sibling file in `sha256sum -c` format, so verifying a download is one command + # and no retyped digest. The asset globs below match *.apk and skip it. The ` *` + # marker is binary mode: Git Bash's sha256sum reads text mode as a licence to + # translate line endings while hashing, which would fail on a valid APK. + printf '%s *%s\n' "$sha256" "$(basename "$apk")" > "$apk.sha256" + + { + echo "checksum=$apk.sha256" + echo "bytes=$bytes" + echo "sha256=$sha256" + } >> "$GITHUB_OUTPUT" + echo "APK is $bytes bytes, sha256 $sha256" + - name: Upload APK artifact uses: actions/upload-artifact@v7 with: name: orca-mobile-apk - path: mobile/android/app/build/outputs/apk/release/*.apk + path: | + mobile/android/app/build/outputs/apk/release/*.apk + mobile/android/app/build/outputs/apk/release/*.apk.sha256 - name: Ensure GitHub release tag if: steps.release.outputs.publish_release == 'true' && !startsWith(github.ref, 'refs/tags/mobile-android-v') @@ -112,18 +145,62 @@ jobs: run: | set -euo pipefail tag="${{ steps.release.outputs.tag }}" + notes_file="$RUNNER_TEMP/android-release-notes.md" + + # printf rather than a heredoc: YAML block indentation would leak into the Markdown. + # Both platforms' commands are named because a reader on macOS has no sha256sum. + verification_section() { + printf '\n### Verify your download\n\n- Size: `%s` bytes\n- SHA-256: `%s`\n\nA "no certificate" or "problem parsing the package" install failure is usually a truncated download — check the size first, then `sha256sum -c app-release.apk.sha256` (`shasum -a 256 -c` on macOS).\n' \ + '${{ steps.apk.outputs.bytes }}' '${{ steps.apk.outputs.sha256 }}' + } + + # The section is always last, so dropping from its heading to EOF leaves the + # generated notes intact. \r* because a body round-tripped through the API has CRLFs. + drop_verification_section() { + sed '/^### Verify your download\r*$/,$d' + } + + # Why: reuse the desktop release path's character-safe truncation so a multi-byte + # character cannot be split at the cap. The section's own length is reserved out of + # that cap, because appending after truncating would push a near-limit body past + # GitHub's API limit and fail the call — on the upload path, after --clobber has + # already replaced the assets. `wc -c` counts bytes, so the section's multi-byte + # characters over-reserve, which errs toward a shorter body. + write_notes_with_verification() { + NOTES_FILE="$notes_file" \ + NOTES_RESERVE="$(verification_section | wc -c | tr -d ' ')" \ + NOTES_MODULE="$GITHUB_WORKSPACE/config/scripts/create-draft-release.mjs" \ + node --input-type=module -e ' + const { readFileSync, writeFileSync } = await import("node:fs") + const { pathToFileURL } = await import("node:url") + const { MAX_RELEASE_BODY_LENGTH, truncateReleaseBody } = await import( + pathToFileURL(process.env.NOTES_MODULE).href + ) + const file = process.env.NOTES_FILE + const max = MAX_RELEASE_BODY_LENGTH - Number(process.env.NOTES_RESERVE) + writeFileSync(file, truncateReleaseBody(readFileSync(file, "utf8"), max)) + ' + verification_section >> "$notes_file" + } if gh release view "$tag" --repo "$GITHUB_REPOSITORY" >/dev/null 2>&1; then gh release upload "$tag" \ --repo "$GITHUB_REPOSITORY" \ --clobber \ - android/app/build/outputs/apk/release/*.apk + android/app/build/outputs/apk/release/*.apk \ + "${{ steps.apk.outputs.checksum }}" + # Why the body is rewritten too: --clobber replaced the APK, so a digest left + # over from the previous build now describes a file nobody can download, and a + # reader comparing against it would reject a good APK. + gh release view "$tag" --repo "$GITHUB_REPOSITORY" --json body --jq .body \ + | drop_verification_section > "$notes_file" + write_notes_with_verification + gh release edit "$tag" --repo "$GITHUB_REPOSITORY" --notes-file "$notes_file" else # Why: release tags live on side branches, so GitHub's automatic # previous-tag detection reaches back several releases; that body # already exceeds the 125000-character API limit and grows each # release. Pin the comparison base and cap the size. - notes_file="$RUNNER_TEMP/android-release-notes.md" previous_tag="$( gh release list --repo "$GITHUB_REPOSITORY" --limit 200 --json tagName --jq '.[].tagName' \ | grep '^mobile-android-v' | grep -Fxv "$tag" | sort -V | tail -1 || true @@ -143,17 +220,7 @@ jobs: if [ ! -s "$notes_file" ]; then printf 'Orca Mobile Android %s\n' "$tag" > "$notes_file" fi - # Why: reuse the desktop release path's character-safe truncation so a - # multi-byte character cannot be split at the cap. - NOTES_FILE="$notes_file" \ - NOTES_MODULE="$GITHUB_WORKSPACE/config/scripts/create-draft-release.mjs" \ - node --input-type=module -e ' - const { readFileSync, writeFileSync } = await import("node:fs") - const { pathToFileURL } = await import("node:url") - const { truncateReleaseBody } = await import(pathToFileURL(process.env.NOTES_MODULE).href) - const file = process.env.NOTES_FILE - writeFileSync(file, truncateReleaseBody(readFileSync(file, "utf8"))) - ' + write_notes_with_verification gh release create "$tag" \ --repo "$GITHUB_REPOSITORY" \ @@ -161,5 +228,6 @@ jobs: --prerelease \ --latest=false \ --notes-file "$notes_file" \ - android/app/build/outputs/apk/release/*.apk + android/app/build/outputs/apk/release/*.apk \ + "${{ steps.apk.outputs.checksum }}" fi diff --git a/config/scripts/create-draft-release.mjs b/config/scripts/create-draft-release.mjs index a25bced199b1..d69f4cdb8cc9 100644 --- a/config/scripts/create-draft-release.mjs +++ b/config/scripts/create-draft-release.mjs @@ -4,7 +4,7 @@ import { execFileSync } from 'node:child_process' import { pathToFileURL } from 'node:url' const API_VERSION = '2022-11-28' -const MAX_RELEASE_BODY_LENGTH = 120_000 +export const MAX_RELEASE_BODY_LENGTH = 120_000 const TRUNCATION_NOTICE = '\n\n---\nRelease notes were truncated because GitHub release bodies are limited to 125,000 characters.' const DESKTOP_RELEASE_TAG_PATTERN = /^v(\d+)\.(\d+)\.(\d+)(?:-rc\.(\d+))?$/ From 95e8725b40d67f83e5c571f144e61b284995cb5e Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 12:48:51 -0700 Subject: [PATCH 18/79] fix(claude): stop Orca making a WSL user's ~/.claude.json world-readable (#23973) * fix(claude): keep a WSL guest's ~/.claude.json mode when Orca writes folder trust * test(claude): assert a skipped trust write leaves no replacement file behind The replacement file is now created before Claude's lock is taken, so the locked path must remove it. --- ...claude-folder-trust-file-wsl-guest.test.ts | 133 ++++++++++++++++++ .../claude/claude-folder-trust-file.test.ts | 3 + src/main/claude/claude-folder-trust-file.ts | 117 ++++++++++----- 3 files changed, 217 insertions(+), 36 deletions(-) create mode 100644 src/main/claude/claude-folder-trust-file-wsl-guest.test.ts diff --git a/src/main/claude/claude-folder-trust-file-wsl-guest.test.ts b/src/main/claude/claude-folder-trust-file-wsl-guest.test.ts new file mode 100644 index 000000000000..e831b7c08eed --- /dev/null +++ b/src/main/claude/claude-folder-trust-file-wsl-guest.test.ts @@ -0,0 +1,133 @@ +import type * as NodeFs from 'node:fs' +import { + chmodSync, + mkdirSync, + mkdtempSync, + readdirSync, + readFileSync, + realpathSync, + rmSync, + statSync, + writeFileSync +} from 'node:fs' +import { tmpdir } from 'node:os' +import { join } from 'node:path' +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest' +import type { WslResult, WslSpec } from '../wsl/wsl-runner' +import { grantClaudeFolderTrust } from './claude-folder-trust-file' + +const { DISTRO, GUEST_PREFIX, guest } = vi.hoisted(() => ({ + DISTRO: 'Ubuntu-24.04', + GUEST_PREFIX: '//wsl.localhost/Ubuntu-24.04', + guest: { root: '' } +})) + +function toGuestDisk(path: string): string { + return path.startsWith(GUEST_PREFIX) ? `${guest.root}${path.slice(GUEST_PREFIX.length)}` : path +} + +// Why: models Windows Node over \\wsl.localhost: a synthetic 0o666 mode, chmod that cannot reach +// the guest bits, and new files at the guest's default 0644 whatever mode was requested. +vi.mock('node:fs', async (importOriginal) => { + const actual = await importOriginal() + const onGuest = (path: unknown): path is string => + typeof path === 'string' && path.startsWith(GUEST_PREFIX) + const disk = (path: string): string => `${guest.root}${path.slice(GUEST_PREFIX.length)}` + const windowsView = (stats: NodeFs.Stats): NodeFs.Stats => + Object.assign(stats, { mode: (stats.mode & ~0o777) | 0o666 }) + return { + ...actual, + existsSync: (path: string) => actual.existsSync(onGuest(path) ? disk(path) : path), + lstatSync: (path: string) => + onGuest(path) ? windowsView(actual.lstatSync(disk(path))) : actual.lstatSync(path), + statSync: (path: string) => + onGuest(path) ? windowsView(actual.statSync(disk(path))) : actual.statSync(path), + realpathSync: (path: string) => + onGuest(path) + ? `${GUEST_PREFIX}${actual.realpathSync(disk(path)).slice(guest.root.length)}` + : actual.realpathSync(path), + readFileSync: (path: string, options: BufferEncoding) => + actual.readFileSync(onGuest(path) ? disk(path) : path, options), + writeFileSync: (path: string, data: string, options?: NodeFs.WriteFileOptions) => { + if (!onGuest(path)) { + return actual.writeFileSync(path, data, options) + } + const created = !actual.existsSync(disk(path)) + const flag = typeof options === 'object' && options ? options.flag : undefined + actual.writeFileSync(disk(path), data, { flag }) + if (created) { + actual.chmodSync(disk(path), 0o644) + } + }, + chmodSync: (path: string, mode: number) => + onGuest(path) ? undefined : actual.chmodSync(path, mode), + renameSync: (from: string, to: string) => + actual.renameSync(onGuest(from) ? disk(from) : from, onGuest(to) ? disk(to) : to), + rmSync: (path: string, options?: NodeFs.RmOptions) => + actual.rmSync(onGuest(path) ? disk(path) : path, options) + } +}) + +vi.mock('proper-lockfile', () => ({ + lock: vi.fn(async () => async () => {}) +})) + +const runWslProcess = vi.hoisted(() => vi.fn<(spec: WslSpec) => Promise>()) +vi.mock('../wsl/wsl-runner', () => ({ runWslProcess })) + +/** Runs `chmod --reference= -- ` as the guest would. */ +async function guestChmod(spec: WslSpec): Promise { + const [reference, separator, target] = spec.args ?? [] + expect(spec).toMatchObject({ distro: DISTRO, loginPath: 'none', program: 'chmod' }) + expect(separator).toBe('--') + const from = `${guest.root}${reference.replace(/^--reference=/, '')}` + chmodSync(`${guest.root}${target}`, statSync(from).mode & 0o7777) + return { environmentResolved: true, code: 0, stdout: '', stderr: '', timedOut: false } +} + +const originalPlatform = process.platform + +describe.skipIf(originalPlatform === 'win32')('grantClaudeFolderTrust on a WSL guest file', () => { + const guestFile = `${GUEST_PREFIX}/home/u/.claude.json` + + beforeEach(() => { + guest.root = realpathSync(mkdtempSync(join(tmpdir(), 'orca-claude-trust-guest-'))) + mkdirSync(join(guest.root, 'home/u'), { recursive: true }) + writeFileSync(toGuestDisk(guestFile), JSON.stringify({ theme: 'dark' })) + chmodSync(toGuestDisk(guestFile), 0o600) + runWslProcess.mockReset().mockImplementation(guestChmod) + Object.defineProperty(process, 'platform', { configurable: true, value: 'win32' }) + }) + + afterEach(() => { + Object.defineProperty(process, 'platform', { configurable: true, value: originalPlatform }) + rmSync(guest.root, { recursive: true, force: true }) + }) + + it("keeps the guest file's owner-only mode through the rewrite", async () => { + await expect( + grantClaudeFolderTrust({ configFile: guestFile, folderKeys: ['/home/u/wt'] }) + ).resolves.toBe('granted') + expect(statSync(toGuestDisk(guestFile)).mode & 0o777).toBe(0o600) + expect(JSON.parse(readFileSync(toGuestDisk(guestFile), 'utf-8'))).toEqual({ + theme: 'dark', + projects: { '/home/u/wt': { hasTrustDialogAccepted: true } } + }) + }) + + it('leaves the file untouched when the guest cannot copy its mode', async () => { + runWslProcess.mockResolvedValue({ + environmentResolved: true, + code: 1, + stdout: '', + stderr: "chmod: unrecognized option '--reference'", + timedOut: false + }) + await expect( + grantClaudeFolderTrust({ configFile: guestFile, folderKeys: ['/home/u/wt'] }) + ).rejects.toThrow(/guest mode/) + expect(readFileSync(toGuestDisk(guestFile), 'utf-8')).toBe(JSON.stringify({ theme: 'dark' })) + expect(statSync(toGuestDisk(guestFile)).mode & 0o777).toBe(0o600) + expect(readdirSync(join(guest.root, 'home/u'))).toEqual(['.claude.json']) + }) +}) diff --git a/src/main/claude/claude-folder-trust-file.test.ts b/src/main/claude/claude-folder-trust-file.test.ts index 1c876be14574..456e8938cb6f 100644 --- a/src/main/claude/claude-folder-trust-file.test.ts +++ b/src/main/claude/claude-folder-trust-file.test.ts @@ -4,6 +4,7 @@ import { lstatSync, mkdirSync, mkdtempSync, + readdirSync, readFileSync, realpathSync, rmSync, @@ -178,6 +179,8 @@ describe('grantClaudeFolderTrust', () => { ) expect(readConfig(file)).toEqual({}) expect(existsSync(lockDir)).toBe(true) + // Why: the replacement file is created before the lock, so a skipped write must remove it. + expect(readdirSync(root).sort()).toEqual(['.claude.json', '.claude.json.lock']) }) it('updates a symlinked config through its target and keeps the link', async () => { diff --git a/src/main/claude/claude-folder-trust-file.ts b/src/main/claude/claude-folder-trust-file.ts index ca35351eeb0c..5ec9bccebccb 100644 --- a/src/main/claude/claude-folder-trust-file.ts +++ b/src/main/claude/claude-folder-trust-file.ts @@ -15,6 +15,7 @@ import { lock } from 'proper-lockfile' import { renameFileWithWindowsRetry } from '../codex-accounts/fs-utils' import { runKeyedSerializedOperation } from '../cli/keyed-promise-queue' import { parseWslUncPath } from '../../shared/wsl-paths' +import { runWslProcess } from '../wsl/wsl-runner' import type { ClaudeRuntimeAuthPreparation } from '../claude-accounts/runtime-auth/runtime-auth-types' export type ClaudeTrustPathStyle = 'posix' | 'win32' @@ -144,20 +145,52 @@ function readConfigObject(target: string): Record | null { } } -function writeConfigAtomically(target: string, config: Record): void { - const mode = statSync(target).mode & 0o777 - const tmpPath = `${target}.orca-trust-${randomUUID()}.tmp` +type ReplacementFile = { target: string; path: string } + +/** + * Creates an empty file beside `target` that already has `target`'s permission bits, so + * renaming it over `target` can never widen them. + */ +async function createReplacementFile(target: string): Promise { + const suffix = `.orca-trust-${randomUUID()}.tmp` + const path = `${target}${suffix}` + const guestFile = parseWslUncPath(target) try { - writeFileSync(tmpPath, `${JSON.stringify(config, null, 2)}\n`, { encoding: 'utf-8', mode }) - if (process.platform !== 'win32') { - // Why: umask may narrow the requested mode; the replacement must match the original exactly. - chmodSync(tmpPath, mode) + if (guestFile) { + // Why: Windows sees a synthetic 0o666 for a guest file and cannot set its bits, and the + // guest gives a file made through \\wsl.localhost its default 0644; only the guest can copy them. + writeFileSync(path, '', { flag: 'wx' }) + const copied = await runWslProcess({ + distro: guestFile.distro, + loginPath: 'none', + program: 'chmod', + args: [`--reference=${guestFile.linuxPath}`, '--', `${guestFile.linuxPath}${suffix}`] + }) + if (copied.code !== 0) { + throw new Error(`could not copy the guest mode of ${target}: ${copied.stderr.trim()}`) + } + } else { + const mode = statSync(target).mode & 0o777 + writeFileSync(path, '', { flag: 'wx', mode }) + if (process.platform !== 'win32') { + // Why: umask may narrow the requested mode; the replacement must match the original exactly. + chmodSync(path, mode) + } } - renameFileWithWindowsRetry(tmpPath, target) } catch (error) { - rmSync(tmpPath, { force: true }) + rmSync(path, { force: true }) throw error } + return { target, path } +} + +function replaceConfig(replacement: ReplacementFile, config: Record): void { + // Why r+: reopening without create/truncate keeps the mode the replacement was given. + writeFileSync(replacement.path, `${JSON.stringify(config, null, 2)}\n`, { + encoding: 'utf-8', + flag: 'r+' + }) + renameFileWithWindowsRetry(replacement.path, replacement.target) } /** @@ -188,37 +221,49 @@ async function grantClaudeFolderTrustNow(args: { return planned === 'refuse' ? 'unreadable' : 'unchanged' } - let release: () => Promise + // Why before the lock: a WSL guest's mode takes a guest process to copy, and Claude's + // lock should stay held only for the synchronous read → rename below. + const replacement = await createReplacementFile(probe.path) try { - release = await lock(args.configFile, { - // Why: Claude locks the literal `.lock`, not a realpath'd one. - lockfilePath: `${args.configFile}.lock`, - realpath: false, - stale: NEVER_STALE_MS, - retries: LOCK_RETRIES, - onCompromised: () => {} - }) - } catch { - return 'locked' - } - try { - // Why: read → rename stays synchronous so Orca's own synchronous auth writer to - // this file cannot interleave and lose an update. - const current = readConfigAt(resolveConfigTarget(args.configFile)) - if (typeof current === 'string') { - return current - } - const change = applyClaudeFolderTrust(current.config, args.folderKeys) - if (change.kind === 'refuse') { - return 'unreadable' + let release: () => Promise + try { + release = await lock(args.configFile, { + // Why: Claude locks the literal `.lock`, not a realpath'd one. + lockfilePath: `${args.configFile}.lock`, + realpath: false, + stale: NEVER_STALE_MS, + retries: LOCK_RETRIES, + onCompromised: () => {} + }) + } catch { + return 'locked' } - if (change.kind === 'unchanged') { - return 'unchanged' + try { + // Why: read → rename stays synchronous so Orca's own synchronous auth writer to + // this file cannot interleave and lose an update. + const current = readConfigAt(resolveConfigTarget(args.configFile)) + if (typeof current === 'string') { + return current + } + // Why: a link retargeted since the replacement copied its mode means Claude asks. + if (current.path !== replacement.target) { + return 'unreadable' + } + const change = applyClaudeFolderTrust(current.config, args.folderKeys) + if (change.kind === 'refuse') { + return 'unreadable' + } + if (change.kind === 'unchanged') { + return 'unchanged' + } + replaceConfig(replacement, change.config) + return 'granted' + } finally { + await release().catch(() => {}) } - writeConfigAtomically(current.path, change.config) - return 'granted' } finally { - await release().catch(() => {}) + // Why: a no-op after the rename; otherwise the unused replacement must not linger. + rmSync(replacement.path, { force: true }) } } From ed462b2caa5e280d6a56b2f9c6fed1b2bc2f86db Mon Sep 17 00:00:00 2001 From: Jinwoo Hong <73622457+Jinwoo-H@users.noreply.github.com> Date: Wed, 30 Sep 2026 17:00:03 -0400 Subject: [PATCH 19/79] fix(relay): re-place hosts off a draining cell without locking its row (#24216) * test(relay): reproduce drain-release contention against director placement Adds a Postgres harness that drives releases from an isolated cell over a 171 ms per-statement pool while five directors re-place reconnecting hosts through the sticky lane. At 18 releases/s placements fall from 20/s to about 5/s and every active director backend is blocked on relay_cells. Moves the per-statement delay pool into a shared test fixture so the rehome target-row test and this harness use one implementation. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 * fix(relay): re-place hosts off a draining cell without locking its row Sticky re-placement of a host whose cell is isolated for a roll locked every relay_cells row. During an Asia drain the source row is held by the cell's own releases for a round trip each, so the placement waited on it, the single sticky slot backed up, and /v1/assign returned 503 fleet-wide. The isolation decision now comes from an unlocked read, and the placement locks only the same-region general rows it can move to. It no longer writes the source row: the host's source leases stay, and each one's own release or expiry takes its units back off the source. The assignment keeps its activity counters and adds one control instead of resetting them. With no same-region headroom the path falls back to the all-rows lock, as before. Dormant hosts hold no units, so their placement also locks only the general rows and skips the zero write to their old cell. The drain harness now asserts the after picture: 20 placements/s at Asia latency with no director lock waits, against 9.2/s and 4.8/s before. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 * fix(relay): take a moved host's units off the cell that holds them After a narrowed re-placement a host keeps leases on its old cell while its assignment row names the new one. Two paths charged the row's whole counted total to the row's cell: aggregate expiry, and the lease deletion in dead-cell and stranded re-placement. Both over-charged the new cell and left the old cell's units stranded. Aggregate expiry now skips hosts that still hold any lease; the lease sweep takes each lease's units off its own cell. Placement frees each deleted lease's units on that lease's cell, charges the old cell only for units no lease backs, and sets the counters from the leases it keeps plus the new control. The narrowed path runs only when the counters already match the leases, so it never needs to write the old cell's row. The all-rows re-placement off an isolated cell follows the same rule, so it no longer decrements the source at placement either. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 * fix(relay): try other regions before the all-rows lock when re-placing off a roll With every same-region neighbour at its connection cap, the narrowed path found no target and fell back to the all-rows lock behind the busy source row, which is the drain brownout again. It now tries a second tier, general cells in every other region, in its own transaction over one ordered lockCellRows, still never the source row. Only when no general cell in any region has room does it fall back to the all-rows path, which keeps the pin. This changes the policy from #21911, which refused to re-place an isolated host across a region. The unit tests that encoded that rule now assert the tier order instead. The drain harness gains a US cell and an arm with every Asia neighbour capped: 200 of 200 dials placed cross-region at 20/s with no lock waits, against 0 placed and 144 sticky rejections on the previous head. Its pass bars are now the rejection share and the lock-waiting share, not the placement rate a slow runner's pacing can move. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 * fix(relay): take the narrowed path for hosts whose counters sit below their leases Main's old placement reset a moved host's counters while keeping its source leases, and those leases' releases floored the counters at zero. Such hosts hold fewer counted units than lease units, and requiring equality sent them down the all-rows path behind the busy source row. The narrowed path now requires only that the host holds no units no lease backs, the one case that needs a write to the old row. Its placement already rebuilds the counters from the kept leases plus the new control, so a drifted host is healed by its next re-placement. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 * test(relay): judge the local drain arm on completion and lock waits, not rate The local-latency arm asserted at least 18 placements/s at a 20/s dial rate, which a slow runner's pacing alone can miss. It now asserts what the Asia arm does: no dial failures, sticky rejections under 10% of dials, every other dial placed, and director lock-waiting under half a backend. Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010 --- ...isolated-cell-replacement-postgres.test.ts | 456 +++++++++++++- ...signment-isolated-cell-replacement.test.ts | 144 ++++- .../src/assignment-store-lock-order.test.ts | 2 +- cloud/apps/relay/src/assignment-store.ts | 223 ++++++- ...lease-cell-row-contention-postgres.test.ts | 579 ++++++++++++++++++ ...al-rehome-target-row-lock-postgres.test.ts | 48 +- .../delayed-postgres-database.ts | 48 ++ 7 files changed, 1422 insertions(+), 78 deletions(-) create mode 100644 cloud/apps/relay/src/drain-release-cell-row-contention-postgres.test.ts create mode 100644 cloud/apps/relay/src/test-fixtures/delayed-postgres-database.ts diff --git a/cloud/apps/relay/src/assignment-isolated-cell-replacement-postgres.test.ts b/cloud/apps/relay/src/assignment-isolated-cell-replacement-postgres.test.ts index 678c6350303a..5c6f69fabbca 100644 --- a/cloud/apps/relay/src/assignment-isolated-cell-replacement-postgres.test.ts +++ b/cloud/apps/relay/src/assignment-isolated-cell-replacement-postgres.test.ts @@ -1,3 +1,4 @@ +import { ASSIGNMENT_LIMITS } from '@orca-cloud/relay-contract' import { createHash } from 'node:crypto' import { afterAll, beforeAll, describe, expect, it } from 'vitest' import { RelayAssignmentStore } from './assignment-store.js' @@ -39,7 +40,14 @@ const TARGETS: RelayCellConfig[] = [ ...CAPPED } ] -const CELLS = [ISOLATED, ...TARGETS] +// The next tier: taken only when the source's region has no general room. +const OTHER_REGION: RelayCellConfig = { + id: 'isolated-replacement-other-region', + url: 'https://isolated-replacement-other-region.example.com', + region: 'asia-east2', + ...CAPPED +} +const CELLS = [ISOLATED, ...TARGETS, OTHER_REGION] const HOST_COUNT = 50 function hostIdentity(index: number): { userId: string; relayHostId: string } { @@ -62,7 +70,98 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => return Number(rows[0]!['reserved_requests']) } - async function heartbeatAll(): Promise { + // The accounting invariant the migration paths assert: a cell's counter is + // exactly the units of the leases held on it. + async function expectReservationAccounting(): Promise { + for (const cell of CELLS) { + const leased = await databases[0]!.query( + `SELECT COALESCE(SUM(request_units), 0) AS units + FROM relay_assignment_activity_leases WHERE cell_id = ?`, + [cell.id] + ) + expect({ cell: cell.id, reserved: await reservedRequests(cell.id) }).toEqual({ + cell: cell.id, + reserved: Number(leased[0]!['units']) + }) + } + } + + async function expectReservationAccountingExcept(cellId: string): Promise { + for (const cell of CELLS.filter(({ id }) => id !== cellId)) { + const leased = await databases[0]!.query( + `SELECT COALESCE(SUM(request_units), 0) AS units + FROM relay_assignment_activity_leases WHERE cell_id = ?`, + [cell.id] + ) + expect(await reservedRequests(cell.id)).toBe(Number(leased[0]!['units'])) + } + } + + async function counters(identity: { + userId: string + relayHostId: string + }): Promise<{ controls: number; splices: number }> { + const rows = await databases[0]!.query( + `SELECT reserved_controls, reserved_splices FROM relay_assignments + WHERE user_id = ? AND relay_host_id = ?`, + [identity.userId, identity.relayHostId] + ) + return { + controls: Number(rows[0]!['reserved_controls']), + splices: Number(rows[0]!['reserved_splices']) + } + } + + // Holds the named relay_cells rows in another session until released, the + // way a release from a far cell holds its row across a round trip. + async function holdCellRows(cellIds: string[]): Promise<() => Promise> { + let release!: () => void + const released = new Promise((resolve) => (release = resolve)) + let held!: () => void + const acquired = new Promise((resolve) => (held = resolve)) + const holder = databases[3]!.transaction(async (transaction) => { + await transaction.queryLocked( + `SELECT cell_id FROM relay_cells WHERE cell_id IN (${cellIds.map(() => '?').join(', ')}) + ORDER BY cell_id ASC`, + cellIds + ) + held() + await released + }) + await acquired + return async () => { + release() + await holder + } + } + + // Puts a host on the source with a live control and a splice, then isolates + // the source for a roll with both targets open. + async function seedActiveHostOnIsolatedSource(index: number): Promise<{ + identity: { userId: string; relayHostId: string } + first: { cellId: string; assignmentEpoch: number } + }> { + const identity = hostIdentity(index) + await applySelector({ [TARGETS[0]!.id]: 'migration-only', [TARGETS[1]!.id]: 'migration-only' }) + const first = await stores[0]!.assign(identity, 'us-central1') + expect(first.cellId).toBe(ISOLATED.id) + await stores[0]!.activateControl(identity, { + cellId: ISOLATED.id, + assignmentEpoch: first.assignmentEpoch, + generation: 1 + }) + await stores[0]!.acquireActivity(identity, { + activityId: 'splice:keep-1', + kind: 'splice', + cellId: ISOLATED.id + }) + await applySelector({ [TARGETS[0]!.id]: 'general', [TARGETS[1]!.id]: 'general' }) + await applySelector({ [ISOLATED.id]: 'migration-only' }, [ISOLATED.id]) + return { identity, first } + } + + // `enforced` puts a cell at its connection cap, which leaves it no headroom. + async function heartbeatAll(enforced: Record = {}): Promise { for (const [index, cell] of CELLS.entries()) { await stores[0]!.recordCellHeartbeat({ cellId: cell.id, @@ -72,10 +171,11 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => ready: true, observedRequests: 0, region: cell.region, - totalConnections: 0, + // The cell checks enforced = total + in-flight + reserved. + totalConnections: enforced[cell.id] ?? 0, inFlightConnections: 0, reservedConnectionUnits: 0, - enforcedConnectionUnits: 0, + enforcedConnectionUnits: enforced[cell.id] ?? 0, connectionHardCap: 600, connectionUnobservedBound: 50 }) @@ -113,7 +213,11 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => else membership.general.push(cellId) } await stores[0]!.applyCellAdmissionSelector({ - attemptId: `isolated-${current.selector.generation}-${Object.keys(states).join('-')}`, + // The id is capped at 128 characters; the generation already makes it unique. + attemptId: `isolated-${current.selector.generation}-${createHash('sha256') + .update(Object.keys(states).join('-')) + .digest('hex') + .slice(0, 16)}`, expectedGeneration: current.selector.generation, ...(current.selector.generation === 0 ? { @@ -140,6 +244,12 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => // isolated would otherwise starve the next one of placement candidates. async function resetFleet(): Promise { await deleteHostRows() + // Only this file's hosts hold leases on these cells, so zero is exact. + await databases[0]!.query( + `UPDATE relay_cells SET reserved_requests = 0 + WHERE cell_id IN (${CELLS.map(() => '?').join(', ')})`, + CELLS.map((cell) => cell.id) + ) await applySelector(Object.fromEntries(CELLS.map((cell) => [cell.id, 'general']))) } @@ -293,10 +403,23 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => expect(grant.assignmentEpoch).toBe(first.get(identity.relayHostId)!.assignmentEpoch + 1) } - expect(await reservedRequests(ISOLATED.id)).toBe(sourceBaseline) + // The placement leaves the source row alone: each host's pending control + // there still counts until the source releases it. + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBaseline + HOST_COUNT) expect( (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) ).toBe(targetBaseline + HOST_COUNT) + await expectReservationAccounting() + for (const identity of identities) { + expect( + await stores[0]!.releaseActivity( + identity, + `control-pending:${first.get(identity.relayHostId)!.assignmentEpoch}` + ) + ).toBe(true) + } + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBaseline) + await expectReservationAccounting() const rows = await databases[0]!.query( `SELECT COUNT(*) AS count FROM relay_assignments @@ -331,13 +454,13 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => expect(granted.length).toBeGreaterThan(0) // However many dials were granted, only one re-placement may have - // committed: the epoch advances by exactly one and the reservation moves - // exactly one unit. + // committed: the epoch advances by exactly one and the target gains exactly + // one unit. The source keeps its lease until the source releases it. const settled = await stores[0]!.resolve(identity) expect(settled?.assignmentEpoch).toBe(first.assignmentEpoch + 1) expect(settled?.cellId).not.toBe(ISOLATED.id) for (const grant of granted) expect(grant.cellId).toBe(settled?.cellId) - expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore - 1) + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore) expect( (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) ).toBe(targetBaseline + 1) @@ -361,4 +484,319 @@ describePostgres('PostgreSQL re-placement off a cell isolated for a roll', () => first.assignmentEpoch + 1 ) }, 30_000) + it('leaves the source row to the source’s own releases and keeps the host’s counters', async () => { + await resetFleet() + const { identity, first } = await seedActiveHostOnIsolatedSource(910) + const sourceBefore = await reservedRequests(ISOLATED.id) + const targetsBefore = + (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) + expect(sourceBefore).toBe(3) + expect(await counters(identity)).toEqual({ controls: 1, splices: 1 }) + + const moved = await stores[1]!.assign(identity, 'us-central1') + expect(TARGETS.map(({ id }) => id)).toContain(moved.cellId) + expect(moved.assignmentEpoch).toBe(first.assignmentEpoch + 1) + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore) + expect( + (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) + ).toBe(targetsBefore + 1) + // One control added on the target; the source's control and splice kept. + expect(await counters(identity)).toEqual({ controls: 2, splices: 1 }) + await expectReservationAccounting() + + // The source closes the host's sockets: each release takes its own units. + expect(await stores[0]!.releaseActivity(identity, `control:${ISOLATED.id}:1`)).toBe(true) + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore - 1) + expect(await counters(identity)).toEqual({ controls: 1, splices: 1 }) + expect(await stores[0]!.releaseActivity(identity, 'splice:keep-1')).toBe(true) + expect(await reservedRequests(ISOLATED.id)).toBe(0) + // The target's control survives the source's releases. + expect(await counters(identity)).toEqual({ controls: 1, splices: 0 }) + await expectReservationAccounting() + }, 30_000) + + it('re-places an isolated incumbent while the source row is held', async () => { + await resetFleet() + const { identity, first } = await seedActiveHostOnIsolatedSource(911) + const sourceBefore = await reservedRequests(ISOLATED.id) + const release = await holdCellRows([ISOLATED.id]) + try { + // Held far past the 500ms bounded wait: a placement that asked for the + // source row could not finish inside this window. + const moved = await Promise.race([ + stores[1]!.assign(identity, 'us-central1'), + new Promise<'blocked'>((resolve) => setTimeout(() => resolve('blocked'), 3_000)) + ]) + expect(moved).not.toBe('blocked') + expect(moved).toMatchObject({ assignmentEpoch: first.assignmentEpoch + 1 }) + } finally { + await release() + } + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore) + await expectReservationAccounting() + }, 30_000) + + it('waits for the target rows it increments', async () => { + await resetFleet() + const { identity, first } = await seedActiveHostOnIsolatedSource(912) + const targetsBefore = + (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) + const release = await holdCellRows(TARGETS.map(({ id }) => id)) + let settled = false + const placement = stores[1]!.assign(identity, 'us-central1').finally(() => { + settled = true + }) + await new Promise((resolve) => setTimeout(resolve, 1_000)) + expect(settled).toBe(false) + await release() + expect(await placement).toMatchObject({ assignmentEpoch: first.assignmentEpoch + 1 }) + expect( + (await reservedRequests(TARGETS[0]!.id)) + (await reservedRequests(TARGETS[1]!.id)) + ).toBe(targetsBefore + 1) + await expectReservationAccounting() + }, 30_000) + + it('keeps the pin when no general cell in any region has headroom', async () => { + await resetFleet() + const { identity, first } = await seedActiveHostOnIsolatedSource(913) + const sourceBefore = await reservedRequests(ISOLATED.id) + await databases[0]!.query( + `UPDATE relay_cells SET reserved_requests = capacity_requests WHERE cell_id IN (?, ?, ?)`, + [...TARGETS, OTHER_REGION].map(({ id }) => id) + ) + try { + expect(await stores[1]!.assign(identity, 'us-central1')).toMatchObject({ + cellId: ISOLATED.id, + assignmentEpoch: first.assignmentEpoch + }) + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore) + expect(await counters(identity)).toEqual({ controls: 1, splices: 1 }) + } finally { + await databases[0]!.query( + `UPDATE relay_cells SET reserved_requests = 0 WHERE cell_id IN (?, ?, ?)`, + [...TARGETS, OTHER_REGION].map(({ id }) => id) + ) + } + }, 30_000) + + it('moves to another region when its own is at the connection cap, without the source row', async () => { + await resetFleet() + const { identity, first } = await seedActiveHostOnIsolatedSource(917) + const sourceBefore = await reservedRequests(ISOLATED.id) + await heartbeatAll(Object.fromEntries(TARGETS.map(({ id }) => [id, 600]))) + const release = await holdCellRows([ISOLATED.id]) + try { + const moved = await Promise.race([ + stores[1]!.assign(identity, 'us-central1'), + new Promise<'blocked'>((resolve) => setTimeout(() => resolve('blocked'), 3_000)) + ]) + expect(moved).toMatchObject({ + cellId: OTHER_REGION.id, + region: 'asia-east2', + assignmentEpoch: first.assignmentEpoch + 1 + }) + } finally { + await release() + await heartbeatAll() + } + expect(await reservedRequests(ISOLATED.id)).toBe(sourceBefore) + expect(await reservedRequests(OTHER_REGION.id)).toBe(1) + await expectReservationAccounting() + }, 30_000) + it('places a dormant host without asking for its old cell row', async () => { + await resetFleet() + const identity = hostIdentity(914) + await applySelector({ [TARGETS[0]!.id]: 'migration-only', [TARGETS[1]!.id]: 'migration-only' }) + const first = await stores[0]!.assign(identity, 'us-central1') + expect(first.cellId).toBe(ISOLATED.id) + // Dormant: no units held and no activity inside the dormancy window. + await databases[0]!.query( + `DELETE FROM relay_assignment_activity_leases WHERE user_id = ? AND relay_host_id = ?`, + [identity.userId, identity.relayHostId] + ) + await databases[0]!.query( + `UPDATE relay_assignments SET reserved_controls = 0, last_activity_at = ? + WHERE user_id = ? AND relay_host_id = ?`, + [NOW - ASSIGNMENT_LIMITS.dormantTtlMs, identity.userId, identity.relayHostId] + ) + await databases[0]!.query(`UPDATE relay_cells SET reserved_requests = 0 WHERE cell_id = ?`, [ + ISOLATED.id + ]) + await applySelector({ [TARGETS[0]!.id]: 'general', [TARGETS[1]!.id]: 'general' }) + await applySelector({ [ISOLATED.id]: 'migration-only' }, [ISOLATED.id]) + + const release = await holdCellRows([ISOLATED.id]) + try { + const moved = await Promise.race([ + stores[1]!.assign(identity, 'us-central1'), + new Promise<'blocked'>((resolve) => setTimeout(() => resolve('blocked'), 3_000)) + ]) + expect(moved).toMatchObject({ assignmentEpoch: first.assignmentEpoch + 1 }) + expect(TARGETS.map(({ id }) => id)).toContain(moved === 'blocked' ? moved : moved.cellId) + } finally { + await release() + } + await expectReservationAccounting() + }, 30_000) + // Moves a seeded host onto TARGETS[0] next to another host whose splice there + // outlives the test, so an over-charge shows instead of clamping at zero. + async function seedMovedHostBesideNeighbour(index: number): Promise<{ + identity: { userId: string; relayHostId: string } + neighbour: { userId: string; relayHostId: string } + }> { + const { identity } = await seedActiveHostOnIsolatedSource(index) + await applySelector({ [TARGETS[1]!.id]: 'migration-only' }) + const neighbour = hostIdentity(index + 50) + expect((await stores[0]!.assign(neighbour, 'us-central1')).cellId).toBe(TARGETS[0]!.id) + await stores[0]!.acquireActivity(neighbour, { + activityId: 'splice:neighbour', + kind: 'splice', + cellId: TARGETS[0]!.id, + expiresAt: NOW + 10 * 60 * 60_000 + }) + expect((await stores[1]!.assign(identity, 'us-central1')).cellId).toBe(TARGETS[0]!.id) + expect(await counters(identity)).toEqual({ controls: 2, splices: 1 }) + await expectReservationAccounting() + return { identity, neighbour } + } + + it('leaves a host with leases on two cells to the lease sweep', async () => { + await resetFleet() + const { identity } = await seedMovedHostBesideNeighbour(915) + // Past every lease but the neighbour's splice. + const later = new RelayAssignmentStore(databases[0]!, () => NOW + 60 * 60_000, { + requireLiveCells: true, + heartbeatTtlMs: 45_000 + }) + try { + // Aggregate expiry first, with the source's leases still present: the + // order a lease sweep skipped on a busy row leaves behind. + await later.releaseExpiredActivity() + expect(await counters(identity)).toEqual({ controls: 2, splices: 1 }) + await expectReservationAccounting() + await later.releaseExpiredActivityLeases() + await later.releaseExpiredActivity() + expect(await counters(identity)).toEqual({ controls: 0, splices: 0 }) + expect(await reservedRequests(ISOLATED.id)).toBe(0) + // Only the neighbour's splice is left on the target. + expect(await reservedRequests(TARGETS[0]!.id)).toBe(2) + await expectReservationAccounting() + } finally { + await applySelector({ [TARGETS[1]!.id]: 'general' }) + } + }, 30_000) + + it('frees the source leases on their own cell when the target dies', async () => { + await resetFleet() + const { identity } = await seedMovedHostBesideNeighbour(916) + // The roll finishes, then the target stops heartbeating before the source + // has released the host. Uncapped, so no fence is needed to move off it. + await applySelector({ [ISOLATED.id]: 'general' }) + await databases[0]!.query(`UPDATE relay_cell_runtime SET ready = 0 WHERE cell_id = ?`, [ + TARGETS[0]!.id + ]) + await databases[0]!.query(`DELETE FROM relay_cell_connection_limits WHERE cell_id = ?`, [ + TARGETS[0]!.id + ]) + try { + const moved = await stores[1]!.assign(identity, 'us-central1') + expect(moved.cellId).toBe(ISOLATED.id) + expect(await counters(identity)).toEqual({ controls: 1, splices: 0 }) + // The deleted source leases took their units with them; a late release + // from the source finds nothing and charges nothing. + expect(await stores[0]!.releaseActivity(identity, `control:${ISOLATED.id}:1`)).toBe(false) + expect(await reservedRequests(ISOLATED.id)).toBe(1) + expect(await reservedRequests(TARGETS[0]!.id)).toBe(3) + await expectReservationAccounting() + } finally { + // Re-registers the connection limit, then the heartbeat restores ready. + await stores[0]!.reconcileCells(CELLS, false) + await heartbeatAll() + await applySelector({ [TARGETS[1]!.id]: 'general' }) + } + }, 30_000) + // A host main's old placement left behind: its counters were reset while its + // source lease stayed, and the lease's release floored them at zero. + async function seedDriftedHost( + index: number, + counted: { controls: number; unbackedSourceUnits: number } + ): Promise<{ identity: { userId: string; relayHostId: string }; epoch: number }> { + const identity = hostIdentity(index) + await applySelector({ [TARGETS[0]!.id]: 'migration-only', [TARGETS[1]!.id]: 'migration-only' }) + const first = await stores[0]!.assign(identity, 'us-central1') + expect(first.cellId).toBe(ISOLATED.id) + await stores[0]!.activateControl(identity, { + cellId: ISOLATED.id, + assignmentEpoch: first.assignmentEpoch, + generation: 1 + }) + await databases[0]!.query( + `UPDATE relay_assignments SET reserved_controls = ? WHERE user_id = ? AND relay_host_id = ?`, + [counted.controls, identity.userId, identity.relayHostId] + ) + // Units the source still counts with no lease behind them. + await databases[0]!.query( + `UPDATE relay_cells SET reserved_requests = reserved_requests + ? WHERE cell_id = ?`, + [counted.unbackedSourceUnits, ISOLATED.id] + ) + await applySelector({ [TARGETS[0]!.id]: 'general', [TARGETS[1]!.id]: 'general' }) + await applySelector({ [ISOLATED.id]: 'migration-only' }, [ISOLATED.id]) + return { identity, epoch: first.assignmentEpoch } + } + + it('heals counters below the leases on the narrowed path', async () => { + await resetFleet() + const { identity, epoch } = await seedDriftedHost(918, { controls: 0, unbackedSourceUnits: 0 }) + expect(await counters(identity)).toEqual({ controls: 0, splices: 0 }) + await expectReservationAccounting() + const release = await holdCellRows([ISOLATED.id]) + try { + const moved = await Promise.race([ + stores[1]!.assign(identity, 'us-central1'), + new Promise<'blocked'>((resolve) => setTimeout(() => resolve('blocked'), 3_000)) + ]) + expect(moved).toMatchObject({ assignmentEpoch: epoch + 1 }) + expect(TARGETS.map(({ id }) => id)).toContain(moved === 'blocked' ? moved : moved.cellId) + } finally { + await release() + } + // The kept source control plus the new one. + expect(await counters(identity)).toEqual({ controls: 2, splices: 0 }) + await expectReservationAccounting() + expect(await stores[0]!.releaseActivity(identity, `control:${ISOLATED.id}:1`)).toBe(true) + expect(await counters(identity)).toEqual({ controls: 1, splices: 0 }) + expect(await reservedRequests(ISOLATED.id)).toBe(0) + await expectReservationAccounting() + }, 30_000) + + it('takes the all-rows path for units no lease backs', async () => { + await resetFleet() + const { identity, epoch } = await seedDriftedHost(919, { controls: 3, unbackedSourceUnits: 2 }) + await expectReservationAccountingExcept(ISOLATED.id) + const release = await holdCellRows([ISOLATED.id]) + const held = stores[1]!.assign(identity, 'us-central1').then( + () => 'placed' as const, + () => 'failed' as const + ) + try { + // It needs the source row to take the unbacked units off it, so while the + // row is held it either waits or times out; it never places. + expect( + await Promise.race([ + held, + new Promise<'waiting'>((resolve) => setTimeout(() => resolve('waiting'), 1_000)) + ]) + ).not.toBe('placed') + } finally { + await release() + } + await held + // Whether the held dial finished after the release or failed, the next one + // lands the single move. + expect(await stores[1]!.assign(identity, 'us-central1')).toMatchObject({ + assignmentEpoch: epoch + 1 + }) + expect(await counters(identity)).toEqual({ controls: 2, splices: 0 }) + await expectReservationAccounting() + }, 30_000) }) diff --git a/cloud/apps/relay/src/assignment-isolated-cell-replacement.test.ts b/cloud/apps/relay/src/assignment-isolated-cell-replacement.test.ts index cc1ae6a26bb0..ccb6ee8ec877 100644 --- a/cloud/apps/relay/src/assignment-isolated-cell-replacement.test.ts +++ b/cloud/apps/relay/src/assignment-isolated-cell-replacement.test.ts @@ -463,9 +463,7 @@ describe('re-placing a host off a cell isolated for a roll', () => { }) }) - it('never re-places across a region boundary, and says so', async () => { - // Both US cells parked and only Asia general: ordinary placement would spill - // to asia-east2 through `preferred[0] ?? candidates[0]`. This path refuses. + it('crosses a region boundary only when its own region has no room', async () => { const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) const { store, isolateForRoll, park } = await setup() const first = await store.assign(IDENTITY, 'us-central1') @@ -473,6 +471,32 @@ describe('re-placing a host off a cell isolated for a roll', () => { await park(other.id, 'migration-only') await isolateForRoll(first.cellId) + warn.mockClear() + expect(await store.assign(IDENTITY, 'us-central1')).toMatchObject({ + cellId: 'asia-c1', + assignmentEpoch: first.assignmentEpoch + 1 + }) + expect(jsonEvents(warn, 'orca_relay_sticky_replaced_off_isolated_cell')).toEqual([ + { + event: 'orca_relay_sticky_replaced_off_isolated_cell', + fromCellId: first.cellId, + fromRegion: 'us-central1', + admissionState: 'migration-only', + toCellId: 'asia-c1', + region: 'asia-east2' + } + ]) + }) + + it('keeps the pin, and says so, when no region has a general cell with room', async () => { + const warn = vi.spyOn(console, 'warn').mockImplementation(() => {}) + const { store, isolateForRoll, park } = await setup() + const first = await store.assign(IDENTITY, 'us-central1') + for (const cell of CELLS) { + if (cell.id !== first.cellId) await park(cell.id, 'migration-only') + } + await isolateForRoll(first.cellId) + warn.mockClear() expect(await store.assign(IDENTITY, 'us-central1')).toMatchObject({ cellId: first.cellId, @@ -489,17 +513,25 @@ describe('re-placing a host off a cell isolated for a roll', () => { expect(jsonEvents(warn, 'orca_relay_sticky_replaced_off_isolated_cell')).toEqual([]) }) - it('keeps a re-placed host in its own region', async () => { + it('prefers its own region over a less loaded cell elsewhere', async () => { + const { store, isolateForRoll } = await setup() + const first = await store.assign(IDENTITY, 'us-central1') + await isolateForRoll(first.cellId) + const moved = await store.assign(IDENTITY, 'us-central1') + expect(moved.region).toBe('us-central1') + expect(moved.cellId).not.toBe(first.cellId) + }) + + it('moves the only cell of a region to another region', async () => { const { store, isolateForRoll } = await setup() const first = await store.assign(IDENTITY, 'asia-east2') expect(first.region).toBe('asia-east2') await isolateForRoll(first.cellId) // asia-c1 is the only Asia cell, so the only in-region candidate is gone. - expect(await store.assign(IDENTITY, 'asia-east2')).toMatchObject({ - cellId: first.cellId, - assignmentEpoch: first.assignmentEpoch - }) + const moved = await store.assign(IDENTITY, 'asia-east2') + expect(moved.region).toBe('us-central1') + expect(moved.assignmentEpoch).toBe(first.assignmentEpoch + 1) }) it('takes the dead-cell path when a stamped cell stops heartbeating', async () => { @@ -606,4 +638,100 @@ describe('re-placing a host off a cell isolated for a roll', () => { } ]) }) + describe('a host whose leases span two cells after the move', () => { + const NEIGHBOUR = { userId: 'user-b', relayHostId: 'host000000000002' } + + async function reserved(database: RelayDatabase, cellId: string): Promise { + const rows = await database.query( + `SELECT reserved_requests FROM relay_cells WHERE cell_id = ?`, + [cellId] + ) + return Number(rows[0]!['reserved_requests']) + } + + async function expectAccounting(database: RelayDatabase): Promise { + for (const cell of CELLS) { + const leased = await database.query( + `SELECT COALESCE(SUM(request_units), 0) AS units + FROM relay_assignment_activity_leases WHERE cell_id = ?`, + [cell.id] + ) + expect({ cell: cell.id, reserved: await reserved(database, cell.id) }).toEqual({ + cell: cell.id, + reserved: Number(leased[0]!['units']) + }) + } + } + + async function counters(database: RelayDatabase): Promise { + const rows = await database.query( + `SELECT reserved_controls, reserved_splices FROM relay_assignments + WHERE user_id = ? AND relay_host_id = ?`, + [IDENTITY.userId, IDENTITY.relayHostId] + ) + return [Number(rows[0]!['reserved_controls']), Number(rows[0]!['reserved_splices'])] + } + + // us-c1 is isolated with the host's control and splice on it; the host + // moves to us-c2, where a neighbour's splice outlives every other lease. + async function movedHost(): Promise { + const harness = await setup() + const { store, isolateForRoll } = harness + const first = await store.assign(IDENTITY, 'us-central1') + expect(first.cellId).toBe('us-c1') + await store.activateControl(IDENTITY, { + cellId: 'us-c1', + assignmentEpoch: first.assignmentEpoch, + generation: 1 + }) + await store.acquireActivity(IDENTITY, { + activityId: 'splice:keep-1', + kind: 'splice', + cellId: 'us-c1' + }) + await isolateForRoll('us-c1') + expect((await store.assign(NEIGHBOUR, 'us-central1')).cellId).toBe('us-c2') + await store.acquireActivity(NEIGHBOUR, { + activityId: 'splice:neighbour', + kind: 'splice', + cellId: 'us-c2', + expiresAt: START_MS + 10 * 60 * 60_000 + }) + expect((await store.assign(IDENTITY, 'us-central1')).cellId).toBe('us-c2') + expect(await counters(harness.database)).toEqual([2, 1]) + await expectAccounting(harness.database) + return harness + } + + it('is left to the lease sweep by aggregate expiry', async () => { + const { store, database, setNow } = await movedHost() + setNow(START_MS + 60 * 60_000) + await store.releaseExpiredActivity() + expect(await counters(database)).toEqual([2, 1]) + await expectAccounting(database) + await store.releaseExpiredActivityLeases() + await store.releaseExpiredActivity() + expect(await counters(database)).toEqual([0, 0]) + expect(await reserved(database, 'us-c1')).toBe(0) + expect(await reserved(database, 'us-c2')).toBe(2) + await expectAccounting(database) + }) + + it('frees the source leases on their own cell when the target dies', async () => { + const { store, database, setNow, heartbeat, restore } = await movedHost() + await restore('us-c1') + // Uncapped, so the dead-cell path needs no fence to move off it. + await database.query(`DELETE FROM relay_cell_connection_limits WHERE cell_id = 'us-c2'`) + setNow(START_MS + HEARTBEAT_TTL_MS + 5_000) + await heartbeat(CELLS[0]!) + await heartbeat(CELLS[2]!) + + expect((await store.assign(IDENTITY, 'us-central1')).cellId).toBe('us-c1') + expect(await counters(database)).toEqual([1, 0]) + expect(await store.releaseActivity(IDENTITY, 'control:us-c1:1')).toBe(false) + expect(await reserved(database, 'us-c1')).toBe(1) + expect(await reserved(database, 'us-c2')).toBe(3) + await expectAccounting(database) + }) + }) }) diff --git a/cloud/apps/relay/src/assignment-store-lock-order.test.ts b/cloud/apps/relay/src/assignment-store-lock-order.test.ts index ea04e016a43f..8379c9177b6f 100644 --- a/cloud/apps/relay/src/assignment-store-lock-order.test.ts +++ b/cloud/apps/relay/src/assignment-store-lock-order.test.ts @@ -480,7 +480,7 @@ describe('RelayAssignmentStore activity lock order', () => { 'cell-b', 'assignment', 'cell-inventory', - 'cell-b', + // No 'cell-b' write: the host holds no units, so there is nothing to release. 'cell-a' ]) }) diff --git a/cloud/apps/relay/src/assignment-store.ts b/cloud/apps/relay/src/assignment-store.ts index 9a5cd9d4165c..e45d15d33575 100644 --- a/cloud/apps/relay/src/assignment-store.ts +++ b/cloud/apps/relay/src/assignment-store.ts @@ -426,7 +426,16 @@ const REGIONAL_REHOME_QUARANTINE_MEMORY_LIMIT = 1_000 const REGIONAL_REHOME_FAILURE_BUDGET = 3 const REGIONAL_REHOME_OBSERVATION_MS = 24 * 60 * 60_000 const ASSIGNMENT_LOCK_RETRY_MAX_DELAY_MS = 50 -type AssignmentInventoryScope = 'none' | 'general' | 'all' +// 'isolated' and 'isolated-other-regions': the general rows a roll-isolated +// incumbent can move to, its own region first, never its own row. The source +// row is the one a drain's releases keep busy. +type AssignmentInventoryScope = + | 'none' + | 'general' + | 'isolated' + | 'isolated-other-regions' + | 'all' +type IsolatedReplacementTier = 'same-region' | 'other-regions' type RetriedAssignmentInventoryScope = Exclude class AssignmentInventoryLockUnavailable extends Error { @@ -435,6 +444,13 @@ class AssignmentInventoryLockUnavailable extends Error { } } +// Postgres holds row locks to COMMIT, so the next tier needs a new transaction. +class AssignmentIsolatedTierExhausted extends Error { + constructor() { + super('assignment_isolated_tier_exhausted') + } +} + class AssignmentInventoryScopeChanged extends Error { constructor() { super('assignment_inventory_scope_changed') @@ -702,6 +718,10 @@ export class RelayAssignmentStore { inventoryScope = 'all' continue } + if (error instanceof AssignmentIsolatedTierExhausted) { + inventoryScope = 'isolated-other-regions' + continue + } if ( !(error instanceof AssignmentInventoryLockUnavailable) || Date.now() >= deadline @@ -938,19 +958,39 @@ export class RelayAssignmentStore { // The retry paths below open with the inventory, so this path takes its // host rows before any of them rather than where the others do. await this.lockControlConnectionReservations(transaction, identity, lockMode) + const isolatedTier: IsolatedReplacementTier = + inventoryScope === 'isolated-other-regions' ? 'other-regions' : 'same-region' + const isolatedScope = inventoryScope === 'isolated' || inventoryScope === 'isolated-other-regions' + const isolatedRetrySource = isolatedScope + ? await this.pinnedCellId(transaction, identity) + : undefined let lockedCells = inventoryScope === 'all' ? await this.lockCellInventory(transaction, lockMode) : inventoryScope === 'general' ? await this.lockGeneralCellInventory(transaction, lockMode) - : undefined + : isolatedScope + ? await this.lockIsolatedReplacementCells( + transaction, + isolatedRetrySource, + isolatedTier, + lockMode + ) + : undefined const existing = await this.assignmentRow( transaction, identity, inventoryScope !== 'none' ) - retryScope = existing ? 'all' : 'general' - if (inventoryScope === 'general' && existing) { + // A dormant host holds no units, so its placement never writes its old + // cell and needs only the rows it could land on. + const dormant = !existing || mayNormallyReassign(activity(existing), now) + retryScope = dormant ? 'general' : 'all' + if ( + (inventoryScope === 'general' && !dormant) || + (isolatedScope && + (!existing || dormant || text(existing, 'cell_id') !== isolatedRetrySource)) + ) { throw new AssignmentInventoryScopeChanged() } const activityLeases = await this.lockAssignmentActivities(transaction, identity, true) @@ -960,7 +1000,46 @@ export class RelayAssignmentStore { let strandedReassignment = false let isolatedIncumbent: CellRow | undefined let isolatedTarget: CellRow | undefined - if (existing && !mayNormallyReassign(activity(existing), now)) { + // Set when only the target rows are locked. The old cell's row is never + // written then, so the host must hold no units that no lease backs. + // Counters below the leases (drift the old reset left) are fine: the + // write below rebuilds them from the leases. + let sourceRowSkipped = false + if (existing && !dormant && inventoryScope !== 'all') { + // Unlocked on purpose: this only chooses which rows to lock. A restore + // racing it lets one host move off a cell that no longer needed it. + if ( + requestUnits(existing) <= leaseRequestUnits(activityLeases) && + (await this.isolatedIncumbentMayMove(transaction, identity, existing, now)) + ) { + const source = await this.unlockedCell(transaction, text(existing, 'cell_id')) + retryScope = isolatedTier === 'same-region' ? 'isolated' : 'isolated-other-regions' + lockedCells ??= await this.lockIsolatedReplacementCells( + transaction, + source.cellId, + isolatedTier, + 'nowait' + ) + isolatedTarget = + (await this.leastLoadedCell( + transaction, + lockedCells, + source.region, + isolatedTier === 'same-region' ? 'require' : 'prefer' + )) ?? undefined + // Same region full: try the other regions. All full: decide under the + // all-rows lock, which keeps the pin as before. + if (!isolatedTarget) { + if (isolatedTier === 'same-region') throw new AssignmentIsolatedTierExhausted() + throw new AssignmentInventoryScopeChanged() + } + isolatedIncumbent = source + sourceRowSkipped = true + } else if (isolatedScope) { + throw new AssignmentInventoryScopeChanged() + } + } + if (existing && !dormant && !sourceRowSkipped) { lockedCells ??= await this.lockCellInventory(transaction, 'nowait') const admission = await cellAdmissionStates(transaction) const currentRow = lockedCells.find( @@ -1006,9 +1085,9 @@ export class RelayAssignmentStore { if (isolatedTarget) { isolatedIncumbent = current } else { - // Keeping the pin is today's behaviour: the host keeps retrying its - // own cell. Scattering a region across the fleet is worse, and it - // cannot be undone without the rehome worker. + // Keep the pin: the host retries its own cell. The narrowed path + // has already tried every region, or declined because the host + // holds units no lease backs; the reason string is a log contract. events.push( JSON.stringify({ event: 'orca_relay_sticky_replacement_deferred', @@ -1089,9 +1168,8 @@ export class RelayAssignmentStore { } } - lockedCells ??= existing - ? await this.lockCellInventory(transaction, 'nowait') - : await this.lockGeneralCellInventory(transaction, 'nowait') + // Only a host with no assignment or a dormant one reaches here unlocked. + lockedCells ??= await this.lockGeneralCellInventory(transaction, 'nowait') // The isolated target was chosen under the same inventory lock, with // region required rather than preferred; re-picking here would reopen the // cross-region spill it exists to refuse. @@ -1113,14 +1191,25 @@ export class RelayAssignmentStore { }) ) } - const previousUnits = existing ? requestUnits(existing) : 0 + // Units come off the cell that holds them. A lease this placement deletes + // frees its units on its own cell; a lease it keeps stays counted until + // its own release or expiry. Only units no lease backs are charged to + // the old cell, which is locked whenever there are any. + const deletesLeases = forcedDeadReassignment || strandedReassignment + const keptLeases = deletesLeases ? [] : activityLeases if (existing) { - await this.adjustCellReservation(transaction, text(existing, 'cell_id'), -previousUnits) - if (forcedDeadReassignment || strandedReassignment) { + const unbackedUnits = requestUnits(existing) - leaseRequestUnits(activityLeases) + if (unbackedUnits > 0) { + await this.adjustCellReservation(transaction, text(existing, 'cell_id'), -unbackedUnits) + } + if (deletesLeases) { // A fenced/dead incarnation cannot own drainable work, and a // stranded host's only leases are the unclaimed grant artifacts of // its own loop. Removing them prevents late expiry from // decrementing the replacement. + for (const [cellId, units] of leaseUnitsByCell(activityLeases)) { + await this.adjustCellReservation(transaction, cellId, -units) + } await transaction.query( `DELETE FROM relay_assignment_activity_leases WHERE user_id = ? AND relay_host_id = ?`, @@ -1130,6 +1219,9 @@ export class RelayAssignmentStore { } await this.adjustCellReservation(transaction, target.cellId, 1) const assignmentEpoch = existing ? integer(existing, 'assignment_epoch') + 1 : 1 + // Counters follow the leases the host still holds, plus the new control. + const counts = activityCounts(keptLeases) + counts.control += 1 const leaseExpiresAt = now + ASSIGNMENT_LIMITS.activityLeaseMs await transaction.query( `INSERT INTO relay_assignments @@ -1153,14 +1245,18 @@ export class RelayAssignmentStore { identity.relayHostId, target.cellId, assignmentEpoch, - leaseExpiresAt, + // Not before a kept lease expires: aggregate expiry keys off this. + Math.max( + leaseExpiresAt, + ...keptLeases.map((lease) => integer(lease, 'expires_at')) + ), now, - 1, - 0, - 0, - 0, - 0, - 0 + counts.control, + counts.splice, + counts.invite, + counts.install, + counts.confirmation, + counts.migration ] ) if (existing) { @@ -7075,10 +7171,17 @@ export class RelayAssignmentStore { const now = this.now() try { return await this.database.transaction(async (transaction) => { + // Only counters no lease backs. A host's leases can sit on more than + // one cell, and the lease sweep takes each one's units off its own. const expired = await transaction.queryLocked( `SELECT * FROM relay_assignments WHERE lease_expires_at <= ? AND (reserved_controls > 0 OR reserved_splices > 0 OR reserved_invites > 0 OR pending_installs > 0 OR pending_confirmations > 0 OR migration_leases > 0) + AND NOT EXISTS ( + SELECT 1 FROM relay_assignment_activity_leases lease + WHERE lease.user_id = relay_assignments.user_id + AND lease.relay_host_id = relay_assignments.relay_host_id + ) AND NOT EXISTS ( SELECT 1 FROM relay_region_rehome_attempts rehome WHERE rehome.user_id = relay_assignments.user_id @@ -7344,6 +7447,66 @@ export class RelayAssignmentStore { return rows } + // The rows a roll-isolated incumbent may move to in one tier: general cells + // in its own region, or in every other region, never its own. Placement + // re-reads admission and headroom under the lock. + private async lockIsolatedReplacementCells( + database: RelayDatabase, + sourceCellId: string | undefined, + tier: IsolatedReplacementTier, + mode: CellInventoryLockMode + ): Promise { + if (sourceCellId === undefined) throw new AssignmentInventoryScopeChanged() + const candidates = await database.query( + `SELECT admission.cell_id FROM relay_cell_admission admission + LEFT JOIN relay_cell_regions region ON region.cell_id = admission.cell_id + WHERE admission.admission_state = 'general' AND admission.cell_id <> ? + AND COALESCE(region.region, ?) ${tier === 'same-region' ? '=' : '<>'} ?`, + [sourceCellId, RELAY_DEFAULT_REGION, await this.cellRegion(database, sourceCellId)] + ) + if (candidates.length === 0) { + if (tier === 'same-region') throw new AssignmentIsolatedTierExhausted() + throw new AssignmentInventoryScopeChanged() + } + return await this.lockCellRows( + database, + candidates.map((row) => text(row, 'cell_id')), + mode + ) + } + + // The same questions the all-rows path asks before moving an isolated + // incumbent, none of which needs a relay_cells lock. + private async isolatedIncumbentMayMove( + database: RelayDatabase, + identity: AssignmentIdentity, + existing: SqlRow, + now: number + ): Promise { + const cellId = text(existing, 'cell_id') + const pinned = await this.pinnedCellAdmission(database, cellId) + if (await this.assignmentStrandedOnUnservedCell(database, identity, existing, now, pinned?.state)) { + return false + } + if (this.requireLiveCells && !(await this.cellIsLive(database, cellId, now))) return false + return await this.incumbentCellIsolatedForRoll( + database, + identity, + existing, + now, + undefined, + pinned + ) + } + + private async unlockedCell(database: RelayDatabase, cellId: string): Promise { + const row = ( + await database.query(`SELECT * FROM relay_cells WHERE cell_id = ?`, [cellId]) + )[0] + if (!row) throw new Error('assigned_cell_missing') + return cell(row, await this.cellRegion(database, cellId)) + } + private async leastLoadedCell( database: RelayDatabase, // Required: the one caller has already locked the inventory it selects from, @@ -7351,9 +7514,8 @@ export class RelayAssignmentStore { rows: SqlRow[], preferredRegion: RelayRegion, // Ordinary placement treats region as a preference and spills globally - // rather than refuse a host a cell. Re-placing off an isolated cell is the - // one caller that must not: a whole region parked migration-only would send - // every one of its hosts to the fallback region, permanently. + // rather than refuse a host a cell. Re-placing off an isolated cell asks for + // its own region first, and leaves it only in a separate second tier. regionMode: 'prefer' | 'require' = 'prefer' ): Promise { const regions = new Map( @@ -8233,6 +8395,19 @@ function activityCounts(rows: SqlRow[]): Record return counts } +function leaseRequestUnits(rows: SqlRow[]): number { + return rows.reduce((total, row) => total + integer(row, 'request_units'), 0) +} + +function leaseUnitsByCell(rows: SqlRow[]): Map { + const units = new Map() + for (const row of rows) { + const cellId = text(row, 'cell_id') + units.set(cellId, (units.get(cellId) ?? 0) + integer(row, 'request_units')) + } + return units +} + function activityUnitsForCell(rows: SqlRow[], cellId: string): number { return rows.reduce( (total, row) => diff --git a/cloud/apps/relay/src/drain-release-cell-row-contention-postgres.test.ts b/cloud/apps/relay/src/drain-release-cell-row-contention-postgres.test.ts new file mode 100644 index 000000000000..92b41649641e --- /dev/null +++ b/cloud/apps/relay/src/drain-release-cell-row-contention-postgres.test.ts @@ -0,0 +1,579 @@ +import { createHash } from 'node:crypto' +import { afterAll, beforeAll, describe, expect, it } from 'vitest' +import { RelayAssignmentStore } from './assignment-store.js' +import { + encodeMembership, + type CellAdmissionMembership, + type CellAdmissionState +} from './cell-admission-selector.js' +import type { RelayCellConfig } from './config.js' +import { + consumeRelayCellInventoryHold, + consumeRelayDatabasePoolPressure, + openRelayDatabase, + type RelayDatabase +} from './database.js' +import { RelayPublicAssignmentAdmission } from './public-assignment-admission.js' +import { + openDelayedPostgresDatabase, + type StatementDelay +} from './test-fixtures/delayed-postgres-database.js' + +// Reproduces the Asia drain: a draining cell's releases each end with a one-row +// UPDATE of the source relay_cells row, held for a round trip to COMMIT, while +// the directors re-place its reconnecting hosts. Placement used to lock every +// relay_cells row and starved (about 9/s at 6 releases/s, 5/s at 18/s, every +// director backend lock-waiting); it now locks only the targets. Prints one +// line per run and asserts the after picture. + +const databaseUrl = process.env.ORCA_RELAY_TEST_POSTGRES_URL +const describePostgres = databaseUrl ? describe : describe.skip + +// asia-east2 cell to the us-central1 database, measured in production. +const ASIA_ROUND_TRIP_MS = 171 +const LOCAL_ROUND_TRIP_MS = 1 +const WINDOW_MS = 10_000 +// Reconnecting hosts dialling the directors during the window. Above the +// ~6-15/s the production tail sustained, so the director side is never idle. +const DIAL_RATE_PER_SECOND = 20 +// Production shapes: five director instances (production.tfvars), each with +// the default pool and sticky lane (variables.tf, config.ts), and an Asia +// cell's pool (validate-relay-asia-topology-plan.mjs). +const DIRECTOR_INSTANCES = 5 +const DIRECTOR_POOL_MAX = 3 +const STICKY_LANE = { maxConcurrent: 1, maxQueued: 64, waitMs: 2_000, minIntervalMs: 2_000 } +const CELL_POOL_MAX = 16 +const DIRECTOR_APPLICATION = 'drain-release-contention-director' + +const USER_PREFIX = 'drain-release-contention' +const NOW = 1_000_000 +const CAPPED = { + capacityRequests: 6_000, + connectionHardCap: 3_000, + connectionUnobservedBound: 60 +} as const +const SOURCE: RelayCellConfig = { + id: 'drain-release-contention-a-source', + url: 'https://drain-release-contention-source.example.test', + region: 'asia-east2', + ...CAPPED +} +const TARGETS: RelayCellConfig[] = ['b', 'c', 'd'].map((suffix) => ({ + id: `drain-release-contention-${suffix}-target`, + url: `https://drain-release-contention-${suffix}-target.example.test`, + region: 'asia-east2', + ...CAPPED +})) +// Next to the database. Taken only once every Asia neighbour is at its cap. +const US_CELL: RelayCellConfig = { + id: 'drain-release-contention-e-us', + url: 'https://drain-release-contention-us.example.test', + region: 'us-central1', + ...CAPPED +} +const CELLS = [SOURCE, ...TARGETS, US_CELL] +const RELEASED_ACTIVITY = `control:${SOURCE.id}:1` + +type Identity = { userId: string; relayHostId: string } +type Tally = Record +type Instance = { database: RelayDatabase; store: RelayAssignmentStore } + +type RunReport = { + roundTripMs: number + releaseRate: number + neighboursCapped: boolean + releases: { attempted: number; ok: number; failed: Tally; p50Ms: number; p95Ms: number } + dials: { + attempted: number + placed: number + placedInWindow: number + placedCrossRegion: number + admissionRejected: Tally + failed: Tally + } + activations: { ok: number; failed: Tally } + placementsPerSecond: number + director: { + lockUnavailable: number + lockTimeouts: number + holds: number + holdMsMax: number + // Mean director backends blocked on a lock per 20ms sample, of those active. + lockWaitingMean: number + activeMean: number + } + sourcePoolWaitersMax: number +} + +function hostIdentity(index: number): Identity { + return { + userId: `${USER_PREFIX}-${index}`, + // Relay host ids are fixed-width opaque ids. + relayHostId: `drainrel${String(index).padStart(8, '0')}` + } +} + +function failureCode(error: unknown): string { + if (error instanceof Error) { + const code = 'code' in error ? error.code : undefined + return typeof code === 'string' ? code : error.message + } + return String(error) +} + +function count(tally: Tally, key: string): void { + tally[key] = (tally[key] ?? 0) + 1 +} + +function total(tally: Tally): number { + return Object.values(tally).reduce((sum, value) => sum + value, 0) +} + +function percentile(values: number[], fraction: number): number { + if (values.length === 0) return 0 + const sorted = [...values].sort((left, right) => left - right) + return Math.round(sorted[Math.min(sorted.length - 1, Math.floor(fraction * sorted.length))]!) +} + +function mean(sum: number, samples: number): number { + return samples === 0 ? 0 : Number((sum / samples).toFixed(2)) +} + +// Fires `operation` at a fixed rate without awaiting it, like independent +// socket closes or dials would. +async function paced( + ratePerSecond: number, + total: number, + operation: (index: number) => Promise +): Promise[]> { + const startedAt = performance.now() + const started: Promise[] = [] + for (let index = 0; index < total; index += 1) { + const dueMs = (index * 1_000) / ratePerSecond - (performance.now() - startedAt) + if (dueMs > 0) await new Promise((resolve) => setTimeout(resolve, dueMs)) + started.push(operation(index)) + } + return started +} + +describePostgres('PostgreSQL drain releases against director placement', () => { + const storeOptions = { requireLiveCells: true, heartbeatTtlMs: 45_000 } + // One switch for every Asia pool: the source cell and each target cell. + const delay: StatementDelay = { enabled: false, delayMs: LOCAL_ROUND_TRIP_MS } + const directors: Instance[] = [] + const servingCells = new Map() + let observer: RelayDatabase + const reports: RunReport[] = [] + + function admin(): Instance { + return directors[0]! + } + + function servingCell(cellId: string): Instance { + const instance = servingCells.get(cellId) + if (!instance) throw new Error(`no cell pool for ${cellId}`) + return instance + } + + async function applySelector( + states: Record, + rollIsolatedCells?: string[] + ): Promise { + const current = await admin().store.inspectCellAdmissionSelector() + // Built from relay_cells: the apply requires exact coverage of a shared fleet. + const fleet = await admin().database.query( + `SELECT cell_id FROM relay_cells ORDER BY cell_id ASC` + ) + const membership: CellAdmissionMembership = { + existingOnly: [], + migrationOnly: [], + general: [] + } + for (const row of fleet) { + const cellId = String(row['cell_id']) + const state = + states[cellId] ?? + (current.selector.membership.existingOnly.includes(cellId) + ? 'existing-only' + : current.selector.membership.migrationOnly.includes(cellId) + ? 'migration-only' + : 'general') + if (state === 'existing-only') membership.existingOnly.push(cellId) + else if (state === 'migration-only') membership.migrationOnly.push(cellId) + else membership.general.push(cellId) + } + await admin().store.applyCellAdmissionSelector({ + attemptId: `drain-release-${current.selector.generation}`, + expectedGeneration: current.selector.generation, + ...(current.selector.generation === 0 + ? { + expectedMembershipSha256: createHash('sha256') + .update(encodeMembership(current.selector.membership)) + .digest('hex') + } + : {}), + membership, + ...(rollIsolatedCells ? { rollIsolatedCells } : {}) + }) + } + + // Other Postgres files write admission through the generation-0 helpers, so + // the selector goes back to generation 0 exactly as found. + async function resetSelectorBoundary(): Promise { + await admin().database.query( + `UPDATE relay_admission_selectors SET generation = 0, attempt_id = NULL + WHERE selector_id = 'general'` + ) + await admin().database.query( + `DELETE FROM relay_admission_selector_intents WHERE attempt_id LIKE 'drain-release-%'` + ) + await admin().store.reconcileCells([], false) + } + + async function deleteHostRows(): Promise { + for (const table of [ + 'relay_control_capabilities', + 'relay_control_connection_reservations', + 'relay_assignment_activity_leases', + 'relay_assignment_migrations', + 'relay_assignment_region_preferences', + 'relay_assignments' + ]) { + await admin().database.query(`DELETE FROM ${table} WHERE user_id LIKE '${USER_PREFIX}-%'`) + } + } + + // `atCap` reports those cells at their connection cap: no headroom left. + async function heartbeatAll(atCap: RelayCellConfig[] = []): Promise { + for (const [index, config] of CELLS.entries()) { + await admin().store.recordCellHeartbeat({ + cellId: config.id, + cellUrl: config.url, + cellIncarnation: `2222222${index}-2222-4222-8222-222222222222`, + startedAt: NOW - 1_000, + ready: true, + observedRequests: 0, + region: config.region, + // The cell checks enforced = total + in-flight + reserved. + totalConnections: atCap.includes(config) ? CAPPED.connectionHardCap : 0, + inFlightConnections: 0, + reservedConnectionUnits: 0, + enforcedConnectionUnits: atCap.includes(config) ? CAPPED.connectionHardCap : 0, + connectionHardCap: CAPPED.connectionHardCap, + connectionUnobservedBound: CAPPED.connectionUnobservedBound + }) + } + } + + // Every host holds a live control lease on the source, then the source is + // isolated for a roll: the state an Asia drain starts from. + async function seed(hosts: Identity[]): Promise { + await deleteHostRows() + await admin().database.query( + `UPDATE relay_cells SET reserved_requests = 0 WHERE cell_id LIKE '${USER_PREFIX}-%'` + ) + // Restoring the source to general also clears the previous run's roll stamp. + await applySelector({ + [SOURCE.id]: 'general', + ...Object.fromEntries(TARGETS.map((target) => [target.id, 'migration-only' as const])) + }) + for (const identity of hosts) { + const grant = await admin().store.assign(identity, 'asia-east2', 'asia-east2') + expect(grant.cellId).toBe(SOURCE.id) + await admin().store.activateControl(identity, { + cellId: SOURCE.id, + assignmentEpoch: grant.assignmentEpoch, + generation: 1 + }) + } + await applySelector( + Object.fromEntries(TARGETS.map((target) => [target.id, 'general' as const])) + ) + await applySelector({ [SOURCE.id]: 'migration-only' }, [SOURCE.id]) + } + + // Samples director backends so the report shows where director time goes: a + // failed or slow placement leaves no hold sample behind. + function sampleDirectorWaits(): { stop: () => Promise<{ waiting: number; active: number }> } { + let running = true + let samples = 0 + let waiting = 0 + let active = 0 + const loop = (async () => { + while (running) { + const rows = await observer.query( + `SELECT wait_event_type FROM pg_stat_activity + WHERE application_name = ? AND state = 'active'`, + [DIRECTOR_APPLICATION] + ) + samples += 1 + active += rows.length + waiting += rows.filter((row) => row['wait_event_type'] === 'Lock').length + await new Promise((resolve) => setTimeout(resolve, 20)) + } + })() + return { + stop: async () => { + running = false + await loop + return { waiting: mean(waiting, samples), active: mean(active, samples) } + } + } + } + + async function run( + roundTripMs: number, + releaseRate: number, + neighboursCapped = false + ): Promise { + const releaseCount = Math.round((releaseRate * WINDOW_MS) / 1_000) + const dialCount = Math.round((DIAL_RATE_PER_SECOND * WINDOW_MS) / 1_000) + const releasing = Array.from({ length: releaseCount }, (_, index) => hostIdentity(index)) + const dialling = Array.from({ length: dialCount }, (_, index) => + hostIdentity(releaseCount + index) + ) + await seed([...releasing, ...dialling]) + if (neighboursCapped) await heartbeatAll(TARGETS) + + const releaseLatencies: number[] = [] + const releaseFailed: Tally = {} + const releases = { attempted: 0, ok: 0, failed: releaseFailed } + const admissionRejected: Tally = {} + const dialFailed: Tally = {} + const dials = { + attempted: 0, + placed: 0, + placedInWindow: 0, + placedCrossRegion: 0, + admissionRejected, + failed: dialFailed + } + const activationFailed: Tally = {} + const activations = { ok: 0, failed: activationFailed } + const activationWork: Promise[] = [] + const stickyLanes = directors.map(() => new RelayPublicAssignmentAdmission(STICKY_LANE)) + for (const instance of directors) consumeRelayCellInventoryHold(instance.database) + consumeRelayDatabasePoolPressure(servingCell(SOURCE.id).database) + delay.delayMs = roundTripMs + delay.enabled = true + const sampler = sampleDirectorWaits() + const startedAt = performance.now() + + const releaseWork = paced(releaseRate, releaseCount, async (index) => { + releases.attempted += 1 + const began = performance.now() + try { + const released = await servingCell(SOURCE.id).store.releaseActivity( + releasing[index]!, + RELEASED_ACTIVITY + ) + if (released) releases.ok += 1 + else count(releases.failed, 'lease_missing') + } catch (error) { + count(releases.failed, failureCode(error)) + } + releaseLatencies.push(performance.now() - began) + }) + // The reconnect path, round-robin over directors: sticky-lane admission, + // the durable-pin check, then the assign the route calls, which re-places + // off the isolated cell. The host then attaches to its new Asia cell. + const dialWork = paced(DIAL_RATE_PER_SECOND, dialCount, async (index) => { + const identity = dialling[index]! + const director = directors[index % directors.length]! + dials.attempted += 1 + let rejection = 'unknown' + const lease = await stickyLanes[index % directors.length]!.acquire( + identity.relayHostId, + (reason) => { + rejection = reason + } + ) + if (!lease) { + count(dials.admissionRejected, rejection) + return + } + try { + if (!(await director.store.resolve(identity))) { + count(dials.failed, 'unverified') + return + } + const grant = await director.store.assign(identity, 'asia-east2', 'asia-east2') + if (grant.cellId === SOURCE.id) { + count(dials.failed, 'kept_on_source') + return + } + dials.placed += 1 + if (grant.region !== SOURCE.region) dials.placedCrossRegion += 1 + if (performance.now() - startedAt <= WINDOW_MS) dials.placedInWindow += 1 + activationWork.push( + servingCell(grant.cellId) + .store.activateControl(identity, { + cellId: grant.cellId, + assignmentEpoch: grant.assignmentEpoch, + generation: 1 + }) + .then( + () => { + activations.ok += 1 + }, + (error: unknown) => count(activations.failed, failureCode(error)) + ) + ) + } catch (error) { + count(dials.failed, failureCode(error)) + } finally { + lease.release() + } + }) + await Promise.allSettled([...(await releaseWork), ...(await dialWork)]) + await Promise.allSettled(activationWork) + const waits = await sampler.stop() + delay.enabled = false + if (neighboursCapped) await heartbeatAll() + + const holds = directors.map((instance) => consumeRelayCellInventoryHold(instance.database)) + const report: RunReport = { + roundTripMs, + releaseRate, + neighboursCapped, + releases: { + ...releases, + p50Ms: percentile(releaseLatencies, 0.5), + p95Ms: percentile(releaseLatencies, 0.95) + }, + dials, + activations, + placementsPerSecond: Number(((dials.placedInWindow * 1_000) / WINDOW_MS).toFixed(1)), + director: { + lockUnavailable: holds.reduce((sum, hold) => sum + hold.cellInventoryLockUnavailable, 0), + lockTimeouts: holds.reduce((sum, hold) => sum + hold.cellInventoryLockTimeouts, 0), + holds: holds.reduce((sum, hold) => sum + hold.cellInventoryHolds, 0), + holdMsMax: Math.round(Math.max(...holds.map((hold) => hold.cellInventoryHoldMsMax))), + lockWaitingMean: waits.waiting, + activeMean: waits.active + }, + sourcePoolWaitersMax: consumeRelayDatabasePoolPressure(servingCell(SOURCE.id).database) + .databasePoolWaitersMax + } + reports.push(report) + console.info(JSON.stringify({ event: 'drain_release_cell_row_contention', ...report })) + return report + } + + beforeAll(async () => { + for (let index = 0; index < DIRECTOR_INSTANCES; index += 1) { + const database = await openRelayDatabase({ + databaseUrl, + dataDir: '', + poolMax: DIRECTOR_POOL_MAX, + applicationName: DIRECTOR_APPLICATION + }) + directors.push({ + database, + store: new RelayAssignmentStore(database, () => NOW, storeOptions) + }) + } + for (const config of CELLS) { + const database = openDelayedPostgresDatabase( + databaseUrl!, + config === US_CELL ? { enabled: false, delayMs: 0 } : delay, + CELL_POOL_MAX + ) + servingCells.set(config.id, { + database, + store: new RelayAssignmentStore(database, () => NOW, storeOptions) + }) + } + observer = await openRelayDatabase({ databaseUrl, dataDir: '', poolMax: 1 }) + await deleteHostRows() + await admin().store.reconcileCells(CELLS, false) + await resetSelectorBoundary() + await heartbeatAll() + }) + + afterAll(async () => { + delay.enabled = false + if (reports.length > 0) console.table(reports.map(flattenReport)) + if (directors.length > 0) { + await deleteHostRows() + // Cells before the selector rebuild, or its membership names missing rows. + for (const config of CELLS) { + for (const table of [ + 'relay_cell_connection_snapshots', + 'relay_cell_connection_runtime', + 'relay_cell_connection_limits', + 'relay_cell_runtime', + 'relay_cell_admission', + 'relay_cell_regions', + 'relay_cells' + ]) { + await admin().database.query(`DELETE FROM ${table} WHERE cell_id = ?`, [config.id]) + } + } + await resetSelectorBoundary() + } + for (const instance of [...directors, ...servingCells.values()]) await instance.database.close() + await observer?.close() + }) + + it('keeps placing during a paced drain when the cells are next to the database', async () => { + for (const rate of [6, 18]) { + const report = await run(LOCAL_ROUND_TRIP_MS, rate) + expect(report.releases.failed).toEqual({}) + expect(report.dials.failed).toEqual({}) + // Completion and contention, not placements/s: pacing on a slow runner + // moves the rate without any lock being involved. + expect(total(report.dials.admissionRejected)).toBeLessThan(0.1 * report.dials.attempted) + expect(report.dials.placed + total(report.dials.admissionRejected)).toBe( + report.dials.attempted + ) + expect(report.director.lockWaitingMean).toBeLessThan(0.5) + } + }, 120_000) + + it('keeps placing while an Asia drain holds the source row', async () => { + // Before the fix: 90 and 140 of 200 dials rejected by the sticky lane and + // about 3.5 of 3.5 active director backends lock-waiting. Pass bars are on + // those, not on placements/s, which a slow runner's pacing alone can move. + for (const [rate, neighboursCapped] of [ + [6, false], + [18, false], + [18, true] + ] as const) { + const report = await run(ASIA_ROUND_TRIP_MS, rate, neighboursCapped) + expect(report.dials.failed).toEqual({}) + expect(total(report.dials.admissionRejected)).toBeLessThan(0.1 * report.dials.attempted) + expect(report.director.lockWaitingMean).toBeLessThan(0.5) + expect(report.dials.placedCrossRegion).toBe(neighboursCapped ? report.dials.placed : 0) + if (rate === 18) { + // Still cell-side: releases on one row serialise at ~1/RTT (~5.8/s) + // and the excess sheds to lease expiry. The director no longer waits. + expect(report.releases.ok).toBeLessThan(0.6 * report.releases.attempted) + } + } + }, 240_000) +}) + +function flattenReport(report: RunReport): Record { + const failed = (tally: Tally): string => + Object.entries(tally) + .map(([key, value]) => `${key}:${value}`) + .join(' ') || '-' + return { + rttMs: report.roundTripMs, + capped: report.neighboursCapped ? 'yes' : 'no', + crossRegion: report.dials.placedCrossRegion, + releasesPerSec: report.releaseRate, + releasesOk: report.releases.ok, + releasesFailed: failed(report.releases.failed), + releaseP95Ms: report.releases.p95Ms, + dialsPlaced: report.dials.placed, + placementsPerSec: report.placementsPerSecond, + dialsFailed: failed(report.dials.failed), + stickyRejected: failed(report.dials.admissionRejected), + lockTimeouts: report.director.lockTimeouts, + lockUnavailable: report.director.lockUnavailable, + lockWaiting: `${report.director.lockWaitingMean}/${report.director.activeMean}`, + activationsFailed: failed(report.activations.failed) + } +} diff --git a/cloud/apps/relay/src/regional-rehome-target-row-lock-postgres.test.ts b/cloud/apps/relay/src/regional-rehome-target-row-lock-postgres.test.ts index 2263820407ad..314b758b17f3 100644 --- a/cloud/apps/relay/src/regional-rehome-target-row-lock-postgres.test.ts +++ b/cloud/apps/relay/src/regional-rehome-target-row-lock-postgres.test.ts @@ -1,13 +1,16 @@ -import pg from 'pg' import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'vitest' import { RelayAssignmentStore } from './assignment-store.js' import { consumeRelayCellInventoryHold, openRelayDatabase, - PostgresDatabase, + type PostgresDatabase, type RelayDatabase } from './database.js' import { readRegionCorrectionOutcomes } from './region-correction-outcomes.js' +import { + openDelayedPostgresDatabase, + type StatementDelay +} from './test-fixtures/delayed-postgres-database.js' const databaseUrl = process.env.ORCA_RELAY_TEST_POSTGRES_URL const describePostgres = databaseUrl ? describe : describe.skip @@ -31,24 +34,23 @@ const TARGET_LOCKED = ['target:relay_cells', 'target:relay_cell_admission'] type Trip = { sql: string; lockable: Record } -type DelayControl = { - enabled: boolean - // Runs before each delayed statement leaves the client, so it sees the locks - // the transaction holds between round trips. - beforeTrip: (sql: string) => Promise -} +type DelayControl = StatementDelay & { beforeTrip: (sql: string) => Promise } describePostgres('PostgreSQL regional rehome target-row lock', () => { let primary: RelayDatabase let observer: RelayDatabase let delayed: PostgresDatabase - const control: DelayControl = { enabled: false, beforeTrip: async () => undefined } + const control: DelayControl = { + enabled: false, + delayMs: STATEMENT_DELAY_MS, + beforeTrip: async () => undefined + } let sequence = 0 beforeAll(async () => { primary = await openRelayDatabase({ databaseUrl, dataDir: '' }) observer = await openRelayDatabase({ databaseUrl, dataDir: '' }) - delayed = openDelayedDatabase(databaseUrl!, control) + delayed = openDelayedPostgresDatabase(databaseUrl!, control) }) beforeEach(async () => { @@ -342,32 +344,6 @@ describePostgres('PostgreSQL regional rehome target-row lock', () => { } }) -// Same session settings as the serving pool, with a fixed delay in front of -// every statement, BEGIN and COMMIT included. -function openDelayedDatabase(url: string, control: DelayControl): PostgresDatabase { - const pool = new pg.Pool({ - connectionString: url, - max: 4, - statement_timeout: 5_000, - lock_timeout: 1_000, - idle_in_transaction_session_timeout: 5_000 - }) - pool.on('error', () => undefined) - pool.on('connect', (client) => { - const query = client.query - Object.assign(client, { - query: async (...args: unknown[]) => { - if (control.enabled) { - await control.beforeTrip(typeof args[0] === 'string' ? args[0] : '') - await new Promise((resolve) => setTimeout(resolve, STATEMENT_DELAY_MS)) - } - return await Reflect.apply(query, client, args) - } - }) - }) - return new PostgresDatabase(pool) -} - const storeOptions = { regionalRehomeCohortPercent: 100, requireLiveCells: true, diff --git a/cloud/apps/relay/src/test-fixtures/delayed-postgres-database.ts b/cloud/apps/relay/src/test-fixtures/delayed-postgres-database.ts new file mode 100644 index 000000000000..eef94c529576 --- /dev/null +++ b/cloud/apps/relay/src/test-fixtures/delayed-postgres-database.ts @@ -0,0 +1,48 @@ +import pg from 'pg' +import { + absorbPostgresIdleClientErrors, + POSTGRES_LOCK_TIMEOUT_MS, + POSTGRES_STATEMENT_TIMEOUT_MS, + PostgresDatabase +} from '../database.js' + +export type StatementDelay = { + enabled: boolean + delayMs: number + // Runs before each delayed statement leaves the client, so it sees the locks + // the transaction holds between round trips. + beforeTrip?: (sql: string) => Promise +} + +// The serving pool's session settings with a fixed delay in front of every +// statement, BEGIN and COMMIT included: one delay per cross-region round trip. +export function openDelayedPostgresDatabase( + url: string, + delay: StatementDelay, + poolMax = 4 +): PostgresDatabase { + const pool = new pg.Pool({ + connectionString: url, + max: poolMax, + // Same bound as openRelayDatabase, so a saturated pool fails acquires the + // way a serving cell does instead of queueing forever. + connectionTimeoutMillis: 2_000, + statement_timeout: POSTGRES_STATEMENT_TIMEOUT_MS, + lock_timeout: POSTGRES_LOCK_TIMEOUT_MS, + idle_in_transaction_session_timeout: 5_000 + }) + absorbPostgresIdleClientErrors(pool) + pool.on('connect', (client) => { + const query = client.query + Object.assign(client, { + query: async (...args: unknown[]) => { + if (delay.enabled) { + await delay.beforeTrip?.(typeof args[0] === 'string' ? args[0] : '') + await new Promise((resolve) => setTimeout(resolve, delay.delayMs)) + } + return await Reflect.apply(query, client, args) + } + }) + }) + return new PostgresDatabase(pool) +} From 007b7c0d32d0c45316e583c3efacc3ae6f393ef0 Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 14:11:41 -0700 Subject: [PATCH 20/79] fix(claude): end a message Claude started but never confirmed, and keep Claude running while it holds one (#23898) * fix(claude): settle a queued send the CLI withdrew from its own cancelled frame Claude reports each uuid-stamped command's lifecycle (queued, started, completed, cancelled). A send it withdraws from its queue gets `cancelled` before the interrupt or cancel_async_message answer, so a lost or failed answer no longer leaves that send pending: it settles as withdrawn, with the same reason and words as the receipt path. A command the CLI already started also ends `cancelled` when its turn is interrupted or fails, so `cancelled` after `started` is not a withdrawal; an echoed send has left the waiter lists and is never reached. Tests replay real 2.1.280 captures, scrubbed. * fix(claude): release a doubted send when the CLI reports its session idle A Claude send whose write ended in doubt is recorded `unknown`, and a live `unknown` reads as work still owed, so the chat showed Working until the child exited. Claude sends `session_state_changed idle` only once its whole queue has drained, so it can no longer be holding that send. The runtime now routes that report to the host's existing release, the same one Codex's thread-stopped report uses; it retires `unknown` only, never `pending`. * fix(claude): keep a command's started mark when a redelivery re-emits queued; fixtures name msg_lifecycle_v1 * fix(claude): settle every terminal lifecycle state of a send the CLI never echoed A send the CLI started, then cancelled before any echo, stayed pending: it may already be in the conversation, so it is released as doubt (unknown, recovered), never withdrawn and never re-sent. A late echo still accepts it. The 2.1.280 schema has two more terminal states. `discarded` (the CLI ended its session with the send still queued) settles as not delivered; `refused` (declined before it queued) settles as not accepted by the provider. After `started`, either one is doubt, as `cancelled` is. The late-settlement path gains an `unknown` outcome, which the host records as released doubt. * fix(claude): release a send the CLI took but left unanswered when it goes idle `session_state_changed idle` comes only once the CLI's queue has drained, so a send it took that is still unanswered there got no echo and never will: a turn that throws can leave `started` with no terminal state. Idle releases it as doubt. What proves the CLI took a send is its lifecycle frame. On a CLI that reports no lifecycle, it is the send's place on stdin: one whose write finished before an interrupt went out was read before the interrupt was, so the first idle after that interrupt releases it too. A send armed ahead of the interrupt but written after it is left alone, since the CLI may still run it. * fix(native-chat): keep the idle sweep off a Claude child that holds a send A Claude retrying a rate-limited request has taken the send but echoes nothing, so no turn row exists yet and the sweep rested the child after the idle window, turning the send into doubt. The adapter now reports whether the CLI holds a send (lifecycle `queued` or `started`, not yet echoed or ended), derived from the live waiters, and owed work counts it. Nothing is stored: every held send leaves the live set on its echo, its terminal lifecycle state, the CLI's idle, or the child's exit, so the hold ends with the send. * fix(claude): count only a started send at idle and as a held send 2.1.280's end-of-turn cleanup can report idle before it re-reads its queue, so a send read in that window goes queued, idle, started. Releasing every taken send at idle doubted that live send and dropped Working. Only a `started` send is released at idle or keeps the child from the idle sweep; a `queued` one ends by starting and echoing, by a terminal lifecycle frame, or with the child. The stdin-order path for CLIs without lifecycle frames is removed: a doubted send retired there disables content matching on CLIs that mint their own echo ids, and no Orca failure called for it. Those CLIs keep the earlier behaviour. Comments that said only a failed write or child exit ends a waiter, or that idle comes only once the queue has drained, now say what ends one. * docs(claude): say only what the CLI's lifecycle frames and idle actually prove * fix(claude): hold the idle sweep while Claude has a send queued, not only started The sweep rested a child whose CLI had queued a follow-up behind a turn, dropping the send it had already taken. The hold now spans the CLI reporting it took the send until its echo, a terminal lifecycle state, or the child's exit. The idle release still covers only started sends: 2.1.280 can report idle before it re-reads its queue. * refactor(native-chat): give provider-proven late dispatch settlement its own module * test(claude): pin a steer a Stop interrupts after it started as doubt, not withdrawn --- .../claude/claude-command-lifecycle.test.ts | 242 +++++++++++++++++- src/main/claude/claude-command-lifecycle.ts | 65 ++++- .../claude/claude-replay-turn-resolution.ts | 14 +- .../claude-structured-dispatch-waiters.ts | 6 +- src/main/claude/claude-structured-dispatch.ts | 51 +++- .../claude-structured-session-acquisition.ts | 2 +- .../claude-structured-session-adapter.ts | 5 + .../claude/claude-structured-session-state.ts | 8 +- .../journal-dispatch-doubt-reasons.ts | 3 + ...structured-agent-session-adapter-router.ts | 3 + .../structured-agent-session-adapter.ts | 3 + ...d-agent-session-conversation-close.test.ts | 1 + ...red-agent-session-conversation-lifetime.ts | 1 + ...structured-agent-session-host-mutations.ts | 46 +--- .../structured-agent-session-host.ts | 2 +- ...tructured-agent-session-idle-sweep.test.ts | 18 ++ .../structured-agent-session-idle-sweep.ts | 3 + .../structured-agent-session-late-dispatch.ts | 49 ++++ .../structured-agent-session-rest-test-rig.ts | 2 + ...ent-session-unanswered-dispatch-release.ts | 2 +- ...e-structured-idle-releases-unknown.test.ts | 89 ++++++- .../structured-agent-session-runtime.ts | 2 +- 22 files changed, 534 insertions(+), 83 deletions(-) create mode 100644 src/main/native-chat/agent-session-wire/structured-agent-session-late-dispatch.ts diff --git a/src/main/claude/claude-command-lifecycle.test.ts b/src/main/claude/claude-command-lifecycle.test.ts index c5570e476fd1..f4182a601747 100644 --- a/src/main/claude/claude-command-lifecycle.test.ts +++ b/src/main/claude/claude-command-lifecycle.test.ts @@ -10,7 +10,13 @@ import { describe, expect, it } from 'vitest' import type { AgentJournalItemBody } from '../../shared/agent-session-journal-types' import { readAgentJournalTurn } from '../../shared/agent-session-turn-record' import { DISPATCH_REJECTED_CANCELLED } from '../../shared/structured-agent-session-dispatch-rejection' +import { + DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED, + DISPATCH_DOUBT_PROVIDER_IDLE +} from '../native-chat/agent-session-journal/journal-dispatch-doubt-reasons' +import { agentSessionFailureFact } from '../../shared/agent-session-failure' import { ClaudeControlRequestError } from './claude-agent-sdk-control-requests' +import { claudeDispatchRejection } from './claude-structured-dispatch-content' import { ClaudeStructuredSessionAdapter } from './claude-structured-session-adapter' import type { ClaudeLateDispatchOutcome } from './claude-structured-session-state' import { @@ -96,6 +102,8 @@ async function replayCapture( omitFrame?: (frame: Record) => boolean /** Frames to deliver right after a captured one, in the capture's own uuids. */ afterFrame?: (frame: Record) => Record[] + /** Deliver this in place of a captured frame, as a CLI reporting another state would. */ + rewriteFrame?: (frame: Record) => Record /** Stop after the control request settles; its tail answers the CLI's own control path. */ stopAfterControl?: boolean } = {} @@ -161,7 +169,7 @@ async function replayCapture( if (options.omitFrame?.(frame)) { return } - const mapped = mapUuids(frame) + const mapped = mapUuids(options.rewriteFrame?.(frame) ?? frame) if (Array.isArray(mapped.capabilities) && options.withoutCapabilities) { mapped.capabilities = mapped.capabilities.filter( (capability) => !options.withoutCapabilities!.includes(String(capability)) @@ -366,6 +374,30 @@ describe('a queued send the CLI withdraws without Orca hearing why', () => { }) }) +/** The auth-failed capture's turn output: without it, its send started and got no echo. */ +function isTurnOutput(frame: Record): boolean { + return frame.type === 'user' || frame.type === 'assistant' || frame.type === 'result' +} + +function isLifecycleFrame(frame: Record, state?: string): boolean { + return frame.type === 'command_lifecycle' && (state === undefined || frame.state === state) +} + +function isIdleFrame(frame: Record): boolean { + return frame.subtype === 'session_state_changed' && frame.state === 'idle' +} + +/** The uuid a capture recorded for one of its sends, as its own frames name it. */ +function capturedUuid(name: string, clientMessageId: string): string { + const dispatch = loadCapture(name).find( + (event) => event.kind === 'dispatch' && event.clientMessageId === clientMessageId + ) + return dispatch?.kind === 'dispatch' ? dispatch.sentUuid : '' +} + +const ENDED_IN_DOUBT = { state: 'unknown', reason: DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED } +const IDLE_IN_DOUBT = { state: 'unknown', reason: DISPATCH_DOUBT_PROVIDER_IDLE } + describe('a send the CLI started, then cancelled', () => { it('stays accepted when its turn failed after the echo', async () => { const replay = await replayCapture('auth-failed') @@ -373,28 +405,220 @@ describe('a send the CLI started, then cancelled', () => { expect(replay.settlementsFor('client-A')).toEqual([acceptedAs(replay.liveUuid('client-A'))]) }) - it('is not read as withdrawn when the cancelled frame came before any echo', async () => { - // The auth-failed capture with the turn's output removed: started, then cancelled. + it('is released as doubt, never withdrawn, when the cancelled frame came before any echo', async () => { + const replay = await replayCapture('auth-failed', { omitFrame: isTurnOutput }) + + expect(replay.settlementsFor('client-A')).toEqual([ENDED_IN_DOUBT]) + expect(replay.adapter.holdsDispatch('session-1')).toBe(false) + }) + + it('stays started when a redelivered command re-emits queued before its cancelled frame', async () => { const replay = await replayCapture('auth-failed', { + omitFrame: isTurnOutput, + afterFrame: (frame) => + isLifecycleFrame(frame, 'started') ? [{ ...frame, state: 'queued' }] : [] + }) + + expect(replay.settlementsFor('client-A')).toEqual([ENDED_IN_DOUBT]) + }) + + it('is still accepted when its echo lands after the doubt', async () => { + const replay = await replayCapture('auth-failed', { omitFrame: isTurnOutput }) + const uuid = replay.liveUuid('client-A') + + replay.connection.handlers.onMessage?.({ + type: 'user', + message: { role: 'user', content: [{ type: 'text', text: 'late echo' }] }, + parent_tool_use_id: null, + session_id: PROVIDER_SESSION_ID, + uuid + }) + + expect(replay.settlementsFor('client-A')).toEqual([ENDED_IN_DOUBT, acceptedAs(uuid)]) + }) + + it.each(['discarded', 'refused'])( + 'is released as doubt when its turn ends %s after it started', + async (state) => { + const replay = await replayCapture('auth-failed', { + omitFrame: isTurnOutput, + rewriteFrame: (frame) => + isLifecycleFrame(frame, 'cancelled') ? { ...frame, state } : frame + }) + + expect(replay.settlementsFor('client-A')).toEqual([ENDED_IN_DOUBT]) + } + ) +}) + +// batch-lead's client-C is a steer: written while A's turn runs, queued, then started as its own +// turn once A completed. Its queued-while-Stopped case is interrupt-lost's client-B, above. +describe('a steer a Stop interrupts after it started', () => { + it('is released as doubt, never withdrawn, when the Stop lands before its echo', async () => { + const steer = capturedUuid('batch-lead', 'client-C') + let started = false + let routes: ControlPoint['routes'] | undefined + const replay = await replayCapture('batch-lead', { + atControl: async (point) => { + routes = point.routes + point.deliverInFlight() + }, + // The Stop lands between the steer's start and its echo: nothing after its start arrives. + omitFrame: () => started, + afterFrame: (frame) => { + started ||= isLifecycleFrame(frame, 'started') && frame.command_uuid === steer + return [] + } + }) + const uuid = replay.liveUuid('client-C') + expect(replay.adapter.holdsDispatch('session-1')).toBe(true) + + // Answered as interrupt-lost's was, with nothing left queued; then that capture's tail for a + // started command a Stop ended (its result dropped: one naming the steer would accept it). + routes!.interrupt = () => ({ still_queued: [], cancelled: [] }) + await replay.adapter.cancelTurn({ sessionId: 'session-1', fence: 7 }) + for (const frame of [ + { + type: 'command_lifecycle', + command_uuid: uuid, + state: 'cancelled', + uuid: 'steer-cancelled' + }, + { type: 'system', subtype: 'session_state_changed', state: 'idle', uuid: 'steer-idle' } + ]) { + replay.connection.handlers.onMessage?.({ ...frame, session_id: PROVIDER_SESSION_ID }) + } + + expect(replay.connection.calls.filter((call) => call.subtype === 'interrupt')).toEqual([ + { subtype: 'interrupt', params: { cancelQueued: true } } + ]) + expect(replay.settlementsFor('client-C')).toEqual([ENDED_IN_DOUBT]) + expect(replay.adapter.holdsDispatch('session-1')).toBe(false) + }) +}) + +describe('a send the CLI ended before starting it', () => { + it.each([ + // The CLI ended its session with the send still queued. + ['discarded', 'notDelivered', false], + // Declined before it queued, so no `queued` frame precedes it. + ['refused', 'providerRejected', true] + ] as const)('settles a %s send as not sent, saying why', async (state, kind, neverQueued) => { + const followUp = capturedUuid('cancel-async', 'client-B') + const replay = await replayCapture('cancel-async', { omitFrame: (frame) => - frame.type === 'user' || frame.type === 'assistant' || frame.type === 'result' + neverQueued && isLifecycleFrame(frame, 'queued') && frame.command_uuid === followUp, + rewriteFrame: (frame) => + isLifecycleFrame(frame, 'cancelled') && frame.command_uuid === followUp + ? { ...frame, state } + : frame }) - expect(replay.settlementsFor('client-A')).toEqual([]) + expect(replay.settlementsFor('client-B')).toEqual([ + { state: 'rejected', ...claudeDispatchRejection(agentSessionFailureFact(kind)) } + ]) + expect(replay.settlementsFor('client-A')).toEqual([acceptedAs(replay.liveUuid('client-A'))]) }) +}) - it('stays started when a redelivered command re-emits queued before its cancelled frame', async () => { +describe('a send the CLI holds', () => { + it('holds its child from the idle sweep until the CLI idles, then is released as doubt', async () => { + // Started, then its turn threw: no echo and no terminal state, as the schema allows. const replay = await replayCapture('auth-failed', { omitFrame: (frame) => - frame.type === 'user' || frame.type === 'assistant' || frame.type === 'result', + isTurnOutput(frame) || isLifecycleFrame(frame, 'cancelled') || isIdleFrame(frame) + }) + expect(replay.adapter.holdsDispatch('session-1')).toBe(true) + expect(replay.settlementsFor('client-A')).toEqual([]) + + replay.connection.handlers.onMessage?.({ + type: 'system', + subtype: 'session_state_changed', + state: 'idle', + uuid: 'idle-after-throw', + session_id: PROVIDER_SESSION_ID + }) + + expect(replay.settlementsFor('client-A')).toEqual([IDLE_IN_DOUBT]) + expect(replay.adapter.holdsDispatch('session-1')).toBe(false) + }) + + // 2.1.280's end-of-turn cleanup can report idle before it re-reads its queue. + it('leaves a queued send the CLI idles before starting to open its turn and be accepted', async () => { + const replay = await replayCapture('auth-failed', { afterFrame: (frame) => - frame.type === 'command_lifecycle' && frame.state === 'started' - ? [{ ...frame, state: 'queued' }] + isLifecycleFrame(frame, 'queued') + ? [ + { + type: 'system', + subtype: 'session_state_changed', + state: 'idle', + uuid: 'idle-before-start', + session_id: frame.session_id + } + ] : [] }) + const uuid = replay.liveUuid('client-A') + expect(replay.settlementsFor('client-A')).toEqual([acceptedAs(uuid)]) + expect(replay.turnStates.get(uuid)).toBe('completed') + }) + + it('holds the child for a send the CLI has only queued, and does not release it at idle', async () => { + const replay = await replayCapture('auth-failed', { + omitFrame: (frame) => + isTurnOutput(frame) || + isLifecycleFrame(frame, 'started') || + isLifecycleFrame(frame, 'cancelled') + }) + + expect(replay.idles).toEqual(['session-1']) + expect(replay.adapter.holdsDispatch('session-1')).toBe(true) expect(replay.settlementsFor('client-A')).toEqual([]) }) + + it('holds a queued send until the CLI withdraws it', async () => { + const held: boolean[] = [] + const replay = await replayCapture('cancel-async', { + atControl: async (point) => { + // The lead was echoed already; only the queued follow-up is left. + held.push(point.adapter.holdsDispatch('session-1')) + point.deliverInFlight() + held.push(point.adapter.holdsDispatch('session-1')) + } + }) + + expect(held).toEqual([true, false]) + expect(replay.settlementsFor('client-B')).toEqual([WITHDRAWN]) + }) + + it('holds a queued send until it starts and is echoed', async () => { + const held: boolean[] = [] + const replay = await replayCapture('batch-lead', { + // Without its turn's end or the idle, only the start and echo can let it go. + omitFrame: (frame) => + frame.type === 'result' || isLifecycleFrame(frame, 'completed') || isIdleFrame(frame), + atControl: async (point) => { + point.deliverInFlight() + // The batch lead is withdrawn; the send behind it is only queued. + held.push(point.adapter.holdsDispatch('session-1')) + } + }) + + expect(held).toEqual([true]) + expect(replay.adapter.holdsDispatch('session-1')).toBe(false) + expect(replay.settlementsFor('client-C')).toEqual([acceptedAs(replay.liveUuid('client-C'))]) + }) + + it.each(['interrupt-lost', 'cancel-async', 'batch-lead', 'auth-failed'])( + 'holds nothing once the %s capture has run to its idle', + async (name) => { + const replay = await replayCapture(name) + + expect(replay.adapter.holdsDispatch('session-1')).toBe(false) + } + ) }) describe('the CLI reporting its session idle', () => { diff --git a/src/main/claude/claude-command-lifecycle.ts b/src/main/claude/claude-command-lifecycle.ts index 652a431d5383..a5d4408799bd 100644 --- a/src/main/claude/claude-command-lifecycle.ts +++ b/src/main/claude/claude-command-lifecycle.ts @@ -1,15 +1,37 @@ // What Claude's per-command `command_lifecycle` frames (msg_lifecycle_v1) settle. // -// `cancelled` is not by itself a withdrawal: a command the CLI already started also ends -// `cancelled` when its turn is interrupted or fails (measured on 2.1.280). Only a command -// cancelled before it started was withdrawn. That frame lands ahead of the control answer, so -// it settles the send even when the interrupt or cancel_async_message answer is lost. +// A terminal state for a command that never started is a verdict: `cancelled` was withdrawn, +// `discarded` was dropped when the CLI ended its session, `refused` was declined before it +// queued. None ran. After `started`, 2.1.280 sends only `cancelled`, when an interrupt or a hard +// failure ends the turn (measured), so the send may be in the conversation: any end is doubt. +// An unstarted send's `cancelled` lands before the Stop's answer, so it settles if that is lost. -import { settleCancelledClaudeDispatchWaiters } from './claude-structured-dispatch' +import { + agentSessionFailureFact, + type SubmissionRejectionKind +} from '../../shared/agent-session-failure' +import { + DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED, + DISPATCH_DOUBT_PROVIDER_IDLE +} from '../native-chat/agent-session-journal/journal-dispatch-doubt-reasons' +import { + rejectClaudeDispatchWaiters, + releaseClaudeDispatchWaitersInDoubt +} from './claude-structured-dispatch' import { readClaudeFrameString } from './claude-structured-init-proof' import type { ClaudeLateDispatchSettlement } from './claude-replay-turn-resolution' import type { ClaudeSession } from './claude-structured-session-state' +const UNSTARTED_VERDICT = { + cancelled: 'cancelled', + discarded: 'notDelivered', + refused: 'providerRejected' +} satisfies Record + +function isTerminalState(state: unknown): state is keyof typeof UNSTARTED_VERDICT { + return state === 'cancelled' || state === 'discarded' || state === 'refused' +} + export function observeClaudeCommandLifecycle( session: ClaudeSession, message: Record, @@ -27,7 +49,36 @@ export function observeClaudeCommandLifecycle( if (state === 'started' || (state === 'queued' && waiter.commandLifecycle !== 'started')) { // Forward only: a redelivered command re-emits `queued`, but it has still started. waiter.commandLifecycle = state - } else if (state === 'cancelled' && waiter.commandLifecycle !== 'started') { - settleCancelledClaudeDispatchWaiters(session, [waiter.sentUuid], onSettledLate) + } else if (isTerminalState(state) && waiter.commandLifecycle === 'started') { + releaseClaudeDispatchWaitersInDoubt( + session, + [waiter], + DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED, + onSettledLate + ) + } else if (isTerminalState(state)) { + rejectClaudeDispatchWaiters( + session, + [waiter.sentUuid], + agentSessionFailureFact(UNSTARTED_VERDICT[state]), + onSettledLate + ) } } + +/** A send the CLI took, until its echo, a terminal state or exit: the sweep must not rest its child. */ +export function claudeHoldsDispatch(session: ClaudeSession): boolean { + return session.dispatchWaiters.some((waiter) => waiter.commandLifecycle !== undefined) +} + +/** + * At idle, only a `started` send still unanswered is doubt: a turn that threw leaves it with no + * terminal state. A `queued` one may still start: 2.1.280's code idles before re-reading its queue. + */ +export function releaseClaudeDispatchesUnansweredAtIdle( + session: ClaudeSession, + onSettledLate?: ClaudeLateDispatchSettlement +): void { + const started = session.dispatchWaiters.filter((waiter) => waiter.commandLifecycle === 'started') + releaseClaudeDispatchWaitersInDoubt(session, started, DISPATCH_DOUBT_PROVIDER_IDLE, onSettledLate) +} diff --git a/src/main/claude/claude-replay-turn-resolution.ts b/src/main/claude/claude-replay-turn-resolution.ts index 9db40feac49e..fe2bb1523f95 100644 --- a/src/main/claude/claude-replay-turn-resolution.ts +++ b/src/main/claude/claude-replay-turn-resolution.ts @@ -3,10 +3,14 @@ // A replay or result is joined to its waiter by the client uuid Claude echoes. // A send Claude FOLDS into the running request cycle is replayed mid-cycle with // the client uuid adopted: once that cycle has done work, that replay is a -// delivery receipt and opens no boundary. A `command_lifecycle` frame opens no turn; its -// `cancelled` can settle a send the CLI withdrew (`claude-command-lifecycle.ts`). +// delivery receipt and opens no boundary. A `command_lifecycle` frame opens no turn; it and the +// CLI going idle settle sends the CLI took but never echoed (`claude-command-lifecycle.ts`). -import { observeClaudeCommandLifecycle } from './claude-command-lifecycle' +import { + observeClaudeCommandLifecycle, + releaseClaudeDispatchesUnansweredAtIdle +} from './claude-command-lifecycle' +import { claudeSessionStateEndsTurn } from './claude-session-state-turn-over' import { forgetRetiredWaiter } from './claude-structured-dispatch-waiters' import { claudeHasReplayContent, @@ -34,6 +38,10 @@ export function resolveClaudeReplayTurn( observeClaudeCommandLifecycle(session, message, onSettledLate) return null } + if (claudeSessionStateEndsTurn(message)) { + releaseClaudeDispatchesUnansweredAtIdle(session, onSettledLate) + return null + } const envelope = readClaudeMessageEnvelope(message) const isUserReplay = envelope?.role === 'user' && diff --git a/src/main/claude/claude-structured-dispatch-waiters.ts b/src/main/claude/claude-structured-dispatch-waiters.ts index 92b6a5048b2f..2c5df3c671eb 100644 --- a/src/main/claude/claude-structured-dispatch-waiters.ts +++ b/src/main/claude/claude-structured-dispatch-waiters.ts @@ -13,8 +13,10 @@ export function forgetRetiredWaiter(session: ClaudeSession, waiter: ClaudeDispat * A waiter with no deadline. A mid-turn send Claude folds into the running turn * is replayed mid-turn; one it runs later is replayed only when its own turn * starts — an interval bounded only by the previous turn. Elapsed time is - * therefore not evidence about delivery, and nothing here expires. Waiters are - * retired by process facts instead: a failed write, or child exit. + * therefore not evidence about delivery, and nothing here expires. Waiters end + * by provider and process facts instead: an echo, a `command_lifecycle` frame, + * the CLI's idle after `started` (`claude-command-lifecycle.ts`), a failed + * write, or child exit. */ export function waitForReplay( session: ClaudeSession, diff --git a/src/main/claude/claude-structured-dispatch.ts b/src/main/claude/claude-structured-dispatch.ts index c208cceccdbf..87e706ddbbc3 100644 --- a/src/main/claude/claude-structured-dispatch.ts +++ b/src/main/claude/claude-structured-dispatch.ts @@ -7,7 +7,7 @@ import { } from './claude-structured-dispatch-waiters' import type { AgentJournalMessageItem } from '../../shared/agent-session-journal-types' import type { AgentSessionDispatchOutcome } from '../native-chat/agent-session-wire/structured-agent-session-adapter' -import type { ClaudeSession } from './claude-structured-session-state' +import type { ClaudeDispatchWaiter, ClaudeSession } from './claude-structured-session-state' import type { ClaudeLateDispatchSettlement } from './claude-replay-turn-resolution' import { claudeDispatchContentKey, @@ -18,7 +18,10 @@ import { } from './claude-structured-dispatch-content' import { dispatchWriteOutcomeUnknownReason } from '../native-chat/agent-session-journal/journal-dispatch-doubt-reasons' import { DISPATCH_REJECTED_QUEUE_FULL } from '../../shared/structured-agent-session-dispatch-rejection' -import { agentSessionFailureFact } from '../../shared/agent-session-failure' +import { + agentSessionFailureFact, + type SubmissionRejectionFact +} from '../../shared/agent-session-failure' import type { AgentJournalDispatchRejection } from '../../shared/agent-session-failure-words' import { claudeUnwrittenUserMessageError, @@ -37,10 +40,25 @@ export function settleCancelledClaudeDispatchWaiters( cancelledUuids: readonly string[], onSettledLate?: ClaudeLateDispatchSettlement ): void { - const cancelled = new Set(cancelledUuids) - const activeWaiters = session.dispatchWaiters.filter((waiter) => cancelled.has(waiter.sentUuid)) + rejectClaudeDispatchWaiters( + session, + cancelledUuids, + agentSessionFailureFact('cancelled'), + onSettledLate + ) +} + +/** Settles sends the CLI proved it will never run, each with the fact that says why. */ +export function rejectClaudeDispatchWaiters( + session: ClaudeSession, + uuids: readonly string[], + fact: SubmissionRejectionFact, + onSettledLate?: ClaudeLateDispatchSettlement +): void { + const named = new Set(uuids) + const activeWaiters = session.dispatchWaiters.filter((waiter) => named.has(waiter.sentUuid)) const retiredWaiters = session.retiredDispatchWaiters.filter((waiter) => - cancelled.has(waiter.sentUuid) + named.has(waiter.sentUuid) ) for (const waiter of activeWaiters) { forgetWaiter(session, waiter) @@ -54,14 +72,31 @@ export function settleCancelledClaudeDispatchWaiters( onSettledLate?.({ clientMessageId: waiter.clientMessageId, state: 'rejected', - ...claudeDispatchRejection(agentSessionFailureFact('cancelled')) + ...claudeDispatchRejection(fact) }) } } } -/** Nothing expires a waiter, so the child's death is what ends every live one. - * Retired rather than dropped: their identities stay joinable, bounded by +/** Releases sends the CLI took and let go without an echo or a verdict: each may have run, so each + * settles as doubt, never re-sent. Retired, not dropped, so a late replay still accepts it. */ +export function releaseClaudeDispatchWaitersInDoubt( + session: ClaudeSession, + waiters: readonly ClaudeDispatchWaiter[], + reason: string, + onSettledLate?: ClaudeLateDispatchSettlement +): void { + for (const waiter of waiters) { + retireWaiter(session, waiter) + waiter.resolve(null) + if (waiter.clientMessageId) { + onSettledLate?.({ clientMessageId: waiter.clientMessageId, state: 'unknown', reason }) + } + } +} + +/** Nothing expires a waiter; the child's death ends every one its echo, a lifecycle frame or the + * CLI's idle has not. Retired rather than dropped: their identities stay joinable, bounded by * `MAX_RETIRED_DISPATCH_WAITERS`. */ export function retireClaudeDispatchWaiters(session: ClaudeSession): void { failClaudeStartup(session, new Error('claude stream-json ended before startup completed')) diff --git a/src/main/claude/claude-structured-session-acquisition.ts b/src/main/claude/claude-structured-session-acquisition.ts index b4adbfccfa5c..0a160d99b934 100644 --- a/src/main/claude/claude-structured-session-acquisition.ts +++ b/src/main/claude/claude-structured-session-acquisition.ts @@ -118,7 +118,7 @@ export async function acquireClaudeSession({ if (message.type === 'result' && sessions.get(sessionId) === liveSession) { persistClaudeTurnResumePoint(sessionId, liveSession, deps) } - // The CLI idles only once its queue drains, so it holds none of this session's sends. + // The CLI's idle releases its doubted sends; a late echo still accepts one it goes on to run. if (claudeSessionStateEndsTurn(message) && sessions.get(sessionId) === liveSession) { deps.onSessionIdle?.({ sessionId }) } diff --git a/src/main/claude/claude-structured-session-adapter.ts b/src/main/claude/claude-structured-session-adapter.ts index 64d3c67c2277..d2788bf40276 100644 --- a/src/main/claude/claude-structured-session-adapter.ts +++ b/src/main/claude/claude-structured-session-adapter.ts @@ -7,6 +7,7 @@ import type { } from '../native-chat/agent-session-wire/structured-agent-session-adapter' import { stopClaudeBackgroundTasks } from './claude-structured-control-actions' import { dispatchClaudeTurn } from './claude-structured-dispatch' +import { claudeHoldsDispatch } from './claude-command-lifecycle' import { releaseClaudeAcquisition } from './claude-structured-acquisition-release' import { acquireClaudeSession } from './claude-structured-session-acquisition' import { supportsClaudeStructuredLocation } from './claude-structured-location-support' @@ -227,6 +228,10 @@ export class ClaudeStructuredSessionAdapter implements StructuredAgentSessionAda } readCommands: NonNullable = (sessionId) => this.sessions.get(sessionId)?.commands.commands + holdsDispatch = (sessionId: string): boolean => { + const session = this.sessions.get(sessionId) + return session ? claudeHoldsDispatch(session) : false + } answerPrompt: StructuredAgentSessionAdapter['answerPrompt'] = (request) => answerClaudeStructuredPrompt({ request, sessions: this.sessions }) setOption: StructuredAgentSessionAdapter['setOption'] = (input) => diff --git a/src/main/claude/claude-structured-session-state.ts b/src/main/claude/claude-structured-session-state.ts index b40aaba81c44..0d43dbb99ec8 100644 --- a/src/main/claude/claude-structured-session-state.ts +++ b/src/main/claude/claude-structured-session-state.ts @@ -87,6 +87,8 @@ export type ClaudeLateDispatchOutcome = providerIdentity: AgentJournalItemIdentity } | ({ clientMessageId: string; state: 'rejected' } & AgentJournalDispatchRejection) + /** The CLI took the send and let it go unanswered: it may have run, so it is never re-sent. */ + | { clientMessageId: string; state: 'unknown'; reason: string } export type ClaudeStructuredSessionAdapterDeps = { resolveLaunch: (input: { @@ -95,7 +97,7 @@ export type ClaudeStructuredSessionAdapterDeps = { onEvent?: (event: ClaudeStructuredSessionEvent) => void /** Direct settlement path for provider-proven late dispatch outcomes. */ onDispatchSettledLate?: (input: { sessionId: string } & ClaudeLateDispatchOutcome) => void - /** The CLI reported `session_state_changed idle`, which it sends only once its queue drains. */ + /** The CLI reported `session_state_changed idle`: its turn is over. */ onSessionIdle?: (input: { sessionId: string }) => void onBackgroundTasksChanged?: ( sessionId: string, @@ -139,7 +141,7 @@ export type ClaudeDispatchWaiter = { requestedAt: number | null /** Set when the provider replay settled this waiter before send returned. */ settledUuid?: string - /** The write failed or the child died, but a replay may still name it. */ + /** Settled with no echo (failed write, child exit, doubt after start); a replay may name it. */ retired?: boolean /** The CLI's last non-terminal `command_lifecycle` state for this send; in memory only. */ commandLifecycle?: 'queued' | 'started' @@ -158,7 +160,7 @@ export type ClaudeSession = { acquisitionGeneration: string prompts: ClaudePromptRegistry dispatchWaiters: ClaudeDispatchWaiter[] - /** Bounded identities for dispatches whose child died or whose write failed. */ + /** Bounded identities for dispatches settled without an echo; a late replay still accepts one. */ retiredDispatchWaiters: ClaudeDispatchWaiter[] /** Once a retired waiter is evicted, legacy content-only replay matching is unsafe. */ replayContentFallbackBlocked: boolean diff --git a/src/main/native-chat/agent-session-journal/journal-dispatch-doubt-reasons.ts b/src/main/native-chat/agent-session-journal/journal-dispatch-doubt-reasons.ts index 6c01e6224dc0..1aaa0b76df95 100644 --- a/src/main/native-chat/agent-session-journal/journal-dispatch-doubt-reasons.ts +++ b/src/main/native-chat/agent-session-journal/journal-dispatch-doubt-reasons.ts @@ -27,6 +27,9 @@ export const DISPATCH_DOUBT_SUBMISSION_MISSING = 'durable_send_submission_missin * left that could still acknowledge the message. */ export const DISPATCH_DOUBT_PROVIDER_IDLE = 'provider_idle_before_acknowledgement' +/** The provider started the message, then ended its turn before acknowledging it. */ +export const DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED = 'provider_ended_before_acknowledgement' + /** The SDK took the frame, but its input pump did not prove whether the write completed. */ export const DISPATCH_DOUBT_WRITE_OUTCOME_UNKNOWN = 'provider_write_outcome_unknown' diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-adapter-router.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-adapter-router.ts index bb003f93427d..9a759ba0411b 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-adapter-router.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-adapter-router.ts @@ -112,6 +112,9 @@ export class StructuredAgentSessionAdapterRouter implements StructuredAgentSessi sessionId ) => this.liveOwnerOrNull(sessionId)?.backgroundTaskState?.(sessionId) + holdsDispatch = (sessionId: string): boolean => + this.liveOwnerOrNull(sessionId)?.holdsDispatch?.(sessionId) ?? false + readCommands: NonNullable = (sessionId) => this.liveOwnerOrNull(sessionId)?.readCommands?.(sessionId) diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-adapter.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-adapter.ts index a689334c8b43..5b8a9b190ffc 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-adapter.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-adapter.ts @@ -331,6 +331,9 @@ export type StructuredAgentSessionAdapter = { taskId?: string }): Promise<{ cancelled: boolean }> backgroundTaskState?(sessionId: string): AgentSessionBackgroundTaskState | null | undefined + /** The provider reported taking a send it has neither answered nor ended, as a queued follow-up + * or a silent retry does. Derived from the live child; false with none. */ + holdsDispatch?(sessionId: string): boolean /** The `/` surface the running provider reports for itself. Undefined when the * provider never reports one, which is what keeps the client on its catalog. */ readCommands?(sessionId: string): AgentSessionSlashCommand[] | undefined diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-close.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-close.test.ts index bf3cd1f96a60..977b0ff55cfd 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-close.test.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-close.test.ts @@ -313,6 +313,7 @@ describe('the wind-down retry with a message queued (P2-31)', () => { deliveryActive: () => true, backgroundTaskState: () => undefined, hasOpenDispatch: () => false, + providerHoldsDispatch: () => false, stopAgent, stopStartingAgent: stopAgent, closeConversation: vi.fn(async () => false), diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts index f69129183b4a..4ecfea0381b4 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-conversation-lifetime.ts @@ -74,6 +74,7 @@ export function createStructuredAgentSessionConversationLifetime(host: { const record = deps().store.getRecord(sessionId) return record !== null && deps().hasOpenDispatch?.(record) === true }, + providerHoldsDispatch: (sessionId) => deps().adapter.holdsDispatch?.(sessionId) === true, stopAgent, // A host stop: the delivery loop waiting on this child writes the one error row and rejects // what is queued with it, both worded from the hostStopped fact. diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-host-mutations.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-host-mutations.ts index 01143eb4bb47..3055bb2d8789 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-host-mutations.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-host-mutations.ts @@ -7,10 +7,7 @@ // admitted without the writer lease; the delivery loop starts the provider child a send needs, and // an operation only the provider can perform starts it before admission. -import type { - AgentJournalItemIdentity, - AgentJournalMessageItem -} from '../../../shared/agent-session-journal-types' +import type { AgentJournalMessageItem } from '../../../shared/agent-session-journal-types' import type { AgentSessionCancelResult, AgentSessionMutationEnvelope, @@ -21,10 +18,7 @@ import type { AgentSessionThreadGoalChange, AgentSessionThreadGoalResult } from '../../../shared/agent-session-wire' -import { - agentSessionFailureWords, - type AgentJournalDispatchRejection -} from '../../../shared/agent-session-failure-words' +import { agentSessionFailureWords } from '../../../shared/agent-session-failure-words' import { agentSessionFailureFact } from '../../../shared/agent-session-failure' import type { AgentSessionPromptRequest } from './structured-agent-session-turns-prompt' import { threadGoalPlan } from './structured-agent-session-thread-goal' @@ -32,7 +26,6 @@ import { isMainAgentWorkingOnceFlushed, performCancel } from './structured-agent-session-turns-cancel' -import { structuredAgentSessionConversationFence } from './structured-agent-session-provider-child' import { admitAndRunAgentSessionMutation, type AgentSessionMutationRequest, @@ -281,41 +274,6 @@ export function changeStructuredAgentSessionThreadGoal( ) } -/** Settle provider-proven delivery independently of an in-flight client mutation. */ -export async function settleStructuredAgentSessionLateDispatch( - context: StructuredAgentSessionMutationContext, - input: { - sessionId: string - clientMessageId: string - } & ( - | { providerIdentity: AgentJournalItemIdentity } - | ({ state: 'rejected' } & AgentJournalDispatchRejection) - ) -): Promise { - const session = context.sessions.get(input.sessionId) - if (!session) { - return - } - const fence = structuredAgentSessionConversationFence(context.deps.store, input.sessionId) - // The journal queue drains before close; the host queue would defer this past teardown. - await session.journal.resolveDispatch( - 'providerIdentity' in input - ? { - clientMessageId: input.clientMessageId, - state: 'accepted', - providerIdentity: input.providerIdentity, - fence - } - : { - clientMessageId: input.clientMessageId, - state: 'rejected', - reason: input.reason, - rejection: input.rejection, - fence - } - ) -} - /** The host's thin mutation surface. Each call re-reads the context, so a session * map or fence that moves between calls is never captured by a stale closure. */ export function structuredAgentSessionMutationDelegates( diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts index bcda340ee79e..ab607328514a 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-host.ts @@ -34,9 +34,9 @@ import type { StructuredAgentSessionAttachContext } from './structured-agent-ses import * as sessionTabs from './structured-agent-session-host-tabs' import { structuredAgentSessionMutationDelegates, - settleStructuredAgentSessionLateDispatch, type StructuredAgentSessionMutationContext } from './structured-agent-session-host-mutations' +import { settleStructuredAgentSessionLateDispatch } from './structured-agent-session-late-dispatch' import { releaseStructuredAgentSessionUnansweredDispatches } from './structured-agent-session-unanswered-dispatch-release' import { flushStructuredAgentSessionHost } from './structured-agent-session-host-teardown' import type { diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.test.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.test.ts index 25288dc64905..1345adc574f7 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.test.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.test.ts @@ -240,6 +240,23 @@ describe('the idle sweep', () => { await vi.waitFor(() => expect(rig.adapter.closeSession).toHaveBeenCalledWith(SESSION)) }) + // A Claude retrying a rate-limited request has taken the send but echoes nothing, so no turn row + // exists yet; only the provider can say it still holds the send. + it('never stops an agent while its provider holds a send, and gives a full window once it lets go', async () => { + await foundRestTestChat(rig) + rig.adapter.holdsDispatch.mockReturnValue(true) + rig.clock.now += 2 * IDLE_MS + + await sweepTicks(12) + expect(rig.adapter.closeSession).not.toHaveBeenCalled() + expect(rig.adapter.holdsDispatch).toHaveBeenCalledWith(SESSION) + rig.adapter.holdsDispatch.mockReturnValue(false) + await sweepTicks() + expect(rig.adapter.closeSession).not.toHaveBeenCalled() + rig.clock.now += IDLE_MS + 1 + await vi.waitFor(() => expect(rig.adapter.closeSession).toHaveBeenCalledWith(SESSION)) + }) + it('keeps a child an unanswered prompt waits on (P2-22 i)', async () => { await foundRestTestChat(rig) providerEvents().appendItem( @@ -304,6 +321,7 @@ describe('the idle sweep with no child running (P2-22 ii)', () => { deliveryActive: () => false, backgroundTaskState: () => undefined, hasOpenDispatch: () => false, + providerHoldsDispatch: () => false, stopAgent, stopStartingAgent: stopAgent, closeConversation, diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.ts index 1fc223808f6b..cdf412a7bcd5 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-idle-sweep.ts @@ -30,6 +30,8 @@ export type StructuredAgentSessionIdleSweepDeps = { backgroundTaskState: (sessionId: string) => AgentSessionBackgroundTaskState | null | undefined /** An orchestration dispatch that still owns this session's worker; derived from its database. */ hasOpenDispatch: (sessionId: string) => boolean + /** The provider holds a send it took and has not answered; see `holdsDispatch`. */ + providerHoldsDispatch: (sessionId: string) => boolean /** Each of these runs inside the session's serialize and never takes it again. */ stopAgent: (sessionId: string) => Promise stopStartingAgent: (sessionId: string) => Promise @@ -145,6 +147,7 @@ export class StructuredAgentSessionIdleSweep { this.queuedOrDelivering(sessionId, session) || agentChildWorkLiveness(this.deps.backgroundTaskState(sessionId)?.tasks) !== null || this.deps.hasOpenDispatch(sessionId) || + this.deps.providerHoldsDispatch(sessionId) || hasPendingStructuredAgentSessionPrompt(items) ) } diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-late-dispatch.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-late-dispatch.ts new file mode 100644 index 000000000000..ba0ab9231261 --- /dev/null +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-late-dispatch.ts @@ -0,0 +1,49 @@ +import type { AgentJournalItemIdentity } from '../../../shared/agent-session-journal-types' +import type { AgentJournalDispatchRejection } from '../../../shared/agent-session-failure-words' +import type { StructuredAgentSessionMutationContext } from './structured-agent-session-host-mutations' +import { structuredAgentSessionConversationFence } from './structured-agent-session-provider-child' + +/** Settle a provider-proven outcome independently of an in-flight client mutation. `unknown` is + * released doubt: the provider let the send go unanswered, so nothing re-sends it. */ +export async function settleStructuredAgentSessionLateDispatch( + context: StructuredAgentSessionMutationContext, + input: { + sessionId: string + clientMessageId: string + } & ( + | { providerIdentity: AgentJournalItemIdentity } + | ({ state: 'rejected' } & AgentJournalDispatchRejection) + | { state: 'unknown'; reason: string } + ) +): Promise { + const session = context.sessions.get(input.sessionId) + if (!session) { + return + } + const fence = structuredAgentSessionConversationFence(context.deps.store, input.sessionId) + // The journal queue drains before close; the host queue would defer this past teardown. + await session.journal.resolveDispatch( + 'providerIdentity' in input + ? { + clientMessageId: input.clientMessageId, + state: 'accepted', + providerIdentity: input.providerIdentity, + fence + } + : input.state === 'unknown' + ? { + clientMessageId: input.clientMessageId, + state: 'unknown', + reason: input.reason, + fence, + recovered: true + } + : { + clientMessageId: input.clientMessageId, + state: 'rejected', + reason: input.reason, + rejection: input.rejection, + fence + } + ) +} diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts index 0c890767ac22..9e5a136d4a45 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-rest-test-rig.ts @@ -45,6 +45,7 @@ export type RestTestAdapter = { NonNullable > backgroundTaskState: Mock> + holdsDispatch: Mock> readOptions: Mock> } @@ -146,6 +147,7 @@ export async function createRestTestRig( dispatch: vi.fn(async () => acceptedDispatch()), acknowledgeSessionRelease: vi.fn(), backgroundTaskState: vi.fn(() => undefined), + holdsDispatch: vi.fn(() => false), readOptions: vi.fn(async () => ({ models: [], current: { model: 'gpt-live' } })) } const hostFor = (overrides: Partial) => diff --git a/src/main/native-chat/agent-session-wire/structured-agent-session-unanswered-dispatch-release.ts b/src/main/native-chat/agent-session-wire/structured-agent-session-unanswered-dispatch-release.ts index 713838c177ec..62078ef853c6 100644 --- a/src/main/native-chat/agent-session-wire/structured-agent-session-unanswered-dispatch-release.ts +++ b/src/main/native-chat/agent-session-wire/structured-agent-session-unanswered-dispatch-release.ts @@ -3,7 +3,7 @@ import type { StructuredAgentSessionHostDeps } from './structured-agent-session- import type { StructuredAgentSessionMutationContext } from './structured-agent-session-host-mutations' /** - * Releases sends the provider can no longer be holding. + * Releases doubted sends once the provider reports it has stopped running. * * A dispatch whose RPC timed out is recorded `unknown` — doubt, never proof of * non-delivery — and a live `unknown` reads as work still owed, so the session diff --git a/src/main/runtime/claude-structured-idle-releases-unknown.test.ts b/src/main/runtime/claude-structured-idle-releases-unknown.test.ts index becdc9f2b389..9fa9cbfcfa6a 100644 --- a/src/main/runtime/claude-structured-idle-releases-unknown.test.ts +++ b/src/main/runtime/claude-structured-idle-releases-unknown.test.ts @@ -1,12 +1,17 @@ // A Claude send whose write ended in doubt is recorded `unknown`, and a live `unknown` reads as -// work still owed. The CLI reports `session_state_changed idle` only once its queue has drained, -// so that report retires the doubt; a send whose dispatch is still `pending` is left alone. +// work still owed. The CLI reporting `session_state_changed idle` retires the doubt; a send whose +// dispatch is still `pending` is left alone unless the CLI had started it. // Against the production runtime, adapter, record store and host, with only the CLI scripted. import { afterEach, describe, expect, it, vi } from 'vitest' import { computeAgentSessionPayloadFingerprint } from '../../shared/agent-session-mutation-envelope' import { claudeSessionIdForOrcaSession } from '../claude/claude-structured-launch-resolution' -import { DISPATCH_DOUBT_WRITE_OUTCOME_UNKNOWN } from '../native-chat/agent-session-journal/journal-dispatch-doubt-reasons' +import { + DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED, + DISPATCH_DOUBT_PROVIDER_IDLE, + DISPATCH_DOUBT_WRITE_OUTCOME_UNKNOWN +} from '../native-chat/agent-session-journal/journal-dispatch-doubt-reasons' +import { hasUnansweredStructuredAgentSessionDispatch } from '../../shared/structured-agent-session-unanswered-dispatch' import { hostTestMessage } from '../native-chat/agent-session-wire/structured-agent-session-host-test-data' import type { StructuredAgentSessionHost } from '../native-chat/agent-session-wire/structured-agent-session-host' import { createScriptedClaudeRuntime } from './structured-claude-scripted-runtime-test-support' @@ -106,3 +111,81 @@ describe('a live Claude chat whose CLI reports idle', () => { expect((await submission(host, pending))?.recovered).toBeUndefined() }) }) + +function lifecycle(commandUuid: string, state: string): Record { + return { + type: 'command_lifecycle', + command_uuid: commandUuid, + state, + uuid: `lifecycle-${state}`, + session_id: PROVIDER_SESSION + } +} + +describe('a live Claude chat whose CLI took a send and let it go without an echo', () => { + it.each([ + ['its turn ended cancelled', 'cancelled', DISPATCH_DOUBT_PROVIDER_ENDED_UNANSWERED], + ['it went idle with no terminal state', 'idle', DISPATCH_DOUBT_PROVIDER_IDLE] + ])('records released doubt when %s', async (_label, ending, reason) => { + const host = await claude.install() + await expect(host.attach(CALLER, claude.attachParams(SESSION, null))).resolves.toMatchObject({ + ok: true + }) + const child = claude.child(SESSION) + let sentUuid = '' + child.connection.send = async (message, beforeDispatch) => { + await beforeDispatch?.() + sentUuid = String(message.uuid) + } + const taken = await send(host, 'retrying') + // The pending row lands before the adapter's write; the write is what names the uuid. + await vi.waitFor(() => expect(sentUuid).not.toBe('')) + expect(await submission(host, taken)).toMatchObject({ dispatchState: 'pending' }) + child.handlers.onMessage?.(lifecycle(sentUuid, 'queued')) + child.handlers.onMessage?.(lifecycle(sentUuid, 'started')) + expect(host.deps.adapter.holdsDispatch?.(SESSION)).toBe(true) + + child.handlers.onMessage?.( + ending === 'idle' ? sessionState('idle') : lifecycle(sentUuid, ending) + ) + + await vi.waitFor(async () => + expect(await submission(host, taken)).toMatchObject({ + dispatchState: 'unknown', + recovered: true, + reason + }) + ) + expect(host.deps.adapter.holdsDispatch?.(SESSION)).toBe(false) + // Released doubt: the chat stops reading working, and nothing re-sends it. + expect( + hasUnansweredStructuredAgentSessionDispatch((await host.journalSnapshot(SESSION)).submissions) + ).toBe(false) + }) + + it('holds a send it has only queued, through an idle that may precede its start', async () => { + const host = await claude.install() + await expect(host.attach(CALLER, claude.attachParams(SESSION, null))).resolves.toMatchObject({ + ok: true + }) + const child = claude.child(SESSION) + let sentUuid = '' + child.connection.send = async (message, beforeDispatch) => { + await beforeDispatch?.() + sentUuid = String(message.uuid) + } + const queued = await send(host, 'behind a turn') + await vi.waitFor(() => expect(sentUuid).not.toBe('')) + // POSITIVE CONTROL: written but not yet taken, nothing holds the child. + expect(host.deps.adapter.holdsDispatch?.(SESSION)).toBe(false) + + child.handlers.onMessage?.(lifecycle(sentUuid, 'queued')) + expect(host.deps.adapter.holdsDispatch?.(SESSION)).toBe(true) + child.handlers.onMessage?.(sessionState('idle')) + await new Promise((resolve) => setTimeout(resolve, 50)) + + expect(host.deps.adapter.holdsDispatch?.(SESSION)).toBe(true) + expect(await submission(host, queued)).toMatchObject({ dispatchState: 'pending' }) + expect((await submission(host, queued))?.recovered).toBeUndefined() + }) +}) diff --git a/src/main/runtime/structured-agent-session-runtime.ts b/src/main/runtime/structured-agent-session-runtime.ts index bb82e070e125..cbf5352f8676 100644 --- a/src/main/runtime/structured-agent-session-runtime.ts +++ b/src/main/runtime/structured-agent-session-runtime.ts @@ -232,7 +232,7 @@ async function installOnJournal( }) ) } - // The provider going idle is what re-derives a doubted send: it can no longer be holding it. + // The provider going idle is what re-derives a doubted send; a late echo still accepts it. const releaseUnansweredDispatches = ({ sessionId }: { sessionId: string }): void => { void host ?.releaseUnansweredDispatches({ From 7cae036ebf75a6afe7cc4f90e144219a27cac9d7 Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 14:13:51 -0700 Subject: [PATCH 21/79] fix(native-chat): a failed Codex turn shows its error, not a raw thread-status row (#23704) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(native-chat): a failed Codex turn shows its error, not a raw thread-status row Codex reports `thread/status/changed` {systemError} just before the `error` frame of a failed turn. The translator read it for session state, then let it fall through to the generic-frame fallback, whose payload check reads `systemError` as a failure and printed "codex · notification:thread/status/changed" in red above the real error row. The translator now owns the notification: it reads the stopped-running verdict exactly as before, then journals nothing for any arm (idle, active, notLoaded, systemError). The error row that follows still carries Codex's sentence and still fails the turn. * refactor(native-chat): the Codex thread status is owned through the typed-translator registry The translator already reads every thread status for session state. Listing the kind beside Claude's background-task frames makes that ownership visible where the fallback consults it, instead of a second method check in the translator. * chore(native-chat): the typed-translator registry key is checked against the provider table A mistyped provider key compiled and silently brought the row back. Also retire the renamed constant from a Claude test comment and say which path the classifier-only fallback test covers. * fix(native-chat): the Codex translator vouches only for the thread status it read The covered flag means this exact frame was handled; passing it for every notification would silently hide any Codex kind later added to the registry. Also say why a state-only kind may be listed, and retitle the cross-provider coverage test now that Codex has entries. --- ...urnal-translation-background-tasks.test.ts | 2 +- ...codex-structured-journal-generic-frames.ts | 17 ++++- ...ctured-journal-translation-streams.test.ts | 63 ++++++++++++------- .../codex-structured-journal-translation.ts | 6 +- .../provider-frame-disposition.test.ts | 18 +++++- .../provider-frame-disposition.ts | 36 ++++++++--- .../unhandled-provider-frame.test.ts | 2 +- 7 files changed, 104 insertions(+), 40 deletions(-) diff --git a/src/main/claude/claude-structured-journal-translation-background-tasks.test.ts b/src/main/claude/claude-structured-journal-translation-background-tasks.test.ts index 312eb6d05a27..c8a9e014950c 100644 --- a/src/main/claude/claude-structured-journal-translation-background-tasks.test.ts +++ b/src/main/claude/claude-structured-journal-translation-background-tasks.test.ts @@ -519,7 +519,7 @@ describe('claude journal translation — background task rows', () => { const { translator, fallbackRows, taskRowIds, taskRowTexts } = harness() playFailedBackgroundCommand(translator) - // ABLATION: drop `message:system:task_*` from CLAUDE_TYPED_TRANSLATOR_KINDS + // ABLATION: drop `message:system:task_*` from TYPED_TRANSLATOR_KINDS // and this is `['claude · message:system:task_updated', 'claude · // message:system:task_notification']` — the reported bug exactly. expect(fallbackRows()).toEqual([]) diff --git a/src/main/codex/codex-structured-journal-generic-frames.ts b/src/main/codex/codex-structured-journal-generic-frames.ts index a8b400a6d93d..7c22a49baadf 100644 --- a/src/main/codex/codex-structured-journal-generic-frames.ts +++ b/src/main/codex/codex-structured-journal-generic-frames.ts @@ -1,4 +1,8 @@ -import { unhandledProviderFrameJournalItem } from '../native-chat/agent-session-wire/unhandled-provider-frame' +import { + unhandledProviderFrameJournalItem, + type UnhandledProviderFrameJournalItemOptions +} from '../native-chat/agent-session-wire/unhandled-provider-frame' +import { DEFAULT_JOURNAL_PAYLOAD_LIMITS } from '../native-chat/agent-session-journal/journal-payload-bounds' import type { StructuredAgentSessionEventSink } from '../native-chat/agent-session-wire/structured-agent-session-event-sink' import type { CodexJournalTranslationAdmission, @@ -71,9 +75,16 @@ export class CodexJournalGenericFrames { appendUnhandled( kind: string, payload: unknown, - threadId: string + threadId: string, + options?: UnhandledProviderFrameJournalItemOptions ): CodexJournalTranslationAdmission { - const translated = unhandledProviderFrameJournalItem('codex', kind, payload) + const translated = unhandledProviderFrameJournalItem( + 'codex', + kind, + payload, + DEFAULT_JOURNAL_PAYLOAD_LIMITS, + options + ) // A frame the classifier declines is deliberately not journaled, which is success. // Failing admission here force-closes the provider through the retry queue. if (!translated) { diff --git a/src/main/codex/codex-structured-journal-translation-streams.test.ts b/src/main/codex/codex-structured-journal-translation-streams.test.ts index ce6fd220132f..54a2c017aa05 100644 --- a/src/main/codex/codex-structured-journal-translation-streams.test.ts +++ b/src/main/codex/codex-structured-journal-translation-streams.test.ts @@ -508,37 +508,56 @@ describe('codex journal translation', () => { ]) }) - it('renders a system error carried by a suppressed status kind', () => { - const { translator, tap } = translatorWith() + it.each(['idle', 'active', 'notLoaded', 'systemError'])( + 'journals no row for a %s thread status', + (type) => { + const { translator, tap } = translatorWith() + + translator.handle( + notification('thread/status/changed', { threadId: THREAD_ID, status: { type } }) + ) + + expect(tap.rows).toEqual([]) + } + ) + + it('a failed turn journals its error, not the thread status Codex reports beside it', () => { + const tap = recorder() + const stopped: string[] = [] + const translator = createCodexJournalTranslator({ + sink: tap.sink, + primaryThreadId: () => THREAD_ID, + onPrimaryThreadStoppedRunning: () => stopped.push(THREAD_ID) + }) + translator.handle(TURN_STARTED) translator.handle( notification('thread/status/changed', { threadId: THREAD_ID, status: { type: 'systemError' } }) ) - - const timeline = projectStructuredItemsToNativeChat( - tap.rows.map((row, index) => ({ - itemId: row.key, - revision: 1, - sequence: index + 1, - observedAt: index + 1, - body: row.body - })) - ) - expect(timeline).toEqual([ - expect.objectContaining({ - role: 'system', - blocks: [ - expect.objectContaining({ - providerFrame: expect.objectContaining({ - kind: 'notification:thread/status/changed' - }) - }) - ] + translator.handle( + notification('error', { + threadId: THREAD_ID, + turnId: TURN_ID, + willRetry: false, + error: { message: 'Selected model is at capacity.' } }) + ) + translator.handle(notification('turn/completed', { turn: { id: TURN_ID, status: 'failed' } })) + + const bodies = tap.rows.map((row) => row.body) + expect(bodies.filter((body) => body.kind === 'status')).toEqual([ + expect.objectContaining({ text: 'Selected model is at capacity.', tone: 'error' }) ]) + expect(bodies.at(-1)).toMatchObject({ + kind: 'turn', + turnId: TURN_ID, + state: 'completed', + outcome: 'failure' + }) + expect(stopped).toEqual([THREAD_ID]) }) it('writes nothing more after dispose', () => { diff --git a/src/main/codex/codex-structured-journal-translation.ts b/src/main/codex/codex-structured-journal-translation.ts index abc094bc3064..be5e53680889 100644 --- a/src/main/codex/codex-structured-journal-translation.ts +++ b/src/main/codex/codex-structured-journal-translation.ts @@ -259,11 +259,13 @@ export function createCodexJournalTranslator( reportPrimaryThreadStoppedRunning() } // A turn-ending `error` is a row inside the turn it names; the failed - // `turn/completed` Codex sends after it is that turn's end. + // `turn/completed` Codex sends after it is that turn's end. The thread + // status was read for state above and prints nothing. const unhandled = genericFrames.appendUnhandled( `notification:${event.method}`, event.params, - event.threadId + event.threadId, + { coveredByTypedTranslator: event.method === 'thread/status/changed' } ) if (unhandled.accepted && event.method === 'error') { commands.errorShown(event.params) diff --git a/src/main/native-chat/agent-session-wire/provider-frame-disposition.test.ts b/src/main/native-chat/agent-session-wire/provider-frame-disposition.test.ts index 55b338366f67..06c5ba73bdca 100644 --- a/src/main/native-chat/agent-session-wire/provider-frame-disposition.test.ts +++ b/src/main/native-chat/agent-session-wire/provider-frame-disposition.test.ts @@ -2,6 +2,7 @@ import { describe, expect, it } from 'vitest' import { DEFAULT_JOURNAL_PAYLOAD_LIMITS } from '../agent-session-journal/journal-payload-bounds' import { classifyProviderFrame, + hasTypedProviderFrameTranslator, isDeltaProviderFrameKind, PROVIDER_FRAME_CLASSIFICATIONS } from './provider-frame-disposition' @@ -199,7 +200,22 @@ describe('typed translator coverage', () => { ).toMatchObject({ classification: 'error-surface' }) }) - it('covers Claude only — the same method name on another provider still falls back', () => { + it('covers the Codex thread status, which reports `systemError` beside the `error` row', () => { + const kind = 'notification:thread/status/changed' + const payload = { threadId: 'thread-1', status: { type: 'systemError' } } + + expect(hasTypedProviderFrameTranslator('codex', kind)).toBe(true) + expect( + unhandledProviderFrameJournalItem('codex', kind, payload, DEFAULT_JOURNAL_PAYLOAD_LIMITS, { + coveredByTypedTranslator: true + }) + ).toBeNull() + expect(unhandledProviderFrameJournalItem('codex', kind, payload)).toMatchObject({ + classification: 'error-surface' + }) + }) + + it('covers a kind only for its own provider — a Claude kind from Codex still falls back', () => { expect( unhandledProviderFrameJournalItem('codex', 'message:system:task_notification', { status: 'failed' diff --git a/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts b/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts index b4d2286145b0..4cc90430d7e2 100644 --- a/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts +++ b/src/main/native-chat/agent-session-wire/provider-frame-disposition.ts @@ -162,23 +162,39 @@ export const PROVIDER_FRAME_CLASSIFICATIONS = { * * Listing a kind before its translator exists deletes the only report of a * failure, so nothing may be added here except together with the code that - * renders it. + * renders it, or that reads it for state while another frame carries its + * failure (the Codex thread status, whose fault arrives on `error`). * - * A frame of a listed kind that names no task writes nothing. The row it + * A Claude frame of a listed kind that names no task writes nothing. The row it * replaces named no task either — it printed the opcode and a raw payload — * and every frame this protocol sends carries the id its own tracker and * roster have always required. */ -const CLAUDE_TYPED_TRANSLATOR_KINDS: ReadonlySet = new Set([ - 'message:system:task_started', - 'message:system:task_updated', - 'message:system:task_progress', - 'message:system:task_notification', - 'message:system:background_tasks_changed' -] satisfies ClaudeStreamJsonFrameKind[]) +const TYPED_TRANSLATOR_KINDS: ReadonlyMap> = new Map< + keyof ProviderFrameClassificationTable, + ReadonlySet +>([ + [ + 'claude', + new Set([ + 'message:system:task_started', + 'message:system:task_updated', + 'message:system:task_progress', + 'message:system:task_notification', + 'message:system:background_tasks_changed' + ] satisfies ClaudeStreamJsonFrameKind[]) + ], + [ + 'codex', + // Every status arm is thread state the translator reads; a fault's sentence arrives on `error`. + new Set([ + 'notification:thread/status/changed' + ] satisfies `notification:${CodexAppServerNotificationMethod}`[]) + ] +]) export function hasTypedProviderFrameTranslator(provider: string, kind: string): boolean { - return provider === 'claude' && CLAUDE_TYPED_TRANSLATOR_KINDS.has(kind) + return TYPED_TRANSLATOR_KINDS.get(provider)?.has(kind) === true } const ERROR_VARIANT_KEYS = new Set(['type', 'status', 'state', 'subtype', 'outcome']) diff --git a/src/main/native-chat/agent-session-wire/unhandled-provider-frame.test.ts b/src/main/native-chat/agent-session-wire/unhandled-provider-frame.test.ts index aefceacbca4b..f0c9309441d3 100644 --- a/src/main/native-chat/agent-session-wire/unhandled-provider-frame.test.ts +++ b/src/main/native-chat/agent-session-wire/unhandled-provider-frame.test.ts @@ -87,7 +87,7 @@ describe('unhandled provider frame journal fallback', () => { }) }) - it('renders codex systemError and Claude error result variants', () => { + it('renders codex systemError and Claude error result variants when no typed translator covers the frame', () => { const codex = unhandledProviderFrameJournalItem('codex', 'notification:thread/status/changed', { threadId: 'thread-1', status: { type: 'systemError' } From afa81dc3ad0b13b553118ab65795fec9fc48e284 Mon Sep 17 00:00:00 2001 From: Brennan Benson <79079362+brennanb2025@users.noreply.github.com> Date: Wed, 30 Sep 2026 14:15:07 -0700 Subject: [PATCH 22/79] fix(native-chat): chat failure messages appear in the app's language (#23674) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * fix(native-chat): a read whose history will not open is refused with its reason History, subscribe, snapshot and options reads reach a chat through one accessor, whose open had no catch: a journal that would not open reached every client as a runtime error carrying the storage's own text (a path, "file is not a database"). The accessor, and the options read's own open, now throw the classified journal refusal: journalCorrupt when SQLite reports damage, journalUnavailable otherwise. The storage text goes to the log only. The wire code stays runtime_error and the message becomes the bare code, as for every thrown refusal; the reason rides in the error's data. * refactor(native-chat): the idle sweep's stop of a hung start carries no hand-written reason The sweep passed an English sentence as the stop's reason. It lived only in memory and nothing read it: the delivery loop words the error row and the rejection from the hostStopped fact. Dropped, with the display-name lookup that built it. * fix(native-chat): a refusal the host throws is worded from its data, never its message A thrown agent-session refusal reaches the client as runtime_error with the bare code as its message and the typed refusal in error.data. Stop, answers, options and goals, a launch's held option pick, the option picker's failure toast, and the Retry line of a chat that could not start now word it from that refusal through the shared notice table. What each caller decides about the outcome is unchanged: only the words move. The Retry line of a failed start no longer prints the host's message or a thrown error's text; it keeps the refusal as a fact and says the cause and step its reason names, or only that the chat could not be started. The option toast keeps a local option surface's own sentence. * fix(native-chat): an unreadable history is worded from its refusal, and damage stops the retry The structured chat's read failure showed the host's text on the status line, and the pane always said Orca keeps trying. The read transport now takes the refusal from the error's data (a stream payload or a thrown RPC error), the reducer keeps it beside the failure text, and the pane and the status line word it through the notice table, once: on the pane when the failure took it, else beside the transcript that stays. A damaged journal says "Unable to load this chat." and the read stops reconnecting for that run; reopening the chat reads again. An open that can clear names its cause without "Try again", since the pane retries on its own. A failure that names no reason keeps today's generic line. Finality comes from the refusal's reason, never its message, which is the bare code for both. * fix(native-chat): a rejected message is worded from its stored fact A message the host recorded and then rejected keeps the host's typed fact beside its reason, but the Retry words re-read the reason alone. Now the fact decides: a hand-over failure says Orca couldn't reach the agent, a kind whose reason may be a legacy marker gets its fact's own sentence (a full queue now says so instead of only "not sent"), and any other kind shows the sentence the host wrote for it, which carries the agent's name and any words the provider wrote for a person. A row with no fact reads as before. * fix(native-chat): each message that did not go through says why on its own row The structured chat showed one Retry strip under the transcript for whichever single entry it picked, so a second failed message had no reason and no Retry of its own. The terminal-backed chat's existing per-row delivery marker now carries a notice and an optional Retry, and the structured pane derives one per message from the outbox on each render: every rejected message, and the one the queue stopped on (read through the drain's own rule, so a Retry never names a message waiting behind it). Each is worded from that message's stored failure. The single strip is deleted. Nothing new is stored, and the shared message projection is untouched. * fix(native-chat): say each chat failure's words where its own control already acts Three wording rules for the desktop chat: - A chat that could not start shows Retry beside its reason, so the reason stops at its cause where the Retry is the step: a reason whose action is to retry, and a start failure's "send your message again". Any other step stays (quit the terminal agent, start a new chat). The start-failure sentences take a retryControl context for this; what the host writes is unchanged. - A history that couldn't open right now still reconnects, so the pane keeps "Orca keeps trying to load it" under its cause. Only a damaged history, which no retry reads past, drops it. - A read failure that names no reason while the transcript is shown is only the pane reconnecting: the status line says "Reconnecting to this chat…" in muted text, not an error. New key components.native-chat.state.reconnecting, hand-translated for es/fr/ja/ko/zh. * fix(native-chat): a rejected message offers Retry only once the queue is moving Each rejected message's row offered its own Retry even while the queue was stopped on another message. Any Retry clears the stopped queue, so pressing a rejected message's Retry also sent the message the queue was holding, which the user had not retried; behind a message whose delivery is unconfirmed, the retried one instead went back into the queue with no notice and waited there. While the queue is stopped, only the message it stopped on offers Retry, as the single Retry strip this replaced did. A rejected message keeps its words on its row and gets its Retry back once the queue moves. * fix(native-chat): a chat whose history will not open logs once, not on every reconnect A reader reconnects every 750 ms while a journal open can clear, and each attempt logged the failure with its full stack. The read door now logs a session's failure once until that session opens, closes, or fails differently; every attempt is still refused with its reason. * fix(native-chat): a message's own Retry is its resend step, so its notice stops at the cause A rejected message offering Retry read "Claude stopped before it finished starting. Send your message to try again." beside that button. Its row now takes the rule the launch strip already follows: beside its own Retry the words leave out sending or trying again, worded from the stored fact with the chat's agent name. The stored fact keeps less than the host wrote from (a refusal, the provider's words), so a reason it cannot rebuild exactly is kept as written. A rejected message without a Retry, while the queue is held, keeps the step. What the host writes and the phone's notice are unchanged. * fix(native-chat): a message's Retry sends only that message, never the one the queue is held on Retry released the queue's refusal hold whichever message it was pressed on. While a queued message waited ahead of a held one, every rejected message offered Retry, and pressing it also sent the held message the person had not retried. Retrying an unconfirmed message ahead of a held one did the same. Retry now releases the hold only for its own message. * fix(native-chat): a not-signed-in failure beside Retry still says to sign in first Beside a Retry the notice dropped the whole next step, so a chat that could not start because the agent was not signed in read only the cause. Pressing Retry without signing in fails the same way again. The words now keep the sign-in step and leave out only the resend, which the Retry button is. * test(native-chat): a rejected message's hidden Retry only avoids waiting unseen * fix(native-chat): every Retry beside a notice leaves out the retry step the same way A message the queue stopped on worded its refusal with no agent name and with its retry step, beside its own Retry, while a rejected message next to it named the agent and left the step to the button. The launch strip and the history pane each had their own copy of the same rule. One wording context now goes through the one notice table for every surface: a Retry beside the words, or a pane that reconnects on its own, is the step for a reason whose action is to retry, and every other step stays. What the phone and the host write is unchanged. * test(native-chat): read the sent message id without a type assertion * fix(native-chat): a rejected message is worded from the journal's own fact, never by comparing sentences A message the host recorded and then rejected kept only the rejection's kind on the message, so its notice was reworded from that smaller copy only when it rebuilt the host's sentence word for word. A different agent name, an older host's wording, or anything the copy dropped (why a start failed, the provider's own words) left the host's sentence in place, beside a Retry that repeated its resend step. The notice now reads the journal's own rejection for that message, found by id, with the pane's agent name and Retry, and shows the provider's words only when they were written for a person. The message's smaller copy words it only when that journal row is not loaded. Nothing new is stored. * fix(native-chat): a message rejected before a restart retries under a new id the first time Whether a Retry needed a new message id was remembered in memory for one message, or read from the journal row when it was loaded. After a restart, or for an older message whose row was not loaded, the first Retry resent under the old id, the host answered with the same settled rejection, and nothing visibly happened. The message now says so itself: one the host recorded and rejected always retries under a new id, including after a restart. A refusal that already gave the message a fresh id, and a message whose delivery is unconfirmed or in flight, keep their id as before. * fix(native-chat): a rejected message older than the loaded history keeps the provider's words When the journal row that rejected a message is not loaded, the message's own copy of the fact has no provider detail or start refusal. For the kinds worded from those, the row now shows the sentence the host wrote for the person instead of a thinner rebuilt one. * test(native-chat): the chat pane words a rejected message from its loaded journal row Nothing covered the pane handing the journal's rows to the per-message notices, so a pane that stopped passing them would quietly fall back to the message's smaller copy of the rejection and show the host's sentence, resend step and all. The new case renders the pane with a rejected message whose journal row is loaded and checks that it reads that row's refusal in the chat's own agent name. * fix(native-chat): a chat whose history won't load says why in one line A read the host refused for a named reason put its sentence under the generic "Could not load conversation" title, so a damaged history read as two lines saying the same thing. The pane's own sentence now takes the title's place; a history that can come back keeps its line saying Orca keeps trying. A failure that names nothing keeps the generic title. * fix(native-chat): a message a failed start rejected says only that it was not sent When an agent stopped before it finished starting, the chat showed the start's red row ("Claude stopped before it finished starting. Send your message to try again.") and then repeated that cause under every message the start rejected. Each of those messages now reads "Your message was not sent." beside its Retry. The match is made on typed facts, not on the words: the host writes the start's row and the rejection of its queued messages from the same failure fact, and the row is keyed by the start. The pane finds the loaded start-failure rows by that key and shortens a message's notice only when its loaded journal submission was rejected with the same fact. Any other rejection, or one whose submission or row is not loaded, keeps its full notice. The row key moves to a shared module so the host that writes it and the pane that reads it use one definition. * fix(native-chat): a chat whose history keeps failing to open retries less often A read the host kept refusing (its history store could not be opened right now) reopened every 750 ms for as long as the chat stayed open, about 40 opens every 30 seconds. Each reconnect now waits twice as long as the last, from 750 ms up to 30 seconds, and never gives up; the first read that delivers anything starts the wait over at 750 ms. A damaged history still stops reconnecting at once. Reset happens on a delivered read, not on connect: a local subscribe resolves before the host's open refuses, so resetting there would keep the 750 ms loop. * test(native-chat): the pane harness types its journal rows without a cast * test(native-chat): the admission test passes no start-failure rows to the notices * test(native-chat): import the journal types once * fix(native-chat): a remote chat reads again as soon as its host is back The read retry doubles its wait up to 30 s during an outage, and nothing reset it when the remote runtime reconnected, so the transcript could lag the reconnect by up to 30 s. The read now watches the runtime status store's contact-regained edges (hostContactEpoch for a same-runtime return, connectionGeneration for a new runtime session) and, when one lands, runs a waiting retry immediately with the wait reset to its base. * refactor(native-chat): build each failure sentence from whole pieces Every sentence agentSessionFailureWords writes is now assembled from a table of whole English pieces, so a reader can supply its own words for each piece. The host still fills them in English, byte for byte as before. * fix(native-chat): translate the failure sentences desktop notices show A refused start and a rejected message now carry their failure fact to the notice instead of its English sentence, and desktop words that fact through translate keys whose English defaults are the host's own pieces. The host keeps writing English into rows and reasons, and a host sentence with no fact beside it still shows as written. * fix(native-chat): the history pane says only that Orca keeps trying When the pane's title already says Orca couldn't open this chat's history right now, the line under it no longer repeats that the transcript could not be read; it says only that Orca keeps trying to load it. The pane with no named reason keeps its two-part line. * test(native-chat): type the failure pieces a refusal notice shares * test(native-chat): the Chinese failure words use no Japanese-only characters * fix(native-chat): every history pane that says it didn't load says only that Orca keeps trying A pane whose title is a code's own words ("This chat's history couldn't be loaded.") now gets the short retrying line too. Only the pane with no named reason keeps the two-part line. * test(native-chat): a provider's words with nesting and markup stay as written in a translated notice * fix(native-chat): French and Spanish say a withdrawn message was withdrawn before the agent began working on it * fix(native-chat): Japanese and Chinese notices run their sentences on without a space A notice joined its sentences with a space in every language, so Japanese and Chinese read "Claude 无法启动。 请重新发送消息。" with a stray gap after the full stop. The failure-sentence builder now takes the joiner alongside its words, and desktop joins in the UI language: no space in Japanese and Chinese (including a plugin pack that declares either), one space elsewhere. The host and the phone keep English, joined with a space as before. * fix(native-chat): a failed /clear or /compact says why in the app's language The line under the composer printed the host's English sentence although the result carries the typed failure beside it. It now words that failure the way the host does (the chat's agent and /clear for a failed /clear, nothing for /compact), in the app's language; an older host that sends no failure keeps its sentence. * fix(native-chat): Spanish says a rate limit, and Korean says a withdrawn message was never processed The Spanish retry notice said the agent hit a usage limit, a different thing from the rate limit the English names. The Korean withdrawn-message notice said the agent had not started, which reads as the agent not launching; it now says the agent had not begun processing the message, as the other languages do. * refactor(native-chat): one rule says which words already say the history didn't load * fix(native-chat): an image size limit says its unit the way the reader's language does * test(native-chat): the option picker's i18n stand-in knows the reader's locale, which a refusal notice now reads * fix(native-chat): a sentence a language pack left in English keeps its space Sentences were joined by the UI language: no space in Japanese and Chinese, one elsewhere. A plugin pack for a Chinese or Japanese variant that predates the failure words falls back to English for them, so a notice read "您的訊息未傳送。Claude couldn't start.Send your message to try again." Each gap now follows the sentence before it: none after a full-width 。！？, one space after anything else. The built-in Japanese and Chinese catalogs end every sentence in 。, so they read as before, and English is unchanged. Since the rule no longer needs the language, one joiner serves every surface and the failure-sentence builder no longer takes one alongside its words. * fix(native-chat): a failed /clear this build only partly understands shows the host's own sentence The line under the composer words a failed /clear or /compact from the fact the host sends beside its sentence. The reader drops any part this build cannot place, such as a refusal code a newer host added, and the rest of the fact can then give different advice: "Run /clear again." where the host said "Start a new chat to continue." When any part the host sent did not survive the read, the line now shows the host's sentence as written, the same as for a host that sends no fact. A fact this build reads whole is still worded in the app's language. * fix(native-chat): a failure fact this build reads only in part shows the host's sentence everywhere The previous check compared only a fact's top-level parts, so a known refusal code carrying a reason a newer host added still counted as read: the reader dropped the reason and the notice re-worded what was left, which can advise differently from the host ("Run /clear again." against "Start a new chat to continue."). One shared reader now answers whether this build read the whole fact: it reads the fact and keeps it only when the read equals what arrived, at every depth. Every place that chooses between wording a fact and showing the host's text uses it: the line under the composer after /clear or /compact, a rejected message's notice from the journal's fact, and the smaller copy a rejected message keeps for when its journal row is not loaded. Matching a rejected message to the start row that already says why still uses what this build can read, since that is identity, not wording. Facts this build reads whole are worded as before. * fix(native-chat): a failed /compact names /compact as its next step on desktop too The host names the command a failed start was waiting on, for /clear and /compact alike. The line under the composer re-worded only /clear with it, so a /compact whose start failed read "The agent couldn't restart. Send your message to try again." in the reader's language. It now words every command the host answers with the agent and the command the host used, so desktop English matches the host and French or Japanese keep /compact. * fix(native-chat): a /compact whose start failed no longer says the operation was not confirmed A /compact on a chat whose agent is not running starts it first, and that start takes a new lease, so the chat's fence moves before the command's reply arrives. The write settles as one for a fence this pane no longer shows, and the composer line read that as "Conversation operation was not confirmed." Such a write now says nothing there, as every other write already does: the chat's own start-failure row says why, and the command's message is rejected in the journal. The sentence it printed is gone from the catalogs. * fix(native-chat): keep the message outbox within its line limit after main's growth * fix(native-chat): a command's own reply is kept when its start moved the fence A /clear or /compact on a chat whose agent is at rest starts the agent first, and that start moves the chat's fence before the command's reply arrives. Every reply from an earlier fence was discarded, so a /clear whose new chat failed to start said nothing at all, and a /compact that started left "/compact" in the composer. A conversation command's reply is now kept while the pane still shows the chat it was sent for; a closed pane or another chat still drops it, and every other write keeps the fence rule. The line under the composer says nothing only when the failure is the chat's own start and that start's loaded row already says why, the rule a message that start rejected already follows. * fix(native-chat): a returned queued message shows the host's sentence for a fact read in part The caption under a returned queued message re-worded its failure from whatever this build could read of the fact. A newer host's fact with a refusal code or reason this build drops read as a shorter sentence with different advice. It now re-words only a fact read whole, and otherwise shows the host's own sentence, as every other surface that re-words a fact does. * test(native-chat): a command reply after any fence move, for /clear and /compact The fence-move tests now state the rule as the code has it: a conversation command's reply is kept whenever the pane still shows its chat, whatever moved the fence. They cover a /clear that completed, and a failed start for each command in French with the fact that command really meets (a /clear's new chat fails to start; a /compact's chat fails to restart). * fix(native-chat): only the reply a pane still waits on outlives a fence move A conversation command's reply was kept across a fence move whenever the pane still showed the same chat. A reply the pane had stopped waiting on, because it left the chat and came back or a newer command replaced it, was applied as if it answered the current one. The pane now remembers the one command request it waits on; only that request's reply is kept after the fence moves, and closing the pane or showing another chat forgets it. Tests now drive the fence move during the request itself rather than through /clear starting an agent, and cover a /clear that stops a running agent. * fix(native-chat): a newer-Orca history error keeps a whole retry line, and the phone shows the host's words for a fact it reads in part When a chat's history was saved by a newer Orca, the pane's title reads "Chats were saved by a newer Orca. Update Orca to keep using them." and the line under it said only "Orca keeps trying to load it.", with nothing for "it" to mean (in French and Spanish the pronoun also disagreed with "chats"). That title names every chat rather than this one, so the pane keeps the full line: "The transcript could not be read. Orca keeps trying to load it." On the phone, a returned queued message whose failure fact this build reads only in part was re-worded from what it could read, dropping advice the host gave; it now shows the host's own sentence, as the desktop card already does. The comment on the reply the pane waits on now says what forgets it: a newer command, or disabling the pane. * fix(native-chat): a command reply this build can't place shows the host's own words A newer host can answer a conversation command with a command name this build doesn't know. This build re-worded that reply from its failure fact as if it were a command it knew, naming the command in its own words, or said nothing when a loaded start row matched the fact. It now shows the host's sentence as written, as it already does for a fact it can read only in part. --- ...le-structured-queued-message-cards.test.ts | 22 ++ .../mobile-structured-queued-message-cards.ts | 8 +- .../native-chat/NativeChatEmptyState.tsx | 30 +- .../native-chat/NativeChatLaunchRetry.tsx | 4 +- .../NativeChatQueuedMessageCard.tsx | 4 +- .../NativeChatQueuedMessageList.test.tsx | 19 + ...eChatStructuredSession.read-error.test.tsx | 34 +- .../NativeChatStructuredSession.tsx | 4 +- .../agent-session-failure-words-text.test.ts | 239 +++++++++++++ .../agent-session-failure-words-text.ts | 228 ++++++++++++ .../agent-session-write-notice-text.test.ts | 67 +++- .../agent-session-write-notice-text.ts | 18 +- ...red-agent-session-delivery-notices.test.ts | 47 +++ ...ructured-agent-session-delivery-notices.ts | 23 +- ...tured-agent-session-read-failure-notice.ts | 6 +- ...ructured-conversation-command-send.test.ts | 178 ++++++++++ .../structured-conversation-command-send.ts | 66 +++- ...ent-session-command-start-failure.test.tsx | 330 ++++++++++++++++++ .../use-structured-agent-session-mutate.ts | 33 +- .../use-structured-agent-session-transport.ts | 3 +- ...tured-agent-session-write-refusal.test.tsx | 37 ++ .../use-structured-agent-session.ts | 26 +- .../src/i18n/en-runtime-required.json | 59 ++++ src/renderer/src/i18n/locales/en.json | 64 +++- src/renderer/src/i18n/locales/es.json | 61 +++- src/renderer/src/i18n/locales/fr.json | 64 +++- src/renderer/src/i18n/locales/ja.json | 64 +++- src/renderer/src/i18n/locales/ko.json | 64 +++- src/renderer/src/i18n/locales/zh.json | 64 +++- .../agent-session-conversation-command.ts | 9 +- src/shared/agent-session-failure-copy.ts | 110 ++++++ src/shared/agent-session-failure-words.ts | 228 ++++++------ src/shared/agent-session-failure.test.ts | 48 +++ src/shared/agent-session-failure.ts | 11 + .../agent-session-refusal-notice.test.ts | 127 ++++--- src/shared/agent-session-refusal-notice.ts | 22 +- src/shared/agent-session-write-notice-copy.ts | 37 +- src/shared/native-chat-empty-state.ts | 3 + src/shared/sentence-joining.ts | 12 + src/shared/structured-agent-session-outbox.ts | 7 +- ...ructured-agent-session-send-disposition.ts | 7 +- 41 files changed, 2246 insertions(+), 241 deletions(-) create mode 100644 src/renderer/src/components/native-chat/agent-session-failure-words-text.test.ts create mode 100644 src/renderer/src/components/native-chat/agent-session-failure-words-text.ts create mode 100644 src/renderer/src/components/native-chat/structured-conversation-command-send.test.ts create mode 100644 src/renderer/src/components/native-chat/use-structured-agent-session-command-start-failure.test.tsx create mode 100644 src/shared/agent-session-failure-copy.ts create mode 100644 src/shared/sentence-joining.ts diff --git a/mobile/src/session/mobile-structured-queued-message-cards.test.ts b/mobile/src/session/mobile-structured-queued-message-cards.test.ts index 2c6173b54a5a..44d1e0b9729f 100644 --- a/mobile/src/session/mobile-structured-queued-message-cards.test.ts +++ b/mobile/src/session/mobile-structured-queued-message-cards.test.ts @@ -239,6 +239,28 @@ describe('mobileQueuedMessageCards', () => { expect(card?.caption).toBe('Words for a kind a newer host added.') }) + it("shows the host's sentence for a fact this build cannot read all of, as the desktop card", () => { + const reason = "Claude couldn't start. Start a new chat to continue." + // As a newer host sends it: a known code with a reason this build doesn't know. + const newer = { + kind: 'startFailed', + refusal: { code: 'agent_session_conflict', details: { reason: 'newerReason' } } + } + const [card] = mobileQueuedMessageCards( + [ + draft({ + messageId: 'a', + state: 'returned', + returnedReason: reason, + returnedRejection: newer + }) + ], + [], + { pendingPrompt: false } + ) + expect(card?.caption).toBe(reason) + }) + it('maps the send-failed pause marker to English and an unknown marker to a plain pause', () => { const cards = mobileQueuedMessageCards( [ diff --git a/mobile/src/session/mobile-structured-queued-message-cards.ts b/mobile/src/session/mobile-structured-queued-message-cards.ts index 16393dbbf8ba..f3edb13e53c8 100644 --- a/mobile/src/session/mobile-structured-queued-message-cards.ts +++ b/mobile/src/session/mobile-structured-queued-message-cards.ts @@ -2,7 +2,7 @@ // The wire carries no hold copy on purpose: the caption is derived here from the // draft's own state plus the live facts the client already holds. -import { readAgentSessionFailureFact } from '../../../src/shared/agent-session-failure' +import { readWholeAgentSessionFailureFact } from '../../../src/shared/agent-session-failure' import type { AgentJournalSubmission } from '../../../src/shared/agent-session-journal-types' import { agentSessionWriteNoticeEnglish } from '../../../src/shared/agent-session-refusal-notice' import { dispatchWasWithdrawn } from '../../../src/shared/structured-agent-session-dispatch-rejection' @@ -37,13 +37,13 @@ function returnedCaption( if (dispatchWasWithdrawn({ dispatchState: 'rejected', reason, rejection })) { return 'Stopped before it was sent' } - // Worded as the desktop card words it: the fact decides, the reason is the fallback, and the - // card's own Send is the retry, so the words leave out sending again. + // Worded as the desktop card words it: a fact read whole decides, the host's reason is the + // fallback, and the card's own Send is the retry, so the words leave out sending again. return agentSessionWriteNoticeEnglish( structuredAgentSessionAttemptFailureParts( { kind: 'rejected', reason }, { retryControl: true }, - readAgentSessionFailureFact(rejection) + readWholeAgentSessionFailureFact(rejection) ) ) } diff --git a/src/renderer/src/components/native-chat/NativeChatEmptyState.tsx b/src/renderer/src/components/native-chat/NativeChatEmptyState.tsx index ec84f554cd76..d6c7094a2961 100644 --- a/src/renderer/src/components/native-chat/NativeChatEmptyState.tsx +++ b/src/renderer/src/components/native-chat/NativeChatEmptyState.tsx @@ -1,13 +1,17 @@ import { MessageSquare, TriangleAlert } from 'lucide-react' import { translate } from '@/i18n/i18n' import { formatAgentTypeLabel } from '@/lib/agent-status' -import { NATIVE_CHAT_EMPTY_STATE_COPY } from '../../../../shared/native-chat-empty-state' +import { + NATIVE_CHAT_EMPTY_STATE_COPY, + NATIVE_CHAT_KEEPS_TRYING_COPY +} from '../../../../shared/native-chat-empty-state' import type { NativeChatSession } from '../../../../shared/native-chat-types' export function NativeChatEmptyState({ kind, message, headline, + headlineSaysUnread = false, agent, retrying = false }: { @@ -15,12 +19,15 @@ export function NativeChatEmptyState({ message?: string /** The chat's own sentence for the failure, said once: it takes the generic title's place. */ headline?: string + /** The headline already says the history didn't load, so the retrying line says only that + * Orca keeps trying. */ + headlineSaysUnread?: boolean agent?: NativeChatSession['agent'] /** The read retries on its own (structured chat), so the error says so instead of pointing * back to the terminal. */ retrying?: boolean }): React.JSX.Element { - const copy = emptyStateCopy(kind, { message, headline }, agent, retrying) + const copy = emptyStateCopy(kind, { message, headline, headlineSaysUnread }, agent, retrying) return (
- {message} - {cause ? ` ${cause}` : null} + {cause ? joinSentences([message, cause]) : message}