Skip to content

fix(orchestration): keep a released worker's closing turn and never leave a blank pane - #34

Merged
kiwidream merged 3 commits into
swaplabs/mainfrom
kiwidream/worker-release-dark-fix
Oct 2, 2026
Merged

kiwidream merged 3 commits into
swaplabs/mainfrom
kiwidream/worker-release-dark-fix

Conversation

@kiwidream

@kiwidream kiwidream commented Oct 2, 2026 •

Copy link
Copy Markdown

ELI5

When a coordinator agent finishes with a helper agent ("worker"), it tells Orca to clean up the worker's terminal. Three things went wrong when that happened:

  • Orca shut the worker down while it was still writing its last message, so that message was lost.
  • On a desktop connected to a remote Orca server, the worker's workspace was left showing an empty black (or white) area with nothing to click.
  • Asking Orca to keep a worker's terminal open didn't stick: Orca's own "what to do next" hint told the coordinator to close it anyway.

This PR fixes all three.

What Changed

1. Release waits for the worker's last turn. A worker reports "done" partway through its final turn, then keeps writing its closing message for a few seconds. orca orchestration worker-release used to save the worker's output and stop the agent immediately, cutting that message off. In the run that prompted this, two of three workers lost it.

  • Before: release saved the output and stopped the agent at once.
  • After: release first waits up to 15 seconds for the agent to go idle at its prompt. This reuses the existing tui-idle terminal wait, the same one orca terminal wait --for tui-idle uses. Then it saves the output and closes the terminal. This applies both to workers on the coordinator's own machine and to workers on another connected Orca server; for the latter, the server running the worker does the wait.
  • If the agent is still busy at 15 seconds: release continues, and the saved output carries a warning saying the closing turn was cut off. worker-read shows that warning.
  • No wait for plain shells (not an agent), structured chat workers, startup recovery, or a release retried after its output was already saved.

2. An emptied workspace offers "New Terminal" instead of a blank pane. When the server closes the last terminal in a workspace, a connected desktop keeps that workspace open.

  • Why it stays open: since upstream fix(session): remember closed and initialized runtime tabs stablyai/orca#21630, Orca deliberately doesn't recreate terminals the user closed in a server-owned workspace.
  • Before: the tab area rendered nothing, so the workspace looked dead.
  • When it shows: only once this workspace's tab list has actually arrived and is explicitly empty. While a connected workspace is still loading, it stays hidden, so it can't race the server into creating a duplicate terminal.
  • After: the empty area shows a centered New Terminal button with its shortcut chip. It matches the floating terminal panel's empty state and reuses the tab bar's existing "New Terminal" string and create command.

3. Kept workers stay kept. worker-list projects a nextAction per row, and it returned worker-release for any settled worker that still owned its terminal. That included workers someone had explicitly kept with worker-retain, and workers retained because their identity couldn't be proven, where it caused a retry loop. Retained rows now get no next action.

The worker-release help text now mentions the wait.

Why

  • Bounded wait vs. other options:
    • Telling every coordinator to poll for idle before releasing is what the guides would otherwise need, and it is easy to get wrong.
    • Not stopping the agent at all would bring back the pile-up of finished workers that release exists to clean up (upstream [Bug]: Orchestration worker terminals stay open after settle (Done stack / retained legacy_ambiguous) — no auto-release stablyai/orca#13047).
    • 15 seconds fits inside the CLI's 60 seconds together with saving the output and the few seconds the process stop takes. For a worker on another connected server, the coordinator now allows that server 40 seconds instead of 30 (the wait plus 25 seconds). An older coordinator's 30 seconds still covers the wait plus the server's own work. A server running an older Orca doesn't wait at all, and the worker-release help says so. There is no wire change.
  • Why the wait is skipped where it is: startup recovery must not stall, and a plain shell never reports idle.
  • Why the wait happens before the identity checks: those checks then look at the terminal as it is after the turn ends.
  • Why existing safeguards still hold: a worker-retain or a user typing into the worker during the wait still wins, because the archive is only committed while the release is still pending. A test covers the retain case.
  • Empty state vs. recreating a terminal on click: recreating would reverse upstream fix(session): remember closed and initialized runtime tabs stablyai/orca#21630, which stopped exactly that because it spawned duplicate terminals. An explicit button keeps that decision and still gives a way back.

Linked Issue

No tracking issue. This came from an operator report: three released workers in one supervised run "went dark." Related upstream context: stablyai#13047, stablyai#21630 and stablyai#21992.

Visual Proof

Captured from the new end-to-end test: a hidden-window Electron desktop client connected to a headless Orca server, Linux.

  • Before (fix reverted, same test): after the server closes the workspace's last terminal, the area under the tab strip is completely blank. It is white in light mode and black in dark mode.
  • After: the same area shows a centered New Terminal button with a Ctrl + T chip in both themes. Clicking it creates a terminal.

The screenshots couldn't be attached from the build host because the gh image extension isn't installed there. They're available and will be added as a comment.

Testing

  • New unit tests in worker-release-closing-turn.test.ts:

    • release waits for idle before saving output or closing;
    • it still releases at the bound and records the cut-off warning, which worker-read returns;
    • a retain during the wait keeps the terminal;
    • a non-agent terminal isn't waited on.

    Three of the four fail on the unfixed code. The fourth guards the plain-shell skip.

  • New federated test federated-release-closing-turn.test.ts: on the server running the worker, release waits before saving and closing, and records the cut-off warning at the bound. Both cases fail without the change.

  • New projection test: a retained, owned, settled worker gets no worker-release next action. It fails on the unfixed code.

  • New Electron end-to-end test tests/e2e/paired-emptied-workspace-empty-state.spec.ts. It runs a headless server and a desktop client, closes the last terminal on the server, expects the empty state, then clicks New Terminal and expects a terminal. It fails with the fix reverted ("element(s) not found") and passes with it.

  • Existing suites: pnpm test over the orchestration, runtime, shared fleet, CLI and tab-group suites passes (379 files).

  • Checks: pnpm tc and pnpm run check:code-quality:changed pass, including the design-system rules. oxlint is clean on the changed files.

  • Not run: the full pnpm lint and full pnpm test.

  • Platforms: Linux only. The shortcut chip uses the existing platform-aware shortcut helper.

  • I manually tested these changes locally (verified through the hidden-window end-to-end test and screenshots, not by hand)

  • Automated tests added/updated, or explained why not below

AI Disclosure

Written with Claude Opus 5.5 in Claude Code.

Review

Not in this PR (follow-ups)

  • Viewing output in the desktop: a released worker's saved output can still only be read with orca orchestration worker-read; the desktop has no view for it.

Agent skill upstream boundary

  • Not applicable, or this change follows docs/reference/agent-skill-sharing-upstream-boundary.md and copies or mechanically translates no upstream skill-installer source, tests, fixtures, registry entries, path tables, comments, or documentation.

Notes

  • Security: no change.
  • SSH/remote: the empty state covers paired/server-owned workspaces, which is the case it fixes. The release wait runs on the host that runs the worker.
  • Mobile: unaffected.
  • Backwards compatibility: no wire or schema change. The warning is one more string in the archive's existing warnings list.
  • Performance: release can take up to 15 seconds longer, and only while the agent is still working.

Checklist

  • This PR is small and focused
  • I explained what changed and why (ELI5, the user-facing before/after, the mechanism, and why over the alternatives)
  • Before/after screenshots or videos attached for UI changes, or N/A with reason (described above; images to follow in a comment)
  • Self-reviewed for correctness, security, and performance
  • Cross-platform, SSH/remote, and path/shortcut impact considered (or N/A)
  • pnpm lint, pnpm typecheck, pnpm test, and pnpm build pass (typecheck, changed-lines lint gate, targeted tests and the e2e build pass locally; full lint/test left to CI)

🤖 Generated with Claude Code

…eave a blank pane

Released supervised workers "went dark" for the operator:

- worker-release froze the archive and SIGTERMed the agent as soon as it
  was called, so a coordinator releasing on receipt of worker_done cut off
  the turn that sent it. Release now waits (bounded, 20 s) for a running
  agent's tui-idle before capturing and closing, and records a warning in
  the archive when the turn was still running at the bound. Recovery,
  structured workers and retried releases skip the wait.
- A paired client keeps the emptied runtime-owned workspace active (by
  design since stablyai#21630), but its tab group body rendered nothing. The
  group now shows a New Terminal action when it has no tabs.
- worker-list's nextAction told coordinators to worker-release a worker
  the user had retained, undoing worker-retain. Retained rows now owe no
  release.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tenki-reviewer

tenki-reviewer Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review complete. No issues found — approved ✅.


Adds a bounded closing-turn wait (with a cut-off archive warning) before worker terminal release so the agent's final message is captured, excludes retained workers from the fleet's release suggestion, and renders a paired-client empty state with a New Terminal action when a runtime workspace's last tab is closed.

Reviewed commit: 5017872

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-02T17:32:42.249505Z 4ac1064 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

tenki-reviewer[bot]
tenki-reviewer Bot previously approved these changes Oct 2, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5017872697

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cli/specs/orchestration-worker-specs.ts Outdated
Codex review: the help text promised the closing-turn wait for every
release, but a worker on a connected server went through the execution
host's federationRelease handler, which captured and closed at once.

The host handler now waits before it inspects the attachment and passes
the outcome to releaseRemoteAttachment, which records the cut-off warning
the same way. The bound drops to 15 s so the host still answers inside
the 30 s a coordinator (old or new) gives a federated release; no wire or
deadline change. The wait's stored-archive check no longer throws, so an
unreadable archive store is still reported by the release itself.
federated-worker-release-host reuses retainedReason instead of its copy.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tenki-reviewer

tenki-reviewer Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review complete. 🟡 3 medium

Reviewed the latest changes.

💬 Inline comments (2)

📍 Outside the diff (1)

  • 🟡 Concurrent releases of one dispatch skip coalescing, double-close (federated-worker-release-host.ts:195–204) — releaseRemoteAttachment has no in-flight coalescing, unlike the local path's activeReleaseByRuntime map (src/main/runtime/rpc/methods/orchestration/worker/worker-release-completion.ts:52-90).

Reviewed commit: f110bd2

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f110bd2884

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/cli/specs/orchestration-worker-specs.ts Outdated
Comment thread src/renderer/src/components/tab-group/TabGroupPanel.tsx Outdated
Comment thread src/main/runtime/rpc/methods/orchestration/worker/worker-release-closing-turn.ts Outdated

@tenki-reviewer tenki-reviewer Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This delta threads the worker closing turn into the federation release path: it waits up to 15s for a closing turn before observation, records a cut-off warning when no stored archive exists, and moves retainedReason handling into worker-release-completion. Three medium-severity issues were found in the delta: the full wait can exceed the 30s federation deadline, hasStoredArchive swallows read errors and silently skips the wait, and concurrent releases of one dispatch skip coalescing and can double-close the terminal.

@tenki-reviewer
tenki-reviewer Bot dismissed their stale review October 2, 2026 17:03

A newer Tenki review found issues on this pull request.

…n wait

- A tui-idle wait that ends on a human-only prompt (satisfied: false)
  no longer counts as a finished turn; the archive records the cut-off.
- An unreadable archive store no longer skips the wait; the release
  path still reports the read failure itself.
- The coordinator gives a federated release the closing-turn bound plus
  25 s, so a full wait still leaves the host room to capture and close.
  Older hosts never wait, which the worker-release help now says.
- The emptied-workspace action needs an accepted, explicitly empty tab
  row for that workspace, so it cannot appear (and race the host into a
  duplicate terminal) before a paired workspace's inventory arrives.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tenki-reviewer

tenki-reviewer Bot commented Oct 2, 2026 •

Copy link
Copy Markdown

Review complete. No blocking issues — 1 nitpick below.

Reviewed the latest changes; 2 prior findings remain open; 1 prior finding resolved.

🧹 Nitpicks (1) — 🟢 1 low
  • 🟢 Stale comment cites 30 s federated release budget (worker-release-closing-turn.ts:6) — The doc comment on WORKER_RELEASE_CLOSING_TURN_TIMEOUT_MS still says it "Fits the 30 s a coordinator gives a federated release" (src/main/runtime/rpc/methods/orchestration/worker/worker-release-closing-turn.ts:6), but this PR raises the coordinator's federated release budget to 40 s (FEDERATED_RELEASE_TIMEOUT_MS = the constant + 25_000, src/main/runtime/rpc/methods/orchestration/federation/federated-worker-release.ts:16).

Delta fixes the closing-turn wait (archive read failures no longer skip it, prompt-blocked turns now count as cut_off) and raises the federated release budget to 40 s, with only a stale 30 s comment left behind.

Reviewed commit: 4ac1064

@kiwidream
kiwidream merged commit 191b47a into swaplabs/main Oct 2, 2026
44 checks passed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ac1064869

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant