Skip to content

[upstream #24928] fix(codex): settle dispatch after unrecoverable reconnect - #307

Open
innocarpe wants to merge 5491 commits into
mainfrom
mac-mini-c3-24791-codex-reconnect-failed-20261003
Open

innocarpe wants to merge 5491 commits into
mainfrom
mac-mini-c3-24791-codex-reconnect-failed-20261003

Conversation

@innocarpe

Copy link
Copy Markdown
Owner

Portfolio mirror of my contribution to upstream stablyai/orca.
Exhibition only — the real review/merge target is upstream.

Upstream

Summary

ELI5 When Codex says its session cannot be restored, Orca now releases the stuck task assignment and tells the coordinator while keeping the terminal and worktree available for diagnosis. ## What Changed - Recognize the exact three-line Codex reconnect-failed banner from the

Note

  • Do not merge this into innocarpe/orca main until the upstream PR is merged.
  • After upstream merges: sync fork from upstream, then close this mirror PR.
  • This open PR exists so visitors see in-flight work on this fork's Pull requests tab.

OrcaWin and others added 30 commits September 30, 2026 23:23
…ase (stablyai#24089)

Lands daemon-protocol-facts.mjs from the Windows update diagnostic branch with a
stricter parser, and adds check-daemon-protocol-crossing.mjs (rule R1): the working
tree must attach the newest release tag's daemon. Rollback crossing is reported only.
Runs in the cross-version-wire job, which already has full tags; tag selection moves
to config/scripts/stable-release-tags.mjs so both use one rule.

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…own (stablyai#24071)

The test re-arms a setImmediate tick while the copy runs and stopped it with a
flag. A tick already queued when the import resolved still ran, and when the
afterEach teardown closed the database first it threw journal_closed as an
unhandled error, failing a CI shard with every test green. Clear the queued
tick instead.
…es the conversation (stablyai#24235)

* fix(claude): a Stop ends Claude's process, and the next message resumes the conversation

Claude's Stop now ends its child after the interrupt, whatever Claude answered, once the stopped turn ends or a 3 s grace runs out. The grace lets Claude's own result move the resume point past that turn, so the next send resumes after it. Background work ends with the child. Codex keeps its child.

* test(native-chat): the router answers stopEndsSession for the session's live owner

* test(native-chat): type the Stop test's close mock as the adapter's optional method

* fix(native-chat): a Claude Stop answers on its interrupt, and the next step on the chat's lane ends the child

The Stop now answers as soon as the interrupt step finishes. Ending Claude's
process runs as a second task on the session's serialized lane, queued in the
same tick as the Stop, so a message sent meanwhile reaches only the resumed
child. That step waits for the stopped turn to end (its result, the CLI's idle,
the child's exit or its close), then ends the child; a failure there is
reported and never fails the Stop, and the wind-down it leaves owed is retried.

The interrupt's answer and the turn's end share one 3 s grace counted from when
the interrupt goes out, so an interrupt Claude never answers ends the child in
about 3 s instead of after the 30 s control deadline. The wait is derived per
call from Claude's open turn: the armed latch and the close's user-stop branch
are gone, and the close keeps its whole deadline for proving the exit.

A Stop naming a turn that has since ended (the phone names the turn it last
saw) now ends the child too when Claude took it, as it does when Claude
interrupts a handed-over follow-up whose turn has not opened.

* test(native-chat): pin a Claude Stop's race, failure and timing cases on the shipping adapter

- Nothing queued while the Stop's first step waits on the interrupt runs before
  the child ends.
- A message sent with the pre-Stop fence during the Stop is accepted with no
  notice, and reaches only the resumed child, after the old child's close.
- An interrupt Claude never answers ends the child within the grace.
- A close that cannot prove the exit leaves the Stop answered with its row; the
  failure goes to the host's error hook.
- A Stop naming the turn that just ended ends the child when Claude interrupts
  the handed-over follow-up.
- A second Stop pressed while the first ends the child stays quiet.

* fix(native-chat): a Stop naming an ended turn ends Claude's child when its interrupt fails or goes unanswered

The phone names the turn it last saw. When Claude interrupted a handed-over
follow-up for that Stop and the answer failed or never came, the child stayed.
Only a provider that answers it did not take the Stop keeps the child now.
Also pins a queue-if-active send made while the Stop ends the child.

* fix(native-chat): a Claude card's Cancel denies an approval and stops on a question, never a bare interrupt

A permission card's Cancel interrupted the turn and kept Claude running, the
old Stop on a second control: a refused interrupt let the turn run on and
background work survived. Now the host asks the provider how a card's Cancel
is answered. For Claude, an approval's Cancel sends the same reply as its Deny
option, so the turn goes on without that tool; a question's Cancel runs the
chat's Stop, which ends the child, and the next send resumes the conversation.
A provider that gives no answer keeps today's path, so Codex is unchanged.

The host decides, so desktops and phones of any version keep sending the same
cancel and get the new behaviour; nothing on the wire changes.

With no Claude caller left, the prompt-cancel interrupt is gone: the bound
claim, the cancellation observation, the admission of the prompt's
cancellation, the prompt's turn binding, and withdrawing a refused stop (every
Claude interrupt now precedes the child's end, so the recorded stop stands).

* fix(native-chat): a Claude approval card's Stop option is the chat's Stop

The approval card's "Stop" option answered Claude with a deny that interrupts
the turn and kept Claude running, the last control on a Claude chat that
interrupted without ending the child. The host now routes it, like the card's
own Cancel, through the provider: for Claude that option is the chat's Stop.
It runs the same Stop body as the Stop button, in the same order (withdraw,
the Stop takes effect, interrupt), and the step that ends the child is queued
in the same synchronous call. The body now lives in one place,
structured-agent-session-chat-stop.ts, shared by the Stop button, a question
card's Cancel and this option.

The host decides, so an old card or client that sends option `cancel` gets the
real Stop; nothing on the wire changes. The interrupting deny reply is gone.

* test(native-chat): the option-route test's answer carries a whole journal resolution

* fix(native-chat): Claude cards drop their Stop option, and a plan card's Cancel asks Claude to wait

- Approval and plan cards no longer offer "Stop". Neither reference offers a
  Stop while an approval is up: the user denies, then stops. An older card's or
  client's `cancel` answer is a plain deny, and the respond path is main's again.
- A plan card's X / Esc no longer answers "Keep planning", which sent Claude
  straight back to revise and re-propose. It dismisses the plan: a deny that
  tells Claude to end its turn and wait for the user. No interrupt; the child
  stays. A tool approval's Cancel stays the Deny reply, and a question's Cancel
  stays the chat's Stop.
- The provider's routing takes the pending card itself
  (`routePromptCancel({ sessionId, prompt })`), so a plan is told from a tool
  approval, instead of a kind plus an optional option id that meant Cancel when
  absent. A routed answer may be one the card does not offer, like the
  dismissal.
- The chat's Stop is reachable only through `mutateWithChatStop`, which owns the
  mutation call and queues the step that ends the child in the same call, so
  no caller can run the Stop and forget the child's end.

* refactor(native-chat): cancelPlan no longer takes a stopChild nothing passes

The Stop's own body ends the child; the plan's default run serves only a card's
interrupt and the background-task stop, which never end it.

* fix(native-chat): a question card's Cancel settles the card in the Stop's first step, and answers Claude when nothing stops

A question card's Cancel runs the chat's Stop. The card stayed pending until
the child ended (up to about 3 s), so it could still be pressed and the second
press was refused once the chat rested. When the Stop found nothing in flight
(Claude asking after its main turn ended), the card stayed pending for good and
Claude's request went unanswered; the phone froze the card with every button
disabled.

Now the card is recorded as cancelled by the user in the Stop's own step, with
the receipt "Cancelled on <device>". A Stop that ends the session leaves
Claude's request to end with the child, so no reply races the interrupt; one
that ends nothing declines the request itself. The provider forgets the card
once the host records it, so neither Claude's own cancel nor the child's end
writes over the user's cancel. The card's Stop stops whatever the chat has in
flight, like the Stop button, instead of a turn the card names that may have
ended.

The same dismissal, a new provider member beside the cancel route, now answers
a plan card's Cancel: recorded as cancelled by the user, and Claude told to end
its turn and wait. That replaces resolving it as an option the card does not
offer.

* test(native-chat): the question-card Stop test has Claude cancel its held request, as it does once interrupted

* fix(native-chat): a Claude Stop before the echo reads Interrupted, and a question card's Stop stays in its own turn

- A Stop pressed after Send but before Claude echoed it read as a normal finish
  ("Worked for 0s", a green done check): with no turn row to wait on, the child
  was ended at once, and the echo that would have opened the stopped turn
  landed on a retired send. The wait before the child ends is now keyed on what
  Claude has in flight (an open turn or a send it has not answered), not on a
  journal turn id, within the same 3 s from the interrupt and woken by the same
  settles; a settle that leaves something in flight waits on. The echo opens
  the turn, the aborted result ends it Interrupted. A Claude that says nothing
  is ended when the grace runs out, the unanswered send doubt as before.
- A question card's Cancel takes the chat's Stop only when the card belongs to
  the turn running now, judged after draining the provider's lifecycle. A card
  a finished turn raised, such as a background agent's, is dismissed instead:
  Claude's request is declined and nothing stops.
- Claude forgets a dismissed card before the host records it, so Claude's own
  cancel landing during that write cannot replace the user's "Cancelled on
  <device>".

* test(native-chat): Stop tests read background work from the host's child records, and guard the optional dismissal

Main's child records replaced the adapter's background-task callback: the
background-work test now reads the host's child record (live before the Stop,
settled after the child ends) and the session's agent status (done, Interrupted,
nothing left for Monitoring).

* fix(native-chat): a tool card's Cancel reads Cancelled, and a Stop's background work reads stopped

- A tool approval card's X / Esc already sent Claude the plain deny, but the
  card read "Deny · Answered on <device>", the same as pressing Deny. It is
  now a dismissal like a plan card's: recorded as cancelled by the user, with
  the same deny reply, no interrupt and the child kept. Every card's Cancel
  now routes to a dismissal or the chat's Stop, so the option route is gone.
- When a Stop ended Claude's child, a background agent or shell it was running
  settled as an unknown ending (a neutral dot), while the same task's own stop
  reads Interrupted. A close Orca asks for (a Stop, a rest, a quit) now ends
  what still runs as stopped; only a session that dies on its own leaves the
  ending unknown. The close reads no stop cause. A rest never meets live child
  work: the idle sweep keeps a chat with any.
- If the host fails to record a dismissed card, Claude gets the card back, and
  a withdrawal Claude made meanwhile closes it, as before the host took it.

* refactor(native-chat): the session mutation path gets its own module, breaking the chat-stop import cycle

chat-stop.ts imported mutateStructuredAgentSession from host-mutations.ts,
which imports mutateWithChatStop back. The mutation context and the one
admit-then-serialize path now live in structured-agent-session-mutation-context.ts,
which both import; host-mutations re-exports the context type for its other
readers.

* fix(claude): a close that saw a descendant survive leaves background work's ending unknown

A Stop, rest or quit ended still-live background tasks as stopped even when the
close found Claude's root gone but a descendant still running. Only a close
that proved the whole tree gone stops them now; otherwise they settle unknown,
as for an exit of the session's own. The stopped ending is marked as Orca's,
so a frame of the child's own still replaces it.

* docs(native-chat): comments describe the Stop as it now works

What re-drives a failed child end, when the child-work decoder stops what is
live, what a Stop's wait re-reads on a result, and why a card's Stop is
unnamed.
…tablyai#24108)

* ci(daemon): gate PRs on daemon protocol crossing from the newest release

Lands daemon-protocol-facts.mjs from the Windows update diagnostic branch with a
stricter parser, and adds check-daemon-protocol-crossing.mjs (rule R1): the working
tree must attach the newest release tag's daemon. Rollback crossing is reported only.
Runs in the cross-version-wire job, which already has full tags; tag selection moves
to config/scripts/stable-release-tags.mjs so both use one rule.

* feat(persistence): run profile backups in the worker whenever its entry is bundled

* refactor(orcad): make profile and native preflight runtime-neutral

The profile preflight parser now takes the expected runtime identity from the
caller (shipped callers pass the pinned Bun identity), and the native
preflight is renamed to orcad-runtime-native-preflight with neutral wording.

* feat(runtime): pin the Node 24.21.0 server runtime with an offline CI check

Add src/shared/node-runtime-pin.ts (NODE_RUNTIME_PIN, SERVER_TARGETS,
NODE_RUNTIME_ASSETS for all 8 server targets plus the headers tarball),
generated by config/scripts/update-node-runtime-pin.mjs from the nodejs.org
and unofficial-builds SHASUMS. check-node-runtime-pin.mjs verifies, with no
network, that the pin tracks the locked Electron, matches engines.node's
major, and covers exactly SERVER_TARGETS; it runs in the static analysis job.

ORCAD_BUN_TARGETS consumers now read SERVER_TARGETS so there is one target
list; orcad's Bun runtime and build output are unchanged.

* test(persistence): skip plain-Node backup selection tests in the Bun profile suite

* fix(runtime): reject a pinned archive that belongs to another target

* ci(daemon): fail PRs that swap a runtime launcher and bump the daemon protocol

D7.1 R3: hosting orcad or the daemon on another runtime is not a protocol change,
so one PR must not do both. The launcher file list lives in the check script; the
allow-runtime-launcher-protocol-bump label overrides it.

* feat(orcad): select pinned-Node slots by a .runtime-node marker

D7.1 R5: a Node slot names its shared runtimes/node-<sha256>/node through
.runtime-node instead of .build-target, so Bun-era clients read it as a legacy
slot rather than exiting 78 on a missing bun-runtime. Nothing builds the marker yet.

* fix(runtime): load the Node pin without the typeless-module warning

check-node-runtime-pin.mjs now requires the pin and takes nodeDistArchiveName from
its own module, so it no longer loads the update script's build graph.

* fix(orcad): resolve Node slots to the design's runtimes/node-<sha>/bin/node layout

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
)

* ci(daemon): gate PRs on daemon protocol crossing from the newest release

Lands daemon-protocol-facts.mjs from the Windows update diagnostic branch with a
stricter parser, and adds check-daemon-protocol-crossing.mjs (rule R1): the working
tree must attach the newest release tag's daemon. Rollback crossing is reported only.
Runs in the cross-version-wire job, which already has full tags; tag selection moves
to config/scripts/stable-release-tags.mjs so both use one rule.

* feat(persistence): run profile backups in the worker whenever its entry is bundled

* refactor(orcad): make profile and native preflight runtime-neutral

The profile preflight parser now takes the expected runtime identity from the
caller (shipped callers pass the pinned Bun identity), and the native
preflight is renamed to orcad-runtime-native-preflight with neutral wording.

* feat(runtime): pin the Node 24.21.0 server runtime with an offline CI check

Add src/shared/node-runtime-pin.ts (NODE_RUNTIME_PIN, SERVER_TARGETS,
NODE_RUNTIME_ASSETS for all 8 server targets plus the headers tarball),
generated by config/scripts/update-node-runtime-pin.mjs from the nodejs.org
and unofficial-builds SHASUMS. check-node-runtime-pin.mjs verifies, with no
network, that the pin tracks the locked Electron, matches engines.node's
major, and covers exactly SERVER_TARGETS; it runs in the static analysis job.

ORCAD_BUN_TARGETS consumers now read SERVER_TARGETS so there is one target
list; orcad's Bun runtime and build output are unchanged.

* test(persistence): skip plain-Node backup selection tests in the Bun profile suite

* fix(runtime): reject a pinned archive that belongs to another target

* ci(daemon): fail PRs that swap a runtime launcher and bump the daemon protocol

D7.1 R3: hosting orcad or the daemon on another runtime is not a protocol change,
so one PR must not do both. The launcher file list lives in the check script; the
allow-runtime-launcher-protocol-bump label overrides it.

* feat(orcad): select pinned-Node slots by a .runtime-node marker

D7.1 R5: a Node slot names its shared runtimes/node-<sha256>/node through
.runtime-node instead of .build-target, so Bun-era clients read it as a legacy
slot rather than exiting 78 on a missing bun-runtime. Nothing builds the marker yet.

* fix(runtime): load the Node pin without the typeless-module warning

check-node-runtime-pin.mjs now requires the pin and takes nodeDistArchiveName from
its own module, so it no longer loads the update script's build graph.

* fix(orcad): resolve Node slots to the design's runtimes/node-<sha>/bin/node layout

* feat(orcad): 8-slot node-pty prebuilds against the pinned Node headers at N-API 8

- build-orcad-prebuilds.mjs adds win32-x64/arm64 (conpty.node, the vendored
  conpty.dll/OpenConsole.exe, upstream's N-API conpty_console_list.node), compiles
  in a scratch copy against the hash-verified pinned headers (node.lib pinned per
  Windows arch) with NAPI_VERSION=8, rejects post-8 node_api_* imports, and writes
  a schema 2 manifest with per-file sha256, N-API level and the glibc need.
- --require-slots [slots] verifies files against hashes; --smoke loads the slot
  under the pinned Node and spawns a PTY; --print-slot names the host slot.
- The slot installer gates on N-API, libc, arch, glibc and file hashes instead of
  the exact NODE_MODULE_VERSION, and installs nested files (conpty/).
- bun-profile-tests.yml builds, verifies and smokes each runner's slot.

* fix(orcad): scope node-pty's glibc .symver pins to glibc on musl prebuild slots

musl's unversioned libc cannot satisfy openpty@GLIBC_* references at link
time, so the Alpine slot compile would fail. Pin the staged pty.cc guard to
__GLIBC__ and assert both musl transforms against the installed patch.

* feat(orcad): run orcad on the pinned Node instead of Bun

A packaged orcad slot now references the pinned Node 24.21.0 by its
executableSha256 (`.runtime-node`, `.server-target`) instead of carrying
bun-runtime, and ships node-pty from the slot's prebuild, only its own
ripgrep, and no Windows Bun PTY gate. The runtime lives beside the slots
at runtimes/node-<sha>/bin/node (node.exe on Windows, upstream name).

- build:orcad (build-orcad-node.mjs) builds the host slot's prebuild when
  missing and places the pinned runtime; the template is schema 3 with
  per-target files.
- handoffToBundledOrcad() resolves the slot's runtime reference and checks
  process.versions.node against the pin; a host Node >= 18 still hands off.
  Startup preflight keys on running as that runtime; callers expect 'node'.
- orcad and its daemon use node-pty (ConPTY + windows-pty-job on Windows);
  the Bun PTY sources, gate entry and canUseBunPty branches are removed.
- SSH deploy uploads the official archive once per pin, extracts and
  hash-checks it on the host, and self-tests it before publishing. Bun
  slots stay launchable for rollback; Node slots never use host Node.
- The runtime materializer is generic over pinned assets; the Bun wrapper
  remains only for the OpenCode vault reader (design Phase 2).
- Cross-runtime test: a profile DB written by Bun 1.4.2 (WAL left by
  SIGKILL) opens and backs up under the pinned Node, and the reverse.

No daemon PROTOCOL_VERSION change (design D7.1 R3).

* docs(ci): name the headless lanes after the pinned Node, drop Bun shard timings

Design D10: ci-demand-rollout.md and ci-runner-efficiency.md follow the
bun-profile-tests.yml -> node-server-tests.yml rename; shard timings drop the
deleted Bun PTY tests and follow the renamed ones.

* chore(ci): count the runtime archive download as a runtime launcher path

* fix(orcad): pin the macOS C++ standard for node-pty prebuilds

The official Node headers' config.gypi sets clang: 0, so common.gypi skips its
gnu++20 xcode_settings and Apple clang 15 (macos-14 runners) compiles
node-addon-api as C++98.

* fix(orcad): resolve the preflight's slot through realpath, as the handoff does

A symlinked orcad.js handed off to its real slot's pinned Node, but the
startup and profile preflights read the symlink's directory, found no
runtime marker there, and silently skipped the readiness check.

* refactor(ssh): drop materializeCachedNodeRuntime, which nothing calls

Deploys upload the verified official archive (design D5); no client path
needs an extracted Node executable cached by digest.

* test(orcad): gate the Bun-to-Node upgrade and Node-to-Bun rollback with live terminals

Design D7.1 R1/R3/R4 and D7.2. The last Bun orcad and this checkout's Node
slot are installed side by side under ~/.orca-remote, launched and stopped
with the client's own deploy commands, and share one data root. Each
direction proves the incoming orcad adopts the outgoing runtime's daemon
(same PID, same shell, output continues), opens its profile database and
backs it up with its own shipped worker, and that GC keeps the slot the
live daemon was forked from.

The node-server Linux lanes provide Bun 1.4.2 and build that Bun orcad
from main, and run with --cross-runtime. --artifact and --cross-runtime
now make their tests fail on a missing input instead of skipping.

* ci(node-server): pin node:24.21.0-alpine by its multi-arch index digest

* test(ssh): name the runtime archive fixture after its role

* test(node-server): load node-pty from the packaged slot in artifact runs

The node-server lane installs dependencies without building node-pty, and
Linux has no upstream prebuild, so the real-PTY failed-I/O teardown test
(picked up by the pty-subprocess selector) could not load pty.node. In
--artifact runs, alias node-pty to out/orcad's shipped slot so the test
exercises the addon orcad actually runs under the pinned Node.

* fix(orcad): let the Windows profile preflight exit after its PTY probe

On Windows, node-pty keeps the conout worker thread and pseudoconsole alive
until kill(), even after the shell exits. The PTY health probe never killed a
cleanly exited probe, so the packaged preflight printed its readiness line
and then hung until the build's 30s timeout, reported with an empty stderr.

- The probe kills its PTY on Windows after exit and uses the bundled ConPTY
  the daemon spawns with.
- The preflight exits once stdout is flushed; its owner reads to EOF.
- Preflight failures now report code, signal, timeout, stdout and stderr.

* test(node-server): load the slot's node-pty in the real-PTY test, not by alias

A vite alias redirected only ESM imports of node-pty; windows-pty-job and
local-pty-utils resolve it through require, so Windows loaded two conpty.node
copies and the Git Bash job-membership proof read an empty job. The failed-I/O
teardown test now loads node-pty through a fixture that picks the packaged slot
in artifact lanes.

The pty-subprocess selector was a prefix that also pulled in its POSIX-host
sibling unit tests, which pr.yml runs and which were never qualified on
Windows. Select the directory plus the two sibling files that belong here.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…ly (stablyai#24320)

* fix(relay): gate the Asia canary on region fallbacks against a pre-canary baseline

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010

* fix(relay): gate the Asia canary on region fallbacks against a pre-canary baseline

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
…promotion (stablyai#24318)

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
…move Bun (stablyai#24128)

* feat(ai-vault): read OpenCode history with the pinned Node instead of Bun

SSH hosts whose Node lacks node:sqlite (or its backup(), which 22.13-22.15
omit) now get the pinned Node in the shared ~/.orca-remote/runtimes/node-<sha>
store orcad uses: POSIX hosts receive the official archive and extract and
hash-verify it on the host; Windows hosts receive the verified node.exe the
client extracted, promoted by host Node with the same hash check. WSL distros
use the same layout and checks under ~/.cache/orca/runtimes/.

The Bun release pin table and its materializer are deleted. Old relays keep
reading their vault-sqlite/<sha>/bun references; nothing deletes those files.
An unconfirmed runtime upload now keeps its stage instead of removing it.

* refactor(sqlite): drop the Bun SQLite adapter; node:sqlite is the only backend

Nothing outside Electron runs on Bun any more (design D4), so SyncDatabase
loses its Bun branch, and bun-sqlite-database, bun-sqlite-statement and
bun-readonly-wal go, with the relay's bun:sqlite external. The profile-state
backup worker admits Electron or an entry that exists, and startup errors
name the pinned Node. The D7 cross-runtime gate still runs Bun 1.4.2, now
reaching Bun's SQLite through its node:sqlite.

* test(native-chat): drop the Bun SQLite driver case now that node:sqlite is the only backend

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
… store GC (stablyai#24130)

* fix(ssh): relay version GC deletes only on an exited verdict and keeps the previous build

The relay records .relay-pid in its version dir once it owns its socket. GC calls a
relay version dir exited only when that PID is provably dead and every relay-*.sock
refuses a connection; a dir without a PID file keeps the test -S rule. The most
recently completed other relay build is pinned like orcad's rollback target.
Design D5 GC liveness.

* feat(ssh): collect the shared runtimes/ Node store and give it its own owner

runtimes/ gets its own owner in the install model, so no version-dir GC (new or old
clients, whose listings are prefix-scoped) can list or delete it. A store pass
removes node-<sha> only when no retained dir references it, it is neither a current
pin nor the newest other verified runtime, and a ps or /proc check ran and found no
process using it. Legacy relay-*/orcad-* dirs are read for references and reported
as diagnostics only (design D10 two-step). Wired behind orcad GC's nodeRuntimePins.

* fix(ssh): runtime store process check holds runtimes reached through a symlinked home

/proc exe resolves symlinks and argv keeps whatever spelling launched the runtime, so
filtering on the exact $root path missed in-use runtimes on hosts like /home -> /var/home.
Filter on the store segment instead; the parser already attributes holds root-agnostically.

* test(ssh): wait for the holder process to spawn instead of a fixed delay

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…compat slot (stablyai#24134)

* build(orcad): build server glibc slots on glibc 2.28 and add the glibc 2.17 compat slot

Design D6: the default linux-{x64,arm64}-glibc node-pty slots now build in
manylinux_2_28 (digest-pinned) and are gated at glibc 2.28 / GLIBCXX_3.4.25
through a floor profile on verify-linux-glibc-floor.cjs; the desktop keeps
its Ubuntu 20.04 (2.31) default.

Adds the opt-in linux-x64-glibc217 compat target: NODE_RUNTIME_COMPAT_ASSETS
pins the unofficial glibc-217 Node (update/check pin scripts cover it, outside
SERVER_TARGETS), and a new CI lane builds the compat slot in manylinux2014
with static libstdc++, gates it at glibc 2.17 with no shared C++ runtime in
DT_NEEDED, and smokes it under the glibc-217 Node.

* refactor(node-runtime-pin): route compat lookups through isCompatServerTarget; keep the glibc doc's slot-name paragraph intact

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…tablyai#24129)

* feat(relay): runtime self-test flag and informational runtime on handshake-ok

relay.js --orca-runtime-selftest <nonce> dlopens pty.node, opens and closes a
PTY, and prints one JSON line (nonce, node, napi, glibcVersionRuntime) for the
client to classify before it launches a daemon on a runtime (design D5).

handshake-ok gains an optional runtime {kind, version}; bridge and daemon
already match exactly on version, so it is informational only (D8.1).

* feat(ssh): opt-in pinned-Node relay with prebuilt addons (D5, D6 rung A, D8.1)

SshTarget.remoteRuntime (legacy | pinned-node, default legacy; env
ORCA_SSH_REMOTE_RUNTIME for development) selects the runtime. On POSIX hosts
the pinned path resolves the target with its glibc major.minor, ensures
~/.orca-remote/runtimes/node-<sha>/bin/node, uploads the relay bundle plus
the target's node-pty slot and @parcel/watcher from the orcad artifact
(no npm or node-gyp on the host), writes .runtime-ref-node-<sha>, and folds
the runtime and addon digests into the relay version so pinned and host-Node
builds never share a dir or socket.

A 30 s self-test (node --version, then the relay self-test) gates
.install-complete. Timeouts and lost channels are unverifiable and never step
down; noexec, missing_lib, libc_floor, illegal_instruction and wrong_libc
refusals fall back to the untouched host-Node path with a logged reason,
remembered for the session.

* fix(ssh): only an answered libc probe steps the pinned relay down

A lost channel during target detection says nothing about the host; descending
would launch a host-Node daemon beside a running pinned one and strand its sessions.

* test(ssh): mark the mocked SSH connection casts in the pinned relay tests

* test(ssh): resolve the pinned runtime mock to an executable path

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…-agent-session-adapter.ts so main's lint passes (stablyai#24323)

structured-agent-session-adapter.ts reached 301 lines once two changes on
main met, one over the 300-line limit, so oxlint fails on main.

The file is the adapter contract: its types, its interface and the typed
failure verdicts. The only logic in it read what a cleanup or a stop proved
about the provider child's exit: rethrowing a failed acquisition with the
verdict its cleanup reached, and reading whether a stop left the provider
root gone. That logic moves unchanged to
structured-agent-session-provider-exit-proof.ts, which imports the verdicts
from the adapter; nothing imports back. Its six importers now import from
the new module. No behavior change.
…setting (stablyai#24133)

* feat(ssh): complete the relay runtime fallback ladder (D6 rungs B slot, C, D)

Rung C runs the relay on the host's Node >= 18 with Orca's prebuilt N-API
addons and no npm (addon-only probe mode). Rung B is a data-driven slot chosen
only when a compat runtime is listed. Rung D fails the connect with a
classified reason carried as a TerminalUnavailableCause. The ladder steps
down only on classified refusals; unanswered probes throw. The rung decision
is persisted per host keyed by (glibc, runtime hash, Orca major), and
ssh_remote_runtime_resolved reports it once per host per session.

* feat(settings): SSH host runtime choice (Auto | Orca-managed Node | Host Node)

* docs(telemetry): describe ssh_remote_runtime_resolved

* fix(ssh): let a passing rung C disprove a remembered noexec; allow glibc-less compat runtimes

A remembered rung A noexec was re-persisted even after rung C self-tested addons from the same
~/.orca-remote tree, so rung A stayed skipped until the key changed. Rung B's evaluator also could
never match a musl compat runtime.

* test(ssh): import node:fs once in the host-node addon test

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…lly receives (stablyai#24343)

The parent workflow collapses canary-apply and batch-apply into the job mode
apply, so the headroom step's canary-apply/batch-apply condition never held and
the gate was skipped on every real roll. Run it wherever the drain runs (apply,
rollback before its restart) and in read-only verify; skip only a resumed
rollback, which drains nothing. A new workflow-shape test fails on any job
step comparing against a mode the parent cannot pass, and on a drain that can
run without the headroom check.

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
* feat(ssh): pinned-Node relay on Windows SSH hosts (D5 Windows, D2)

Windows hosts opted into remoteRuntime 'pinned-node' now get the same rung A
relay POSIX hosts do, instead of an early host-Node fallback.

- Runtime store: the official node-v24.21.0-win-<arch>.zip is uploaded to a
  stage under %USERPROFILE%\.orca-remote\runtimes, verified against the pinned
  archive hash, node.exe extracted with System32 tar.exe (Expand-Archive
  fallback), hashed with Get-FileHash, run once, and published with
  node.exe + .verified by one Directory.Move. One powershell.exe per phase via
  the existing powerShellCommand helper; the probe also creates the stage. No
  new -EncodedCommand site, no -ExecutionPolicy, no Add-Type. node.exe keeps
  its real name at runtimes\node-<sha>\node.exe.
- Bytes that change or vanish after Orca wrote and verified them are reported
  as ORCA_NODE_RUNTIME_SECURITY_MODIFIED and become a remembered
  'security_software' refusal (fallback to the host-Node relay); application
  control blocks classify as 'noexec'.
- Addons: the win32 slot's conpty.node, conpty_console_list.node,
  conpty\conpty.dll + OpenConsole.exe, watcher and windows-process-tree.node
  ride with the relay; the orcad template now carries the win32 targets.
- Self-test on Windows is one powershell.exe running relay.js on node.exe; the
  report must name the pinned Node. The relay self-test loads conpty.node and
  opens a PTY with useConptyDll, and reports a missing bundled ConPTY file as a
  load failure. A pinned relay's terminals use the bundled ConPTY too; host-Node
  relays are unchanged.
- describeRelayRuntime recognizes the Windows store layout.

* fix(ssh): skip the redundant stage-cleanup powershell.exe after a Windows runtime promote

The promote script already removes its stage on every path, so the client-side
cleanup only runs when promote never returned (upload failure, abort, timeout).

* test(ssh): expect the ladder's remembered flag and pin check on Windows pinned relays

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…ack (stablyai#24136)

* feat(ssh): run runtime-store GC after a pinned relay launch, under a store lock (D5)

The pinned-Node relay deploy now collects runtimes/ after a successful
launch, keeping the pin it runs. Promotion in ensureRemoteOrcadNodeRuntime
and GC deletion both hold runtimes/.store-lock (install-lock primitives,
20-minute stale rule); GC only tries the lock and skips when busy. A cold
pinned install re-checks its runtime under the lock once the relay ref is
visible, closing the ensure-then-ref window. GC also sweeps runtimes/.stage-*
dirs nothing has written to within the stale rule.

* feat(ssh): stream relay and runtime uploads over exec stdin when SFTP is refused (D5)

On the bundled ssh2 transport to a POSIX host, a definite SFTP refusal
(subsystem refused, sftp-server exited during the handshake, or a chrooted
view answering NO_SUCH_FILE for a shell-created path) now falls back to
writing through an exec channel's stdin, reusing makePosixWriteFileCommand
with a byte-count check and atomic rename. Transport loss, timeouts and
aborts never select the fallback. execCommand gains a stdin option.

* test(ssh): answer the runtime store lock in the OpenCode runtime setup test

Promotion now runs under runtimes/.store-lock, so the mocked host must grant
the lock and the stage-exhaustion case makes four more round trips.

* test(ssh): rung C relays never take the runtime store lock

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…ials (stablyai#24347)

* fix(relay): let a draining cell pass restart-safe through refused redials

A draining cell refuses every control and host proof, so once no session,
splice, or queued byte remains, an in-flight or reserved connection unit can
only belong to a dial the cell is about to refuse. The restart-safe wait no
longer resets its pace-window streak on those units, and its progress line
now prints every counter the gate reads.

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010

* test(relay): pin fail-closed parsing of handshake counters on a draining cell

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
…4146)

* fix(ssh): classify a musl host missing libstdc++ as missing_lib, not wrong_libc

musl's loader follows each missing-library line with one 'Error relocating ... symbol
not found' per unresolved symbol, and the relocation pattern was checked first. Check
missing libraries before relocation errors; the ld-linux/ld-musl interpreter case stays
wrong_libc.

* build(orcad): allow a partial deployment template for CI

build-orcad-template --targets a,b builds and verifies only the named slots, so a CI job
that can fill just the x64 Linux prebuild slots can still materialize rung A/C addons.
Without the flag every target is still built and verified.

* ci(ssh): hostile-host matrix for the relay runtime ladder

Drives the real client-side relay deploy against Docker sshd targets and asserts the
design D6 rung each lands on: Debian 10 and AlmaLinux 8 (glibc 2.28) and Alpine (musl)
on rung A; Alpine without libstdc++ refused missing_lib down to D; Ubuntu 22.04 with a
host Node 20 and a noexec home straight to D (home_noexec); CentOS 7 (glibc 2.17)
refused libc_floor at A and C, falling to a host-npm path with no Node; and a
no-egress Debian 10 still on rung A. Launched cells also prove the terminal echoes,
no npm or compiler ran, a second connect reuses the uploaded runtime, and runtime GC
keeps the in-use runtime while collecting an idle one.

New workflow ssh-hostile-hosts.yml runs on dispatch and on path-filtered PRs.

* test(ci): pin the hostile-host workflow to the headless-server builder images

The matrix builds its runtime slots in copies of the node-server lanes' Alpine
and manylinux images; this contract fails when NODE_RUNTIME_PIN or either
builder digest moves in one workflow and not the other.

* test(ssh): reconnect as the same client and retry a grace-held PTY owner in the hostile-host matrix

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
… can run (stablyai#24147)

* feat(ssh): connect in plain SSH mode when no Orca runtime can run on the host

Runtime ladder rung D (design D6): instead of failing the connect, register an
ssh2 shell-channel PTY provider and an SFTP-only filesystem provider and publish
the classified reason on the SSH connection state.

* fix(ssh): harden plain SSH mode against stale reconnects, host sleep and tilde cwd

- Only the current connect or reconnect attempt may enter plain SSH mode; a superseded
  reconnect whose ladder ends at rung D no longer registers a second provider set.
- Host-sleep resume probes a plain session over SFTP instead of always reconnecting,
  which ended every open plain shell.
- A home-relative cwd keeps its tilde outside the quotes so the shell expands it.
- The SFTP provider implements folder download, which the connect state advertises.

* docs(ssh): rung D now means plain SSH mode, not a failed connect

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
Co-authored-by: m4air <m4air@Mac.localdomain>
… Windows (stablyai#24149)

* fix(ssh): collect the pinned-Node runtime store on Windows hosts

Windows SSH hosts now run runtime-store GC instead of skipping it: one
PowerShell inventory reads .runtime-ref-node-<sha> and .runtime-node refs from
every version dir, and one Get-CimInstance Win32_Process query filtered on an
image path under runtimes\ adds process holds (never by image name; a failed
query keeps everything). Stale upload stages are swept with the same rule as
POSIX. Promotion and the post-upload hold check now take the store lock on
Windows too, and the lock's own commands run unwrapped there.

Windows relay version-dir liveness now honours .relay-pid (design D5): a live
PID answers ALIVE before any pipe is touched, a dead one (ESRCH) plus refusing
pipes is exited, anything else is unverifiable. The runtime probe adopts a
pinned node.exe an earlier vault reader left without a .verified marker after
running it.

* fix(ssh): Windows stage fencing and vault runtime go through the verified node.exe

Upload-stage file identity on Windows no longer compiles an Add-Type P/Invoke
helper when the relay runs on Orca's verified pinned node.exe: the stage
commands run a fixed fs.lstatSync(..., {bigint:true}) script through it. It
prints the legacy helper's vol:high:low lowercase hex, and identity files are
compared after normalising hex spelling, so old and new clients recover each
other's stages. Host-Node relays keep the legacy helper; the choice is
documented in windows-edr-posture.md.

The Windows OpenCode vault reader now installs the pinned runtime through
ensureRemoteOrcadNodeRuntime (official zip, host-side extraction, .verified,
store lock) instead of uploading a client-extracted node.exe, and the relay dir
gains a .runtime-ref-node-<sha> so store GC keeps the runtime the vault uses.

* test(ssh): run the Windows stage-identity and store-GC tests on the Windows lane

The legacy/node.exe identity compatibility test and the Win32_Process hold path
were gated to win32 but no CI lane ran them. Add both files to the Windows
package lane and a real running-node.exe hold test.

* test(ssh): tear down Windows-lane temp trees through removeTreeSync

* test(ssh): grant the store lock to the Windows OpenCode runtime setup test

The Windows promote now runs under runtimes/.store-lock, so the mocked host
must answer the lock's CreateNew step.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
Co-authored-by: m4air <m4air@Mac.localdomain>
…uthorisation, not job startup (stablyai#24349)

* fix(relay): anchor same-cap monitor evidence freshness to the run's authorisation, not job startup

The same-cap gate now verifies the dry-run on its own clock and records the
authorisation instant in the single-use consumed marker; each cell job checks
the evidence age at that instant and bounds its own start after it.

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010

* fix(relay): refuse a re-run same-cap gate before it consumes evidence; tighten order tests

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
…ablyai#24155)

* build(orcad): merge per-runner prebuild slot trees into one matrix

Each node-server lane builds only its own node-pty slot. Release CI needs
their union before `build:orcad-prebuilds --require-slots` and the
template build can run; merge-orcad-prebuilds.mjs verifies every lane's
files against its own manifest, refuses duplicate slots and mismatched
node-pty/N-API/Node-header builds, then writes one merged manifest.

* build(orcad): keep agent-browser out of the desktop deployment template

The template rides inside every desktop build (design D2). Seven ~10 MB
agent-browser binaries would be ~76 MB, more than the rest of the template;
design D2's package contents never listed it, and a slot without one
already reports no headless browser. ORCAD_OMIT_AGENT_BROWSER=1 skips the
copy; standalone build:orcad still includes it.

* feat(packaging): ship the orcad deployment template in desktop builds

Design D2: the server JS and every target's addons ship inside the app,
as out/relay does; the ~120 MB Node runtimes stay excluded and are
downloaded on demand. electron-builder copies out/orcad-template to
Resources/orcad-template on every desktop OS, which is the first path
materializeOrcadArtifact tries (process.resourcesPath).

Platform signing rewrites native bytes the template manifest hashes:
- macOS: the tree is signIgnored (codesign rejects its ELF/PE payloads);
  afterPack signs the darwin targets' Mach-O files with the app identity,
  as notarization requires, then reseals only those manifest entries.
- Windows: SignPath signs after packaging, so release CI reseals from the
  inner-signing list (packaged-orcad-template.cjs --reseal-signed).
Every other file must still match the build's hashes; afterPack verifies.

ORCA_REQUIRE_ORCAD_TEMPLATE=1 makes a missing template fail beforePack and
afterPack; without it a build ships none and SSH relays keep the legacy
path. verify-packaged-orcad-template.test.mjs's "unused, excluded"
contract is reversed on purpose.

* ci(release): build the orcad template from qualified lanes and package it

node-server-tests.yml becomes callable with a ref and build_template.
With build_template, each lane that owns a release slot (macOS, Windows,
the glibc 2.28 and Alpine lanes, and the glibc 2.17 compat lane) uploads
its qualified out/orcad-prebuilds, the Windows lane also uploads both
process-table addons, and desktop_template merges them, gates the full
matrix plus the compat slot with --require-slots, runs
build:orcad-template and uploads the orcad-template artifact.

release-cut calls it at the release tag beside the other gates. The
build and build-mac jobs wait for it, download it into out/orcad-template
(the mac workflow from the parent run), and require it via
ORCA_REQUIRE_ORCAD_TEMPLATE. The Windows signing staging skips the
template's Linux/macOS payloads, and a reseal step records SignPath's
bytes before the installer rebuild. A template-scoped concurrency group
keeps a release call and main's push runs from cancelling each other.

* test(orcad): keep the packaged-lookup imports clear of the compat-slot import edits

* ci(orcad): let a rerun lane replace its template artifacts

upload-artifact v4 refuses a second upload under an existing name in the same
run, so rerunning a flaky node-server lane during a release would fail at the
upload instead of re-qualifying the slot.

* ci(node-server): build the template's Windows addons before the lane switches to Node 18

The addon build script imports TypeScript, which Node 18 cannot load, so every
build_template run (release-cut included) failed on windows-2022.

* fix(build): ship the orcad template's shared node_modules

electron-builder's extraResources filter always drops the root node_modules of
a source directory, so packaged apps lost orcad-template/node_modules and the
afterPack verify failed. Copy it through its own resource entry.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…ned relay (stablyai#24180)

* ci(ssh): import the private Windows OpenSSH provisioning harness

Copied unchanged from origin/OrcaWin/np-windows-ssh-provider-diagnostic
(config/ci/windows-ssh-provider/preview-ssh/ at 1242f3c, commits 78b3857,
c24adcc, 069aa7b): a private LocalSystem sshd service on 127.0.0.1 for a
dedicated standard user, either the Microsoft-signed Win32-OpenSSH
10.0.0.0p2-Preview ZIP (archive and every binary pinned by sha256) or the inbox
OpenSSH.Server capability binaries. The following commits extend it for the
pinned-Node relay host lanes.

* test(ssh): run hostile-host cells through a host-agnostic driver

The Docker matrix drove the relay deploy and inspected the container with
inline docker exec calls, so no other host could reuse it. Split it into:

- ssh-hostile-host-test-harness.ts: the deploy, ladder observation, terminal
  echo (per-shell probe), runtime reuse and GC-keeps-in-use assertions, now
  also capturing every command the deploy sent the host.
- ssh-hostile-host-observer.ts: how a driver inspects the host outside SSH;
  docker exec for containers, the local filesystem for a loopback host.
- a legacy_opt_out outcome: the ladder never runs and nothing enters the
  pinned store, whatever the host-Node path does.

Launched cells now also check the runtime's sha256 on the host and that every
slot file (the Windows bundled ConPTY pair included) landed in the relay dir.

* ci(ssh): Windows SSH-host lanes for the pinned-Node relay

Phase 2 exit gate, Windows half: the real deployAndLaunchRelay through a real
SshConnection against Win32-OpenSSH on 127.0.0.1, on windows-2022 (x64) and
windows-11-arm (arm64), for both the inbox OpenSSH.Server capability and the
Microsoft-signed 10.0.0.0p2-Preview release (ZIP; archive and each binary
pinned by sha256 and Authenticode, as in the imported harness).

Builds on the provisioning harness from
origin/OrcaWin/np-windows-ssh-provider-diagnostic (previous import commit):
- one private standard account per cell, so every cell starts from an empty
  runtime store;
- -HiddenTools: the private sshd service's own Environment carries a PATH
  without any machine PATH entry holding node/npm/compilers, led by logging
  .cmd shims; a session probe fails the job if node.exe still resolves;
- DefaultShell set per cell by invoke-pinned-relay-cells.ps1 and restored at
  cleanup (dispatch proven per cell via %COMSPEC%).

Cells (src/main/ssh/ssh-windows-host-cells.ts): pinned-cmd (stock sshd),
pinned-powershell (DefaultShell = Windows PowerShell) expect rung A on the
pinned node.exe with the relay self-test passing, terminal echo, runtime
reuse, GC keeping the in-use runtime, stage identity through node.exe and no
Add-Type in any decoded session command; legacy-opt-out expects the ladder
never to run and an untouched pinned store.

* fix(ssh-ci): tolerate absent-drive PATH entries and retry Windows userData teardown

Join-Path throws on a machine PATH entry naming a drive the runner lacks,
which would abort provisioning before any cell ran; the toolchain split now
probes with [IO.File]::Exists over [IO.Path]::Combine, and the self-test
covers an absent drive. The hostile-host harness removes its throwaway
userData with removeTreeSync so a transient Windows lock cannot fail the
lane's afterAll.

* fix(ssh-ci): stop the account list rebinding the typed -Accounts param

PowerShell variable names are case-insensitive, so $accounts=[List[hashtable]] assigned into
the [int]$Accounts parameter and every Windows host job died before provisioning. Rename the
list and make the provisioning self-test reject script-scope assignments that shadow a param.

* fix(ssh-ci): hide the host toolchain by ACL, since sessions ignore the service PATH

Win32-OpenSSH builds a session's PATH from the machine and user registry values, so the private
service's Environment never reached SSH sessions and host node.exe stayed visible. Deny the private
accounts the toolchain PATH directories, put the logging shims on each account's own PATH, and
record failing sshd and client log lines so a refused login is diagnosable from the receipt.

* fix(ssh): resolve the upload root before checking entries stay inside it

uploadDirectory compared each entry's realpath against the root as given, so a root reached
through a symlink, junction or Windows 8.3 short name (C:\Users\RUNNER~1 in TEMP) rejected every
entry as escaped and the pinned runtime upload never started.

* fix(ssh-ci): fail cells on a vitest failure and give each account its own keys file

The cells script read $LASTEXITCODE under the workflow's GetNewClosure callback, which sees a
stale captured copy, so failed cells reported exit 0 and the job passed. Read the global value.
Inbox sshd 8.1 checks authorized_keys with read_ok=0, refusing a file other accounts can read;
use one keys file per account via %u.

* fix(ssh-ci): keep the account name in inbox mode and surface the WMI launch gap

The inbox binary-verification loop reused $name, so later SSH and SFTP probes logged in as
'sftp-server.exe'. Before the cells run, probe whether a standard SSH user can call WMI
Win32_Process.Create (the Windows relay launch path); when refused, warn and grant the cell
accounts Remote Enable on root\cimv2 for the run so the remaining assertions execute.

* test(ssh): keep the first terminal session answering keepalives through GC

The hostile-host driver disposed the first session's multiplexer before the GC and reconnect
steps, so a slow Windows GC let the relay reap the silent owner as 'local' and the reconnect then
waited out the full owner grace. Keep the session live until the connection closes, as the app
does, and resend the terminal probe until the shell evaluates it: ConPTY PowerShell can drop
typeahead sent before its first prompt.

* ci(ssh): keep each cell's relay logs in the receipts

* fix(relay): detach an ended socket client as peer-closed before destroying it

The listener destroyed a socket on 'end' but detached its client only on 'close'. A relay write
in that window failed with 'Relay socket is closed', and the dispatcher closed the client as
'local', so its PTY owner kept the full 30s grace instead of the peer-closed floor and a quick
reconnect was refused. The Windows host lanes logged this race on the named-pipe endpoint.

* test(relay): drive the peer-end listener test with a real dispatcher instead of a cast stub

The stub was an unchecked 'as unknown as RelayDispatcher' that failed the changed-code casting gate.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…with EPIPE or ECONNRESET (stablyai#24209)

On the Windows arm64 lane a relay write to a named-pipe client could fail with EPIPE or ECONNRESET
before EOF arrived. The dispatcher's writer then closed the client as 'local', so its PTY owner kept
the full 30s grace instead of the 250ms peer-closed floor and a quick reconnect was refused.

The reconnect listener now detaches the client as peer-closed before settling a write that failed
with EPIPE/ECONNRESET, or a write attempted after a peer reset destroyed the socket (so
ERR_STREAM_DESTROYED counts only via socket.errored). ECANCELED, ECONNABORTED and a plain
destroy() stay 'local', since our own teardown can raise them.

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…a symlinked root (stablyai#24179)

* test(ssh): upload a root reached through a symlinked parent

The upload-root realpath fix landed with stablyai#24180; this keeps macoshost's case
where the root is passed explicitly beneath a symlinked parent.

* ci(ssh): macOS hostile-host lane on a loopback user-level sshd

Adds local-sshd cells for darwin-arm64 (macos-14) and darwin-x64
(macos-15-intel): a non-root sshd on 127.0.0.1 logs in as the runner user
with SetEnv PATH=<shims>:/usr/bin:/bin:/usr/sbin:/sbin and an empty HOME, so
no rc file restores Homebrew. The driver asserts rung A, terminal echo,
cached runtime reuse, GC keeping the in-use runtime, no toolchain or xattr
calls, and that the SFTP-uploaded Node carries no quarantine and runs as
uploaded. Docker cells are unchanged; each machine runs only cells it can host.

* test(ssh): fail a hostile-host run that would skip every named or hostable cell

A cell named for the wrong OS or arch was silently skipped, so a macOS job on a
mismatched runner went green having deployed nothing. Named cells must now be
hostable here, and a gated run must select at least one cell.

---------

Co-authored-by: m4air <m4air@m4airs-Air.localdomain>
…ded cells without rewriting the MIG (stablyai#24373)

* fix(relay): accept MIG version-name reconciliation and recreate stranded cells without rewriting the MIG

The stranded-rollback recovery ran a gcloud rolling action, which renames the
MIG version outside Terraform. Every later plan for that cell then reverted the
label, and the capacity-plan validator refused the revert as an unreviewed MIG
change, so the cell could be neither rolled nor rolled back.

The validator now accepts a MIG field moving back to what relay-gce-cells.tf
declares (version name and update policy), in every mode, and a test pins those
values to the Terraform file. The stranded branch recreates the cell's single
instance with recreate-instances, which leaves the MIG untouched.

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010

* fix(relay): let a label-only MIG plan through and recreate on it in a stranded rollback

A stranded rollback whose template is already in place plans only the version
name revert. The validator still required the MIG template to move, so that
plan was refused, and the recreate gate (changes == 0) would have skipped a
plan of one change and left the drain flag set. Require the template move only
when no declared field reconciles, and recreate whenever the template was not
replaced (changes < 2).

Claude-Session: ced32ebb-7155-4413-adad-1eccd14c2010
OrcaWin and others added 10 commits October 2, 2026 18:31
…he (stablyai#24669)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
* fix(jira): expire cached attachment images while idle

* test(jira): preserve cache bounds and site expiry after clears

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: Neil <neil@stably.ai>
…gs (stablyai#24680)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: Neil <neil@stably.ai>
…stablyai#24686)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…yai#24694)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…yai#24851)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…tablyai#24721)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…tablyai#24805)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…tablyai#24833)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
@innocarpe innocarpe added the bug Something isn't working label Oct 3, 2026
brennanb2025 and others added 9 commits October 2, 2026 18:47
…red Orca servers (stablyai#24204)

* fix(native-chat): a host admits structured sessions by client capability, not its own chat setting

A host's experimentalStructuredNativeChat decided whether any paired client could reach
agentSession.* at all, and whether session.tabs.* showed it structured tabs. That setting is the
host user's own launch preference: whether a new agent opens as a chat or a terminal is decided by
whoever launches it. Using it as admission control meant a client whose own preference was
"structured chat" was refused on a host whose preference was "terminal", and chats opened while
the setting was on were withheld from mobile once it was turned off.

The gate now asks one thing: did the client advertise agent-session.structured.v1 (in-process
callers negotiate nothing and are always admitted). Tab projection and restore follow the same
rule. With the setting no longer gating anything, the separate cleanup gate (close, cancel,
unsubscribe, release), which existed only so those kept working after the setting was switched
off, is identical to the main gate and is folded into it. The settings listener that republished
tabs when the setting changed is removed, since projection no longer depends on it.

The host setting still picks the default for launches that start on the host itself
(agent.launch from mobile, orchestration worker-start).

* fix(native-chat): the desktop declares structured chat support to paired hosts

The desktop renderer advertised agent-session.structured.v1 (and the Claude, turn-item and
background-task capabilities that go with it) to its own main process but not to a paired Orca
server. The server therefore refused every agentSession.* call from the desktop and stripped
structured chat tabs out of the tab list it published to it, so a structured chat running on a
paired server never appeared on the desktop, even though the renderer already mirrors a host's
agent-session tabs and drives each one against the server that owns its workspace.

The same renderer reads structured chats on either host, so the remote Electron list now carries
the same structured-session capabilities as the local one, and the capability test pins that
nothing is advertised only locally.

* fix(native-chat): negotiate client-chosen launch mode so released phones and old servers keep terminals

Hosts advertise agent-session.structured.client-launch-mode.v1: they admit
structured sessions by client capability alone. A remote client that does
not advertise it (phones released before agent.launch) asks createSupport
to pick the launch mode, so the host keeps answering that with its own
setting, exactly as before. Cleanup methods keep their own named gate so a
future admission condition cannot make close or cancel refusable.

* refactor(runtime): keep the Electron client capability list in its own module

protocol-version.ts is at its line budget; the list is what the desktop
advertises to paired hosts, not the host's own contract.

* fix(native-chat): the desktop declares it picks each launch mode itself

Paired hosts and the desktop's own main process then answer createSupport
by the workspace rather than by their own chat setting.

* chore(native-chat): justify the two type assertions this change's lines touch

* fix(native-chat): chats that already exist keep showing whatever the chat setting says

The structured chat setting decides only what new agents open as. With it
off, this machine's structured chats used to be hidden while the host,
which no longer reads the setting, still reported them to the workspace
activation gate, so a workspace holding only a chat opened empty. The
local chat mirror and its startup restore now run whatever the setting
says, the continue-after-restart offer follows the chats that exist, and
the setting's copy says it applies to new agents.

* test(native-chat): pin that a host advertises the client-chosen launch mode

* fix(native-chat): mirror this machine's chats only where it holds them

Round 1 ran the local chat mirror for everyone so existing chats show
whatever the setting says. That gave every desktop a permanent
session-tabs listener, which turns on the runtime's phone replication
paths, plus two full session-tab censuses at startup, and made the
browser client mirror its remote host a second time.

The runtime now says whether it holds structured chats: its structured
host is built only when saved chats were restored at startup or a client
created one here, and it announces the moment one is built. The mirror,
the startup restore and the continue-after-restart offer run only when
the setting launches chats or the host holds some, and never in the
browser client. A chat a paired client creates here with the setting off
still appears at once. The chat behaviour settings show wherever chats
exist, and the setting's copy says it picks what new agents open as. The
toggle-off teardown this made dead is removed.

* test(native-chat): record install listeners without a cast

* fix(native-chat): mirror this machine's chats only once it holds one, not once its host is built

Session history, resume preparation, terminal resume commands and replay-safe phone launches all
build the structured host for users who never had a chat, which turned on the chat mirror and the
structured-only settings rows until the next restart. The signal is now derived from the host's
records (or a records file still owed its import) and pushed when the first chat is restored or
created. A throwing listener no longer fails the install that fired it.

* feat(native-chat): createSupport reports the saved selection a new chat on this host starts with

A chat on a paired server starts with the server's saved model and options, which the desktop could
not read, so its picker showed a guess. createSupport's answer, which the desktop already waits for
before a paired launch, now also carries that seed as a new optional field (older clients ignore it).
Create and createSupport read it through one resolver so they cannot drift.

* refactor(protocol): move the Electron remote client capability list into its own module

Merging main left protocol-version.ts one line over the max-lines limit on this branch. The list of
capabilities the desktop advertises to a paired host moves, unchanged, into
electron-remote-runtime-client-capabilities.ts, the module the next PR in the stack already uses
for it; importers point there.

* test(cross-version): stub the launch seed resolver createSupport now reads

* test(protocol): pin the desktop capability divergence against what a paired server receives

Every paired transport sends the shared remote base plus the Electron list, so the
divergence test now compares that union with the renderer's local list instead of
the declared Electron list. A capability added only to the shared base can no
longer slip past it. The two base-only capabilities it surfaced are recorded:
skills.install-result.v2 has no local caller; the authoritative-inventory label is
read by the local tabs sync but dropped by main, and is marked unsettled.

The turn-item and both background-task-stop capabilities were already sent through
the shared base, so the Electron list no longer repeats them. The wire set is
unchanged; this PR's real change on the wire is structured.v1, the Claude
structured capability and the client launch-mode capability.

* fix(native-chat): the desktop tells its own host it picks each launch mode, so retrying an existing chat works with the setting off

* docs(native-chat): name the real exit for the released-phone createSupport rule

* test(cross-version): a released client still gets the host-setting createSupport answer; a launch-mode client gets supported plus the seed
Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
…tablyai#24781)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: Orca Worker <orca-worker@localhost>
…#24625)

* fix(mobile): release callbacks retained by canceled streams

* test(mobile): cover delayed frames after stream cancellation

Preserve late-ready cleanup while rejecting canceled scrollback and terminal routing; cover unsent cancellation across browser, client events and session tabs.

* test(mobile): model unavailable queued stream transport

The encrypted sender rejects writes until connection setup completes. Keep existing session-tab unsubscribe attempts and assert canceled queued openers are never replayed.

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: Neil <neil@stably.ai>
Co-authored-by: m4air <m4air@Mac.localdomain>
…blyai#24665)

* fix(mobile): reduce base64 allocations for encrypted text frames

* Correct screencast encoder comment after byte-codec extraction

---------

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: m4air <m4air@Mac.localdomain>
…tablyai#24670)

* fix(cursor): restore exact conversation after startup snapshot

* fix(terminal): preserve ready snapshot reattach and isolate bridge fixtures

* test(cursor): seed the real startup bridge after module resets

* test(terminal): settle startup snapshots in remote restore fixtures

Signed-off-by: Neil <neil@stably.ai>

---------

Signed-off-by: Neil <neil@stably.ai>
…tablyai#24707)

Co-authored-by: OrcaWin <293788423+OrcaWin@users.noreply.github.com>
Co-authored-by: m4air <m4air@Mac.localdomain>
…chat launch (stablyai#24564)

* fix(native-chat): remove the "still starting" notice that flashed on chat launch

Every structured chat passes through a starting phase, and the pane showed
"<agent> is still starting. Messages wait until it is ready; close this chat to
give up on it." for it. A 5 s grace period only narrowed the flash to starts
that finish just past it. A message sent while the agent starts already counts
as working, so the chat's working indicator and Stop cover that state; the
notice only repeated it.

Removes the notice, its copy in every locale, the delayed-status hook that
existed only for it, and the per-child key the chat read only to reset it.

* docs(agents): no pop-up notices for transient or internal states

Records the rule the removed startup notice broke, so agents building UI
show transient states through existing surfaces instead of new messages.

* docs(agents): allow the common delayed loading placeholder

The rule against delaying a flashing message should not forbid a quiet
skeleton or spinner that waits a moment before appearing.

* fix(native-chat): stop publishing which provider child is starting

hostExecutionChild existed only to re-key the removed starting notice's delay.
Older clients read it as optional, so omitting it only changes when their
notice appears after a still-starting child is replaced.

* docs(agents): narrow the status-notice rule's wording

Ban 'execution host' rather than ordinary host copy, scope 'confirming' to a
connection, and keep the no-delay rule where the common pattern delays.

* docs(agents): let a control's own busy state wait, per the style guide

The status-notice rule covers notices and status lines; a delayed in-place
label swap like "Saving…" stays as STYLEGUIDE.md describes.

* docs(agents): keep the status-message guideline out of the repo
@innocarpe
innocarpe force-pushed the mac-mini-c3-24791-codex-reconnect-failed-20261003 branch from aa2ef77 to a9ee501 Compare October 3, 2026 02:53
…tart, no "Not reported" caption (stablyai#24588)

* fix(native-chat): stop the loading flash, the doubled launch failure, and the "Not reported" caption

- A structured chat no longer paints "Start a chat" or the full-pane
  "Loading conversation..." text before its first read settles; the pane
  stays blank until the history (or its absence) is known. A chat this
  view just started keeps its empty state through publish.
- A failed chat start is said once, by the chat's own Retry line. The
  "Could not open <agent> chat" toast that fired beside it, and again on
  every failed Retry, is removed; the raw error still goes to the log.
- The Fast mode switch in the options menu no longer carries a
  "Not reported" / "Default" caption.

* fix(native-chat): show a quiet spinner if a chat's first read lasts

The blank pane stayed blank with no cue for as long as the first read
took, including a slow remote read or a pending resume. The pane now
holds a textless spinner (announced as "Loading chat") that a CSS
animation delay keeps invisible for the first 250 ms, so a quick read
still paints nothing. A cancelled launch no longer holds the pane in
that state; it shows what the chat shows when not launching.

* test(native-chat): a refused new-tab chat start reports in its chat, not a toast

The new-tab launch opens (or reuses) the chat tab before the start can
fail, so its Retry line is the one report. The guard tests now pin that
the chat tab exists and no toast fires, for both a refused start and one
whose outcome could not be confirmed.

* fix(native-chat): leave a failed resume's pane blank beside its Retry line

A resumed chat whose start failed or could not be confirmed kept the
loading spinner turning forever above "Chat could not be started.", though
nothing was reading its history. The spinner now shows only while a read or
the resuming launch is in flight.

Also log a chat tab that throws while opening during worktree creation, and
drop a comment that still described the removed launch-failure toast.
@innocarpe

Copy link
Copy Markdown
Owner Author

Sync update (a9ee5016de0)

Addressed the terminal reuse concern in a9ee5016. Reconnect detection resets its bounded output context when the active dispatch changes, so an earlier banner retained by the PTY cannot settle a later dispatch. Regression coverage exercises terminal reuse and a banner split across the dispatch boundary. A detected session failure marks the task failed pending coordinator review and prevents automatic replay while the original PTY remains alive.

brennanb2025 and others added 4 commits October 2, 2026 19:56
…lyai#24594)

* fix(native-chat): plain wording for chat errors and status rows

Replaces Orca-internal words (host, journal, transcript, outbox, process,
unverifiable, "no contact", byte budgets) in native chat refusals, status
rows, the skills menu, subagent and background-task state labels and the
history-load error with plain language, and routes the two hard-coded
English composer errors through translate(). Copy only; no behaviour change.

* fix(native-chat): match chat copy to what happens and to the sidebar's words

- The held-message row says Orca keeps checking, which it does: the idle
  sweep retries the unproven stop and a landed retry sends what waited.
- An unsettled earlier message reads as unconfirmed, not undelivered.
- The skills-unavailable line names SSH chats, its only cause.
- Subagent and background-task rows say "no recent update", the sidebar's
  words for the same state, and "status unavailable" after a count; the
  row no longer repeats the state as a reason.

* test(native-chat): find the repair row by its own text, not the old wording

* fix(native-chat): word the history-repair and too-large rows in the reader's language

Both rows were finished English the host wrote into the chat, so nothing could
translate them. Each now names itself with a presentation, the way the
compaction row does, and the chat says it through translate(). The English text
stays on the row for clients that predate these presentations and for the phone.

* fix(native-chat): say composer send errors with the chat's notice sentences

The composer's two errors had their own wording file beside the sentence table
every other chat notice uses. The send outcome now carries notice parts, worded
by the same function as the rest. A refused redelivery says "Orca couldn't
confirm your message reached the agent. Check the chat, then send it again if
needed." (the same sentence the failed-send rework uses), and a message this
client couldn't store says "Couldn't save your message. Try again."

* fix(native-chat): drop the retry line, keep one name for a lost task, and say only true causes

- The history error pane no longer adds "Orca keeps trying to load this chat."
  under its title: the read still retries on its own, but the pane says only
  that the chat didn't load.
- A write refused as unsupported asks for an Orca update only when no reason
  came back, which means the host is older. A named reason (a location or agent
  that can't run there, no chat host, a client missing the capability) now reads
  "This isn't available in this chat.", since updating doesn't fix it.
- A task Orca lost track of is "no recent update" everywhere; after a count it
  reads "2 agents with no recent update" instead of a second name.
- The skills menu announces the same sentence it shows, including in SSH chats.
- The row for messages held behind a previous agent reads "{{agent}} from before
  may still be running. Your messages will send once it stops."

* fix(native-chat): a chat whose host can't run it says its history didn't load

A history read refused as unsupported with a named reason left the error pane
saying only "This isn't available in this chat.", which never said the chat
failed to load and named nothing the reader asked for. A read now says "This
chat's history couldn't be loaded."; other writes keep the shorter sentence.
@innocarpe
innocarpe force-pushed the mac-mini-c3-24791-codex-reconnect-failed-20261003 branch from a9ee501 to fd0c6d8 Compare October 3, 2026 03:17
@innocarpe

Copy link
Copy Markdown
Owner Author

Sync update (fd0c6d83a69)

Addressed CodeRabbit's fallback concern in fd0c6d8: the PTY-to-handle fallback now receives ptyId, with regression coverage for a missing cached PTY index and a retained synthetic handle. The issue body records three workers' observed banner after a shared Codex app-server restart. This PR matches the exact reported banner in live onPtyData PTY chunks rather than rescanning retained preview text. I did not capture a new local Codex restart transcript, as stated in Testing. Decision on retained terminals: keep the failed worker terminal owned and unreleased for diagnosis. The task stays failed pending coordinator review to prevent automatic replay. A later --terminal dispatch may transfer it only by explicit opt-in; after transfer, the new dispatch owns the terminal and the failed dispatch no longer has release authority. This is intentional.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

8 participants