Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
85 changes: 85 additions & 0 deletions src/main/runtime/orchestration/__snapshots__/preamble.test.ts.snap
Original file line number Diff line number Diff line change
@@ -1,5 +1,90 @@
// Vitest Snapshot v1, https://vitest.dev/guide/snapshot.html

exports[`buildDispatchPreamble > renders a stable snapshot of a chat worker preamble 1`] = `
"You are working inside Orca, a multi-agent IDE. You are a dispatched worker.
Your coordinator's Orca session ID is: orca_session_id:COORD
Your task ID is: task_SNAP
Your Orca session ID is: orca_session_id:WORKER

The coordinator cannot see this chat, so reach it with the \`orca orchestration\`
commands below; a question or result left only in this chat never gets to it.
Don't post to Slack, GitHub, or other channels during the run; report through these commands.

=== CLI COMMANDS ===

\`\`\`sh
# Report the task outcome (required, exactly once).
#
# --body must be a 3-sentence executive summary (what you did,
# what you found, what's left). Never send an empty body; the coordinator
# reads the body first and only opens artifacts if it needs more detail.
# Append --files-modified only when files changed, and append --report-path
# only when you produced a durable report. Always pass real values; do not
# send the example placeholders literally.
#
# Send worker_done exactly once. Use --outcome succeeded when the
# requested work is done, or replace it with --outcome failed when it is not.
# Never encode failure only in prose and never silently exit.
# Include BOTH taskId and dispatchId in the payload so a late completion
# from a failed retry cannot complete the current dispatch.
orca orchestration send --from orca_session_id:WORKER --type worker_done --subject "<short status>" --body "<3-sentence summary: what you did, what you found, what's left>" --task-id task_SNAP --dispatch-id ctx_SNAP --outcome succeeded

# Send a heartbeat every 5 minutes
# while actively working on the task. The coordinator uses this to
# distinguish "still thinking" from "hung / crashed." Skip heartbeats only
# while blocked inside \`check --wait\` or \`ask\` — those calls are
# themselves liveness signals.
#
# Include BOTH taskId and dispatchId in the payload: the coordinator
# attributes the heartbeat to the specific dispatch context, not just
# the task, so a straggler heartbeat from a previously-failed dispatch
# cannot mask a hung retry.
orca orchestration send --from orca_session_id:WORKER --type heartbeat --subject "alive" --task-id task_SNAP --dispatch-id ctx_SNAP --phase "<short: investigating|implementing|reviewing|waiting>"

# Ask the coordinator a question and block until it answers.
#
# Use this instead of AskUserQuestion: that opens a local prompt the
# coordinator cannot see or answer, so the task would stall until someone
# happened to look at this chat. Send every question through \`ask\`.
#
# The \`ask\` verb durably records a question in this Dispatch's Run and
# blocks until the coordinator replies, then prints the reply body. If the
# call times out or disconnects, resume with the returned message ID instead
# of creating a duplicate question.
orca orchestration ask --from orca_session_id:WORKER --question "<your question>" --options "<optional,comma,separated>" --timeout-ms 600000

# Escalate a blocker or failure (pre-completion, when you need the
# coordinator to do something before you can continue):
orca orchestration send --from orca_session_id:WORKER --type escalation --subject "Blocked: <reason>" --body "<details>" --task-id task_SNAP --dispatch-id ctx_SNAP

# Read coordinator follow-ups. Nothing interrupts you: a durable message only
# arrives when you look, so run this at each natural checkpoint — before you
# start a new file and after a test run — and once more immediately before
# you send worker_done, so a redirect lands before the task settles.
orca orchestration check --terminal orca_session_id:WORKER --json
\`\`\`

=== AFTER YOU SEND worker_done ===

worker_done ends your turn for this task. Your dispatched work is complete:
stop, return to an idle prompt, and take no further actions — do NOT start
new or unrelated work, do NOT run a sleep/poll loop, and do NOT keep calling
\`orca orchestration check\`. The coordinator has already recorded your
completion and expects no further output.

A direct instruction from the user takes precedence over this idle rule.
Treat it as new user-owned work: follow it without coordinator approval or a
fresh Dispatch, and do not send lifecycle messages using the settled task or
Dispatch IDs. Never refuse a direct user request because you were a worker.

If the coordinator has more for you, it will send this chat a fresh
preamble + TASK block, which arrives as a new message. Treat that as supervised
work under the new Dispatch; ignore stale follow-ups from the settled task.

=== TASK ===
TASK_BODY"
`;

exports[`buildDispatchPreamble > renders a stable snapshot of the full preamble 1`] = `
"You are working inside Orca, a multi-agent IDE. You are a dispatched worker.
Your coordinator's terminal handle is: term_COORD
Expand Down
28 changes: 22 additions & 6 deletions src/main/runtime/orchestration/agent-facing-parity.test.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
/**
* A chat agent and a terminal agent run the same orchestration process: the same preamble, the
* same pointer text and the same guide. The one difference is how each is named: a terminal by its
* handle, exactly as on main, and a session by its Orca session ID. Orca absorbs everything else.
* same pointer text and the same guide. They differ in how each is named (a terminal by its handle,
* exactly as on main, a session by its Orca session ID) and in the preamble calling a chat a chat.
* Orca absorbs everything else.
*/
import { readdirSync, readFileSync } from 'node:fs'
import { join } from 'node:path'
Expand All @@ -23,7 +24,11 @@ import {
import { OrchestrationDb } from './db'
import { formatMessagePointer } from './formatter'
import { OrchestrationStructuredMailboxPointerDelivery } from './structured-mailbox-pointer-delivery'
import { buildDispatchPreamble } from './preamble'
import {
buildDispatchPreamble,
CHAT_REDISPATCH_PARAGRAPH as CHAT_REDISPATCH,
TERMINAL_REDISPATCH_PARAGRAPH as TERMINAL_REDISPATCH
} from './preamble'
import { ORCA_SESSION_ID_AS_ADDRESS } from '../../../shared/orca-session-id-wording-test-fixture'

const sent = vi.hoisted((): { preambles: string[] } => ({ preambles: [] }))
Expand Down Expand Up @@ -142,15 +147,26 @@ async function renderChatPointer(mailbox: string): Promise<string> {
return texts[0]!
}

describe('a chat agent and a terminal agent see the same text but for how each is named', () => {
it("teaches a chat worker a terminal worker's preamble but for its identity lines", async () => {
describe('a chat agent and a terminal agent see the same text but for how each is named and called', () => {
it("teaches a chat worker a terminal worker's preamble but for its identity lines and the chat wording", async () => {
const chat = await renderPreamble('chat')
const terminal = await renderPreamble('terminal')
const withoutRedispatch = chat.replace(CHAT_REDISPATCH, TERMINAL_REDISPATCH)

expect(terminal).not.toContain('Orca session ID')
expect(chat).toContain(`Your task ID is: task_1${SELF_LINE}\n`)
expect(chat).not.toContain(CHAT_WORKER_HANDLE)
expect(chat.replace(SELF_LINE, '').split(CHAT_ADDRESS).join(TERMINAL_HANDLE)).toBe(terminal)
expect(chat).not.toContain('this terminal')
expect(withoutRedispatch).not.toBe(chat)
expect(withoutRedispatch.split('this chat')).toHaveLength(4)
expect(
withoutRedispatch
.replace(SELF_LINE, '')
.split(CHAT_ADDRESS)
.join(TERMINAL_HANDLE)
.split('this chat')
.join('this terminal')
).toBe(terminal)
})

it.each([
Expand Down
42 changes: 38 additions & 4 deletions src/main/runtime/orchestration/preamble.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,11 @@ import { spawnSync } from 'node:child_process'
import remarkParse from 'remark-parse'
import { unified } from 'unified'
import { describe, expect, it } from 'vitest'
import { buildDispatchPreamble } from './preamble'
import {
buildDispatchPreamble,
CHAT_REDISPATCH_PARAGRAPH as CHAT_REDISPATCH,
TERMINAL_REDISPATCH_PARAGRAPH as TERMINAL_REDISPATCH
} from './preamble'

function baseParams(overrides: Partial<Parameters<typeof buildDispatchPreamble>[0]> = {}) {
return {
Expand Down Expand Up @@ -349,6 +353,17 @@ describe('buildDispatchPreamble', () => {
})
expect(result).toMatchSnapshot()
})

it('renders a stable snapshot of a chat worker preamble', () => {
const result = buildDispatchPreamble({
taskId: 'task_SNAP',
dispatchId: 'ctx_SNAP',
taskSpec: 'TASK_BODY',
coordinatorHandle: 'orca_session_id:COORD',
workerHandle: 'orca_session_id:WORKER'
})
expect(result).toMatchSnapshot()
})
})

describe('sub-dispatch section', () => {
Expand Down Expand Up @@ -417,15 +432,34 @@ describe('how the preamble names the worker and its coordinator, by kind', () =>
)
})

it("teaches a session worker a terminal worker's text but for how it is named", () => {
it("teaches a chat worker a terminal worker's text but for its name and the chat wording", () => {
const terminal = buildDispatchPreamble(baseParams())
const session = buildDispatchPreamble(baseParams({ workerHandle: SESSION_WORKER }))
const chat = buildDispatchPreamble(baseParams({ workerHandle: SESSION_WORKER }))
const withoutRedispatch = chat.replace(CHAT_REDISPATCH, TERMINAL_REDISPATCH)

expect(withoutRedispatch).not.toBe(chat)
expect(withoutRedispatch.split('this chat')).toHaveLength(4)
expect(
session
withoutRedispatch
.replace(`\nYour Orca session ID is: ${SESSION_WORKER}`, '')
.split(SESSION_WORKER)
.join('term_worker')
.split('this chat')
.join('this terminal')
).toBe(terminal)
})

it('tells a chat worker nothing about a terminal or a shell, even as a bare-shell worker', () => {
for (const workerKind of ['prompt-returning-agent', 'bare-shell'] as const) {
const chat = buildDispatchPreamble(baseParams({ workerHandle: SESSION_WORKER, workerKind }))

expect(chat).not.toContain('this terminal')
expect(chat).not.toContain('exit the shell')
expect(chat).not.toContain('Exit the shell')
expect(chat).not.toContain('Your terminal')
expect(chat).toContain('return to an idle prompt')
expect(chat).toContain(`check --terminal ${SESSION_WORKER}`)
expect(afterWorkerDoneSection(chat).trimEnd().endsWith(CHAT_REDISPATCH)).toBe(true)
}
})
})
46 changes: 32 additions & 14 deletions src/main/runtime/orchestration/preamble.ts
Original file line number Diff line number Diff line change
Expand Up @@ -44,15 +44,21 @@ export type PreambleParams = {
// cadence tuning is a single-line change (Q1 in DESIGN_DOC_PREAMBLE_FIX.md).
const HEARTBEAT_INTERVAL_MIN = 5

/** A session address names a chat worker; any other handle is a terminal. */
function isSessionHandle(handle: string): boolean {
return handle.startsWith(ORCA_SESSION_ADDRESS_PREFIX)
}

/** Terminal agents keep their handle's wording; only a session is named by its Orca session ID. */
function dispatchIdentityLines(params: PreambleParams): string {
const isSession = (handle: string) => handle.startsWith(ORCA_SESSION_ADDRESS_PREFIX)
return [
isSession(params.coordinatorHandle)
isSessionHandle(params.coordinatorHandle)
? `Your coordinator's Orca session ID is: ${params.coordinatorHandle}`
: `Your coordinator's terminal handle is: ${params.coordinatorHandle}`,
`Your task ID is: ${params.taskId}`,
...(isSession(params.workerHandle) ? [`Your Orca session ID is: ${params.workerHandle}`] : [])
...(isSessionHandle(params.workerHandle)
? [`Your Orca session ID is: ${params.workerHandle}`]
: [])
].join('\n')
}

Expand All @@ -66,9 +72,12 @@ export function buildDispatchPreamble(params: PreambleParams): string {
// socket. Without this, agents inside the dev Electron app would call the
// production CLI and talk to the wrong Orca instance (Section 6.4).
const cli = params.devMode ? 'orca-dev' : (params.cliCommand ?? 'orca')
const chat = isSessionHandle(params.workerHandle)
const surface = chat ? 'chat' : 'terminal'
const postDoneInstructions = buildPostWorkerDoneInstructions({
cli,
workerKind: params.workerKind ?? 'prompt-returning-agent'
workerKind: params.workerKind ?? 'prompt-returning-agent',
chat
})

// Why: one-line recipes paste unchanged in POSIX shells, PowerShell, and cmd.exe.
Expand All @@ -78,8 +87,8 @@ export function buildDispatchPreamble(params: PreambleParams): string {
const header = `You are working inside Orca, a multi-agent IDE. You are a dispatched worker.
${dispatchIdentityLines(params)}

The coordinator cannot see this terminal, so reach it with the \`${cli} orchestration\`
commands below; a question or result left only in this terminal never gets to it.
The coordinator cannot see this ${surface}, so reach it with the \`${cli} orchestration\`
commands below; a question or result left only in this ${surface} never gets to it.
Don't post to Slack, GitHub, or other channels during the run; report through these commands.

=== CLI COMMANDS ===
Expand Down Expand Up @@ -117,7 +126,7 @@ Don't post to Slack, GitHub, or other channels during the run; report through th
#
# Use this instead of AskUserQuestion: that opens a local prompt the
# coordinator cannot see or answer, so the task would stall until someone
# happened to look at this terminal. Send every question through \`ask\`.
# happened to look at this ${surface}. Send every question through \`ask\`.
#
# The \`ask\` verb durably records a question in this Dispatch's Run and
# blocks until the coordinator replies, then prints the reply body. If the
Expand Down Expand Up @@ -170,16 +179,28 @@ export function dispatchPreambleSendOptions(requestId: string): DispatchPreamble
}
}

export const TERMINAL_REDISPATCH_PARAGRAPH = `Do not exit the shell. Your terminal stays available, and if the
coordinator has more for you it will re-engage this terminal with a fresh
preamble + TASK block, which arrives as new input. Treat that as supervised
work under the new Dispatch; ignore stale follow-ups from the settled task.`

export const CHAT_REDISPATCH_PARAGRAPH = `If the coordinator has more for you, it will send this chat a fresh
preamble + TASK block, which arrives as a new message. Treat that as supervised
work under the new Dispatch; ignore stale follow-ups from the settled task.`

function buildPostWorkerDoneInstructions({
cli,
workerKind
workerKind,
chat
}: {
cli: string
workerKind: NonNullable<PreambleParams['workerKind']>
chat: boolean
}): string {
// Why: re-dispatch reaches idle agents as terminal input; inbox polling
// Why: re-dispatch reaches idle agents as new input; inbox polling
// after completion cannot receive that new TASK block and looks hung.
if (workerKind === 'bare-shell') {
// Why chat first: a chat has no shell to exit, whatever workerKind says.
if (workerKind === 'bare-shell' && !chat) {
return `=== AFTER YOU SEND worker_done ===

worker_done ends your turn for this task. Your dispatched work is complete:
Expand All @@ -206,10 +227,7 @@ Treat it as new user-owned work: follow it without coordinator approval or a
fresh Dispatch, and do not send lifecycle messages using the settled task or
Dispatch IDs. Never refuse a direct user request because you were a worker.

Do not exit the shell. Your terminal stays available, and if the
coordinator has more for you it will re-engage this terminal with a fresh
preamble + TASK block, which arrives as new input. Treat that as supervised
work under the new Dispatch; ignore stale follow-ups from the settled task.`
${chat ? CHAT_REDISPATCH_PARAGRAPH : TERMINAL_REDISPATCH_PARAGRAPH}`
}

// Why the whole section is omitted rather than softened when nesting is off: a
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@
*
* Two properties are pinned here, because both were false at some point in this lane:
*
* - a worker is TAUGHT the same thing whichever mode it runs in, byte for byte once how it is named
* and the dispatch id are normalised. The sub-dispatch section used to be withheld from a structured
* - a worker is TAUGHT the same thing whichever mode it runs in, byte for byte once how it is named,
* where it runs (chat or terminal) and the dispatch id are normalised. The sub-dispatch section used to be withheld from a structured
* worker, which is a two-tier capability model dressed as a preamble tweak;
* - a structured worker can actually BE a coordinator. `worker-start` used to resolve `--from`
* through `showTerminal`, which needs a PTY, so the capability the preamble withheld was in fact
Expand All @@ -19,6 +19,10 @@ import {
structuredWorkerIdentities,
structuredWorkerProcessIncarnation
} from '../../structured-worker-identity'
import {
CHAT_REDISPATCH_PARAGRAPH,
TERMINAL_REDISPATCH_PARAGRAPH
} from '../../orchestration/preamble'
import { ORCHESTRATION_METHODS } from './orchestration'
import { readStructuredWorkerOutput } from './orchestration-structured-worker-lifecycle'
import { inspectWorkerTerminal } from './orchestration/worker/worker-observation'
Expand Down Expand Up @@ -107,6 +111,11 @@ function normalizePreamble(preamble: string, handle: string, dispatchId: string)
.join('<dispatch>')
.replace(/dcap_[\w-]+/g, '<capability>')
.replace(/task_[0-9a-f]+/g, '<task>')
.split(CHAT_REDISPATCH_PARAGRAPH)
.join('<redispatch>')
.split(TERMINAL_REDISPATCH_PARAGRAPH)
.join('<redispatch>')
.replace(/\bthis (chat|terminal)\b/g, 'this <surface>')
}

describe('a worker cannot tell which mode it is running in', () => {
Expand Down Expand Up @@ -224,6 +233,10 @@ describe('a worker cannot tell which mode it is running in', () => {
expect(structuredPreamble).toContain(`${selfLine}\n`)
expect(structuredPreamble).not.toContain(STRUCTURED_HANDLE)
expect(terminalPreamble).not.toContain('Orca session ID')
// Only the surface words differ: a chat is told about its chat, a terminal keeps main's text.
expect(structuredPreamble).toContain(CHAT_REDISPATCH_PARAGRAPH)
expect(terminalPreamble).toContain(TERMINAL_REDISPATCH_PARAGRAPH)
expect(structuredPreamble).not.toContain('this terminal')
// The section the structured lane used to withhold, asserted by name so the equality above
// cannot pass by both preambles losing it.
expect(structuredPreamble).toContain('=== SUB-DISPATCH ===')
Expand Down
Loading
Loading