Repository navigation
fix(cli): warn that fleet release does not tear down Cloud sandboxes #1978
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
NilotpalK
wants to merge
1
commit into
AgentWorkforce:main
Choose a base branch
from
NilotpalK:fix/1922-fleet-release-warning
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
37 changes: 37 additions & 0 deletions
37
.agentworkforce/trajectories/issue-1922/compacted/compact_ridr6adoo0go_2026-10-15.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,37 @@ | ||
| { | ||
| "id": "compact_ridr6adoo0go", | ||
| "version": 1, | ||
| "type": "compacted", | ||
| "compactedAt": "2026-10-15T19:57:17.694Z", | ||
| "sourceTrajectories": [ | ||
| "traj_fesap23qytsg" | ||
| ], | ||
| "dateRange": { | ||
| "start": "2026-10-15T19:49:55.679Z", | ||
| "end": "2026-10-15T19:56:51.350Z" | ||
| }, | ||
| "summary": { | ||
| "totalDecisions": 1, | ||
| "totalEvents": 2, | ||
| "uniqueAgents": [ | ||
| "default" | ||
| ] | ||
| }, | ||
| "decisionGroups": [ | ||
| { | ||
| "category": "api", | ||
| "decisions": [ | ||
| { | ||
| "question": "Warn once after a dispatched release acknowledgement, before optional identity cleanup", | ||
| "chosen": "Warn once after a dispatched release acknowledgement, before optional identity cleanup", | ||
| "reasoning": "The release endpoint acknowledges worker dispatch only. Emitting through the existing stderr warning sink preserves the single JSON stdout document and covers delete-agent variants without invoking Cloud/provider deletion.", | ||
| "fromTrajectory": "traj_fesap23qytsg" | ||
| } | ||
| ] | ||
| } | ||
| ], | ||
| "keyLearnings": [], | ||
| "keyFindings": [], | ||
| "filesAffected": [], | ||
| "commits": [] | ||
| } |
18 changes: 18 additions & 0 deletions
18
...tworkforce/trajectories/issue-1922/compacted/compact_ridr6adoo0go_2026-10-15.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,18 @@ | ||
| # Trajectory Compaction: Oct 16, 2026 - Oct 16, 2026 | ||
|
|
||
| ## Summary | ||
| - Sessions: 1 | ||
| - Decisions: 1 | ||
| - Events: 2 | ||
| - Agents: default | ||
| - Files: 0 | ||
| - Commits: 0 | ||
|
|
||
| ## Api | ||
| - Warn once after a dispatched release acknowledgement, before optional identity cleanup -> Warn once after a dispatched release acknowledgement, before optional identity cleanup (traj_fesap23qytsg) | ||
|
|
||
| ## Key Learnings | ||
| - None | ||
|
|
||
| ## Key Findings | ||
| - None |
5 changes: 5 additions & 0 deletions
5
.agentworkforce/trajectories/issue-1922/completed/2026-10/traj_fesap23qytsg/compaction.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,5 @@ | ||
| { | ||
| "trajectoryId": "traj_fesap23qytsg", | ||
| "compactedInto": "compact_ridr6adoo0go", | ||
| "compactedAt": "2026-10-15T19:57:17.709Z" | ||
| } |
33 changes: 33 additions & 0 deletions
33
...orkforce/trajectories/issue-1922/completed/2026-10/traj_fesap23qytsg/summary.md
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| # Trajectory: Warn when fleet release leaves Cloud sandboxes running | ||
|
|
||
| > **Status:** ✅ Completed | ||
| > **Task:** 1922 | ||
| > **Confidence:** 90% | ||
| > **Started:** October 16, 2026 at 01:19 AM | ||
| > **Completed:** October 16, 2026 at 01:26 AM | ||
|
|
||
| --- | ||
|
|
||
| ## Summary | ||
|
|
||
| Added the dispatched-release sandbox cost warning without changing JSON stdout or requesting provider teardown. Added seven regressions, one RelayFlow bugfix proof, and a patch changelog entry. Independent review found no actionable issues. Full Windows suite limitations are documented for maintainer validation. | ||
|
|
||
| **Approach:** Standard approach | ||
|
|
||
| --- | ||
|
|
||
| ## Key Decisions | ||
|
|
||
| ### Warn once after a dispatched release acknowledgement, before optional identity cleanup | ||
| - **Chose:** Warn once after a dispatched release acknowledgement, before optional identity cleanup | ||
| - **Reasoning:** The release endpoint acknowledges worker dispatch only. Emitting through the existing stderr warning sink preserves the single JSON stdout document and covers delete-agent variants without invoking Cloud/provider deletion. | ||
|
|
||
| --- | ||
|
|
||
| ## Chapters | ||
|
|
||
| ### 1. Work | ||
| *Agent: default* | ||
|
|
||
| - Warn once after a dispatched release acknowledgement, before optional identity cleanup: Warn once after a dispatched release acknowledgement, before optional identity cleanup | ||
| - The seven new regressions and all 18 release tests pass. Both proof observations match the expected bug/fixed behavior. Typecheck and formatting pass; full Windows validation has 342 failures and a worker OOM, and the fleet Unix-path mismatch is confirmed on unchanged main. |
69 changes: 69 additions & 0 deletions
69
.agentworkforce/trajectories/issue-1922/completed/2026-10/traj_fesap23qytsg/trajectory.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,69 @@ | ||
| { | ||
| "id": "traj_fesap23qytsg", | ||
| "version": 1, | ||
| "task": { | ||
| "title": "Warn when fleet release leaves Cloud sandboxes running", | ||
| "source": { | ||
| "system": "plain", | ||
| "id": "1922" | ||
| } | ||
| }, | ||
| "status": "completed", | ||
| "startedAt": "2026-10-15T19:49:55.679Z", | ||
| "completedAt": "2026-10-15T19:56:51.350Z", | ||
| "agents": [ | ||
| { | ||
| "name": "default", | ||
| "role": "lead", | ||
| "joinedAt": "2026-10-15T19:51:26.854Z" | ||
| } | ||
| ], | ||
| "chapters": [ | ||
| { | ||
| "id": "chap_w5couri5corw", | ||
| "title": "Work", | ||
| "agentName": "default", | ||
| "startedAt": "2026-10-15T19:51:26.854Z", | ||
| "endedAt": "2026-10-15T19:56:51.350Z", | ||
| "events": [ | ||
| { | ||
| "ts": 1792093886855, | ||
| "type": "decision", | ||
| "content": "Warn once after a dispatched release acknowledgement, before optional identity cleanup: Warn once after a dispatched release acknowledgement, before optional identity cleanup", | ||
| "raw": { | ||
| "question": "Warn once after a dispatched release acknowledgement, before optional identity cleanup", | ||
| "chosen": "Warn once after a dispatched release acknowledgement, before optional identity cleanup", | ||
| "alternatives": [], | ||
| "reasoning": "The release endpoint acknowledges worker dispatch only. Emitting through the existing stderr warning sink preserves the single JSON stdout document and covers delete-agent variants without invoking Cloud/provider deletion." | ||
| }, | ||
| "significance": "high" | ||
| }, | ||
| { | ||
| "ts": 1792094210071, | ||
| "type": "reflection", | ||
| "content": "The seven new regressions and all 18 release tests pass. Both proof observations match the expected bug/fixed behavior. Typecheck and formatting pass; full Windows validation has 342 failures and a worker OOM, and the fleet Unix-path mismatch is confirmed on unchanged main.", | ||
| "raw": { | ||
| "confidence": 0.9 | ||
| }, | ||
| "significance": "high", | ||
| "tags": [ | ||
| "confidence:0.9" | ||
| ] | ||
| } | ||
| ] | ||
| } | ||
| ], | ||
| "retrospective": { | ||
| "summary": "Added the dispatched-release sandbox cost warning without changing JSON stdout or requesting provider teardown. Added seven regressions, one RelayFlow bugfix proof, and a patch changelog entry. Independent review found no actionable issues. Full Windows suite limitations are documented for maintainer validation.", | ||
| "approach": "Standard approach", | ||
| "confidence": 0.9 | ||
| }, | ||
| "commits": [], | ||
| "filesChanged": [], | ||
| "projectId": "AgentWorkforce/relay", | ||
| "tags": [], | ||
| "_trace": { | ||
| "startRef": "c1ffabbc4a933585ca2ddda729b22a34e3b2a3f2", | ||
| "endRef": "c1ffabbc4a933585ca2ddda729b22a34e3b2a3f2" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
21 changes: 21 additions & 0 deletions
21
tests/relayflows/cases/1922-fleet-release-sandbox-warning/case.json
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,21 @@ | ||
| { | ||
| "version": 1, | ||
| "id": "1922-fleet-release-sandbox-warning", | ||
| "kind": "bugfix", | ||
| "title": "Warn that fleet release does not tear down Cloud sandboxes", | ||
| "runner": { | ||
| "command": ["node", "tests/relayflows/cases/1922-fleet-release-sandbox-warning/run.mjs"] | ||
| }, | ||
| "requirements": [], | ||
| "timeoutSeconds": 900, | ||
| "expected": { | ||
| "base": { | ||
| "outcome": "bug", | ||
| "signature": "fleet_release_missing_sandbox_warning" | ||
| }, | ||
| "head": { | ||
| "outcome": "fixed", | ||
| "signature": "fleet_release_warns_sandbox_may_keep_running" | ||
| } | ||
| } | ||
| } |
62 changes: 62 additions & 0 deletions
62
tests/relayflows/cases/1922-fleet-release-sandbox-warning/probe.test.mts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,62 @@ | ||
| import fs from 'node:fs'; | ||
|
|
||
| import { Command } from 'commander'; | ||
| import { expect, test, vi } from 'vitest'; | ||
|
|
||
| // The runner copies this probe into <target>/.relay-pr-proof/. | ||
| import { registerFleetCommands } from '../packages/cli/src/cli/commands/fleet.js'; | ||
|
|
||
| test('a dispatched fleet release keeps JSON stdout and reports sandbox costs on stderr', async () => { | ||
| const observationPath = process.env.RELAY_PR1922_OBSERVATION_PATH; | ||
| if (!observationPath) throw new Error('Missing RELAY_PR1922_OBSERVATION_PATH.'); | ||
|
|
||
| const stdout: string[] = []; | ||
| const stderr: string[] = []; | ||
| const errors: string[] = []; | ||
| const receipt = { | ||
| invocationId: 'inv_release_proof', | ||
| actionName: 'agent.release', | ||
| status: 'dispatched', | ||
| }; | ||
| const release = vi.fn(async () => receipt); | ||
| const deleteSandbox = vi.fn(); | ||
| const program = new Command(); | ||
| program.exitOverride(); | ||
| // Exercise the production default log/warn sinks; only the remote lifecycle API is stubbed. | ||
| const log = vi.spyOn(console, 'log').mockImplementation((message) => stdout.push(String(message))); | ||
| const warn = vi.spyOn(console, 'warn').mockImplementation((message) => stderr.push(String(message))); | ||
| const error = vi.spyOn(console, 'error').mockImplementation((message) => errors.push(String(message))); | ||
|
|
||
| try { | ||
| registerFleetCommands(program, { | ||
| createFleetWorkspaceClient: (() => ({ agents: { release } })) as never, | ||
| deleteCloudFleetSandbox: deleteSandbox as never, | ||
| }); | ||
| await program.parseAsync(['fleet', 'release', 'proof-worker', '--workspace-key', 'rk_proof'], { | ||
| from: 'user', | ||
| }); | ||
|
|
||
| expect(errors).toEqual([]); | ||
| expect(stdout).toHaveLength(1); | ||
| expect(JSON.parse(stdout[0]!)).toEqual(receipt); | ||
| expect(release).toHaveBeenCalledExactlyOnceWith({ | ||
| name: 'proof-worker', | ||
| reason: expect.stringMatching(/^fleet agent released \(actor: .+\)$/), | ||
| deleteAgent: false, | ||
| }); | ||
| expect(deleteSandbox).not.toHaveBeenCalled(); | ||
|
|
||
| const missingWarning = stderr.length === 0; | ||
| const hasWarning = | ||
| stderr.length === 1 && | ||
| stderr[0]!.includes('Release was dispatched') && | ||
| stderr[0]!.includes('Cloud/provider sandbox may keep running and incurring cost') && | ||
| stderr[0]!.includes('`fleet release` does not tear it down'); | ||
| expect(missingWarning || hasWarning, JSON.stringify(stderr)).toBe(true); | ||
| fs.writeFileSync(observationPath, JSON.stringify({ missingWarning, hasWarning })); | ||
| } finally { | ||
| log.mockRestore(); | ||
| warn.mockRestore(); | ||
| error.mockRestore(); | ||
| } | ||
| }); |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟡 Completed releases omit sandbox cost warning
When
agents.releasereturnscompleted, the CLI omits the sandbox warning despite a successful release. Sandbox-backed workers can keep incurring cost without notice; apendingrelease dispatched later has the same gap.Learn more
The workspace lifecycle client passes through release receipts from the server without converting their status. A release can return
completedimmediately, as shown by the lifecycle test;pendingwork can also dispatch after this command returns. Neither state triggers the warning, although the release command never calls sandbox deletion.Example: A sandbox-backed worker receives
{status: 'completed'}from the release API. The command prints its receipt and exits without mentioning that the Cloud sandbox can remain running.Recommended fix: Warn on successful release acknowledgements without requiring the transient
dispatchedstate. For pending receipts, use wording that does not claim dispatch has already occurred; retain the one-document stdout contract.Was this helpful? React with 👍 or 👎 to provide feedback.