Repository navigation
fix: dispatch's second review — no sandbox without an id, no turn deciding nothing, a way out of a clean-up that cannot finish - #63
Merged
Conversation
…nnot finish never leave a run deciding nothing The second review of #61 and #62 (findings 1 to 12) and the real-Boat run at 9b3ff2e: - A `boat new` that fails or is cut off without an id: what it printed is kept, the sandbox is looked up by name, and the run stays unsettled until the id is known or Boat's whole list says there is none (a page with hasMore does not). The watcher waits longer than `new` may take. - A turn Boat no longer knows is decided by the runner's `read`; one that cannot be followed for 10 minutes makes the run stuck. A failed turn waits for a fresh pull. - machine_not_running during clean-up is nothing to remove; a credential that cannot be removed is recorded and makes `stop` exit 1; `stop --forget` is the owner's way out. - A turn that started while the run was stopped is ended and the sandbox stopped again. - The lease outlasts a clean-up, the end entry waits for the stop, the watcher leaves only when the project is unregistered, a live run from an older kontext is ended, and `status` shows the watcher line while any run is not settled. - The start script installs gh stack; a failed start script's reason is one line. - The fake boat answers as real Boat did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…st reach, and what --forget does Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The second review of #61 and #62 found 12 problems (head 9b3ff2e). Two of them leave a sandbox billing. This PR fixes them, plus what the real-Boat run at 9b3ff2e showed. It stacks on #62; the owner merges.
What changes
Sandboxes that kept billing
boat newwithout an id (finding 1).newis asked, that a sandbox may exist without an id.stopexits non-zero, and the watcher stays.newcut off by its time limit keeps what it printed, so itscreatedline still gives the id.boat listpage that sayshasMoreand lacks the name counts as "not known yet".boat list --helphas no cursor option. An answer of another shape counts the same way.newmay take: 16 minutes instead of 2.read <n>now decides it. Anexitfile gives the verdict. Nopidfile means the turn never ran, which is stuck. A pid that still runs (the runner now saysalive=) is followed.Smaller fixes
boat execruns into a sandbox the state records as stopped") claims more. I did not edit feat: the dispatch coordinator runs as Claude Code through boat exec #61. The accurate wording is the line above, and the DESIGN text.machine_not_runningfrom anexecduring clean-up means nothing is left to kill or remove there. It is reported, not counted as a failure, and the stop goes on.kontext dispatch stop <epic> --forget, the owner's way out.forgotten, which counts as clean.rmfails for another reason, the run recordscredential_left.stopexits 1 and says to replace the credential (claude setup-token, thenkontext dispatch credential set).statusshows it, and the stuck question does not suggestboat resume./dev/shm. What keeps the credential out of a stopped sandbox is the removal before kontext stops it.kontext clonekeeps the (revoked) kontext token in the sandbox's kontext config.id(written by an older kontext) gets its sandbox stopped, and the run is stuck with "dispatched by an older kontext".statusshows the "watcher is not running" line whenever some run is not settled. The overdue refusal names the command that starts it.project.tomlthat cannot be read is logged and tried again. An error fromdone()no longer ends it either.after()skips a run that holds a lease. The end entry ("its sandbox stopped") is written only once the run is clean, bystopand by the watcher alike.Failedturn seen on a failed pull waits for a fresh pull, asEndeddoes. "Every failure is final" stays, and DESIGN says so.credential setrefuses a bare prefix and NUL or control bytes.stop's message is gone.--jsonfailure before the reservation shows"run": nullinstead of an older run.dispatchcan hold the foreground for about 20 minutes, and that the watcher finishes a killed one.From the real run
kontext-dispatch:line, else the last non-empty line. The whole text, redacted, goes in the watcher's log or the--jsonoutput'sdetail.boat history <id>and~/.kontext-dispatch/turn-<n>/.boat eventsprinted nothing in the real run.gh stackwhengh extension listlacks it. A failure is reported on stderr and is not fatal.[[repos]]remote is cloned as written.Found while fixing: a stop's clean-up could clear a pending mark that the watcher's
give_uphad just set again. Fixed with alookedmark (the clean-up clears only the mark it looked for) and a fresh mark ingive_up. No test; checked by reading.The fake
boatnow answers as real Boat did in the facts file:execon a stopped sandbox exits 1 withmachine_not_running;stopon a stopped sandbox exits 0;--killof an exited process exits 0 withsignaled:false;list --jsonis one{"pageInfo","sandboxes"}document.New steering files:
fail-new-silent,list-more,gone-<id>-<n>,hang-detach. The fake'sinfonow JSON-escapes a multi-linesetupError.Tests
a_credential_that_cannot_be_removed_still_lets_the_sandbox_stopis now…_stops_the_sandbox_and_says_to_replace_it, and expects exit 1.a_turn_that_cannot_be_followed_decides_nothingis nowa_turn_boat_forgot_is_read_from_the_sandbox_and_an_unfollowed_one_ends.a_failed_turn_is_stuck_at_once_with_its_reasonalso checks the stuck question's hint.gh stackinstall.Test plan
cargo fmt --checkcargo clippy --all-targets -- -D warnings(fresh target dir)KONTEXT_REQUIRE_SERVER_TESTS=1 cargo test, full suite, fresh target dir: 544 passed, 0 failedtests/dispatch.rsandtests/fixtures/fake-boat.shrun against 9b3ff2e's code (scratch worktree, own target dir): the 17 new or changed tests fail and the other 29 pass. Each fails on its first assertion that the old code breaks, so a later assertion in the same test was not red-checked: the spaces instop's message (behind finding 10's assertion), finding 3's "stuck after 10 minutes" and "turn never ran" (behind its first half; the old suite'sa_turn_that_cannot_be_followed_decides_nothingpinned the opposite), and the forget test's exit-2 refusal of a live run (old clap exits 2 on the unknown flag; a later assertion fails). The new unit tests (lease length,list_lookup, credential checks,failure_line, the new grace) use symbols 9b3ff2e does not have, so they were not run there.Run against real Boat at 9b3ff2e, before this PR's changes, by the session that ordered it
git clone403: the sandbox's GitHub login had no access to a new private repo):dispatchexit 1, credential removed, sandbox stopped, token revoked,tmp/empty, watcher gone.gh api user; "coordinator started … turn 1", exit 0; the coordinator did both tasks, pushed a branch and closed them in about 90 s; the watcher saw the epic closed and ranexec --kill,exec … rm -f <credential paths>,stop --yes, token revoked. Runclosed.tmp/empty; the watcher took the start over, waited for the start script and started turn 1.--resume, same session) ended → stuck, with a question to the pm, credential removed, sandbox stopped, token revoked.boat new: the sandbox existed anyway; the watcher found it by name 5 s later and finished the start.dispatch stopduring turn 1:exec --kill <pid>,rm,stop, token revoked; exit 0; runstopped.kxt1.orsk-ant-in the argument log,boat history,boat events, the state folder, the dispatch output or the ledger; in the sandbox the credential was only in/dev/shm/kontext-dispatch-agent-credential(0600).state.jsonin docs: kontext dispatch in DESIGN, the README and the pm's skills #58's format (one closed run) is read bydispatch status.dispatch credential rmremoves the file.Run against real Boat at 57dffcb (this PR's head), one sandbox, by the session that ordered it
dispatch status.dispatchof an epic stopped earlier, with a credential that is not valid:boat newthrough the new create path gives the id; the start script finishes; turn 1 fails;dispatchexits 1 with one line ("the coordinator did not start: Claude Code said: Failed to authenticate. API Error: 401 …"); the credential is removed, the sandbox stopped, the token revoked; the run isstuckwith that reason.gh stackin the sandbox: after a resume,gh extension listsaysgh stack github/gh-stack v0.3.0(so the script'sgrep gh-stackmatches) andgh stack --helpworks./dev/shmor under$HOME.Last run against real Boat at 57dffcb, three sandboxes, a real credential and a private GitHub repo
gh stack; the coordinator opened two draft pull requests, the second based on the first, in one GitHub stack, and closed the epic; the watcher killed the turn, removed the credential, stopped the sandbox and revoked the token. Checked from outside the sandbox withgh.boat historyhint.boat stopwhile turn 1 ran:dispatchgotmachine_not_running, warned that the credential stays in that sandbox's saved state, stopped (exit 0 from Boat), revoked the token and exited 1; the run isstuckand clean; a laterdispatch stopis refused with exit 2.kxt1.orsk-ant-inboat history,boat events, the argument log, the dispatch output or the state folder.Not checked against real Boat
boat stopsays for an archived or deleted sandbox;boat execreaches a sandbox that isstopping;boat newprints itscreatedline before its time limit cuts it off (finding 1's fix reads it from the partial output; the look-up by name covers the rest);alive=check through/proc/<pid>/cmdline;dispatchhas returned, seen by the watcher (it was seen bydispatchitself, above).🤖 Generated with Claude Code