Skip to content

feat: the dispatch watcher closes, resumes and stops sandboxes by the ledger - #57

Merged
OmarAlex24 merged 2 commits into
dispatch/boat-adapterfrom
dispatch/watcher
Nov 17, 2026
Merged

OmarAlex24 merged 2 commits into
dispatch/boat-adapterfrom
dispatch/watcher

Conversation

@OmarAlex24

@OmarAlex24 OmarAlex24 commented Nov 16, 2026 •

Copy link
Copy Markdown
Owner

Third of the kontext dispatch stack: the watcher, which decides by the ledger, never by what the agent says.

What changed and why

  • kontext dispatch watch [--detach] (src/dispatch/watch.rs): a background process on the owner's machine, started by dispatch when it is not running. Its lock file is its liveness: status says whether it is alive and, when it is not and runs are live, the command to start it again.
  • Each tick it syncs with the server, then for each live run, in order:
    • the epic is closed in the ledger: boat stop (never delete), revoke the token, closed;
    • the time limit passed (8 h, [dispatch] hours): stuck;
    • the coordinator's turn ended with the epic open: resume once in the same conversation (boat prompt --resume <conversation>), state resumed; if that turn ends with the epic still open: stuck;
    • stuck: a question to the pm as agent:dispatcher ("Epic X is stuck in its sandbox: ", with how to look and how to redispatch), then stop and revoke.
      A turn only counts as ended after a sync that succeeded, so a stale ledger never resumes or sticks an epic that closed. Each action is taken under the state lock after re-reading the run, so a watcher and dispatch stop cannot both act on it.
  • Restart: a restarted watcher reads the saved state and the turns' files and picks up where it was; a stop or revoke that failed is retried on the next ticks.
  • dispatch watch --detach counts as started a watcher that wrote its pid, even if it already did all there was to do and left (commit 7cfe98c: CI's faster runner caught ensure reporting such a watcher as "did not start"; reproduced locally by delaying the first check, and fixed).
  • It exits once nothing is live (it drops its lock first, under the state lock, so a new dispatch never sees a watcher that is about to leave).

Departures from the brief

  • A turn that was lost (its process died, the Mac slept, boat prompt failed) neither resumes nor makes the epic stuck: the watcher logs it once and goes on, and the time limit still applies.

Test plan

  • cargo fmt --check, cargo clippy --all-targets -- -D warnings and the full cargo test (42 suites, 497 tests) passed with the fix
  • The epic's tasks are closed and the epic closed in the ledger: the watcher stops the sandbox, revokes the token, writes the end entry, and exits
  • The coordinator's turn ends with the epic open: resumed once with --resume and the same conversation id; it ends again: stuck, the pm gets the question from agent:dispatcher, the sandbox is stopped and the token revoked
  • A lost turn decides nothing
  • A watcher killed and started again with the clock past the limit marks the run stuck on the time limit
  • The same test with ensure's first check delayed 2 s (a temporary local edit, not committed): it failed before the fix with CI's message and passes after
  • By hand, end to end, against the fake boat and server/dev.mjs: the refusals, a dispatch, status, the resume, the close once the tasks were done, kontext tree, the watcher's log; no kxt1. in the fake's calls, the state or the watcher's log

🤖 Generated with Claude Code

@OmarAlex24
OmarAlex24 added this pull request to stack #59 November 16, 2026 22:42
OmarAlex24 and others added 2 commits November 16, 2026 16:53
… ledger

`kontext dispatch watch` (started detached by `dispatch`, one per project
and machine, under a lock) pulls the ledger and, for each live sandbox:
stops it and revokes the token when the epic closes; resumes the
coordinator once in the same conversation when its turn ended with the
epic open; marks it stuck when it stops again or its time limit passes (a
question to the pm as agent:dispatcher, then stop and revoke). A turn that
could not be followed, or one whose end was seen while the pull failed,
decides nothing. Everything is in the saved state and the turns' files, so
a restarted watcher (`dispatch watch --detach`) picks up; `dispatch status`
says when it is not running and how to start it. It exits when nothing is
live, deciding that under the state lock.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… as started

On a fast machine `dispatch watch --detach` could start a watcher that took
its lock, marked the last live run stuck and exited before the 50 ms check
saw the lock, and then reported "the watcher did not start". It now counts a
watcher that wrote its own pid as started, and says it ran and stopped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@OmarAlex24
OmarAlex24 marked this pull request as ready for review November 17, 2026 14:30
@OmarAlex24
OmarAlex24 merged commit 27b7d92 into main Nov 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant