fix(subagents): drain owned batch stop across cancellation

[poijygfdyy]

Why

A caller can cancel SubagentRuntime.stop() after the runtime has already hidden its owned batch submitter but before the owned SubagentBatchService.stop() has actually finished.

Current main sets _batch_started = False before directly awaiting the service stop. If that await is interrupted, the runtime now reports no batch submitter and any later stop() call returns immediately, while the owned service may still be part-way through stopping. The real service has cancellable awaits before active executions are cancelled and its ownership maps are cleared, so this can leave runtime-owned batch work alive after the runtime itself has released lifecycle ownership.

What changed

• Run the owned batch-service stop in a strongly-owned task.
• Shield and drain that task across caller cancellation, including repeated Task.cancel().
• Preserve the first caller CancelledError, but only propagate it after the owned service stop reaches a terminal state.
• Add a deterministic regression that blocks the owned stop, cancels the runtime twice, and verifies the runtime cannot return before the service finishes.

Surface area

• Frontend UI — page / component / setting / interaction under frontend/
• Backend API — endpoint / SSE event / request-response shape under backend/app
• Agents / LangGraph — direct SubagentRuntime lifecycle ownership
  • Prompt-layer self-check: no prompt or trust-channel changes.
• Sandbox — docker/ or sandboxed execution
• Skills — change under skills/
• Dependencies — new/upgraded entry in backend/pyproject.toml or frontend/package.json
• Default behavior change — cancelling runtime shutdown no longer releases ownership before the owned batch worker stops
• Docs / tests / CI only — runtime behavior changes

Screenshots / Recording

N/A — backend lifecycle/cancellation fix.

Bug fix verification

Test: backend/tests/test_subagent_runtime.py::test_runtime_stop_drains_owned_batch_worker_across_repeated_cancellation.

Deterministic schedule:

1. Start a caller-owned SubagentRuntime with an owned batch service.
2. Block service.stop() after it starts.
3. Cancel runtime.stop() once, then again while cleanup is still blocked.
4. Assert the runtime stop task remains pending and the submitter remains hidden.
5. Release the service stop and assert the original cancellation propagates only after cleanup finishes.

Red on current main 16f154f32b6104ddeea525c713445565e648aeee: yes. The task becomes cancelled/done after the first cancellation while the service stop is still blocked.

Green on this branch: yes. The runtime keeps lifecycle ownership through repeated cancellation until the owned stop completes.

Collision audit

Re-checked immediately before submission:

• no open issue or PR found implementing cancellation-safe SubagentRuntime.stop();
• open fix(subagents): close stream before releasing resources #5221 currently changes stream cleanup / executor paths, does not touch subagents/runtime.py, and explicitly scopes hard Task.cancel(), batch persistence, and Gateway shutdown outside that PR;
• open fix(subagents): tear down dispatched child before batch retry #5253 changes batch-service retry supervision and executor teardown, not SubagentRuntime.stop();
• no maintainer handoff or active implementation for this exact direct-runtime ownership boundary was found.

Validation

• Rebased onto current upstream main 16f154f32b6104ddeea525c713445565e648aeee; the two touched file blobs are unchanged from the validated head, so the rebase only changed the parent commit.
• Current head is 66cd99b38bb73b9077a178252726b5f965845b02, 1 commit ahead / 0 behind; diff is 2 files, production change +21/-3.
• Current-main regression: 1 failed as expected, with runtime stop released ownership after the first cancellation.
• Patched focused suites: tests/test_subagent_runtime.py tests/test_subagent_batch_service.py — 10 passed.
• ruff check on both changed files — clean.
• ruff format --check on both changed files — clean.
• python scripts/check_agent_guidance.py — 28 AGENTS.md, 0 errors, 0 warnings.
• git diff --check — clean.

AI assistance

Tool(s) used: ChatGPT

How you used it: Audited the current runtime lifecycle, constructed and ran the deterministic cancellation schedule on current main, performed collision checks against active subagent work, implemented the minimal shield/drain fix, rebased it onto current main, and ran focused tests/lint.

• I've read and understand every line of this change and take responsibility for it — it's not unreviewed AI output.

[willem-bd]

Verified empirically at head 66cd99b: the new regression test fails on base 16f154f with exactly the reported message and passes on this branch; the focused suites (test_subagent_runtime.py, test_subagent_batch_service.py) plus the broader subagent lifecycle/executor suites pass (160 passed; 5 pre-existing env-only collection errors in test_extension_subagent_lifecycle.py also occur on base), and ruff check/format are clean. The fix itself looks correct: the lifecycle lock is held for the whole drain, so concurrent start()/stop() callers still serialize and a second stop() still returns only after the service reaches a terminal state; hiding the submitter immediately matches the stated intent, and the first caller cancellation is preserved. Two minor points below on the unbounded drain and an exception-guard asymmetry — neither blocks.

[willem-bd]

Nit/suggestion on robustness: this drain is intentionally unbounded — once the first cancellation is recorded, stop() can never be abandoned until service.stop() reaches a terminal state. That holds for today's SubagentBatchService.stop() (it cancels the poller/children and gathers with return_exceptions=True), but one pathological await inside the service — a repository call that ignores cancellation, or a gathered child task that suppresses CancelledError without exiting — would now make runtime (and by extension Gateway) shutdown permanently un-cancellable, where old main would at least escape (buggily). Consider either bounding the drain (e.g., asyncio.wait({stop_task}, timeout=...), then re-raising the recorded cancellation regardless) or documenting the invariant that the owned service's stop() must always terminate.

[willem-bd]

Nit on an asymmetry: if stop_task itself ends cancelled, stop_task.result() re-raises CancelledError as a BaseException, which bypasses this except Exception guard — so the recorded first caller cancellation is silently dropped and the raise cancellation from exc chaining never happens. With the current SubagentBatchService.stop() that state is unreachable (the shield keeps outer cancellation off the inner task, and the service gathers with return_exceptions=True), so it's theoretical today, but checking stop_task.cancelled() (or catching BaseException) would make the "preserve the first caller CancelledError" contract hold unconditionally.