Skip to content

feat(#663): activate MCP-recover for low-cadence agents (R1 enable-gate + R2 checkable) - #665

Merged
olegbrok merged 2 commits into
mainfrom
feat/663-mcp-recover-activation
Jun 2, 2026
Merged

feat(#663): activate MCP-recover for low-cadence agents (R1 enable-gate + R2 checkable)#665
olegbrok merged 2 commits into
mainfrom
feat/663-mcp-recover-activation

Conversation

@olegbrok

@olegbrok olegbrok commented Jun 2, 2026

Copy link
Copy Markdown
Collaborator

What & why

Follow-up to #664 (merged, 83d054f). #664 shipped MCP-orphan auto-recovery flag-gated default-OFF so it could soak on one agent (barsik) before fleet rollout. The post-merge review surfaced two blockers that make flipping mcp_recover on barsik a no-op. This PR fixes both so the flag is actually actionable for a low-cadence agent.

barsik's live shape (the soak target): watchdog_config={"enabled": false}, heartbeat_interval=0, wake_interval=3600.

R1 — watchdog enable-gate (session_watchdog._evaluate)

mcp_recover is its own opt-in, independent of the master cfg.enabled switch. Previously if not cfg.enabled: return at the top of _evaluate skipped the MCP-bind branch entirely for any agent with the progress watchdog off — so barsik (enabled=false) could never MCP-recover.

Now: return early only when both enabled and mcp_recover are off. The MCP-bind branch runs before the enabled gate; the progress/transition watchdog stays gated on enabled. (Reordering note: during a BOOTING/RECONNECTING transition snap.connected is False, so the MCP-bind branch is a cheap no-op there and the transition branch below still handles that wedge class.)

R2 — checkable signal (api._watchdog_mcp_bind_status)

checkable was heartbeat_interval > 0. barsik carries heartbeat_interval == 0 but heartbeats in practice (wake-driven), so it was wrongly excluded — Murzik Finding #2. Now checkable is also true when the agent emitted a recent agent-origin heartbeat (get_latest_agent_heartbeat, which excludes the scheduler's synthetic server_presence rows — so a wedged agent can't look checkable-and-bound). Extracted as the pure, unit-tested helper compute_mcp_checkable (window: DEFAULT_MCP_CHECKABLE_HEARTBEAT_MAX_AGE = 1800s).

Safety

  • Still flag-gated; mcp_recover default OFF. No agent's behavior changes until the flag is set.
  • Existing fixed-interval agents: unaffected (hb > 0 still → checkable).
  • Recovery rails from feat(#663): auto-recover MCP-orphaned sessions (watchdog force-fresh) #664 unchanged: 240s sustained-unbound deadline, per-agent post-recover grace, global rate-limit, swallowed status errors.

Tests

tests/test_session_watchdog.py52 pass, ruff clean.

  • R1: MCP branch fires with enabled=False, mcp_recover=True; fully-off agent is a true no-op (no status lookup, no state entry); progress/transition watchdog stays gated off when disabled.
  • R2: compute_mcp_checkable policy matrix (no agent / no epoch / cadence agent / recent beat / stale beat / no history / window boundary).

⚠️ Key review question (the soak risk) — for @murzik

The honest tension I want a second pair of eyes on before we flip the flag on barsik:

A healthy-but-idle barsik and a wedged barsik both present as bound=False right after a gateway rebuild. The only difference is the healthy one eventually rebinds (on its next MCP call) while the wedged one never does. With R2, a recently-active barsik is checkable for 30 min, so during the 240s sustained-unbound window after a rebuild there's a false-positive risk: if a healthy barsik doesn't make any MCP round-trip for 240s post-resume, the watchdog would force-fresh restart it.

Mitigations I'm relying on:

  1. On resume, barsik processes the wake prompt and almost always touches MCP (load_my_context / heartbeat / save_my_context) within the first turn → rebinds in seconds, far inside 240s.
  2. bound goes sticky-true on the first MCP call of the epoch and stays true → the risk window is only the brief post-rebuild rebind gap, not steady-state.
  3. 240s deadline + global rate-limit + post-recover grace bound the blast radius to at most one restart.

Questions: (a) Is passive heartbeat-recency checkable soak-safe enough for barsik, or should barsik enablement wait for the phase-2 active mcp_probe elicitation (which removes the ambiguity by forcing an MCP round-trip at launch)? (b) Is 1800s a sane window, or would you tie it to a multiple of the unbound deadline instead? (c) Any objection to the _evaluate reorder (MCP-bind now before the transition branch)?

Closes the activation blockers for #663. Does not include phase-2 (active probe / launch-time directive) — tracked separately.

🤖 Generated with Claude Code

Oleg and others added 2 commits June 2, 2026 14:58
…te + R2 checkable)

Two refinements that let `mcp_recover` be enabled on an agent whose
progress/transition watchdog is OFF (barsik's shape) — the activation
blockers found in the #664 post-merge review.

R1 — watchdog enable-gate (session_watchdog._evaluate):
`mcp_recover` is its own opt-in, independent of the master `cfg.enabled`
switch. The MCP-bind branch now runs before the `enabled` gate (return
early only when BOTH `enabled` and `mcp_recover` are off); the
progress/transition watchdog stays gated on `enabled`. During a
BOOTING/RECONNECTING transition the branch is a cheap no-op (connected
is False), so the transition branch still handles that wedge.

R2 — checkable signal (api._watchdog_mcp_bind_status): `checkable` was
`heartbeat_interval > 0`, which wrongly excludes wake-driven sidekicks
that carry interval==0 yet heartbeat in practice (barsik; Murzik
Finding #2). Now also true when the agent emitted a recent AGENT-ORIGIN
heartbeat (get_latest_agent_heartbeat — excludes synthetic
server_presence rows, so a wedged agent can't look checkable-and-bound).
Extracted as the pure, unit-tested helper `compute_mcp_checkable`.

Still flag-gated (mcp_recover default OFF). Tests: R1 enable-gate routing
+ progress-gated-off, R2 checkable policy matrix. 52 pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Proves the R1 _evaluate reorder doesn't steal lifecycle transitions: with
enabled=True + mcp_recover=True and a RECONNECTING (connected=False) snap,
the MCP-bind branch no-ops and the transition branch still owns the wedge.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@olegbrok
olegbrok merged commit 667d5b4 into main Jun 2, 2026
11 checks passed
@olegbrok
olegbrok deleted the feat/663-mcp-recover-activation branch June 2, 2026 22:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant