Skip to content

fix(mcp): bound reconnect transport setup - #63823

Closed
oralzl wants to merge 1 commit into
NousResearch:mainfrom
oralzl:fix/mcp-reconnect-setup-timeout
Closed

fix(mcp): bound reconnect transport setup#63823
oralzl wants to merge 1 commit into
NousResearch:mainfrom
oralzl:fix/mcp-reconnect-setup-timeout

Conversation

@oralzl

@oralzl oralzl commented Jul 13, 2026

Copy link
Copy Markdown

What does this PR do?

Prevents a long-lived MCP server task from hanging forever when a fresh transport wedges before publishing server.session.

The reconnect loop previously awaited _run_http() / _run_stdio() directly. If SDK, OAuth, or transport setup blocked without returning or raising, the loop never advanced past reconnect attempt 1; later reconnect signals had no effect and only a gateway restart recovered the server.

This change adds a setup-only watchdog:

  • the transport still runs in one owning task, preserving AnyIO's same-task cleanup requirement;
  • only the interval before self.session is published is bounded by connect_timeout;
  • after publication, the transport remains long-lived exactly as before;
  • timeout cancellation is awaited before the outer retry loop continues;
  • upstream lifecycle results such as "recycle" pass through unchanged.

This is complementary to #63495, which fixes the known OAuth inner-generator leak behind #38193. That root fix prevents one concrete deadlock; this PR keeps any future SDK/OAuth/transport setup wedge from freezing Hermes' entire reconnect state machine.

Related Issue

Related: #38193, #31987, #49543
Complementary to: #63495

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ✨ New feature (non-breaking change that adds functionality)
  • 🔒 Security fix
  • 📝 Documentation update
  • ✅ Tests (adding or improving test coverage)
  • ♻️ Refactor (no behavior change)
  • 🎯 New skill (bundled or hub)

Changes Made

  • tools/mcp_tool.py
    • add _run_transport_with_connect_timeout();
    • bound only pre-session transport setup;
    • preserve transport lifecycle return values;
    • route both HTTP and stdio setup through the watchdog.
  • tests/tools/test_mcp_reconnect_signal.py
    • prove a transport that never publishes a session times out;
    • prove lifecycle results remain transparent after session publication.

How to Test

  1. Run the MCP tool suite:

    python -m pytest tests/tools/test_mcp_reconnect_signal.py tests/tools/test_mcp*.py -q

    Result: 582 passed, 4 pre-existing unknown-mark warnings.

  2. Run static and cross-platform checks:

    ruff check tools/mcp_tool.py tests/tools/test_mcp_reconnect_signal.py
    python scripts/check-windows-footguns.py --diff origin/main
    python -m py_compile tools/mcp_tool.py tests/tools/test_mcp_reconnect_signal.py
    git diff origin/main...HEAD --check

    Result: all pass.

  3. Production validation: on macOS arm64 / Python 3.11.15, an OAuth-backed MCP transport emitted the AnyIO lock-cleanup error and then logged only reconnect attempt 1 indefinitely. With the setup watchdog loaded, the gateway restarted cleanly and the live Discord session completed MCP format-guide, search, and memo-by-ID calls directly without a fresh-process fallback.

Checklist

Code

Documentation & Housekeeping

  • Documentation update — N/A; behavior is internal and documented in code
  • cli-config.yaml.example update — N/A; no config keys added
  • CONTRIBUTING.md / AGENTS.md update — N/A
  • Cross-platform impact considered; Windows footgun scan passes
  • Tool descriptions/schemas update — N/A

Screenshots / Logs

Observed failure signature:

RuntimeError: The current task is not holding this lock
MCP server '<oauth-server>' connection lost (attempt 1/5), reconnecting in 1s

No attempt 2 or fresh-session registration followed until gateway restart.

@alt-glitch alt-glitch added type/bug Something isn't working comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint tool/mcp MCP client and OAuth P2 Medium — degraded but workaround exists labels Jul 13, 2026

@tonydwb tonydwb left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review Summary

Verdict: LGTM

Bounding MCP reconnect transport setup prevents a runaway retry loop from blocking connection. 2 files, 95 lines. Title matches diff.

Strengths

  • A timeout on the reconnect transport path is the correct place — not the broader connection lifecycle.

Reviewed by Hermes Agent in batch mode

@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the focused MCP reconnect analysis and regression coverage.

Automated hermes-sweeper review found the reported pre-session wedge already addressed on current main:

  • 4638f3b433b9e7d498912c0a99226ec737039825 bounds the session.initialize() handshake with connect_timeout in all transport paths.
  • Current tools/mcp_tool.py:2316, :2612, :2669, and :2701 apply that bound before assigning self.session.
  • Current tools/mcp_tool.py:3048 also cancels the independently-created run() task when startup is cancelled, preventing a pre-ready transport from continuing detached.

The proposed outer watchdog is therefore redundant with the current-main behavior for this reconnect failure mode.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent runtime: loop, agent_init, prompt builder, context-compression, responses endpoint P2 Medium — degraded but workaround exists sweeper:implemented-on-main Sweeper: behavior already present on current main tool/mcp MCP client and OAuth type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants