Skip to content

MCP: record per-backend initialization failures in metrics #2724

Description

@lucazz

Business Value

Operators need to detect when an MCP route silently loses one backend while the composite client session still succeeds through the remaining backends.

Current behavior

The composite initialization path initializes backends in parallel. When one backend fails, it logs failed to create MCP session, leaves an empty entry, removes that entry, and continues if any other backend initialized successfully. The code has a TODO asking whether a metric should be recorded for this case.

Successful initialization records backend-scoped initialization duration, method count, request duration, and capabilities. Failure paths return before those backend-scoped metrics are recorded. A failure while forwarding notifications/initialized has the same gap.

This means dashboards can show a successful client initialization while the tools from one backend have disappeared from the catalog.

Related: #1661 fixed method-level error accounting after requests reach normal MCP handling, but it does not cover backend initialization failures in the composite setup path.

Suggested behavior

  • Record a backend-scoped failed/error initialization count when initialize fails.
  • Record backend-scoped failure for notifications/initialized separately from initialize.
  • Record failure duration so timeout and immediate rejection cases can be distinguished.
  • Add a test where one backend fails, another succeeds, and the client session remains usable while failure metrics identify the omitted backend.

The metric names and semantic attributes should follow the existing MCP metrics conventions, including mcp.backend, method name, and status.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    area/mcpMCP proxy, MCPRoute, and MCP spec conformancearea/observabilityMetrics, tracing, logging, GenAI observabilityenhancementNew feature or request

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions