Skip to content

MCP: locate_bounding_boxes backend description names unavailable backends on local-only servers #35

Description

@ghchinoy

Follow-up to #23 / #1 (cosmetic, low priority).

On a local-only server (dgem mcp --local, dgem serve --local), visionBackendChoicesSchema removes local (no vision tower) and ends up with no backends. applyBackendChoices returns early on an empty list, so locate_bounding_boxes keeps the static struct-tag description:

enum=null desc=Optional inference backend selector: 'vertex_first' (default), 'vertex', or 'cloudrun' (note: 'local' Metal does not support SigLIP vision).

(from tools/list on v0.1.2-4-gd1a3246 with --local)

On this server, agents are told to use backends that don't exist. A call returns backend "vertex" is not enabled on this gateway (available: local), so nothing breaks, but the text is misleading. The other three tools correctly advertise only local.

Suggested fix: when the vision list is empty, set the description to something like "No vision-capable backend on this server ('local' has no vision tower); this tool will return an error." Possibly also mention it in the tool description, so agents skip the tool. Keeping the field without an enum is fine (an empty enum would be invalid for Gemini).

Test: extend TestMCPBackendSchemaListsOnlyConfiguredBackends to assert the local-only locate_bounding_boxes description doesn't name vertex_first.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions