Skip to content

TODO: Add provider-aware dynamic LLM routing for agent calls #86

Description

@MuJiongan

Goal

Let the orchestrator choose the best available LLM for each agent call instead of relying on one fixed default node model.

The routing decision should be constrained to the providers and models the user has connected, and should account for the needs of the call (for example: task complexity, tool use, vision, context size, latency, and cost).

TODO

  • Make the connected provider/model inventory and relevant model capabilities available to the orchestrator without exposing credentials.
  • Define a routing contract for each ctx.agent(...) call (explicit model override vs. automatic selection).
  • Have the orchestrator assign a provider, model, and reasoning variant when creating/configuring an agent call.
  • Validate that the selected model is available and compatible with the call's requirements.
  • Add a deterministic fallback when the preferred provider/model is unavailable or fails.
  • Persist the selected provider/model/variant in call traces, including enough routing metadata to debug the choice.
  • Preserve explicit user/node model selections and keep continued subagent chats pinned unless the user changes them.
  • Add tests for capability matching, disconnected providers, explicit overrides, fallbacks, and trace persistence.

Acceptance criteria

  • Every agent call can use automatic routing or an explicit model override.
  • Automatic routing only selects from currently connected providers/models.
  • Tool, vision, context, and reasoning requirements are respected.
  • A missing or failed preferred model falls back predictably and is visible in the trace.
  • Existing workflows that use the configured default node model remain compatible.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions