Skip to content

Add provider binding validation and live-chat agent config#124

Merged
ManuelKugelmann merged 1 commit into
mainfrom
claude/test-models-providers-ANVcO
Mar 18, 2026
Merged

Add provider binding validation and live-chat agent config#124
ManuelKugelmann merged 1 commit into
mainfrom
claude/test-models-providers-ANVcO

Conversation

@ManuelKugelmann
Copy link
Copy Markdown
Owner

Summary

This PR adds comprehensive validation to ensure all agents have properly configured providers and models, with a new test suite that verifies provider bindings against the librechat configuration. It also explicitly configures the live-chat agent to use Anthropic's Claude Sonnet 4.6 model.

Key Changes

  • New test class TestProviderBinding: Validates that every agent has an effective provider and model by resolving overrides from agent-models.json against base agents.json configuration
  • Provider validation logic: Ensures all agent providers either:
    • Are native providers (OpenAI, Anthropic) requiring only .env keys
    • Exist as custom endpoints in librechat-user.yaml
  • Live-chat agent configuration: Explicitly assigns the main chat agent to Anthropic's Claude Sonnet 4.6 model with a safeguard test to prevent downgrading to cheaper providers
  • GitHub Models PAT documentation: Updated comment to clarify that classic PATs or fine-grained tokens with "Models" read permission are both acceptable

Implementation Details

  • The effective_assignments fixture resolves the provider/model hierarchy: base agent config → layer defaults → per-agent overrides
  • The NATIVE_PROVIDERS constant identifies providers that don't require YAML endpoint configuration
  • The test_live_chat_not_downgraded test acts as a safeguard to prevent accidental cost-cutting that would degrade the primary chat experience
  • All validation tests skip gracefully if librechat-user.yaml is not present (e.g., in minimal test environments)

https://claude.ai/code/session_01CYT2ttbEzMGq2uoZgRHo2X

The Trading Assistant (live-chat, L5) was getting overridden from
anthropic/claude-sonnet to GitHub Models/gpt-4o-mini by the L5 layer
default in agent-models.json, causing a 401 "models permission required"
error. Add explicit per-agent override to keep it on Anthropic.

Also adds TestProviderBinding tests to catch agent-to-provider binding
mismatches in CI, and fixes misleading GitHub PAT scope comment.

https://claude.ai/code/session_01CYT2ttbEzMGq2uoZgRHo2X
@ManuelKugelmann ManuelKugelmann merged commit 383e601 into main Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants