[omerhochman]

Summary

Exposes LangChain's native .withFallbacks() primitive through a new fallbackModels: [] config field so users can declare ordered backup LLMs (e.g. OpenRouter as a fallback when Groq hits a 429) without patching node_modules or running a LiteLLM sidecar.

model: "llama-3.3-70b-versatile"
modelProvider: "groq"
fallbackModels:
  - model: "anthropic/claude-sonnet-4.5"
    modelProvider: "openai"
    baseUrl: "https://openrouter.ai/api/v1"
    apiKey: "${OPENROUTER_API_KEY}"

How it works

1. The agent tries the primary model.
2. On a transient error (429, 503, network blip, etc.), the primary's own retry (llmRetryAttempts, default 3) kicks in.
3. If the retries are exhausted — or the error is non-transient — the next entry in fallbackModels is tried, with its own retry budget.
4. The chain continues until a model succeeds or all fallbacks are exhausted.

Each fallback entry supports model, modelProvider, baseUrl, and apiKey, so a single fallback can target a different provider account or an OpenAI-compatible gateway (OpenRouter, etc.) without leaking into the primary's env vars. For openai/azure_openai providers, baseUrl maps to the SDK's configuration.baseURL; other providers receive baseUrl directly.

Files changed

• src/config/schema.ts — new FallbackModelSchema + fallbackModels: [] array on ConfigSchema.
• src/config/types.ts — exports the new FallbackModel type.
• src/agent/model-factory.ts — refactors createChatModel to build the primary + each fallback through a shared buildSingleModel helper, then wraps with primary.withFallbacks({ fallbacks }). Adds buildInitOptions (exported, tested) that normalizes baseUrl/apiKey per provider.
• docs/configuration.md + README.md — new “Fallback Models” section with semantics, fields table, and OpenRouter example.
• Test suites updated and expanded:
  • tests/unit/config/schema.test.ts — 7 new tests covering defaults, single/multi entries, OpenRouter-style entry, and validation errors.
  • tests/unit/agent/model-factory.test.ts — 14 new tests covering the empty-array no-op, retry composition, baseUrl mapping for openai vs other providers, and apiKey pass-through. New buildInitOptions suite with 6 unit tests.
  • Six existing Config literals updated to include fallbackModels: [] so they continue to satisfy the type.

Test plan

• bun run typecheck clean
• bun run test — 17 files / 223 tests passing
• Manual smoke: configure a real Groq primary + OpenRouter fallback, force a 429 on the primary, confirm the fallback executes.

Tradeoff worth flagging

.withFallbacks() falls through on any error from the primary, including 401 Unauthorized. This keeps the suite running through a misconfigured primary, but means a stray credential mistake will silently route traffic (and cost) to the fallback. The first-run logs will surface it, but we may want to add a fallbackOn: ['transient', 'rate-limit'] filter in a follow-up.

🤖 Generated with Claude Code

---

Generated by Claude Code