Skip to content

feat(subagent): surface unset model-fallback on retryable failures + ARMORY_FLEET_MODEL_FALLBACK=auto (#58) - #70

Merged
rz1989s merged 2 commits into
mainfrom
feat/58-model-fallback-visibility
Aug 29, 2026
Merged

rz1989s merged 2 commits into
mainfrom
feat/58-model-fallback-visibility

Conversation

@rz1989s

@rz1989s rz1989s commented Aug 29, 2026

Copy link
Copy Markdown
Member

Problem (#58)

A retryable provider failure (stopReason 'error') with neither a per-dispatch modelFallback nor the global defaultModelFallback silently skipped auto-retry — the dispatch just failed and the operator had no signal that a safety feature was off.

Changes (per the issue's rec: visibility first, auto as opt-in)

  1. Visibilitytools/subagent.ts: when a dispatch fails retryable and NO fallback is configured, the surfaced error now appends:
    (no modelFallback configured — pass modelFallback or set ARMORY_FLEET_MODEL_FALLBACK to enable one-shot auto-retry)
    …exactly when the missing fallback matters. Mutually exclusive with fix(subagent): surface primary error when fallback retry masks it + journal failure reason (#59) #69's composed primary+fallback error (a retry implies a fallback existed).

  2. auto sentinelARMORY_FLEET_MODEL_FALLBACK=auto resolves a fallback per session from the runtime's configured+available snapshot via a new pure helper resolveAutoFallback (engine/auto-fallback.ts):

    • prefers the first model from a different provider than the session model (a real fallback family — the "Ollama primary + OpenRouter fallback" pattern, without hardcoding providers, unlike the enhancement: builtin model tiers hardcode Ollama — default to an "inherit" sentinel so they're provider-agnostic #64 tiers anti-pattern);
    • falls back to a different model id on the same provider when no other provider is configured;
    • undefined on single-model setups → auto-retry stays off + a one-time session warning says why.
      Wiring: index.ts init uses non-auto env values verbatim (unchanged); session_start resolves auto (needs the session model to differ from).

Tests

  • New test/auto-fallback.test.mts (5): different-provider preference, same-provider fallback, single-model → undefined, empty snapshot → undefined, never picks the session model itself.
  • Tool tests (2): retryable failure without fallback → hint present (keeps original failure text, names the env var); with per-dispatch fallback → hint absent.
  • 709/709 pass, pnpm typecheck clean.

Closes #58

…ARMORY_FLEET_MODEL_FALLBACK=auto

#58: a retryable provider failure with neither a per-dispatch modelFallback
nor the global default silently skipped auto-retry — the operator had no
signal. Two changes, per the issue's rec:

- Visibility: the tool appends 'no modelFallback configured - pass
  modelFallback or set ARMORY_FLEET_MODEL_FALLBACK' to a retryable
  failure's error, exactly when the missing fallback matters.
- auto sentinel: ARMORY_FLEET_MODEL_FALLBACK=auto resolves a fallback
  from the runtime's configured+available snapshot per session — a
  different provider than the session model preferred, else a different
  id; unresolvable (single-model) stays off with a one-time warning.

resolveAutoFallback is a pure helper (engine/auto-fallback.ts) + tests;
wiring in index.ts init (verbatim values) + session_start (auto).
@rz1989s
rz1989s merged commit 996ea26 into main Aug 29, 2026
1 check passed
@rz1989s
rz1989s deleted the feat/58-model-fallback-visibility branch August 29, 2026 04:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

subagent: defaultModelFallback unset by default — provider rate-limits fail the dispatch with no auto-retry

1 participant