Summary
The CLI ships a hardcoded model catalogue that has drifted out of date. Two concrete consequences:
conductor doctor advertises Anthropic models that do not exist — both return 404 from the
Anthropic API.
conductor agent init always writes model: openai/gpt-4o regardless of which providers are
actually configured, so the documented init → run flow fails out of the box for anyone
without an OpenAI key.
Found while regression-testing CLI branch cleanup/remove-skill-command (#100) against server
3.32.0-rc.23.
1. doctor advertises nonexistent models
cmd/doctor.go hardcodes per-provider model lists in aiProviders:
{name: "Anthropic", envVars: []string{"ANTHROPIC_API_KEY"},
models: []string{"anthropic/claude-sonnet-4-20250514", "anthropic/claude-3-5-sonnet-20241022"}},
doctor presents these as available:
$ conductor doctor
AI Providers
ok Anthropic (ANTHROPIC_API_KEY)
anthropic/claude-sonnet-4-20250514
anthropic/claude-3-5-sonnet-20241022
Both are gone. Verified against the live Anthropic API with a valid key, with a control:
| Model |
GET /v1/models/<id> |
claude-sonnet-4-20250514 |
404 |
claude-3-5-sonnet-20241022 |
404 |
claude-haiku-4-5-20251001 (control) |
200 |
Copying a model straight out of doctor output produces a failed agent execution:
reasonForIncompletion: Task ... failed: Anthropic Messages API failed with status 404:
{"type":"error","error":{"type":"not_found_error","message":"model: claude-sonnet-4-20250514"}}
Note this failure is server-side at execution time, so the user gets a FAILED workflow rather than
a validation error — an expensive way to learn the model name was wrong.
The other providers' lists are likely stale too (openai/gpt-4o-mini,
google_gemini/gemini-1.5-pro, azure_openai/gpt-4o); I only verified Anthropic, which is the key
I had.
2. agent init default ignores configured providers
cmd/agent.go:38:
defaultInitModel = "openai/gpt-4o"
agent init writes that unconditionally:
$ conductor doctor | grep -A1 'AI Providers' -A14 | grep ' ok'
ok Anthropic (ANTHROPIC_API_KEY) # only Anthropic configured
$ conductor agent init probe && cat probe.yaml
model: openai/gpt-4o # unusable here
$ conductor agent run --config probe.yaml "hi"
# fails at execution time — no OPENAI_API_KEY
Nothing cross-checks the init default against what doctor already knows is configured, even though
both live in the same binary. The command even prints
Run with: conductor agent run --config probe.yaml "your prompt here" — an instruction that cannot
succeed.
Impact
First-run experience for the agent feature is broken in the common case where a user has exactly one
provider key and it isn't OpenAI. The two defects compound: doctor says Anthropic is ready and
names two models, both invalid; agent init then picks a third model from a provider that isn't
configured at all.
Fix options
Model catalogue
- A. Drop the model lists from
doctor. It only needs to report which providers are configured;
naming specific models is a maintenance liability that will drift again. Smallest, most durable
fix.
- B. Fetch models from the provider at runtime (e.g. Anthropic's
GET /v1/models). Accurate, but
adds network calls and per-provider code to a diagnostic command.
- C. Refresh the hardcoded lists. Restores correctness today; guarantees a repeat of this issue.
Recommend A, or A plus a pointer to each provider's model docs.
agent init default
- D. Derive the default from configured providers — reuse the
aiProviders detection doctor
already performs, and pick a model for the provider whose key is present.
- E. Require
--model when the default provider isn't configured, with an error naming what is
configured.
Recommend D with E as the fallback when nothing is configured.
Test coverage
cmd/doctor_test.go exists but does not assert model-string validity (it can't, offline). Worth
noting the model lists are untestable-by-construction while hardcoded, which is an argument for
option A.
Summary
The CLI ships a hardcoded model catalogue that has drifted out of date. Two concrete consequences:
conductor doctoradvertises Anthropic models that do not exist — both return 404 from theAnthropic API.
conductor agent initalways writesmodel: openai/gpt-4oregardless of which providers areactually configured, so the documented
init→runflow fails out of the box for anyonewithout an OpenAI key.
Found while regression-testing CLI branch
cleanup/remove-skill-command(#100) against server3.32.0-rc.23.
1.
doctoradvertises nonexistent modelscmd/doctor.gohardcodes per-provider model lists inaiProviders:{name: "Anthropic", envVars: []string{"ANTHROPIC_API_KEY"}, models: []string{"anthropic/claude-sonnet-4-20250514", "anthropic/claude-3-5-sonnet-20241022"}},doctorpresents these as available:Both are gone. Verified against the live Anthropic API with a valid key, with a control:
GET /v1/models/<id>claude-sonnet-4-20250514claude-3-5-sonnet-20241022claude-haiku-4-5-20251001(control)Copying a model straight out of
doctoroutput produces a failed agent execution:Note this failure is server-side at execution time, so the user gets a FAILED workflow rather than
a validation error — an expensive way to learn the model name was wrong.
The other providers' lists are likely stale too (
openai/gpt-4o-mini,google_gemini/gemini-1.5-pro,azure_openai/gpt-4o); I only verified Anthropic, which is the keyI had.
2.
agent initdefault ignores configured providerscmd/agent.go:38:agent initwrites that unconditionally:Nothing cross-checks the init default against what
doctoralready knows is configured, even thoughboth live in the same binary. The command even prints
Run with: conductor agent run --config probe.yaml "your prompt here"— an instruction that cannotsucceed.
Impact
First-run experience for the agent feature is broken in the common case where a user has exactly one
provider key and it isn't OpenAI. The two defects compound:
doctorsays Anthropic is ready andnames two models, both invalid;
agent initthen picks a third model from a provider that isn'tconfigured at all.
Fix options
Model catalogue
doctor. It only needs to report which providers are configured;naming specific models is a maintenance liability that will drift again. Smallest, most durable
fix.
GET /v1/models). Accurate, butadds network calls and per-provider code to a diagnostic command.
Recommend A, or A plus a pointer to each provider's model docs.
agent initdefaultaiProvidersdetectiondoctoralready performs, and pick a model for the provider whose key is present.
--modelwhen the default provider isn't configured, with an error naming what isconfigured.
Recommend D with E as the fallback when nothing is configured.
Test coverage
cmd/doctor_test.goexists but does not assert model-string validity (it can't, offline). Worthnoting the model lists are untestable-by-construction while hardcoded, which is an argument for
option A.