thread spawn accepts a nonexistent model: rc=0, provisions a worktree, fails as an agent message

[MGrin]

Summary

bb thread spawn --model <id> accepts a model id that does not exist: it returns rc=0, provisions a full git worktree, starts the turn, and surfaces the failure as an ordinary agent message rather than an error. The same gap accepts a reasoningLevel the chosen model does not advertise.

An execution setting appears to be validated nowhere between the CLI and the provider — even though bb already has the rule elsewhere.

1. A model id that does not exist

Two threads were spawned with --model claude-does-not-exist-9. In both cases bb:

• returned rc=0 from bb thread spawn
• ran the full provisioning sequence — Preparing worktree, Fetching origin/main, status: completed
• started the turn
• delivered the failure as an item/completed agentMessage:

There's an issue with the selected model (claude-does-not-exist-9). It may not exist or you may not have access to it.

The cost of a typo is a worktree provision plus an agent turn, and every status along the way says success.

2. A reasoning level the model does not advertise

A dispatch preset was created as claude-haiku-4-5-20251001 with reasoningLevel: medium. Haiku's supportedReasoningEfforts has exactly one entry, low. bb accepted the preset and never complained for sixteen days, until it was noticed by hand.

Why this looks like an oversight rather than a decision

bb already enforces this rule on a different path. bb workflows validate rejects a bad literal model with Model mismatch for literal workflow selection. So the check exists — it just isn't applied on the spawn path or on preset creation.

Reproduction

bb thread spawn --project <p> --model claude-does-not-exist-9 --prompt 'hello'
echo "rc=$?"          # 0

Then, to confirm it was accepted rather than rejected:

sqlite3 ~/.bb/bb.db "select json_extract(data,'$.execution.model'), count(*)
  from events where type='client/turn/requested' group by 1 order by 2 desc;"

The bogus id appears alongside the real ones. Cross-check the real catalogue with bb provider models claude-code --json.

Suggested fix

Validate --model and --reasoning-level against the resolved provider's catalogue at thread spawn and at preset create|update, failing non-zero before provisioning. supportedReasoningEfforts is already published per model, so the reasoning-level half needs no new data.

Why it matters more now than it used to

While every spawn uses one project default this is survivable — on our machine 10,126 of 11,011 turns in 14 days ran the same model. It stops being survivable the moment anything routes deliberately: that is exactly when a typo appears, and it currently costs a worktree and a turn while reporting success.

Not verified

Whether other paths (bb automation, workflow agent() calls) validate the model. Only thread spawn and preset create|update were measured.

🤖 Generated with Claude Code