fix(ai): the platform LLM was calling a model that no longer exists - #772
Merged
Conversation
Found by asking the Cat a real question in production and reading why it answered with its fallback: platform-llm: model call failed status: 404 model: llama-3.3-70b-versatile Groq stopped serving that id. Confirmed against the live API: 14 models available, and it is not among them. This was not the Cat's problem. Every callPlatformJson caller returned null — the offer engine, both writing engines, prompt suggestions, platform feedback, image suggestions, the voice intent router, and the Cat. Each is written to "degrade gracefully", so eight features degraded gracefully into doing nothing, and the log line was a warn. Three changes, and the id is the least important one. FAILOVER. resolveProvider returned the FIRST provider whose key existed and stopped, so a dead Groq id took everything down while OpenRouter sat configured and unused. It now returns a LIST and the call walks it. The ordering is unchanged and is not the point — the point is that there is a second entry at all. A 404 is now an ERROR, not a warn, and says what it means: the pinned id has rotted. A model that no longer exists is a configuration fault, not a hiccup; it will fail identically until a human changes a constant, so it must not share a log level with a timeout. check:ai-models asks each provider whether it still serves what we pinned, reading the ids out of source so the check cannot drift from the constants. Deliberately NOT in `verify`: it needs the network and a provider key, and a gate that goes red when an API hiccups is a gate that gets switched off. It runs where the keys live, and a provider it cannot ask is reported as SKIPPED on its own line — because the failure mode of this entire class of check is a skip being mistaken for a pass. Proven both ways against the live APIs: green on the new id, and red on the old one with "'llama-3.3-70b-versatile' is NOT served (14 models available)". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
| } | ||
|
|
||
| async function servedIds(url, key) { | ||
| const res = await fetch(url, { headers: { Authorization: `Bearer ${key}` } }); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found by asking the Cat a real question in production and reading why it answered with its fallback:
Groq stopped serving that id. Confirmed against the live API — 14 models available, and it is not among them.
This was never the Cat's problem
Every
callPlatformJsoncaller returnednull:services/cat/offer-engine.tsservices/cat/writing-engine.tsservices/cat/writing-revise.tsservices/cat/prompt-suggestions.tsservices/cat/platform-feedback.tsservices/images/image-suggest.tsservices/voice/intent-router.tsservices/mentions/cat-reply.tsEach is written to "degrade gracefully" — so eight features degraded gracefully into doing nothing, and the log line for it was a
warn.Three changes, and the model id is the least important
Failover.
resolveProviderreturned the first provider whose key existed and stopped. So a dead Groq id took everything down while OpenRouter sat configured and unused. It now returns a list and the call walks it. The ordering is unchanged and isn't the point — the point is that there's a second entry at all.A 404 is now an
error, not awarn, and says what it means: the pinned id has rotted. A model that no longer exists is a configuration fault, not a hiccup — it will fail identically until a human changes a constant, so it must not share a log level with a timeout.check:ai-modelsasks each provider whether it still serves what we pinned, reading the ids out of source so the check can't drift from the constants.Deliberately not in
verify: it needs the network and a provider key, and a gate that goes red when an API hiccups is a gate that gets switched off. It runs where the keys live, and a provider it cannot ask is reported asSKIPPEDon its own line — because the failure mode of this entire class of check is a skip being mistaken for a pass.Proven both ways, against the live APIs
and with the old id restored:
Verification
npm run verify— exit 0 (2369 tests)🤖 Generated with Claude Code