Skip to content

fix(ai): the platform LLM was calling a model that no longer exists - #772

Merged
github-actions[bot] merged 1 commit into
mainfrom
fix/platform-llm-model-rot
Aug 26, 2026
Merged

fix(ai): the platform LLM was calling a model that no longer exists#772
github-actions[bot] merged 1 commit into
mainfrom
fix/platform-llm-model-rot

Conversation

@catomean

Copy link
Copy Markdown
Collaborator

Found by asking the Cat a real question in production and reading why it answered with its fallback:

platform-llm: model call failed  status: 404
model: llama-3.3-70b-versatile

Groq stopped serving that id. Confirmed against the live API — 14 models available, and it is not among them.

This was never the Cat's problem

Every callPlatformJson caller returned null:

feature file
offer engine services/cat/offer-engine.ts
writing engine services/cat/writing-engine.ts
writing revise services/cat/writing-revise.ts
prompt suggestions services/cat/prompt-suggestions.ts
platform feedback services/cat/platform-feedback.ts
image suggestions services/images/image-suggest.ts
voice intent router services/voice/intent-router.ts
the Cat's replies services/mentions/cat-reply.ts

Each is written to "degrade gracefully" — so eight features degraded gracefully into doing nothing, and the log line for it was a warn.

Three changes, and the model id is the least important

Failover. resolveProvider returned the first provider whose key existed and stopped. So a dead Groq id took everything down while OpenRouter sat configured and unused. It now returns a list and the call walks it. The ordering is unchanged and isn't the point — the point is that there's a second entry at all.

A 404 is now an error, not a warn, and says what it means: the pinned id has rotted. A model that no longer exists is a configuration fault, not a hiccup — it will fail identically until a human changes a constant, so it must not share a log level with a timeout.

check:ai-models asks each provider whether it still serves what we pinned, reading the ids out of source so the check can't drift from the constants.

Deliberately not in verify: it needs the network and a provider key, and a gate that goes red when an API hiccups is a gate that gets switched off. It runs where the keys live, and a provider it cannot ask is reported as SKIPPED on its own line — because the failure mode of this entire class of check is a skip being mistaken for a pass.

Proven both ways, against the live APIs

[check-ai-models] groq: openai/gpt-oss-120b — served (14 models)
[check-ai-models] openrouter: nvidia/nemotron-3-super-120b-a12b:free — served (417 models)
[check-ai-models] OK — every pinned model is still served.

and with the old id restored:

[check-ai-models] FAIL: 1 pinned model(s) no longer exist:
  groq: 'llama-3.3-70b-versatile' is NOT served (14 models available)

Verification

  • npm run verify — exit 0 (2369 tests)
  • gate proven red-then-green against the real provider APIs

🤖 Generated with Claude Code

Found by asking the Cat a real question in production and reading why it
answered with its fallback:

  platform-llm: model call failed  status: 404
  model: llama-3.3-70b-versatile

Groq stopped serving that id. Confirmed against the live API: 14 models
available, and it is not among them.

This was not the Cat's problem. Every callPlatformJson caller returned
null — the offer engine, both writing engines, prompt suggestions,
platform feedback, image suggestions, the voice intent router, and the
Cat. Each is written to "degrade gracefully", so eight features degraded
gracefully into doing nothing, and the log line was a warn.

Three changes, and the id is the least important one.

FAILOVER. resolveProvider returned the FIRST provider whose key existed
and stopped, so a dead Groq id took everything down while OpenRouter sat
configured and unused. It now returns a LIST and the call walks it. The
ordering is unchanged and is not the point — the point is that there is a
second entry at all.

A 404 is now an ERROR, not a warn, and says what it means: the pinned id
has rotted. A model that no longer exists is a configuration fault, not a
hiccup; it will fail identically until a human changes a constant, so it
must not share a log level with a timeout.

check:ai-models asks each provider whether it still serves what we
pinned, reading the ids out of source so the check cannot drift from the
constants. Deliberately NOT in `verify`: it needs the network and a
provider key, and a gate that goes red when an API hiccups is a gate that
gets switched off. It runs where the keys live, and a provider it cannot
ask is reported as SKIPPED on its own line — because the failure mode of
this entire class of check is a skip being mistaken for a pass.

Proven both ways against the live APIs: green on the new id, and red on
the old one with "'llama-3.3-70b-versatile' is NOT served (14 models
available)".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
}

async function servedIds(url, key) {
const res = await fetch(url, { headers: { Authorization: `Bearer ${key}` } });
@github-actions
github-actions Bot merged commit 9221eec into main Aug 26, 2026
7 checks passed
@github-actions
github-actions Bot deleted the fix/platform-llm-model-rot branch August 26, 2026 12:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants