feat(ai): pick a model from what the key can use, instead of typing one - #459
Merged
Merged
Conversation
Connecting a provider failed at the model ID, and the product gave the user no way out of it. `gemini-1.5-flash` was PARTHA's Gemini default and is not offered at all to a Google AI Studio project created today, so a new key met "AI provider rejected the request, most likely because of an unsupported model ID" -- an error that names the problem and nothing that would fix it. Every provider default carried the same fault, because a hardcoded model ID is a fact with a shelf life. Providers publish their own model lists, so nothing has to be guessed. `POST /ai/models` asks the provider what this key can use and returns the list, plus one recommendation taken from that list and never invented. It shares `config_for_test`, so a key typed into the form and a key already saved both work -- the list is fetchable before anything is saved, which is when it is needed. Discovery is a GET through the same egress policy, the same allowlisted host and the same pinned connection a completion already uses; `/v1/models` and `/v1beta/models` were permitted under each provider's base path already. The sender gained a method parameter and nothing else. POST still routes through `post`, so every existing sender -- including the fakes in tests -- is untouched. Gemini lists embedding-only models alongside chat ones; offering those would produce a failure a user could not diagnose, so anything without generateContent never reaches the list. In the UI, credentials now come before the model, because that is the order setup actually runs in: you cannot list models until you have the key. Fetch models turns the free-text box into a picker, and a model the provider no longer offers is replaced rather than left in place to be rejected again. Defaults are refreshed too, but they are only a starting point now. The two I checked against a live catalogue before committing: the Gemini default I first wrote (gemini-2.0-flash) does not exist any more either, and the lowest current tier is gemini-2.5-flash.
parthrohit22
requested review from
SHAURYAKSHARMA24 and
hardikuppal04
as code owners
September 13, 2026 10:11
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reported live: Gemini could not be connected, and neither could anything else.
Why it failed
gemini-1.5-flashwas PARTHA's Gemini default. Google stopped offering the 1.5 models to projects created after they were retired, so a key made today gets:An error that names the problem and nothing that would fix it. There is no way to discover a valid ID from inside the product, so "confirm the model ID" is advice the user cannot act on.
Every provider default carried the same fault. A hardcoded model ID is a fact with a shelf life, and this one had expired.
The fix
Providers publish their own model lists, so none of this has to be guessed.
POST /ai/modelsasks the provider what this key can use and returns the list plus one recommendation — taken from that list, never invented. It sharesconfig_for_test, so a key typed into the form and a key already saved both resolve the same way: the list is fetchable before anything is saved, which is exactly when it is needed.Security surface is unchanged. Discovery is a GET through the same egress policy, the same allowlisted host and the same IP-pinned connection a completion already uses —
/v1/modelsand/v1beta/modelswere already permitted under each provider's base path.SecureProviderHttpSendergained amethodparameter and nothing else; POST still routes throughpost, so every existing sender (including the fakes in tests) is untouched.Gemini lists embedding-only models alongside chat ones. Offering one would produce a failure a user could not diagnose, so anything without
generateContentnever reaches the list.UI
Credentials now come before the model, because that is the order setup actually runs in — you cannot list models until you have the key. Fetch models turns the free-text box into a picker, and a model the provider no longer offers is replaced rather than left in place to be rejected again. The list is dropped when the provider changes, so one provider never offers another's models.
Defaults
Refreshed — but they are only a starting point now, not the only route to a working config.
Worth flagging: the Gemini default I first wrote was also wrong. I checked
gemini-2.0-flashagainst a live catalogue before committing and it no longer exists either; the lowest current tier isgemini-2.5-flash. That is the second time a hand-picked default was stale within this one change, which is the whole argument for discovery.gpt-4.1-miniclaude-haiku-4-5-20251001claude-haiku-4.5present upstreamgemini-2.5-flashopenai/gpt-4.1-minillama3.2Verified
The parser ran against real provider data — OpenRouter's public catalogue, 445 models, parsed correctly, recommendation resolved to
openai/gpt-4.1-mini. The Gemini endpoint answers 403 "needs a key" rather than 404, confirming host and path. I did not test with the user's own key and did not ask for it.Tests
8 backend (each provider's list shape, Gemini's embedding filter, an empty list reported rather than returned, missing credential per provider, and that the recommendation is always drawn from the returned list) and 4 frontend (fetch and select, keep a still-offered choice, drop the list on provider change, surface a failure without leaving a stale list).
One existing test changed on purpose: a secret-scanning guard on
/ai/providersflags any 24+ character token-shaped string, and a dated Anthropic model id matches that shape. Exempted by exact value against the registry rather than by loosening the pattern, which would have let a real leak through.1187 backend / 468 frontend passing;
ruff,mypy,tsc,eslintclean; API contract regenerated.