Skip to content

[Bug]: Default Gemini model (gemini-2.5-pro) is unavailable to new API keys, breaking every AI feature on a fresh install #2

Description

@And3m

Surface

Local main branch


What happened?

On a fresh clone with a newly created Gemini API key, every AI feature fails with HTTP 502. The cause is that PROVIDER_CATALOG ships gemini-2.5-pro as the Gemini default (backend/app/llm/catalog.py:17), and Google has since closed the entire Gemini 2.5 family to newly created API keys. The provider returns 404 NOT_FOUND with the message "This model models/gemini-2.5-pro is no longer available to new users." Because the default model is what a first-time user gets before touching settings, the practical effect is that graph generation, closure quizzes, and the assistant are all dead on arrival for any new user — the exact first-run path the quickstart walks people through.

This is not a quota or billing problem. The key authenticates fine and GET /v1beta/models lists both gemini-2.5-pro and gemini-2.5-flash; they only fail at generateContent. Listing a model is not the same as having access to it, which makes this easy to misdiagnose.

The same stale list is duplicated in the frontend at frontend/src/components/SettingsModal.tsx:11-14, and SettingsModal.tsx:216 selects nextOptions[0] when the provider dropdown changes — so a user who opens Settings and re-selects "gemini" is auto-assigned gemini-2.5-pro, i.e. steered straight back into the broken model.

Steps to reproduce

  1. Create a new Gemini API key at https://aistudio.google.com/apikey (an older key will not reproduce this — existing users retain 2.5 access).
  2. git clone, cp .env.example .env, set KG_GEMINI_API_KEY=..., start the stack.
  3. Open the frontend, leave the default model untouched (gemini-2.5-pro).
  4. Open the starter mathematics-demo graph, select Arithmetic and number sense, and start a closure quiz.
  5. Observe HTTP 502 and "closure quiz generation failed: unexpected provider error". Any other AI action (expand graph, ingest topics, assistant) fails the same way.

Expected behavior

A fresh install with a valid, newly created provider key should be able to generate on the default model without the user first discovering that the default is unusable.

Concretely, I'd suggest:

  1. Move the Gemini default_model to a model still available to new keys — gemini-3-flash-preview is already in model_options and works.
  2. Refresh model_options to drop or de-prioritise the 2.5 entries.
  3. Ideally, surface the provider's own message. The 404 text is fully actionable, but the user only sees "unexpected provider error" (backend/app/services/quiz_service.py:261) unless they enable debug mode and read logs/logs.log. Given the project's "fail-closed beats fake success" principle, this fails closed correctly but doesn't say why.

I'm happy to open a PR for (1) and (2) — and (3) separately if you want it scoped apart.

Logs, screenshots, or traces

Direct provider check, confirming this is availability and not quota:

POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-pro:generateContent
404 {
  "error": {
    "code": 404,
    "message": "This model models/gemini-2.5-pro is no longer available to new users. Please update your code to use a newer model for the latest features and improvements. We recommend you to use the Interactions API (https://ai.google.dev/gemini-api/docs/migrate-to-interactions).",
    "status": "NOT_FOUND"
  }
}

gemini-2.5-flash and gemini-2.5-flash-lite return the identical error. What I measured on a new key:

Model generateContent
gemini-2.5-pro 404 — not available to new users
gemini-2.5-flash 404 — not available to new users
gemini-pro-latest 429 — quota
gemini-3-pro-preview 429 — quota
gemini-flash-latest 503 — overloaded
gemini-3-flash-preview 200 OK

From logs/logs.log on the failing quiz:

{"kind": "server", "level": "error",
 "title": "POST /api/v1/graphs/mathematics-demo/topics/arithmetics/quiz/start",
 "message": "Closure quiz generation failed", "status_code": 502,
 "response_excerpt": "{\"detail\":\"closure quiz generation failed: unexpected provider error\",\"diagnostics\":{\"model\":\"gemini-2.5-pro\",\"error_type\":\"ClientError\",\"error_message\":\"404 NOT_FOUND ... no longer available to new users\"}}"}

Confirming the fix — after POST /api/v1/workspace/config {"default_model":"gemini-3-flash-preview"}, the same quiz call returns 200 with 12 well-formed questions and "generator": "gemini-3-flash-preview". No code change was needed, which is why I think this is just a catalog default that reality moved out from under.

Commit or branch

main @ 41ad232

Regression?

Yes

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions