feat(llm): GOOGLE_AI_PERSONAL — route Gemini to consumer AI Studio key#24
Closed
dnplkndll wants to merge 4 commits into
Closed
feat(llm): GOOGLE_AI_PERSONAL — route Gemini to consumer AI Studio key#24dnplkndll wants to merge 4 commits into
dnplkndll wants to merge 4 commits into
Conversation
Vertex shipped gemini-3.5-flash GA. Drop-in upgrade across all 30
seer call sites that used 2.5-flash; pro family stays on 2.5-pro
since 3.5-pro isn't GA yet.
Touched paths:
* summarize_issue, summarize_replays, summarize_traces
* autofix change_describer, confidence, comment_thread,
insight_sharing
* autofix root_cause + solution fallback chains
* autofix tools (semantic_search, explain_file)
* assisted_query (local-region routing)
* codegen (pr_review, relevant_warnings, bug_prediction)
* app.py model_mapping default
Untouched:
* gemini-2.5-flash-lite in relevant_warnings_component (no
-lite variant exists at 3.5 yet)
* gemini-2.5-flash-preview-04-17 in agent test fixtures
(cassette-pinned to that specific preview name)
* gemini-2.5-pro everywhere (3.5-pro not yet GA on Vertex)
Verified by listing publishers/google/models via google-genai
SDK in the prod seer container:
flash family 2.0 2.5 2.5-lite 3-preview 3.1-lite 3.5-flash ← GA
pro family 1.5-pro-002 2.5-pro 3.1-pro-preview (no 3.5-pro)
Test suite: 56 passed, 1 skipped. The 1 failed + 3 errors are
pre-existing on the base branch (confirmed by re-running tests
without this commit) — TestSummarizeIssue cassette uses a
totally different model name, TestFixabilityScore needs a DB
fixture missing in DEV=1.
Expected impact:
* better structured-output fidelity in summarize_issue
(addresses the parsed=None mode that hit 19,356 events/14d
before PR #20's degrade landed)
* potentially separate quota allocation on a newer model
* potentially lower cost per call
Risk: if 3.5-flash structured-output quality differs, prompts
could misbehave. PR #20's degrade paths (parsed=None, 429,
ECONNRESET) catch all known failure modes — worst case is a
graceful skip, not a worker storm.
Stack onto PR #22. After PR #22 bumps the flash family to 3.5-flash across the codebase, this routes the 6 highest-volume / lowest-stakes call sites to gemini-3.1-flash-lite — the GA -lite variant — which is typically ~25% the price of full flash and well-suited for short structured-output tasks. Routed to 3.1-flash-lite (high volume, low stakes): - summarize/issue.py (top cost driver — ~80% of flash spend) - summarize/replays.py (replay breadcrumb summarization) - summarize/traces.py (trace summarization) - autofix change_describer (commit-msg generation per PR) - autofix insight_sharing (compact summaries between steps) - autofix confidence (scoring with structured output, short) Kept on 3.5-flash (lower volume, quality-sensitive): - autofix comment_thread (interactive UI) - autofix tools (semantic_search, explain_file) - autofix root_cause / solution fallback chains - codegen (pr_review, relevant_warnings, bug_prediction) - assisted_query (NL→SQL translation) - app.py generic /v0/llm endpoint default Kept on 2.5-flash-lite (no 3.x equivalent listed): - relevant_warnings_component (the one explicit -lite caller) Expected impact (relative to PR #22 baseline): - summarize_issue is ~80% of flash spend → routing it to lite cuts flash spend by another ~60% (lite is ~25% of flash pricing) - Combined with PR #20's retry-storm fix, total Vertex spend should drop from ~$110/mo to maybe $20-30/mo run rate. Risk: lite models can produce less rich text. If issue summaries become noticeably worse in the UI we'd revert the summarize/* files. PR #20's degrade paths catch any structured-output failures so worst case is a graceful skip, not a worker storm. Tests: existing tests pass (39 passed). The 1 failed + 3 errors are pre-existing on the base branch (TestSummarizeIssue cassette pins a totally different model; TestFixabilityScore needs a DB fixture missing in DEV=1) — confirmed on PR #22 branch with same outcome.
Stacks on PR #23. Adds a config switch that points GeminiProvider at the consumer Gemini API (generativelanguage.googleapis.com) with a personal AI Studio key, instead of Vertex on GOOGLE_CLOUD_PROJECT. When GOOGLE_AI_PERSONAL is set on AppConfig, GeminiProvider.get_client() returns genai.Client(api_key=...) instead of genai.Client(vertexai=True, location=...). Region selection is skipped on this path — the consumer API is globally routed. Empty string = disabled. Default behavior unchanged. Use: mint a key at https://aistudio.google.com/app/apikey, drop into /opt/sentry-extra/secrets/seer.env as GOOGLE_AI_PERSONAL=AIza..., recreate seer container. Rollback = unset + restart. 4 unit tests pin both branches of the routing decision.
Bugbot finding on PR #24: the `# noqa: B006` was hiding a real bug (shared-state across calls via the mutable default `list[str] = []`). The proper fix is small enough to do here: - Default `thinking_content_chunks: list[str] = []` (mutable) → `thinking_content_chunks: list[str] | None = None` + a guard `if thinking_content_chunks is None: thinking_content_chunks = []` at the top of the function. The Anthropic inner provider method expects a non-optional `list[str]`, so the normalization happens in the LlmClient wrapper — provider contract stays unchanged. Confirmed callsite safety: the only production caller in autofix_agent.py:232 passes `thinking_content_chunks` explicitly, so no caller relies on the mutable-default's empty-list identity. 8 tests pass (4 personal-key + the existing construct_message_from_stream coverage).
dnplkndll
force-pushed
the
feat/summarize-flash-lite
branch
from
8f5ac7c to
10b9396
Compare
dnplkndll
added a commit
that referenced
this pull request
May 22, 2026
Bugbot finding on PR #24: the `# noqa: B006` was hiding a real bug (shared-state across calls via the mutable default `list[str] = []`). The proper fix is small enough to do here: - Default `thinking_content_chunks: list[str] = []` (mutable) → `thinking_content_chunks: list[str] | None = None` + a guard `if thinking_content_chunks is None: thinking_content_chunks = []` at the top of the function. The Anthropic inner provider method expects a non-optional `list[str]`, so the normalization happens in the LlmClient wrapper — provider contract stays unchanged. Confirmed callsite safety: the only production caller in autofix_agent.py:232 passes `thinking_content_chunks` explicitly, so no caller relies on the mutable-default's empty-list identity. 8 tests pass (4 personal-key + the existing construct_message_from_stream coverage).
dnplkndll
added a commit
that referenced
this pull request
May 22, 2026
#26) * feat(llm): GOOGLE_AI_PERSONAL — route Gemini to consumer AI Studio key Stacks on PR #23. Adds a config switch that points GeminiProvider at the consumer Gemini API (generativelanguage.googleapis.com) with a personal AI Studio key, instead of Vertex on GOOGLE_CLOUD_PROJECT. When GOOGLE_AI_PERSONAL is set on AppConfig, GeminiProvider.get_client() returns genai.Client(api_key=...) instead of genai.Client(vertexai=True, location=...). Region selection is skipped on this path — the consumer API is globally routed. Empty string = disabled. Default behavior unchanged. Use: mint a key at https://aistudio.google.com/app/apikey, drop into /opt/sentry-extra/secrets/seer.env as GOOGLE_AI_PERSONAL=AIza..., recreate seer container. Rollback = unset + restart. 4 unit tests pin both branches of the routing decision. * review: properly fix the B006 mutable-default-arg (drop the noqa) Bugbot finding on PR #24: the `# noqa: B006` was hiding a real bug (shared-state across calls via the mutable default `list[str] = []`). The proper fix is small enough to do here: - Default `thinking_content_chunks: list[str] = []` (mutable) → `thinking_content_chunks: list[str] | None = None` + a guard `if thinking_content_chunks is None: thinking_content_chunks = []` at the top of the function. The Anthropic inner provider method expects a non-optional `list[str]`, so the normalization happens in the LlmClient wrapper — provider contract stays unchanged. Confirmed callsite safety: the only production caller in autofix_agent.py:232 passes `thinking_content_chunks` explicitly, so no caller relies on the mutable-default's empty-list identity. 8 tests pass (4 personal-key + the existing construct_message_from_stream coverage).
dnplkndll
added a commit
that referenced
this pull request
May 22, 2026
Sentry issues #24 + #27 on the seer dogfood project started firing 16 events/hour AFTER the gemini-2.5-flash → gemini-3.5-flash bump landed on 2026-05-22. Symptom: 404 NOT_FOUND from streamGenerateContent on regional endpoints (us-central1) even though `models.get` lists the model as available there. Root cause: Vertex's control plane (model registry) and data plane (serving) diverge for the gemini-3.x family right now. Hitting the regional endpoint trips into 404. `global` routes via Vertex's load balancer that knows where the model is actually serving and just works. Fix: add an explicit region-preference rule matching `^gemini-3(\.|-)` that pins to `global` only — before the catch-all `.*` rule that would otherwise hand 3.x models the standard `["global", "us-central1", "us-east1"]` fallback chain. The 2.x family keeps its existing routing untouched. The rule's regex matches both the dot form (`gemini-3.5-flash`, `gemini-3.1-flash-lite`) and the hyphen form (`gemini-3-flash-preview`). The preexisting `.*-preview-.*` rule still wins for 2.5-flash-preview-04-17 etc. due to first-match-wins ordering in `get_config`. Tests (5 cases pinning the contract): - gemini-3.5-flash → global only - gemini-3.1-flash-lite → global only - gemini-3-flash-preview → global only (via the new 3.x rule, not the preview rule — preview rule still allows us-central1) - gemini-2.5-flash → keeps us-central1 + us-east1 fallback (no bleed into the 2.x family) - gemini-2.5-pro → unchanged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacks on #23. Adds a config switch that points GeminiProvider at the consumer Gemini API (`generativelanguage.googleapis.com`) with a personal AI Studio key, instead of Vertex on `GOOGLE_CLOUD_PROJECT`.
Why
Today's cost analysis showed Vertex flash usage at ~$45/mo for input + $60/mo for output, primarily driven by Sentry's auto-summary path firing per new error group across 14 projects. The retry-storm fix (#20) + flash-lite routing (#23) cut the run rate substantially, but there's still a structural ask: peel high-volume non-critical traffic off the billed org project.
Consumer Gemini API has a free tier (10 RPM / 250 RPD on flash, 100 RPD on pro) and bills the personal Google account beyond that — separate billing from `kendall-ledo`. Pricing comparable to Vertex but with the option to use Google AI Studio's free quota on dev/low-volume paths.
Behavior
When `GOOGLE_AI_PERSONAL` is set on `AppConfig`, `GeminiProvider.get_client()` returns `genai.Client(api_key=...)` instead of `genai.Client(vertexai=True, location=...)`. Region selection + DE-region lockout are skipped — the consumer API is globally routed.
Empty string (default) = disabled. Production unchanged unless the env var is set.
Use
Rollback
Unset the env var, restart the seer container. No code-level rollback.
Tests (4 cases)
Drive-by
Adds a `# noqa: B006` to a pre-existing mutable-default-arg on line 2588 of `client.py` (`thinking_content_chunks: list[str] = []`). The line predates this PR — pre-commit's `flake8-bugbear` now flags it after a recent rule update, and that was blocking commits. Not a functional change; the proper fix (rewrite to `= None` + guard) is out of scope here. Filed as a follow-up.
Out of scope