fix(llm): pin gemini-3.x family to global region only#27
Merged
Conversation
Sentry issues #24 + #27 on the seer dogfood project started firing 16 events/hour AFTER the gemini-2.5-flash → gemini-3.5-flash bump landed on 2026-05-22. Symptom: 404 NOT_FOUND from streamGenerateContent on regional endpoints (us-central1) even though `models.get` lists the model as available there. Root cause: Vertex's control plane (model registry) and data plane (serving) diverge for the gemini-3.x family right now. Hitting the regional endpoint trips into 404. `global` routes via Vertex's load balancer that knows where the model is actually serving and just works. Fix: add an explicit region-preference rule matching `^gemini-3(\.|-)` that pins to `global` only — before the catch-all `.*` rule that would otherwise hand 3.x models the standard `["global", "us-central1", "us-east1"]` fallback chain. The 2.x family keeps its existing routing untouched. The rule's regex matches both the dot form (`gemini-3.5-flash`, `gemini-3.1-flash-lite`) and the hyphen form (`gemini-3-flash-preview`). The preexisting `.*-preview-.*` rule still wins for 2.5-flash-preview-04-17 etc. due to first-match-wins ordering in `get_config`. Tests (5 cases pinning the contract): - gemini-3.5-flash → global only - gemini-3.1-flash-lite → global only - gemini-3-flash-preview → global only (via the new 3.x rule, not the preview rule — preview rule still allows us-central1) - gemini-2.5-flash → keeps us-central1 + us-east1 fallback (no bleed into the 2.x family) - gemini-2.5-pro → unchanged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Production Sentry issues #24 + #27 (on the seer dogfood project) started firing 16 events/hour AFTER PR #22's
gemini-2.5-flash → gemini-3.5-flashbump deployed at 2026-05-22T12:00Z.Symptom
Fired from
root_cause_step.root_cause_taskafter the primary 2.5-pro call failed over to a flash fallback.Root cause
Vertex's control plane (
client.models.get("gemini-3.5-flash")) lists the model as available in us-central1. The data plane (streamGenerateContent/generateContent) returns 404 NOT_FOUND when actually hitting that regional endpoint. Probed all six US/EU regions from inside the prod seer container:Same pattern for
gemini-3.1-flash-lite.globalis the only reliably-serving endpoint for the 3.x family right now.Fix
Add a region-preference rule matching
^gemini-3(\.|-)BEFORE the catch-all.*. Pinned toglobalonly. 2.x family is untouched, keeps its["global", "us-central1", "us-east1"]fallback chain for redundancy.Regex coverage
The
^gemini-3(\.|-)pattern matches:gemini-3.5-flash(dot separator)gemini-3.1-flash-lite(dot separator)gemini-3-flash-preview(hyphen separator)gemini-3.1-pro-preview(dot separator)The pre-existing
.*-preview-.*rule still wins forgemini-2.5-flash-preview-04-17(first-match-wins inget_config).Tests
5 cases pinning the contract (all pass):
gemini-3.5-flash → global onlygemini-3.1-flash-lite → global onlygemini-3-flash-preview → global onlygemini-2.5-flash → keeps us-central1 + us-east1gemini-2.5-pro → unchangedExpected production impact
After deploy, Sentry issues #24 + #27 should stop firing immediately. The 12 other 'fixed' Sentry issues from today's optimization sprint are already resolved-with-auto-reopen — they verify that #20/#22/#25's degrade paths are still working.
Rollback
Revert this commit → image rebuild → recreate seer container. The 3.x models would fall back to the wider region chain again.