feat(llm): route high-volume low-stakes paths to gemini-3.1-flash-lite#25
Merged
Merged
Conversation
Stack onto PR #22. After PR #22 bumps the flash family to 3.5-flash across the codebase, this routes the 6 highest-volume / lowest-stakes call sites to gemini-3.1-flash-lite — the GA -lite variant — which is typically ~25% the price of full flash and well-suited for short structured-output tasks. Routed to 3.1-flash-lite (high volume, low stakes): - summarize/issue.py (top cost driver — ~80% of flash spend) - summarize/replays.py (replay breadcrumb summarization) - summarize/traces.py (trace summarization) - autofix change_describer (commit-msg generation per PR) - autofix insight_sharing (compact summaries between steps) - autofix confidence (scoring with structured output, short) Kept on 3.5-flash (lower volume, quality-sensitive): - autofix comment_thread (interactive UI) - autofix tools (semantic_search, explain_file) - autofix root_cause / solution fallback chains - codegen (pr_review, relevant_warnings, bug_prediction) - assisted_query (NL→SQL translation) - app.py generic /v0/llm endpoint default Kept on 2.5-flash-lite (no 3.x equivalent listed): - relevant_warnings_component (the one explicit -lite caller) Expected impact (relative to PR #22 baseline): - summarize_issue is ~80% of flash spend → routing it to lite cuts flash spend by another ~60% (lite is ~25% of flash pricing) - Combined with PR #20's retry-storm fix, total Vertex spend should drop from ~$110/mo to maybe $20-30/mo run rate. Risk: lite models can produce less rich text. If issue summaries become noticeably worse in the UI we'd revert the summarize/* files. PR #20's degrade paths catch any structured-output failures so worst case is a graceful skip, not a worker storm. Tests: existing tests pass (39 passed). The 1 failed + 3 errors are pre-existing on the base branch (TestSummarizeIssue cassette pins a totally different model; TestFixabilityScore needs a DB fixture missing in DEV=1) — confirmed on PR #22 branch with same outcome.
This was referenced May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Reopens #23 (auto-closed by GitHub when its base branch was deleted on PR #22 merge). Same diff, rebased onto current feature/explorer-endpoints.
Routes 6 high-volume / low-stakes call sites to gemini-3.1-flash-lite (the GA -lite variant, ~25% of full flash pricing):
Kept on full flash (3.5-flash) — comment_thread, semantic_search, explain_file, root_cause/solution fallbacks, codegen, assisted_query, app.py default.
Expected impact: summarize_issue is ~80% of flash spend; lite is ~25% of flash pricing → ~60% additional cut on the post-#22 baseline.
Risk: lite quality. PR #20's degrade paths catch any structured-output regressions.
Bugbot review pass: no findings (pure routing swap, no logic changes).