feat: Add multi-model provider support#1
Open
frannovo wants to merge 1 commit into
Open
Conversation
Replace hardcoded Gemini integration with a provider abstraction supporting Gemini, Claude (Anthropic), Grok (xAI), and Ollama. Provider is selected at startup via AI_PROVIDER env var. - Extract RewriteRequest model to src/models.py for clean imports - Add src/providers.py with lazy SDK imports and error handling - Make guardrails security validation lazy and optional (ENABLE_SECURITY) - Propagate provider API errors with original status codes - Validate OLLAMA_BASE_URL scheme and AI_PROVIDER value (fail fast) - Fix deprecated Guard.use_many call - Add tests for provider dispatch, error handling, and API endpoint Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> Signed-off-by: Fran Novo <fjnovo@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
AI_PROVIDERenvironment variableENABLE_SECURITYenv varMotivation
The service was hardcoded to Google Gemini, limiting flexibility. This change allows users to choose their preferred AI provider based on availability, cost, privacy (local models via Ollama), or preference.
Overview of changes
src/providers.py(new): Provider dispatch module with lazy SDK imports so missing optional dependencies don't crash unused providers. Each provider function catches SDK-specific errors and wraps them inProviderErrorwith the original status code.OLLAMA_BASE_URLis validated for scheme, andAI_PROVIDERis validated on every call (fail fast).src/models.py(new): ExtractedRewriteRequestPydantic model fromsecurity.pyto decouple request validation from guardrails dependencies.src/security.py: Lazy initialization of guardrails guard. Falls back to passthrough with a logged warning if hub validators are not installed. Replaced deprecatedGuard.use_manywithGuard.use.src/main.py: Usesgenerate()from providers module. CatchesProviderErrorand returns the provider's status code and message. Security validation gated byENABLE_SECURITY.pyproject.toml: Added optional dependency groups (claude,grok,ollama,all).tests/: Added 20 new test cases covering provider selection, validation, error propagation, API endpoint behavior, and security toggle.Test plan
uv run pytest -v-- 25 passed, 5 skipped (guardrails hub validators require auth + API key)GEMINI_API_KEY=... uv run pytest tests/test_security.py -v-- all 10 passedAI_PROVIDER=ollama ENABLE_SECURITY=falseconfirmed workingThis PR description was generated with AI assistance (Claude Opus 4.6).