Switch to Mistral Small 4 as default LLM#168
Merged
Merged
Conversation
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
Co-authored-by: Copilot <copilot@github.com>
…-api into mistral-small-4 Collaborative fix
Collaborator
Author
|
Yes, I know, I would squash and merge... ;) |
Contributor
There was a problem hiding this comment.
Pull request overview
This PR switches the app’s primary LLM integration from Azure-backed chat models to Mistral, updating runtime wiring, agent construction, config, and supporting dependencies. It mainly affects the chat/tutor stack and the infra layer that brokers LLM calls.
Changes:
- Replaced Azure/LiteLLM-backed chat initialization with Mistral-backed clients for tutor, chat, and agent flows.
- Refactored agent execution to use newer LangChain/LangGraph agent APIs and added a chat history endpoint.
- Updated config, tests, deployment values/secrets, and Python dependencies to support the new Mistral setup.
Reviewed changes
Copilot reviewed 19 out of 20 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
src/app/tutor/service/tutor.py |
Swaps tutor model initialization from Azure chat to ChatMistralAI. |
src/app/tests/services/tutor/test_utils.py |
Updates tutor utility tests for async cases and current UploadFile construction. |
src/app/tests/services/test_llm_proxy.py |
Rewrites proxy unit tests around Mistral completions. |
src/app/tests/services/test_agent.py |
Removes tests tied to deleted conversation trimming logic. |
src/app/shared/infra/llm_proxy.py |
Replaces LiteLLM completion logic with direct Mistral client calls. |
src/app/shared/infra/abst_chat.py |
Migrates agent creation/execution to newer LangChain APIs and adds history retrieval. |
src/app/services/agent.py |
Removes the old message-trimming hook. |
src/app/models/chat.py |
Extends agent response payloads with thread_id. |
src/app/core/lifespan.py |
Initializes the app LLM proxy with Mistral settings at startup. |
src/app/core/config.py |
Adds Mistral env vars to settings. |
src/app/api/api_v1/endpoints/chat.py |
Adds chat history endpoint and thread-id handling for agent conversations. |
pytest.ini |
Adds Mistral test environment variables. |
pyproject.toml |
Upgrades LangChain/LangGraph stack and adds Mistral dependencies. |
poetry.lock |
Locks the updated dependency graph for the migration. |
k8s/welearn-api/values.yaml |
Adds default Mistral model config to deployment values. |
k8s/welearn-api/secrets.staging.yaml |
Adds staging Mistral API secret and refreshes SOPS metadata. |
k8s/welearn-api/secrets.prod.yaml |
Adds production Mistral API secret and refreshes SOPS metadata. |
k8s/welearn-api/secrets.dev.yaml |
Adds dev Mistral API secret and refreshes SOPS metadata. |
Dockerfile |
Upgrades pip before dependency installation. |
.env.example |
Documents new Mistral env vars and cleans obsolete examples. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+77
to
+79
| else: | ||
| # We assume that if it's not an Azure model, it's a Mistral model for now. This can be extended in the future to support other types of models. | ||
| return await self.mistral_completion(messages) |
Comment on lines
+39
to
+41
| # MISTRAL ENV VARS | ||
| MISTRAL_API_KEY: str | ||
| MISTRAL_LLM_MODEL_NAME: str |
Comment on lines
28
to
30
| yield | ||
| await app.state.qdrant.close() | ||
| await app.state.llm.close_client() |
| messages=[{"role": "user", "content": "Hello"}], | ||
| ) | ||
| self.assertIsInstance(response, str) | ||
| self.assertEqual(response, '{"key": "value"}') |
Comment on lines
+335
to
+351
| @router.get("/chat/history") | ||
| async def get_chat_history( | ||
| thread_id: UUID, | ||
| chatfactory=Depends(get_chat_service), | ||
| ) -> list[Dict[str, str | list[Dict[str, str]] | None]]: | ||
| if thread_id: | ||
| async with await psycopg.AsyncConnection.connect( | ||
| DB_URI, autocommit=True, prepare_threshold=0, row_factory=dict_row | ||
| ) as conn: | ||
| await conn.execute("SET SEARCH_PATH to agent_related") | ||
| await conn.commit() | ||
|
|
||
| memory = AsyncPostgresSaver(conn) | ||
| res = await chatfactory.agent_get_history( | ||
| thread_id=thread_id, memory=memory | ||
| ) | ||
| return res |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request migrates the backend from using Azure-based LLMs to Mistral models, updates related environment variables and configuration, and upgrades several dependencies to newer versions. It also removes unused code and adapts the agent and LLM proxy logic for the new Mistral integration. The most important changes are summarized below.
Mistral LLM Integration:
.env.example,k8s/welearn-api/secrets.dev.yaml,k8s/welearn-api/values.yaml, andpytest.inito addMISTRAL_API_KEYandMISTRAL_LLM_MODEL_NAMEvariables for Mistral LLM configuration; removed obsolete Azure Mistral Sweden variables. [1] [2] [3] [4]src/app/core/config.pyandsrc/app/core/lifespan.pyto use Mistral LLM as the default chat model, updating the initialization logic accordingly. [1] [2]src/app/shared/infra/llm_proxy.pyto support Mistral as a backend, adding a Mistral client and a new completion method, and removing the previous LiteLLM-based logic. [1] [2] [3] [4]Agent and Chat Logic Updates:
src/app/shared/infra/abst_chat.pyto uselangchain-mistralai'sChatMistralAI, switched fromcreate_react_agenttocreate_agent, and replaced message handling with the new format. [1] [2] [3]trim_conversation_historyfunction and its references fromsrc/app/services/agent.pyand related tests. [1] [2] [3]Dependency Upgrades:
langchain-core,langchain-azure-ai, andlanggraphto newer versions; addedlangchain-mistralaiandmistralaidependencies; removedhuggingface-hubandlangchain-huggingface. [1] [2] [3] [4]Testing and Miscellaneous:
src/app/tests/services/test_llm_proxy.pyto mock Mistral completions instead of LiteLLM, and removed obsolete test code.pip install --upgrade pipstep in theDockerfileto ensure the latest pip is used.Infrastructure and Housekeeping:
.env.example.These changes collectively modernize the backend to use Mistral LLMs, simplify agent logic, and keep dependencies up to date.