Repository navigation
Feat/ollama robustness - #7
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Pull Request
Summary
The Ollama adapter spoke the native
POST /api/chatendpoint but only forwarded messages, tools, and temperature. Verified against the official Ollama API documentation (docs.ollama.com), this PR closes every gap in the chat/stream surface:reasoning_effortmaps to the nativethinkparameter (minimal/low→"low",medium,high), athinking_budget_tokensrequest enablesthink: true, andOllamaAdapter(think=...)sets an adapter default (including"max").message.thinkingreturns asThinkingBlockcontent and livethinkingstream events.chat(..., response_format=...)translates OpenAI-style shapes to Ollama'sformat("json"or a JSON schema), mirroring the OpenAI adapter's keyword.max_tokens→options.num_predict; newdefault_optionsconstructor passthrough forseed,top_p,top_k,num_ctx, and friends with per-call precedence;keep_aliveforwarded.image_urldata-URIs, which the native endpoint silently ignores. They now go out as per-message base64imagesarrays, so multimodal models (e.g.llava) actually receive the image.OLLAMA_API_KEYenv orapi_key=sendsAuthorization: Bearerfor ollama.com cloud; local servers are unaffected.{"error": ...}objects (delivered on an established HTTP 200) now surface as terminalerrorstream events instead of a silently truncated answer; HTTP 404 names the missing model with the exactollama pullcommand;done_reason: "length"maps tostop_reason="max_tokens".Change type
Compatibility
OllamaAdapterconstructor parameters (api_key,keep_alive,think,default_options) and a new optionalresponse_formatkeyword onchat(). Existing positional arguments and behavior unchanged.LLMChatOptions, theLLMAdapterprotocol, CLI, YAML/TOML, checkpoints, durable run data: none.Verification
15 new offline tests in
tests/test_providers.py(mockedhttpx.AsyncClient, no network), including the previously untestedstream()path: thinking events, tool-call accumulation, usage capture, mid-stream error stop, 404 guidance, Bearer-header presence/absence, options precedence,formattranslation for all three input shapes, and native image payload assertions.uv run python -m pytestuv run python -m ruff check .uv run python -m ruff format --check src/uv run python -m pyrightuv run python -m mkdocs build --strict(plusscripts/check_docs.py)Documentation and release notes
site_docs/llms.txtupdated where needed (guides/providers.mdgained an "Ollama: local and cloud" section;guides/reasoning-models.mdno longer lists Ollama thinking as "Ignored" and documents the think-level mapping)[Unreleased]changelog entry added, or no user-visible changeReviewer notes
errorkey before parsing.httpx.HTTPStatusErrorwith the same request/response objects, so the resilience layer's retry classification is unchanged (404 stays terminal)./api/tagsmodel listing and/api/embedembeddings (theLLMAdapterprotocol has no list/embed surface), and echoing thinking traces back into message history (Ollama has no signature requirement)._build_payload(first commit) before any behavior change, so each subsequent commit is a small reviewable diff.