Skip to content

Feat/ollama robustness - #7

Merged
AliMuhammadOfficial merged 8 commits into
mainfrom
feat/ollama-robustness
Jul 24, 2026
Merged

AliMuhammadOfficial merged 8 commits into
mainfrom
feat/ollama-robustness

Conversation

@AliMuhammadOfficial

Copy link
Copy Markdown
Member

Pull Request

Summary

The Ollama adapter spoke the native POST /api/chat endpoint but only forwarded messages, tools, and temperature. Verified against the official Ollama API documentation (docs.ollama.com), this PR closes every gap in the chat/stream surface:

  • Thinking: reasoning_effort maps to the native think parameter (minimal/low → "low", medium, high), a thinking_budget_tokens request enables think: true, and OllamaAdapter(think=...) sets an adapter default (including "max"). message.thinking returns as ThinkingBlock content and live thinking stream events.
  • Structured outputs: chat(..., response_format=...) translates OpenAI-style shapes to Ollama's format ("json" or a JSON schema), mirroring the OpenAI adapter's keyword.
  • Sampling options: max_tokens → options.num_predict; new default_options constructor passthrough for seed, top_p, top_k, num_ctx, and friends with per-call precedence; keep_alive forwarded.
  • Images fixed: image blocks were emitted as OpenAI image_url data-URIs, which the native endpoint silently ignores. They now go out as per-message base64 images arrays, so multimodal models (e.g. llava) actually receive the image.
  • Cloud auth: OLLAMA_API_KEY env or api_key= sends Authorization: Bearer for ollama.com cloud; local servers are unaffected.
  • Errors and stop reasons: mid-stream NDJSON {"error": ...} objects (delivered on an established HTTP 200) now surface as terminal error stream events instead of a silently truncated answer; HTTP 404 names the missing model with the exact ollama pull command; done_reason: "length" maps to stop_reason="max_tokens".
  • Corrected module/class docstrings that falsely claimed the OpenAI-compatible endpoint.

Change type

  • Bug fix
  • Backward-compatible feature
  • Documentation
  • Breaking change
  • Test or internal maintenance
  • Security-sensitive change handled through a private advisory

Compatibility

  • Version impact: minor
  • Python API: additive only — new keyword-only OllamaAdapter constructor parameters (api_key, keep_alive, think, default_options) and a new optional response_format keyword on chat(). Existing positional arguments and behavior unchanged. LLMChatOptions, the LLMAdapter protocol, CLI, YAML/TOML, checkpoints, durable run data: none.
  • Deprecation or migration path: none required.
  • Rollback or persisted-data impact: none — no persisted formats touched. The one observable payload change is the image-encoding fix, which replaces a shape the server ignored.

Verification

15 new offline tests in tests/test_providers.py (mocked httpx.AsyncClient, no network), including the previously untested stream() path: thinking events, tool-call accumulation, usage capture, mid-stream error stop, 404 guidance, Bearer-header presence/absence, options precedence, format translation for all three input shapes, and native image payload assertions.

  • Regression or behavior tests added or updated
  • uv run python -m pytest
  • uv run python -m ruff check .
  • uv run python -m ruff format --check src/
  • uv run python -m pyright
  • uv run python -m mkdocs build --strict (plus scripts/check_docs.py)

Documentation and release notes

  • Public docstrings and reference pages match the implementation
  • Guides, examples, README, and site_docs/llms.txt updated where needed (guides/providers.md gained an "Ollama: local and cloud" section; guides/reasoning-models.md no longer lists Ollama thinking as "Ignored" and documents the think-level mapping)
  • [Unreleased] changelog entry added, or no user-visible change
  • Security, configuration, or release documentation updated where needed (no changes required; the API key stays in env configuration per existing policy)

Reviewer notes

  • The mid-stream error contract matters most: Ollama reports model failures inside the 200 NDJSON stream, so clients that only check HTTP status silently lose output. The stream loop now checks every chunk for an error key before parsing.
  • 404 handling re-raises httpx.HTTPStatusError with the same request/response objects, so the resilience layer's retry classification is unchanged (404 stays terminal).
  • Deliberately out of scope: /api/tags model listing and /api/embed embeddings (the LLMAdapter protocol has no list/embed surface), and echoing thinking traces back into message history (Ollama has no signature requirement).
  • Payload construction was deduplicated into _build_payload (first commit) before any behavior change, so each subsequent commit is a small reviewable diff.

@AliMuhammadOfficial
AliMuhammadOfficial merged commit cc1a61a into main Jul 24, 2026
29 of 30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant