Skip to content

feat(adapter): Ollama (local) preset via /v1/chat/completions OpenAI-compat - #44

Merged
romer8 merged 1 commit into
mainfrom
feat/ollama-local-openai-compat-preset
May 28, 2026
Merged

feat(adapter): Ollama (local) preset via /v1/chat/completions OpenAI-compat#44
romer8 merged 1 commit into
mainfrom
feat/ollama-local-openai-compat-preset

Conversation

@romer8

@romer8 romer8 commented May 28, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds a new provider preset Ollama (local) that targets Ollama's OpenAI-compat endpoint (/v1/chat/completions) through the Django proxy. Fixes the multi-round tool-call regression observed against qwen3:30b-a3b-instruct-2507-q4_K_M on local Ollama >0.16.2:

Error: 400 json: cannot unmarshal object into Go struct field
.messages.tool_calls.function.arguments of type string

This is the architectural fix #23 explicitly named in its follow-up:

"the right fix is switch the Ollama adapter to use /v1/chat/completions (OpenAI-compat endpoint) instead of the native /api/chat, since the OpenAI-compat path has a stable, OpenAI-spec wire contract."

Implemented as a new preset rather than swapping the existing ollama provider so Ollama Cloud (which is happy on /api/chat) is untouched. Local Ollama participants pick "Ollama (local)" from the dropdown.

Why

Ollama's native /api/chat flipped the inbound shape of tool_calls[].function.arguments between versions — object on <=0.16.2, string on >0.16.2. PR #22 stringified universally → broke 0.16.2's chat-template parser → PR #23 reverted. The two versions can't be satisfied by a single static wire format. /v1/chat/completions follows OpenAI spec stably across all Ollama versions (arguments always stringified).

Changes

  • storage/llmProviderStorage.js — new ollama_local preset.
  • engine/adapters/ollamaLocal.js (new) — OpenAI SDK wrapper with baseURL = ${location.origin}/apps/tethysdash/ollama-proxy/v1 and defaultHeaders carrying x-csrftoken, x-ollama-host, x-ollama-key so the Django proxy can route to the user-configured Ollama host.
  • engine/index.jsollama_local → ollamaLocalStreamChat in PROVIDER_ADAPTERS.
  • helpers/index.jslistModels ollama_local branch reuses the existing /api/tags + /api/show discovery flow (same x-ollama-host routing).
  • components/LLMProviderPanel.jsx — show URL field for ollama_local alongside ollama/custom.
  • engine/adapters/ollamaLocal.test.js (new) — 9 tests including the regression catcher PR Revert "fix(engine): stringify tool_call arguments for Ollama /api/chat wire format (#22)" #23 named ("the test that would have caught this regression"): multi-round messages → second-round request's prior assistant tool_calls[0].function.arguments must be a string.

Requires

Companion controller in tethysapp-tethys_dash adding @controller(url="tethysdash/ollama-proxy/v1/chat/completions/") to proxy /v1/chat/completions upstream. Without it, this preset returns 404.

Test plan

  • npx vitest run engine/adapters/ollamaLocal.test.js → 9/9 passed
  • npx vitest run → 669/669 passed (no regressions)
  • npm run build → dist rebuilt clean
  • After the tethysdash companion lands, manual smoke against local Ollama with the original failing prompt: List available dates for cfe_nom model should now succeed and the assistant reaches the tool dispatch
  • Workshop image rebuild + republish → participants see the new preset in the dropdown

…compat

Adds a new provider preset "Ollama (local)" that targets Ollama's
OpenAI-compatibility endpoint (/v1/chat/completions) through the
Django proxy, fixing the multi-round tool-call regression on local
Ollama >0.16.2:

  Error: 400 json: cannot unmarshal object into Go struct field
  .messages.tool_calls.function.arguments of type string

Background: Ollama's native /api/chat flipped the inbound shape of
`tool_calls[].function.arguments` between versions — object on
<=0.16.2, string on >0.16.2. PR #22 tried to fix the newer shape by
stringifying universally, but PR #23 reverted it because that broke
0.16.2's chat-template parser. PR #23's follow-up note named the
right architectural fix: route Ollama chat through /v1/chat/completions
(OpenAI spec, stable wire contract).

Changes:

- storage/llmProviderStorage.js: new `ollama_local` preset, baseUrl
  default "http://localhost:11434", label "Ollama (local)".
- engine/adapters/ollamaLocal.js (new): OpenAI SDK wrapper with
  baseURL = `${location.origin}/apps/tethysdash/ollama-proxy/v1` and
  defaultHeaders carrying x-csrftoken, x-ollama-host, x-ollama-key so
  the Django proxy can route to the user-configured Ollama host.
- engine/index.js: route `ollama_local` → ollamaLocalStreamChat in
  PROVIDER_ADAPTERS.
- helpers/index.js listModels: `ollama_local` reuses the existing
  /api/tags + /api/show discovery via the same Django proxy headers —
  same x-ollama-host routing, no separate proxy needed.
- components/LLMProviderPanel.jsx: show the URL field for ollama_local
  alongside ollama/custom.
- engine/adapters/ollamaLocal.test.js (new): 9 tests covering the
  regression-catcher PR #23 explicitly recommended — second-round
  request must include prior assistant tool_calls with
  function.arguments as a JSON string. Plus header forwarding, SDK
  baseURL routing through the proxy (not direct to localhost:11434
  which would hit CORS), and the streaming contract.

Requires a companion controller in tethysapp-tethys_dash that adds
@controller(url="tethysdash/ollama-proxy/v1/chat/completions/") to
proxy /v1/chat/completions upstream. Without that controller this
preset returns 404.

Existing "Ollama Cloud" preset (provider=ollama, /api/chat) is
unchanged — Cloud still works via the OpenAI-incompatible native API
because the cloud version's wire format is stable.
@romer8
romer8 merged commit 35a5d02 into main May 28, 2026
2 checks passed
romer8 added a commit to tethysplatform/tethysapp-tethys_dash that referenced this pull request May 28, 2026
0.15.3 ships the new `ollama_local` provider preset
(Aquaveo/chatbox-core#44 merged 2026-05-28). The companion proxy
controller for /ollama-proxy/v1/chat/completions/ is in the previous
commit in this PR.

Without bumping the npm pin, the devcontainer base image (which the
workshop image layers on top of) ships with chatbox-core 0.15.2 and
the new "Ollama (local)" preset is missing from the chatbox provider
dropdown.
romer8 added a commit to tethysplatform/tethysapp-tethys_dash that referenced this pull request May 28, 2026
…-compat (#133)

* feat(controllers): proxy route for Ollama /v1/chat/completions OpenAI-compat

Companion to chatbox-core feat/ollama-local-openai-compat-preset. Adds
the Django proxy route the new "Ollama (local)" preset depends on:

    /apps/tethysdash/ollama-proxy/v1/chat/completions/
      → ${X-Ollama-Host}/v1/chat/completions

The new route mirrors the existing /ollama-proxy/api/{tags,show,chat}/
routes (same _proxy_to_ollama helper, same X-Ollama-Host/X-Ollama-Key
header contract), but forwards to Ollama's OpenAI-compatibility
endpoint instead of /api/chat.

Why a new endpoint instead of swapping /api/chat: newer Ollama versions
rejected the existing chat path's tool_call argument format mid-turn
("cannot unmarshal object into Go struct field ... of type string"),
while older versions reject the OpenAI-spec string format. The two
versions can't be satisfied by a single static wire format on the
native endpoint. The OpenAI-compat endpoint follows OpenAI spec stably
across versions. See chatbox-core PR #23 follow-up note for the
architectural rationale.

The Ollama Cloud preset (existing /ollama-proxy/api/chat/ route) is
unchanged — Cloud's wire format is stable.

Includes test_ollama_v1_chat_completions_proxy_forwards_to_openai_compat_path
asserting the upstream URL ends with /v1/chat/completions and respects
the X-Ollama-Host header.

* chore(devcontainer): bump @aquaveo/chatbox-core 0.15.2 → 0.15.3

0.15.3 ships the new `ollama_local` provider preset
(Aquaveo/chatbox-core#44 merged 2026-05-28). The companion proxy
controller for /ollama-proxy/v1/chat/completions/ is in the previous
commit in this PR.

Without bumping the npm pin, the devcontainer base image (which the
workshop image layers on top of) ships with chatbox-core 0.15.2 and
the new "Ollama (local)" preset is missing from the chatbox provider
dropdown.
romer8 added a commit to Aquaveo/ciroh_devcon_2026_workshop that referenced this pull request May 28, 2026
The v2026.05.5 image (published by publish-image.yml on tag push of
v2026.05.5) bakes tethysapp-tethys_dash @ tip of
feature/tethysdash-mcp-server, which carries:

  * @aquaveo/chatbox-core 0.15.3 — adds the "Ollama (local)"
    provider preset routing chat through Ollama's OpenAI-compat
    /v1/chat/completions endpoint (Aquaveo/chatbox-core#44).
  * New Django proxy controller /ollama-proxy/v1/chat/completions/
    forwarding to the same upstream path
    (tethysplatform/tethysapp-tethys_dash#133).

Together these fix the multi-round tool-call failure on local Ollama
>0.16.2 ("cannot unmarshal object into Go struct field
.messages.tool_calls.function.arguments of type string"). The
participant-facing change is one dropdown pick: select the new
"Ollama (local)" preset, URL http://localhost:11434.

Definitions.md updates both the provider description and the cheat
sheet row to reflect the preset name + URL (no /v1 suffix; the
preset appends it internally).

Participants run `bash scripts/update.sh` to pull the new image and
pick up the working preset.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant