Skip to content

feat: add Ollama runtime support - #9

Draft
stamsam wants to merge 2 commits into
ammaarreshi:mainfrom
stamsam:codex/add-ollama-runtime-stacked
Draft

feat: add Ollama runtime support#9
stamsam wants to merge 2 commits into
ammaarreshi:mainfrom
stamsam:codex/add-ollama-runtime-stacked

Conversation

@stamsam

@stamsam stamsam commented Apr 29, 2026

Copy link
Copy Markdown

Summary

  • add an Ollama runtime that streams through Ollama’s OpenAI-compatible chat completions endpoint
  • add provider-aware model metadata and model picker labels
  • discover locally installed Ollama models and surface them in the app
  • document how to use Ollama models locally

Depends on #8 for the MLX port-collision fix. This PR is easiest to review after #8 lands; until then GitHub may show the shared port-selection commits in this diff.

Verification

@rickmanelius

Copy link
Copy Markdown

Damn it. Posted my review to the wrong area. See feedback on original issue #7 (comment)

@stamsam

stamsam commented Apr 30, 2026

Copy link
Copy Markdown
Author

Updated the draft with a provider-aware first-run setup flow:

  • first boot now preselects an installed Ollama model when one is detected
  • setup screen has explicit Ollama / MLX choices instead of a flat download-first model list
  • installed Ollama models are labeled as installed and shown before suggested Ollama models
  • MLX downloads remain available from the MLX tab

Re-verified with npm run typecheck and npm run build.

@stamsam stamsam changed the title feat: add Ollama runtime support feat: add oMLX and external runtime support Apr 30, 2026
@stamsam

stamsam commented Apr 30, 2026

Copy link
Copy Markdown
Author

Updated this draft to center oMLX instead of treating Ollama as the main external runtime:

  • renamed the backend integration to external OpenAI-compatible runtimes
  • added first-class oMLX discovery from http://127.0.0.1:8000/v1/models
  • added OMLX_HOST and OMLX_API_KEY support
  • first-run setup now has oMLX / MLX / Ollama choices
  • oMLX models discovered from the server can be selected directly without downloading managed MLX weights

I also built a local packaged app with npm run pack.

@stamsam

stamsam commented Apr 30, 2026

Copy link
Copy Markdown
Author

Added setup-time oMLX key entry and model refresh:

  • users can paste an oMLX API key in first-run setup
  • the key is applied in the main process immediately, without restarting
  • Refresh re-queries /v1/models and updates the model list
  • empty oMLX state now tells users to enter a key if needed and refresh

Rebuilt the local packaged app with npm run pack.

@stamsam
stamsam force-pushed the codex/add-ollama-runtime-stacked branch from c26c1c5 to 95b8820 Compare April 30, 2026 02:38
@stamsam stamsam changed the title feat: add oMLX and external runtime support feat: add Ollama runtime support Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants