Add an OpenLLM preset to the local server step of Connect models (localhost:8787)

[Jonathan (apewithllm)]

Summary

I'd like to add OpenLLM as a preset in the local server step of Connect models, next to llama.cpp, vLLM, llama-swap, LM Studio and mlx-lm. I'm opening this first, per CONTRIBUTING, to agree on the shape before sending a PR to dev.

What OpenLLM is

A local daemon with an OpenAI-compatible API at http://127.0.0.1:8787/v1. It serves a user's Claude, ChatGPT, Cursor, Kimi and Grok subscriptions, their provider API keys, and their local models through that one endpoint. GET /v1/models returns the usual { "object": "list", "data": [{ "id": ... }] } shape that your discovery expects. Fallback aliases (ultra, plus, lite) move to the next model in a chain when a provider is rate limited or down. The local daemon accepts requests with no Authorization header, and a Bearer key also works.

To be upfront, given the local-first charter: OpenLLM can forward requests to the hosted providers its user has connected, so the README's note about remote providers seeing prompts applies. The preset is opt-in and runs the same user-triggered Test connection probe as the others.

Proposed change

1. frontend/src/components/ConnectModels.tsx: add { id: 'openllm', name: 'OpenLLM', baseUrl: 'http://localhost:8787/v1' } to LOCAL_PRESETS. The flow already builds the definition from the preset and the probed /models list.
2. docs/35-PROVIDER_CONNECTIONS.md: a short OpenLLM section covering Test connection, the authless local default, and the Docker note (host.docker.internal instead of localhost).
3. A focused regression test, as AGENTS.md asks. I didn't find an existing test covering LOCAL_PRESETS, so let me know where you'd like it.

If you'd rather have it bundled like llama-cpp (plugins/openllm.json plus its trust-anchor hash and the regenerated capability inventory), I can do that instead. I think the preset is enough.

One thing I noticed: the preset flow caps the stored model_map at 50 models (.slice(0, 50)), and an OpenLLM account can list more (mine lists 72). Discovery refreshes the catalog later, so it's probably fine. I'll check it while testing.

I haven't run Libre WebUI against OpenLLM end to end yet. I'll do that, natively and in Docker, before opening the PR, and say exactly what I ran.

Disclosure: I'm a cofounder of OpenLLM.

[rob (rob-x-ai)]

Thanks for opening this first and for the disclosure.

I don't want to change Libre's code for this, so no preset and no bundled plugin. The local server step is meant for things that run models on your own machine, like llama.cpp, vLLM and LM Studio. OpenLLM mostly forwards to hosted providers and subscriptions, so listing it there would give people the wrong idea about where their prompts go. Bundled plugins are also ones we maintain and ship to everyone, which isn't a fit here.

What you can do is propose a plugin. Libre supports imported plugins, so an OpenLLM plugin JSON works without touching the codebase. plugins/llama-cpp.json is a good template: set base_url to http://localhost:8787/v1 and leave the auth header empty. Admins install it from Settings → Plugins, and you keep control of updates on your side. Feel free to post it here.