feat(serve): qmd serve shared model server + RemoteQMD client (supersedes #511)

[brettdavies]

qmd serve: shared model server, continuing #511

Continuation note: Five commits on this branch are @jaylfc's work, cherry-picked with -x (preserving his Author: line and recording the source SHAs): three rebased from #511, plus two follow-ups he pushed as ready-to-pick branches on this PR thread: a /vsearch dimension-mismatch guard that returns a 409 with a rebuild hint instead of a raw sqlite-vec 500, and a test-mock fixture alignment that brings tsc --noEmit back to main's baseline. This PR rebases #511 onto current main, applies the changes he's been iterating on, and adds work on top: a store-layer routing fix and the completion of the embed-over-remote story (RemoteQMD no longer has to refuse embed; qmd serve handles it directly, and tests share the running daemon's models instead of allocating their own LlamaCpp on the GPU). Per @jaylfc's reply, this PR supersedes #511.

Why

Multiple QMD clients each cold-load the embed, generate, and rerank models into their own process. On a 16 GB device running three agents, that is 3× the model memory; on ARM64 / headless servers, each client has to compile node-llama-cpp against whatever GPU drivers are installed locally.

qmd serve centralises the three models behind an HTTP server. Clients (CLIs in LXC containers, agents on separate hosts) talk to a single warm instance via the RemoteQMD class introduced in this PR, activated by QMD_REMOTE_URL=http://host:7832. The remote client implements the same LLM interface as LlamaCpp, so it slots in transparently anywhere the SDK accepts an LLM.

Use cases:

• Multi-agent setups sharing one embedding server.
• LXC / Docker containers reaching a host-level GPU / NPU without compiling node-llama-cpp inside each container.
• ARM64 / headless servers: bypass local llama.cpp build entirely.
• ARM SBCs (Orange Pi RK3588 with NPU) where RAM is scarce.

Running this fork on an RTX 3090 Ti alongside an Ollama Gemma 4 26B; the low-vram combination (this PR plus the engine flag in #662, carried together in #927) makes the three-model fit possible.

What this PR adds

1. qmd serve subcommand: long-running HTTP server with two backends.
   • local (default): loads GGUF models via node-llama-cpp.
   • ollama: proxies to any Ollama-compatible REST endpoint (Ollama itself, rkllama on RK3588 NPU, etc).
2. RemoteQMD (src/remote-qmd.ts): LLM-interface implementation that POSTs to the server. Auto-activates via QMD_REMOTE_URL (or --remote-url <url> per-invocation). Fetches the server's actual model names from /health in the background at construction time so embed call sites tag vectors with the server's model, not DEFAULT_EMBED_MODEL.
3. Store-layer routing through getDefaultLLM so QMD_REMOTE_URL actually reaches RemoteQMD at every call site. Previously Store.llm and four call sites in src/store.ts hardcoded getDefaultLlamaCpp(), bypassing the polymorphic accessor one layer deeper. Invisible on cards with headroom (clients silently paid for two LLM backends); on a 24 GB GPU with ~21 GB held by Ollama, this surfaced as Failed to create any rerank context during rerank.
4. Embed-over-remote: the gate that previously refused qmd embed when QMD_REMOTE_URL was set is gone. LLM.embedBatch accepts options, LLM.ready?() exposes async warmup so RemoteQMD can populate model names from /health before generateEmbeddings reads them, and the server's /embed-batch handler threads options through to the local backend. chunkDocumentByTokens char-truncates by the actual chars-per-token ratio when the active LLM has no detokenize (RemoteQMD never had it), eliminating the last LlamaCpp-specific dependency in the embed flow.
5. Tests route through qmd serve when QMD_REMOTE_URL is set: a vitest setup file registers a RemoteQMD as the default LLM, so previously-GPU-bound integration tests share the daemon's resident models. The hybridQuery + session-rerank tests that failed under VRAM pressure pass cleanly via the daemon.

Endpoints

Endpoint	Method	Purpose
/embed, /embed-batch	POST	embedding (single + batch)
/rerank	POST	document reranking
/expand	POST	query expansion (lex/vec/hyde)
/tokenize	POST	token count
/health	GET	server status + loaded models
/status	GET	index health (doc counts, embedding status)
/collections	GET	collection list
/search?q=…	GET	FTS5 keyword search
/vsearch	POST	vector similarity search
/browse	GET	paginated chunk listing

Default bind is 127.0.0.1; --bind 0.0.0.0 exposes to network. Body size capped at 50 MB. Strict input validation on every endpoint.

Relationship to other open PRs

This PR is independent of the engine --low-vram PR (#662): both base directly on main and merge cleanly in either order. #662 adds lowVram to LlamaCppConfig, surfaces it as a global flag / env, and adds the two-pass on-demand reclaim path. The serve-side surface for that flag (help text, startup log tagging models resident vs on-demand, a warning when combined with --backend ollama) lives in the combined PR (#927) together with both stacks; that PR is the production shape for anyone running serve + low-vram on one GPU, and closes as redundant once #662 and this PR both land.

This PR also overlaps in problem space with #608 ("Daemon-aware CLI fast-path"). Both keep models warm across requests. The honest comparison:

	qmd mcp --http --daemon (+ #608)	qmd serve (this PR)
Protocol	MCP over Streamable HTTP	Plain REST
What it exposes	High-level tools: query, get, multi_get	Low-level model primitives + index endpoints
Who calls it	MCP agents + same-machine CLI (after #608)	RemoteQMD qmd clients (multi-host, LXC)
Default bind	localhost (PID-file discovery)	designed for 0.0.0.0
Backend swap	local node-llama-cpp only	local or ollama-compat proxy

A real question (open below) is whether these should remain two separate daemons or whether the model-primitive endpoints + RemoteQMD should land as an extension of the MCP daemon instead. The substance of @jaylfc's work (the primitive endpoints, the RemoteQMD client, the ollama proxy backend) survives either shape; only the top-level serve subcommand vs mcp --http --daemon choice changes.

This PR also shares a class name with #705 ("OpenAI-compatible remote embedding, reranking & query expansion"), and the two sit on orthogonal axes. #705 is the model-backend axis (where inference runs): its RemoteLLM + HybridLLM let a qmd process offload embedding / reranking / expansion to an OpenAI-compatible provider (vLLM, Ollama, OpenAI) while keeping local generation. This PR is the client axis (where qmd runs): RemoteQMD is a thin client that talks to a remote qmd serve over HTTP. They compose rather than compete: a RemoteQMD client points at a remote qmd serve, and that server chooses its model source: local node-llama-cpp, ollama-compat, or (with #705) an OpenAI-compatible backend via HybridLLM. Both build on the same getDefaultLLM / setDefaultLLM seam, which is exactly where #705's hybrid would slot in. To let both land without a class-name collision, this PR renames its client RemoteLLM → RemoteQMD (src/llm-remote.ts → src/remote-qmd.ts, matching the repo's QMDStore casing); #705 keeps RemoteLLM for the backend tier. RemoteQMD also leaves room to grow search / vsearch methods if the client later pulls the corpus, not just the models.

Architecture

This branch bases directly on current main: no engine commits, no dependency on #662.

Commits on this branch:

7fcc630  Brett   refactor(llm): rename RemoteLLM client to RemoteQMD
1932276  @jaylfc test: align mock LLM fixtures with the EmbeddingResult/RerankResult contracts
925dd52  @jaylfc fix(serve): return a clear 409 on /vsearch embedding-dimension mismatch
0f45dd8  Brett   test: route llm/mcp tests through qmd serve when QMD_REMOTE_URL is set
bcd93e4  Brett   feat(serve): wire embed through RemoteLLM end-to-end
a12944f  Brett   feat(llm): widen LLM interface so embed flows through any backend
873ad62  Brett   fix(remote): route store layer through getDefaultLLM so QMD_REMOTE_URL reaches RemoteLLM
b7f55c8  @jaylfc refactor: rename rkllama backend to ollama-compat — generic Ollama API naming
06b642c  @jaylfc feat(serve): add /vsearch endpoint for semantic vector search
933110f  @jaylfc feat: remote model server (qmd serve) and client support

The earlier commit subjects still say RemoteLLM because they predate the rename; 7fcc630 flips the class, interface, and src/llm-remote.ts → src/remote-qmd.ts in one move (see #705 below).

Design notes

Store-layer wiring through getDefaultLLM

The CLI sets setDefaultLLM(new RemoteQMD(...)) correctly, but the store layer was hardcoding getDefaultLlamaCpp() at every embed / expandQuery / rerank call site. The polymorphic accessor was being set and then bypassed one layer deeper. This PR:

• Routes the four store call sites through getDefaultLLM() and widens their llmOverride?: LlamaCpp params to LLM. Store.llm widens to LLM to match the documented "Can be LlamaCpp or RemoteQMD" comment.
• Guards cli/qmd.ts:getStore so it skips the eager setDefaultLlamaCpp(new LlamaCpp(...)) when QMD_REMOTE_URL is set. The --remote-url flag mirrors into the env so one source of truth gates the guard.
• Adds optional embedModelName / generateModelName / rerankModelName readonly accessors to the LLM interface so existing consumers keep their ?? DEFAULT_* fallbacks without casting.
• Replaces a buggy lazy require("./remote-qmd.js") in getDefaultLLM (which fails under vitest ESM resolution) with a static import. No runtime cycle: remote-qmd.ts only type-imports from llm.ts.

Widened LLM interface so embed flows through any backend

The last LlamaCpp-specific dependency in the embed flow was detokenize in chunkDocumentByTokens. That lifts in this PR:

• chunkDocumentByTokens switches from getDefaultLlamaCpp() to getDefaultLLM(). The pathological-single-line fallback that previously needed llm.detokenize(tokens.slice(...)) now char-truncates by the actual chars-per-token ratio when the active LLM has no detokenize (RemoteQMD never had it; dummy remote tokens couldn't have round-tripped anyway).
• LLMSessionManager, LLMSession, withLLMSessionForLlm, and getSessionManager widen from LlamaCpp to LLM. Pure type widening (LlamaCpp implements LLM), so no behavior change for local callers.
• LLM.embedBatch accepts EmbedOptions for parity with embed.
• New optional LLM.ready?() hook lets backends with async metadata (RemoteQMD fetching model names from /health) warm up before callers read embedModelName. generateEmbeddings awaits it so vectors get tagged with the server's actual embedding model, not DEFAULT_EMBED_MODEL.
• generateEmbeddings and maybeAdoptLegacyEmbeddingFingerprint drop the (previously overly defensive) requireLlamaCpp gate. Both already used only LLM-interface methods.

Tests routed through serve via vitest setup

Previous behavior: vitest.config.ts and src/test-preload.ts unset QMD_REMOTE_URL so developer shells could not accidentally route tests through a real daemon. Useful as a default but blocks the case this PR wants: tests that share serve's resident models instead of allocating their own LlamaCpp on the GPU. This PR:

• Adds src/test-setup-remote.ts: when QMD_REMOTE_URL is set, register a RemoteQMD as the default LLM via setDefaultLLM, so withLLMSession-driven tests route through serve.
• vitest.config.ts stops scrubbing QMD_REMOTE_URL. Unset (CI default) → hermetic local LlamaCpp; set → route through serve.
• test/llm.test.ts LlamaCpp Integration describe block now also skips when QMD_REMOTE_URL is set, since those tests bind directly to getDefaultLlamaCpp() and would still try to load local models even though serve owns the VRAM.

Tests

Engine-level concurrency and reclaim tests for the low-vram mode live in #662. Synthetic tests there exercise the two-pass reclaim against fake llama backends so they run without GPU; an opt-in real-GPU integration test gated by QMD_RECLAIM_INTEGRATION=1 exercises the reclaim path end-to-end.

On this branch:

• test/store-remote-llm.test.ts (8 cases): covers both llmOverride and getDefaultLLM() routing for expandQuery and rerank, the Store adapter routing through Store.llm, generateEmbeddings accepting a real RemoteQMD (no requireLlamaCpp gate), and chunkDocumentByTokens char-truncating when detokenize is absent. All pass under Node / vitest.
• Full suite on the dev box (QMD_REMOTE_URL unset): 1228 / 1229 pass. The one failure (mcp.test.ts content-type on the 2025-era no-session initialize path) reproduces identically on unmodified main, so it predates this branch.

tsc --noEmit clean.

Backwards compatibility

• New serve subcommand and RemoteQMD class. Purely additive.
• QMD_REMOTE_URL and --remote-url are new (renamed from QMD_SERVER / --server in @jaylfc's earlier draft per a naming discussion; see the rename commit for the rationale). Nothing in main reads them.
• The existing qmd query path is unchanged: with no QMD_REMOTE_URL set, behaviour is identical to today.
• LLM interface widening is additive: embedBatch gains an optional EmbedOptions parameter, ready?() is a new optional method, embedModelName / generateModelName / rerankModelName are new optional readonly accessors. Existing implementations (LlamaCpp) satisfy the wider interface without changes; existing consumers (no opt-in needed) are unaffected.

Things I deliberately did not do

• Auth. Bind defaults to 127.0.0.1; if you want network exposure (--bind 0.0.0.0) it is behind a firewall / Tailscale / etc. Adding token auth or mTLS is a separate concern.
• Unify with the MCP daemon. Real question worth maintainer input; see open questions below.
• Replace --backend rkllama with anything more specific. @jaylfc's rename to ollama-compat is the right generic name; specific NPU implementations like rkllama remain compatible via that interface.

Open questions for you

1. Should this and qmd mcp --http --daemon be one process or two? They are both long-running HTTP servers that hold models warm. The differences (MCP protocol vs REST, agent tools vs model primitives, localhost vs network) are real but orthogonal: one daemon could expose both surfaces. If you would prefer a single canonical daemon, the substance of this PR (primitive endpoints + RemoteQMD client + ollama proxy backend) could land as an extension of the MCP daemon instead of a new serve subcommand. Happy to reshape if that is the call.
2. /v1/ versioning, like Daemon-aware CLI fast-path: ~4× speedup for qmd query #608 introduced? This PR uses unversioned routes (/embed, /rerank, etc.) matching @jaylfc's original. If you want /v1/embed etc. for consistency with Daemon-aware CLI fast-path: ~4× speedup for qmd query #608's /v1/search, easy rename.
3. Authoring credit. Three commits are @jaylfc's verbatim with (cherry picked from commit …) trailers. If you would rather take feat: remote model server (qmd serve) for shared inference across clients #511 directly instead of this rebased + stacked version, I am fine to close this and re-open a smaller PR against feat: remote model server (qmd serve) for shared inference across clients #511's branch with just my additions.

Changelog

Added

• qmd serve subcommand: long-running HTTP server exposing embed, rerank, expand, tokenize, vsearch, and read-only index endpoints. Two backends (local for in-process node-llama-cpp; ollama for any Ollama-compatible REST endpoint).
• RemoteQMD client (src/remote-qmd.ts) implementing the LLM interface against a qmd serve instance, auto-activated by QMD_REMOTE_URL or --remote-url <url>.
• LLM.embedBatch accepts EmbedOptions and a new optional LLM.ready?() hook for backends with async metadata warm-up.
• Optional LLM.embedModelName / generateModelName / rerankModelName accessors so consumers can read the active backend's model identifiers without casting.

Changed

• Store-layer call sites (src/store.ts embed / expandQuery / rerank paths) route through getDefaultLLM() instead of hardcoding getDefaultLlamaCpp(). Store.llm widens to LLM to match the documented polymorphism.
• chunkDocumentByTokens no longer requires detokenize; it char-truncates by the actual chars-per-token ratio when the active LLM lacks the method. Eliminates the last LlamaCpp-specific dependency in the embed flow.
• generateEmbeddings and maybeAdoptLegacyEmbeddingFingerprint no longer go through requireLlamaCpp; they already used only LLM-interface methods.
• Vitest test setup registers RemoteQMD as the default LLM when QMD_REMOTE_URL is set, so integration tests share the running daemon's models instead of allocating their own LlamaCpp on the GPU.

Type of Change

• feat: New feature (non-breaking change which adds functionality)
• fix: Bug fix
• BREAKING CHANGE: Breaking API change

Related Issues/Stories

• Story: n/a
• Issue: n/a
• Architecture: n/a
• Related PRs: continues feat: remote model server (qmd serve) for shared inference across clients #511; independent of feat(llm): --low-vram mode with on-demand reclaim on embed VRAM pressure #662 (the engine --low-vram machinery) since the split; the combined shape lives in feat: low-vram engine + qmd serve combined (production shape of #662 + #663) #927; overlaps in problem space with Daemon-aware CLI fast-path: ~4× speedup for qmd query #608 (compared above).

Testing

• Unit tests added/updated
• Integration tests added/updated
• Manual testing completed
• All tests passing

Files Modified

Modified:

• src/cli/qmd.ts: qmd serve subcommand wiring, --remote-url flag mirroring into env, guarded eager setDefaultLlamaCpp when QMD_REMOTE_URL is set.
• src/llm.ts: LLM interface widening (embedBatch options, ready?(), optional model-name accessors). getDefaultLLM now uses static import for remote-qmd.js.
• src/store.ts: embed / expandQuery / rerank call sites route through getDefaultLLM(); Store.llm widens to LLM. chunkDocumentByTokens switches to getDefaultLLM() with char-truncation fallback. generateEmbeddings and maybeAdoptLegacyEmbeddingFingerprint drop the requireLlamaCpp gate.
• vitest.config.ts: stops scrubbing QMD_REMOTE_URL.

Created:

• src/serve.ts: HTTP server entry, request validation, backend dispatch (LocalBackend and OllamaCompatBackend adapter classes live here).
• src/remote-qmd.ts: RemoteQMD client.
• src/test-setup-remote.ts: vitest setup that registers RemoteQMD as the default LLM when QMD_REMOTE_URL is set.
• test/store-remote-llm.test.ts: 8 cases covering the store-layer routing and RemoteQMD accept-paths.

Renamed:

• None at the net diff (src/remote-qmd.ts is net-new). Two renames happen inside the branch history (rkllama → ollama-compat in the backend module, and the client src/llm-remote.ts → src/remote-qmd.ts), but neither target exists in base, so both land as created, not renamed.

Deleted:

• None.

Breaking Changes

• No breaking changes.

Deployment Notes

• No special deployment steps required. qmd serve is opt-in; existing qmd query workflows are unchanged with QMD_REMOTE_URL unset.

[jaylfc]

Thanks for picking this up Brett, and for the care with the credit. The rebase is cleaner than my original #511, and the embed-over-remote work closes the gap I never got around to finishing.

On your open questions:

Which PR / credit (3): please take this one and close #511. The stacked version is strictly better than what I had, no need to reshape it back onto my branch.

One daemon or two (1): I'd keep them separate. They've drifted into genuinely different jobs in practice: the MCP daemon is the agent-facing tool surface, serve is the raw model + index primitive layer that other processes embed against. Folding the primitives into the MCP daemon would mean every RemoteLLM client drags in the MCP protocol surface just to reach an embed call. Two small servers with one clear job each has been easier to reason about on our side than one server wearing both hats.

Versioning (2): no strong feeling. Happy to match whatever you settle on for #608, including /v1/ across the board if that is the direction.

One thing I can offer that isn't on the branch yet: a small guard on /vsearch for the dimension-mismatch case. When an index was built with a different embed model than the server is currently running, sqlite-vec throws a raw "Dimension mismatch" that surfaces as an opaque 500. I catch it around the searchVec call and return a 409 with the query model + dim and a rebuild hint, so the operator immediately sees "rebuild the index or wrong model configured" instead of debugging a 500. About 30 lines. Happy to push it onto this branch or send it as a follow-up once this lands, whichever you prefer.

For what it's worth, we have been running this in production for a few weeks: serve with the ollama-compat backend in front of an NPU on an ARM board, a few agents in containers sharing one warm instance via RemoteLLM. The multi-host, scarce-RAM case in your "why" is exactly what we use it for and it has held up well. Good to see it heading upstream.

[jaylfc]

Follow-up on the dim-guard offer: rather than make you wait, I rebased it onto this branch's head and pushed it ready to pull: https://github.com/jaylfc/qmd/tree/dim-guard-on-663 (one commit, 01cb340, applies clean on 3a7718d). tsc --noEmit is error-neutral against your head (same 57 pre-existing test-file errors with and without it). Cherry-pick it in if you want it on this PR, or ignore it and I'll send it as a follow-up once this lands. Either way works for me.

[jaylfc]

One more ready branch while I was in there: this PR's head carries 6 new tsc --noEmit errors over main's baseline (51 -> 57), all type-shape drift in the new test mocks (EmbeddingResult.model, RerankDocumentResult.index, RerankResult.model, and two cast signatures in the low-vram test). Fixed in one commit, tests still 31/31: https://github.com/jaylfc/qmd/tree/fix-663-test-types (81d2e29, applies clean on 3a7718d, brings the branch back to main's baseline). Same deal as the dim-guard: cherry-pick if useful, ignore if not.

[brettdavies]

Hi @jaylfc, thanks for both of the commits. Pushing them as ready-to-pick branches instead of just describing them saved real time. 663 PR body updated to reflect their addition. Appreciate you confirming that #511 can be closed in favor of this pr. Great to hear that you've been running it in production for a few weeks. Hard to ask for better signal than that.

[brettdavies]

Rebased onto current main (v2.6.3). The #662 lowVram engine commits dropped on their own, since git's patch-id caught them as already-applied.

Renamed the bundled client RemoteLLM → RemoteQMD (src/llm-remote.ts → src/remote-qmd.ts). #705 introduces its own RemoteLLM for the model-backend tier (qmd → OpenAI-compatible provider); this one is the client tier (qmd → remote qmd serve). Two different things that both wanted the name, so ours moves. Left getDefaultLLM/setDefaultLLM shared, since that's the seam #705's HybridLLM would plug into as a backend. Full reasoning in the "Relationship to other open PRs" section.

One test fix while rebasing: main's new _ciMode guard throws from embed/rerank under CI, which tripped the low-vram reclaim tests (they mock the *Impl methods, so nothing real loads). Isolated the suite from ambient CI/QMD_LOW_VRAM so it runs under CI again. Low-vram suite and the full store suite pass under CI=true locally.

[soundadam]

I tested this head with Node 22 / pnpm 10: TypeScript passes and the focused suite passes (218 passed, 13 skipped).

Requesting changes for the HTTP security boundary:

• serve.ts sets Access-Control-Allow-Origin: * for every response.
• All index and inference routes are unauthenticated, including /browse, /search, /collections, /vsearch, /embed, /rerank, and /expand.
• /browse exposes document paths and snippets. With wildcard CORS, a hostile web page can read a QMD instance bound to 127.0.0.1; binding to a LAN address additionally exposes it to unauthenticated network clients.

Please make CORS deny-by-default (or explicit trusted origins), require authentication for every non-health route, and refuse non-loopback binding unless authentication is configured or a clearly named unsafe override is supplied. Add browser-origin, missing-token, invalid-token, and non-loopback-negative tests.

The shared-daemon/client direction is useful, but this boundary needs to fail closed before merge.

[brettdavies]

Thanks for the careful pass @soundadam. The security boundary is real: serve.ts sets wildcard CORS on every response,
the index and inference routes are unauthenticated, and the CLI default bind is 0.0.0.0, so out of the box a serve
instance is reachable across the LAN with /browse handing back document paths and snippets.

I scoped this pr to the shared model server and the client path, so I'd rather keep the auth/CORS
hardening as its own concern than grow this one. @tobi, is fail-closed auth a merge blocker for qmd serve, or would
you rather land this and take the hardening as a fast-follow? Happy either way. Just want your call on sequencing before
I build it.

If it's required, the minimal shape I'd propose (no new deps, keeps the loopback dev path zero-config):

1. Default the CLI bind to 127.0.0.1 instead of 0.0.0.0.
2. Optional shared bearer token (QMD_SERVE_TOKEN / --token); when set, require it on every route except /health.
3. Refuse to bind a non-loopback address unless a token is set or --allow-unauthenticated is passed explicitly.
4. CORS deny-by-default: echo Access-Control-Allow-Origin only for origins in an allowlist (--allow-origin), nothing
   otherwise. Plus tests for browser-origin, missing/invalid token, and the non-loopback refusal.

That leaves TLS, multi-user, and rate limiting out, and keeps the MCP --http server (same binding smell) as a separate
follow-up. I can fold it into this pr or ship it right after this lands.

[brettdavies]

@jaylfc thinking about splitting the low-vram engine (#662) back out of #663 so each lands single-concern, the way you
scoped them in #511. Before I move anything: in prod are you actually running the local --low-vram mode, or is it
serve + ollama end to end? If you're leaning on both, I'll file the combined shape (same as #663 today) as its own PR
for you to move to, so you're not stranded on a branch I've rewritten.

[brettdavies]

Split executed, as proposed above. This PR is now serve-only: the #662 engine commits are gone from the branch, along with the --low-vram serve surface (help text, startup log tags, ollama warning) and the low-vram test-isolation commit. What remains is the shared model server and client path: qmd serve, RemoteQMD, the ollama-compat backend, store-layer routing through getDefaultLLM, and embed-over-remote. 10 commits, based directly on current main: no dependency on #662 in either direction. The two PRs now merge cleanly in either order.

The combined shape (engine + serve + the integration surface, i.e. this PR as it stood before the split) lives in #927 as promised. @jaylfc, that branch is the drop-in for production; this one no longer carries the engine.

Also rebased onto current main (post-v2.8.3). Integration notes from the rebase:

• getStore keeps main's trust gate on project-local model URIs (fix(trust): gate project-local collection paths and model URIs #892) and its store.llm assignment, both inside the new QMD_REMOTE_URL guard: with a remote URL set, no local LlamaCpp is constructed and store.llm stays unset, so the store layer falls through to getDefaultLLM()'s remote client.
• searchVec gained an llm parameter on main (fix(store): embed searchVec queries with the store's pinned model #863); it widens from LlamaCpp to LLM here, matching the rest of the store layer.
• chunkDocumentByTokens keeps main's surrogate-pair stripping (fix(store): don't split UTF-16 surrogate pairs at chunk boundaries (#777) #855) on top of the char-truncation fallback for backends without detokenize.
• expandQuery call sites follow main's intent-removal (fix(store): keep caller intent out of query expansion (#818) #870): intent stays out of expansion and its cache key, and still reaches the reranker prefix.

@jaylfc's five commits carry through with authorship intact.

[jaylfc]

@brettdavies sorry for the slow answer on your July question — it deserved a faster one.

Prod shape on my side: serve + ollama-compat end to end. qmd serve --backend ollama fronting rkllama on an RK3588 NPU box, with the clients (CLIs in LXC containers plus agents on other hosts) going through QMD_REMOTE_URL / RemoteQMD. I'm not running the local --low-vram mode in production — that combination was my GPU dev box scenario, so don't let it block anything on my account.

Given that, the split you executed is exactly right: #663 serve-only and #662 engine-only, independently mergeable, with #927 as the tracked combined branch. That matches how I scoped #511 originally, and the serve-only shape is the one I depend on.

On your open question 3 (credit): the cherry-picks with -x and preserved Author lines are exactly what I'd want — no need to close this and re-stack on #511's branch. The RemoteLLM → RemoteQMD rename to make room for #705 also looks right to me; the two compose cleanly on the getDefaultLLM seam as you described.

Happy to test the serve-only branch against my rkllama setup if that helps it land.