Skip to content

fix(controller): install prebuilt CUDA llama.cpp on Windows with staged, digest-verified swaps#213

Open
luminary19 wants to merge 6 commits into
sybil-solutions:mainfrom
luminary19:up/llamacpp-install
Open

fix(controller): install prebuilt CUDA llama.cpp on Windows with staged, digest-verified swaps#213
luminary19 wants to merge 6 commits into
sybil-solutions:mainfrom
luminary19:up/llamacpp-install

Conversation

@luminary19

Copy link
Copy Markdown
Contributor

Summary

Adds a Windows prebuilt-CUDA install path for managed llama.cpp and makes
upgrades survive a locked running binary. On win32, installManagedLlamacpp
now queries the latest ggml-org/llama.cpp release, detects the driver's CUDA
version from the nvidia-smi banner, picks the newest compatible
bin-win-cuda-*-x64 build plus its matching cudart runtime, downloads both,
verifies each asset's sha256 digest, extracts with the bundled
System32\tar.exe, and installs by staged rename swap: extract to a staging
dir, rename the old install aside, rename staging into place, then delete the
retired copy. Because the swap never writes over the live executable, an upgrade
while a model is loaded no longer fails on a locked llama-server.exe; a still
-locked retired copy is detected up front with a clear "stop it and retry"
message. Downloads and extraction are bounded by timeouts.

The driver-CUDA banner parser is shared with the wave-1 cuda-version.ts
extractor (one regex, CUDA (?:UMD )?Version) instead of a second copy. Config
help now resolves through resolveLlamaServerHelpBinary, which falls back to the
managed install location when llama_bin is unset and the binary is not on
PATH, so the help panel works right after a managed install.

POSIX is untouched: non-win32 still requires git/cmake and builds from
source exactly as before. Stacked on wave-1 cuda-umd-banner (#185); rebased
onto main.

Validation

bun run typecheck and bun run lint clean. The managed-llamacpp unit suite
(13 tests — asset selection, CUDA-version compatibility picking, digest gating,
help-binary resolution) passes.

Verified live on Windows 11 (RTX 4070): a real managed install downloaded
llama-b9988-bin-win-cuda-13.3-x64.zip + matching cudart, digest-verified,
extracted, staged-swapped, and the installed llama-server.exe --version
reported version: 9988 (f4253ef96). With no llama_bin set,
resolveLlamaServerHelpBinary then resolved that managed binary and
--help exited 0. Total install ~156 s over the network.

The POSIX branch is unchanged, so Linux CI builds llama.cpp from source as
before.

UI changes

None (the existing Install/Upgrade and config-help flows now succeed on
Windows).

Risks / rollout notes

  • Win32 install fetches from api.github.com and the release CDN; behind a
    proxy/offline box it fails with a clear error and the source-build /
    LOCAL_STUDIO_LLAMA_BIN / LOCAL_STUDIO_LLAMACPP_UPGRADE_CMD escape hatches
    still apply.
  • Picks the newest CUDA build at or below the detected driver version; if
    the driver banner is unreadable it falls back to the lowest-CUDA build.
  • The staged swap needs room for one extra copy of the install under the data
    dir during the swap; the retired copy is deleted immediately after.

Fixes #212.

@luminary19
luminary19 requested a review from 0xSero as a code owner July 13, 2026 19:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] Managed llama.cpp install is build-from-source only — no Windows prebuilt path, and upgrades fail on a locked running binary

1 participant