fix(controller): install prebuilt CUDA llama.cpp on Windows with staged, digest-verified swaps#213
Open
luminary19 wants to merge 6 commits into
Open
fix(controller): install prebuilt CUDA llama.cpp on Windows with staged, digest-verified swaps#213luminary19 wants to merge 6 commits into
luminary19 wants to merge 6 commits into
Conversation
…uda banner, staged swap, download timeouts)
…ed binary survives upgrades
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Windows prebuilt-CUDA install path for managed llama.cpp and makes
upgrades survive a locked running binary. On win32,
installManagedLlamacppnow queries the latest
ggml-org/llama.cpprelease, detects the driver's CUDAversion from the
nvidia-smibanner, picks the newest compatiblebin-win-cuda-*-x64build plus its matchingcudartruntime, downloads both,verifies each asset's
sha256digest, extracts with the bundledSystem32\tar.exe, and installs by staged rename swap: extract to a stagingdir, rename the old install aside, rename staging into place, then delete the
retired copy. Because the swap never writes over the live executable, an upgrade
while a model is loaded no longer fails on a locked
llama-server.exe; a still-locked retired copy is detected up front with a clear "stop it and retry"
message. Downloads and extraction are bounded by timeouts.
The driver-CUDA banner parser is shared with the wave-1
cuda-version.tsextractor (one regex,
CUDA (?:UMD )?Version) instead of a second copy. Confighelp now resolves through
resolveLlamaServerHelpBinary, which falls back to themanaged install location when
llama_binis unset and the binary is not onPATH, so the help panel works right after a managed install.
POSIX is untouched: non-win32 still requires
git/cmakeand builds fromsource exactly as before. Stacked on wave-1
cuda-umd-banner(#185); rebasedonto
main.Validation
bun run typecheckandbun run lintclean. The managed-llamacpp unit suite(13 tests — asset selection, CUDA-version compatibility picking, digest gating,
help-binary resolution) passes.
Verified live on Windows 11 (RTX 4070): a real managed install downloaded
llama-b9988-bin-win-cuda-13.3-x64.zip+ matching cudart, digest-verified,extracted, staged-swapped, and the installed
llama-server.exe --versionreported
version: 9988 (f4253ef96). With nollama_binset,resolveLlamaServerHelpBinarythen resolved that managed binary and--helpexited 0. Total install ~156 s over the network.The POSIX branch is unchanged, so Linux CI builds llama.cpp from source as
before.
UI changes
None (the existing Install/Upgrade and config-help flows now succeed on
Windows).
Risks / rollout notes
api.github.comand the release CDN; behind aproxy/offline box it fails with a clear error and the source-build /
LOCAL_STUDIO_LLAMA_BIN/LOCAL_STUDIO_LLAMACPP_UPGRADE_CMDescape hatchesstill apply.
the driver banner is unreadable it falls back to the lowest-CUDA build.
dir during the swap; the retired copy is deleted immediately after.
Fixes #212.