Llama.cpp cant start

[geomi737]

I installed odysseus in docker, downloaded cuda toolkit, did that command that sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker did this -> scripts/check-docker-gpu.sh --enable-nvidia-overlay, ensured that nvidia line appeared in .env, builded completely new instance via docker compose up -d --build and llama.cpp is still doesnt works

OS: CachyOS(Arch)

[odysseus] HF token: [redacted] SET — gated/private models will be denied. Add one in O
dysseus Cookbook -> Settings -> HuggingFace Token.
Native llama-server not found — building from source (one-time, may take a few m
inutes)...
[odysseus] No matching prebuilt llama-server for this host (arch=x86_64) — will
build from source.
[odysseus] Vulkan-capable GPU detected (no ROCm/CUDA toolchain installed) — buil
ding llama-server with Vulkan support...
-- The C compiler identification is GNU 14.2.0
-- The CXX compiler identification is GNU 14.2.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMAKE_BUILD_TYPE=Release
-- Found Git: /usr/bin/git (found version "2.47.3")
-- The ASM compiler identification is GNU
-- Found assembler: /usr/bin/cc
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Warning: ccache not found - consider installing it for faster compilation or
disable this warning with GGML_CCACHE=OFF
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- GGML_SYSTEM_ARCH: x86
-- Found OpenMP_C: -fopenmp (found version "4.5")
-- Found OpenMP_CXX: -fopenmp (found version "4.5")
-- Found OpenMP: TRUE (found version "4.5")
-- Including CPU backend
-- x86 detected
-- Adding CPU backend variant ggml-cpu: -march=native
CMake Error at /usr/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake
:233 (message):
  Could NOT find Vulkan (missing: Vulkan_LIBRARY Vulkan_INCLUDE_DIR glslc)
  (found version "")
Call Stack (most recent call first):
  /usr/share/cmake-3.31/Modules/FindPackageHandleStandardArgs.cmake:603 (_FPHSA_
FAILURE_MESSAGE)
  /usr/share/cmake-3.31/Modules/FindVulkan.cmake:595 (find_package_handle_standa
rd_args)
  ggml/src/ggml-vulkan/CMakeLists.txt:9 (find_package)


-- Configuring incomplete, errors occurred!
llama-server build failed — installing Python bindings as fallback...
WARNING: The directory '/app/.cache/pip' or its parent directory is not owned or
 is not writable by the current user. The cache has been disabled. Check the per
missions and owner of that directory. If executing pip with sudo, you should use
 sudo's -H flag.
ERROR: llama.cpp serving is not available after install/build attempts.

=== Process exited with code 127 ===

Maybe i dont know how to read but i dont have literally any clues about whats going on

Also it installs cpu version every time

[odysseus] HF token: [redacted] SET — gated/private models will be denied. Add one in O
dysseus Cookbook -> Settings -> HuggingFace Token.
Looking in indexes: https://pypi.org/simple, https://abetlen.github.io/llama-cpp
-python/whl/cpu
Requirement already satisfied: llama-cpp-python[server] in ./.local/lib/python3.
14/site-packages (0.3.34)

[joellara4577-blip]

I utilize claude to go into detail about these kind of errors but if i had to take a guess, i would say it has something to do with the error in vulcan and in the specific directory /app/ .cache/pip. something tells me you're either feeding it the wrong command for the cache and you may be missing dependencies in vulcan. that would be my guess. Honest guess and if i were you, i would load a lot of these into a different AI until you use it independently. best of luck!

[geomi737]

Solved it by adding Nvidia minimal-build into dockerfile. Why I need to do it myself?