Dears,
may i know please if it's possible to run ollama without GPU as i tried this by using below in helm chart
ollama:
models:
- "qwen3:1.7b"
gpu:
enabled: false
but i received below error at pod level
2025/06/25 13:00:03 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2025-06-25T13:00:03.810Z level=INFO source=images.go:753 msg="total blobs: 0"
time=2025-06-25T13:00:03.813Z level=INFO source=images.go:760 msg="total unused blobs removed: 0"
time=2025-06-25T13:00:03.817Z level=INFO source=routes.go:1200 msg="Listening on [::]:11434 (version 0.3.12)"
time=2025-06-25T13:00:03.818Z level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu cpu_avx cpu_avx2 cuda_v11 cuda_v12]"
time=2025-06-25T13:00:03.818Z level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
time=2025-06-25T13:00:03.821Z level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2025-06-25T13:00:03.821Z level=INFO source=types.go:107 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="7.5 GiB" available="5.2 GiB"
[GIN] 2025/06/25 - 13:00:03 | 200 | 37.185µs | 127.0.0.1 | HEAD "/"
[GIN] 2025/06/25 - 13:00:03 | 200 | 98.415µs | 127.0.0.1 | GET "/api/ps"
[GIN] 2025/06/25 - 13:00:03 | 200 | 20.774µs | 127.0.0.1 | HEAD "/"
[GIN] 2025/06/25 - 13:00:04 | 200 | 1.021242121s | 127.0.0.1 | POST "/api/pull"
Dears,
may i know please if it's possible to run ollama without GPU as i tried this by using below in helm chart
ollama:
models:
- "qwen3:1.7b"
gpu:
enabled: false
but i received below error at pod level
2025/06/25 13:00:03 routes.go:1153: INFO server config env="map[CUDA_VISIBLE_DEVICES: GPU_DEVICE_ORDINAL: HIP_VISIBLE_DEVICES: HSA_OVERRIDE_GFX_VERSION: HTTPS_PROXY: HTTP_PROXY: NO_PROXY: OLLAMA_DEBUG:false OLLAMA_FLASH_ATTENTION:false OLLAMA_GPU_OVERHEAD:0 OLLAMA_HOST:http://0.0.0.0:11434 OLLAMA_INTEL_GPU:false OLLAMA_KEEP_ALIVE:5m0s OLLAMA_LLM_LIBRARY: OLLAMA_LOAD_TIMEOUT:5m0s OLLAMA_MAX_LOADED_MODELS:0 OLLAMA_MAX_QUEUE:512 OLLAMA_MODELS:/root/.ollama/models OLLAMA_NOHISTORY:false OLLAMA_NOPRUNE:false OLLAMA_NUM_PARALLEL:0 OLLAMA_ORIGINS:[http://localhost https://localhost http://localhost:* https://localhost:* http://127.0.0.1 https://127.0.0.1 http://127.0.0.1:* https://127.0.0.1:* http://0.0.0.0 https://0.0.0.0 http://0.0.0.0:* https://0.0.0.0:* app://* file://* tauri://*] OLLAMA_SCHED_SPREAD:false OLLAMA_TMPDIR: ROCR_VISIBLE_DEVICES: http_proxy: https_proxy: no_proxy:]"
time=2025-06-25T13:00:03.810Z level=INFO source=images.go:753 msg="total blobs: 0"
time=2025-06-25T13:00:03.813Z level=INFO source=images.go:760 msg="total unused blobs removed: 0"
time=2025-06-25T13:00:03.817Z level=INFO source=routes.go:1200 msg="Listening on [::]:11434 (version 0.3.12)"
time=2025-06-25T13:00:03.818Z level=INFO source=common.go:49 msg="Dynamic LLM libraries" runners="[cpu cpu_avx cpu_avx2 cuda_v11 cuda_v12]"
time=2025-06-25T13:00:03.818Z level=INFO source=gpu.go:199 msg="looking for compatible GPUs"
time=2025-06-25T13:00:03.821Z level=INFO source=gpu.go:347 msg="no compatible GPUs were discovered"
time=2025-06-25T13:00:03.821Z level=INFO source=types.go:107 msg="inference compute" id=0 library=cpu variant=avx2 compute="" driver=0.0 name="" total="7.5 GiB" available="5.2 GiB"
[GIN] 2025/06/25 - 13:00:03 | 200 | 37.185µs | 127.0.0.1 | HEAD "/"
[GIN] 2025/06/25 - 13:00:03 | 200 | 98.415µs | 127.0.0.1 | GET "/api/ps"
[GIN] 2025/06/25 - 13:00:03 | 200 | 20.774µs | 127.0.0.1 | HEAD "/"
[GIN] 2025/06/25 - 13:00:04 | 200 | 1.021242121s | 127.0.0.1 | POST "/api/pull"