Skip to content

installation.md

Latest commit

 

History

History
70 lines (47 loc) · 1.35 KB

installation.md

File metadata and controls

70 lines (47 loc) · 1.35 KB

Installation

Docker Compose (Recommended)

Prerequisites

  • Docker and Docker Compose
  • NVIDIA GPU with CUDA support (recommended for faster inference)
  • At least 40GB disk space for LLM model storage

Setup

git clone https://github.com/dbcls/bsllmner-mk2.git
cd bsllmner-mk2

# Build and start containers
docker compose up -d --build

GPU Configuration

The compose.yml reserves all available NVIDIA GPUs by default. To restrict which GPUs are used, edit the CUDA_VISIBLE_DEVICES environment variable:

# compose.yml (ollama service)
environment:
  - CUDA_VISIBLE_DEVICES=0,1  # Use GPU 0 and 1

For Ollama performance tuning options, see Configuration - Ollama Performance Tuning.

uv (Local Development)

Prerequisites

  • Python 3.10+
  • uv
  • Ollama server running locally or remotely

Setup

uv sync

# Install with test/development dependencies
uv sync --all-extras

If the Ollama server is running on a different host, set the OLLAMA_HOST environment variable:

export OLLAMA_HOST=http://<ollama-host>:11434

Verify Installation

For Docker Compose:

# Check containers are running
docker compose ps

For uv (local):

# Check the CLI is available
uv run bsllmner2_extract --help