What LLM models are supported?

[scottconverse]

Q: The docs mention Gemma 4 as the recommended model. Can I use something else?

A: CivicRecords AI is model-agnostic. It works with any chat-completion model exposed through Ollama. As of v1.2.0 the installer presents a locked 4-model picker with gemma4:e4b as the default.

Supported Gemma 4 tags (Scott-verified against the Ollama registry 2026-04-21):

Tag	Class	Effective params	Disk	Min RAM (advisory)	Supportable @ 32 GB baseline
gemma4:e2b	Edge	2.3 B	7.2 GB	16 GB	✅ yes
gemma4:e4b (default)	Edge	4.5 B	9.6 GB	20 GB	✅ yes
gemma4:26b	Workstation · MoE	25.2 B total / 3.8 B active	18 GB	32 GB	⚠️ selectable with "stronger hardware" acknowledgement
gemma4:31b	Workstation · Dense	30.7 B	20 GB	40 GB	⚠️ selectable with "stronger hardware" acknowledgement

Correction: earlier answers in this thread mentioned gemma4:12b and gemma4:27b as installer defaults. Those tags don't exist in the Ollama registry and were purged repo-wide in T5C (part of v1.2.0). Don't try to ollama pull gemma4:12b — it will 404.

RAM values are advisory in the picker UI. The installer re-verifies empirically against the actual host RAM at install time; gemma4:26b / gemma4:31b require an explicit operator acknowledgement even when RAM is technically adequate.

Other models (non-Gemma, selectable manually):

Llama 3, Mistral, Qwen, and other Ollama-registry models work for search and synthesis. Swap models from the admin panel — no code change, no restart. The chat model is separate from the embedding model.

Embedding model. nomic-embed-text (auto-pulled by the installer alongside the selected chat model). It's required for semantic search and runs on minimal resources. Not switchable in the UI — if you want to change it, edit .env and restart the stack.

GPU acceleration. AMD: ROCm on Linux, DirectML via a host Ollama instance on Windows. NVIDIA: Ollama's native CUDA. CPU-only is a supported configuration and the locked target profile (32 GB baseline, GPU optional). See scripts/detect_hardware.sh / .ps1 for what the installer detects.

Authoritative source for tag/size re-verification: https://ollama.com/library/gemma4/tags — upstream tags and disk sizes can change.