A free Windows program that searches your own documents and answers questions about them — fully local, no cloud, no data ever leaving your machine. Under the hood it is a complete RAG system (Retrieval-Augmented Generation) with semantic + keyword search, a cross-encoder reranker and source citations.
Website & download: https://kreutzweb.de/ki-assistent.html (German · English)
Many teams — especially public authorities and companies with sensitive data — hold huge knowledge in tickets, manuals and documents, but cloud AI is off-limits for that data. The SWX KI-Assistent brings "ChatGPT for your internal documents" onto your own PC or server: after setup it runs entirely offline.
- Import your own data: PDF, Word, Excel, text/CSV/Markdown, OTRS ticket exports (JSON) — chunked and indexed locally (ChromaDB + bge-m3 embeddings)
- Hybrid search: semantic (by meaning) + BM25 keyword search, fused and re-ranked by a cross-encoder — rare technical terms and error codes are found
- Answers with sources: a local LLM (any GGUF model via llama.cpp) answers exclusively from the retrieved passages and cites ticket numbers / files; if nothing fits, it honestly says so
- Fast: answer cache (repeated questions answered instantly), switchable INT8/ONNX search backend for CPU-only servers, incremental re-imports (only new/changed entries are re-embedded)
- Practical: native Windows app (bilingual DE/EN) + web UI + HTTP API, setup wizard, knowledge spaces, runs standalone / as a network server / as a Windows service, optional direct OTRS ticket links
- Private by design: no cloud, no telemetry; suitable for air-gapped networks (full package ships all models for offline installation)
Two packages, both from the website:
| Package | Size | For |
|---|---|---|
| Full | ~3.8 GB | Offline/air-gapped servers — everything included |
| Lite | ~85 MB | Normal PCs with internet — models download on first start |
Unzip into any folder, run KI-Assistent.exe — a setup wizard guides you
through the rest. Requirements: Windows 10/11 or Server, ~8 CPU cores and
16 GB RAM recommended. A GPU is optional (CUDA/Vulkan accelerate).
- Import — files are split into meaningful sections and embedded locally
- Retrieve — for each question, the best sections are found (meaning + keywords) and re-ranked
- Generate — a local language model writes the answer from exactly those sections, with sources
The language model never sees your full document store and is never trained on your data — knowledge lives only in the local vector database.
Free for private individuals and small businesses under the PolyForm Small Business License 1.0.0 (source-available; free for organisations with fewer than 100 people and under USD 1M annual revenue — larger companies need a paid license via kreutzweb.de). Built on excellent open-source software: llama.cpp, ChromaDB, sentence-transformers, FastAPI, Streamlit and others (MIT/Apache/BSD — see LIZENZEN.txt in the package).
Deutsche Fassung: README.de.md