LLM Council

The idea of this repo is that instead of asking a question to your favorite LLM provider (e.g. OpenAI GPT 5.1, Google Gemini 3.0 Pro, Anthropic Claude Sonnet 4.5, xAI Grok 4, eg.c), you can group them into your "LLM Council". This repo is a simple, local web app that essentially looks like ChatGPT except it uses local LLMs via Ollama to send your query to multiple models, it then asks them to review and rank each other's work, and finally a Chairman LLM produces the final response.

In a bit more detail, here is what happens when you submit a query:

1. Stage 1: First opinions. The user query is given to all LLMs individually, and the responses are collected. The individual responses are shown in a "tab view", so that the user can inspect them all one by one.
2. Stage 2: Review. Each individual LLM is given the responses of the other LLMs. Under the hood, the LLM identities are anonymized so that the LLM can't play favorites when judging their outputs. The LLM is asked to rank them in accuracy and insight.
3. Stage 3: Final response. The designated Chairman of the LLM Council takes all of the model's responses and compiles them into a single final answer that is presented to the user.

Vibe Code Alert

This project was 99% vibe coded as a fun Saturday hack because I wanted to explore and evaluate a number of LLMs side by side in the process of reading books together with LLMs. It's nice and useful to see multiple responses side by side, and also the cross-opinions of all LLMs on each other's outputs. I'm not going to support it in any way, it's provided here as is for other people's inspiration and I don't intend to improve it. Code is ephemeral now and libraries are over, ask your LLM to change it in whatever way you like.

Setup

1. Install Ollama

First, install Ollama to run local LLMs:

# On Linux
curl -fsSL https://ollama.com/install.sh | sh

# On macOS
brew install ollama

# Or download from https://ollama.com/download

Start the Ollama server:

ollama serve

Pull some models for your council:

# Council members - pull 3-4 different models for variety
ollama pull llama3.2:3b        # Fast, lightweight
ollama pull mistral:7b         # Good reasoning
ollama pull qwen2.5:7b         # Strong analytical
ollama pull gemma2:9b          # Google's model

# Chairman model - for synthesis
ollama pull llama3.1:8b

2. Install Project Dependencies

The project uses uv for project management.

Backend:

uv sync

Frontend:

cd frontend
npm install
cd ..

3. Configure Models (Optional)

Edit backend/config.py to customize which local models to use:

COUNCIL_MODELS = [
    "llama3.2:3b",
    "mistral:7b",
    "qwen2.5:7b",
    "gemma2:9b",
]

CHAIRMAN_MODEL = "llama3.1:8b"

You can use any models you've pulled with ollama pull. List available models with ollama list.

Running the Application

Option 1: Use the start script

./start.sh

Option 2: Run manually

Terminal 1 (Backend):

uv run python -m backend.main

Terminal 2 (Frontend):

cd frontend
npm run dev

Then open http://localhost:5173 in your browser.

Tech Stack

• Backend: FastAPI (Python 3.10+), async httpx, Ollama API
• Frontend: React + Vite, react-markdown for rendering
• Storage: JSON files in data/conversations/
• LLM Runtime: Ollama (local inference)
• Package Management: uv for Python, npm for JavaScript