WhatIff

An open-source, self-hostable AI assistant where your continuity and memory are not tied to a single model provider.

WhatIff keeps the persistent AI state in your stack: conversations, long-term memory, personalities, tools, and scheduled agents. The model that generates the next response is interchangeable, so you can move between supported providers without making any one of them the owner of your AI's continuity.

---

WhatIff is the open-source core that powers whatiff.chat, a Go API server and an Angular web app you can run yourself. It owns the continuity layer independently of the model provider: semantic memory, personalities, conversation state, tools, and recurring work stay with the WhatIff deployment while supported models can be selected per user and per chat. Bring your own model API keys and host the whole thing on your own infrastructure.

Want the story behind it? Read the Imagination Foundry blog.

✨ Features

🤖 Multi-provider chat

• Chat with models from OpenAI, Anthropic, Google (Gemini), Z.ai (GLM), Mistral, DeepSeek, Qwen, and Xiaomi — configurable per user and per chat
• Streaming responses with Markdown rendering and an emoji picker
• Multiple chat sessions with sidebar navigation, inline renaming, and per-chat settings

🧠 Semantic memory

• Long-term memory stored as vector embeddings (pgvector) and recalled by conversation context
• Filter and search memories by date, scope, chat, and content
• Memory merging and compaction with checkpoints so context stays coherent over time

🎭 Personalities

• Define reusable AI personas with custom system prompts and behavior
• Switch personalities per chat or set a default; attach reference documents to a persona
• Full create/edit/organize UI

🔮 Rituals

• Reusable prompt templates you can invoke with customizable keyboard shortcuts
• Bind a ritual to a personality and auto-enrich messages with its context

🛠️ Tools & agents

• File attachments with 30+ supported types, plus semantic search over uploaded documents
• Code interpreter, image generation, and web search available to the assistant
• Agents — scheduled and background AI jobs with a distributed-lock-aware scheduler
• AI-generated files saved straight back into the conversation

🔗 Integrations

• Remote MCP connectors with per-chat activation controls
• Webhook API tokens (stored as one-way hashes) with static bearer auth isolated from session JWTs, each limited to the scopes it was created with (messages:write, chat:read)
• Webhook trigger modes for existing chats: user, assistant (write-only), and background (async agent trigger)
• Webhook read routes (scope chat:read): threads (filterable by persona), persona names, messages, and job status; see the webhooks guide and the integration reference

🔧 Built for operators

• OpenAPI specification and structured logging (Zap)
• Ent ORM with automatic, non-destructive database migrations
• Docker & Docker Compose for local development
• GitHub Actions CI

🏗️ Tech stack

Layer	Technologies
Backend	Go 1.27 · Gorilla Mux · Ent ORM · PostgreSQL 16 + pgvector · JWT auth · Zap
Frontend	Angular 22 · TypeScript 5 · Tailwind CSS 4 · RxJS · ngx-markdown
AI	OpenAI, Anthropic, Google, Z.ai, Mistral, DeepSeek, Qwen, Xiaomi (bring your own keys)
Infra	Docker & Docker Compose · GitHub Actions

🚀 Quick start (Docker)

The fastest way to run the full stack locally is Docker Compose.

Prerequisites: Docker, and an OpenAI API key, which you add in the app.

# Start the stack. On first run it generates this install's secrets, runs
# migrations, and seeds the model catalog.
docker compose up --build

Then open http://localhost:8081, create an account, and add your OpenAI key when the app asks (it lives in Profile & Settings → API keys, where you can add other providers' keys too). Keys belong to your account and are stored encrypted; there is nothing to put in .env.

The JWT and token-encryption secrets are generated once per install and kept in a Docker volume; nothing is shipped in the repository. Values you set in your shell or .env take precedence. docker compose down -v deletes them along with the database.

Service	URL
Frontend	http://localhost:8081
API	http://localhost:8080
PostgreSQL	localhost:5432

Compose is for local development only. Its defaults (DESTRUCTIVE_MIGRATION=false, DB_SSL_MODE=disable, no secret fallbacks) are deliberately conservative — every deployed environment must set its own real secrets and connection settings explicitly.

🧑‍💻 Local development

Run the backend and frontend directly for a faster inner loop.

Backend (Go 1.27+, a running PostgreSQL 16 with pgvector):

go run ./cmd/api-server
# Connects to Postgres, enables pgvector, runs non-destructive migrations
# (AUTO_MIGRATE=true), seeds a default model, and serves on :8080.

Frontend (Node.js 20+):

cd web/app
npm install
npm start          # Angular dev server on :4200, proxying the API on :8080

🔧 Configuration

Configuration is via environment variables. The essentials:

Variable	Purpose
OPENAI_API_KEY and other provider keys	Not used by default: each account adds its own keys in the app. A deployment that supplies keys for everyone binds providerkeys.New to these instead
JWT_SECRET, JWT_REFRESH_SECRET	JWT access/refresh token signing (≥32 chars). Generated automatically under Docker Compose
TOKEN_ENCRYPTION_SECRET	Encryption key for at-rest secrets (≥32 chars). Generated automatically under Docker Compose
DB_HOST, DB_PORT, DB_USER, DB_PASSWORD, DB_NAME	PostgreSQL connection
SERVER_HOST, SERVER_PORT	API bind address (default localhost:8080)
ENVIRONMENT	development | staging | production
AUTO_MIGRATE	Run migrations on boot (true/false)
DESTRUCTIVE_MIGRATION	Allow dropping columns/indexes during auto-migrate — keep false in deployed envs

See .env.example and docker-compose.yml for the full set.

📁 Project structure

what-iff/
├── cmd/api-server/       # Application entry point
├── internal/
│   ├── agent/            # Tool-using AI agent, tools, and file processing
│   ├── agentjobs/        # Scheduled & background agent jobs (scheduler)
│   ├── auth/             # JWT, password hashing, token utilities
│   ├── datastore/        # Database operations (repository pattern over Ent)
│   ├── handlers/         # HTTP request handlers
│   ├── memoryutil/       # Semantic memory helpers
│   ├── metering/         # Usage-metering seam (no-op in the open-source build)
│   ├── middleware/       # Auth, CORS, logging
│   ├── models/           # Data transfer objects
│   ├── providers/        # Model provider integrations
│   └── server/           # HTTP server wiring & routing
├── ent/schema/           # Ent database schema (generated code is not committed)
├── web/app/  # Angular frontend
├── docs/                 # Architecture and reference docs
└── openapi.yaml          # OpenAPI specification

📚 Documentation

• ARCHITECTURE.md — system design, patterns, and internals
• openapi.yaml — full REST API specification
• CONTRIBUTING.md — how to contribute
• SECURITY.md — reporting vulnerabilities

🤝 Contributing

Contributions are welcome! Please read CONTRIBUTING.md and our Code of Conduct before opening a pull request. Issues and feature ideas are equally appreciated.

📄 License

Licensed under the Apache License 2.0.

Built by the team behind WhatIff · Blog