Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions docs/alive/_category_.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
{
"label": "Alive",
"position": 0.0004,
"link": {
"type": "generated-index",
"description": "Avatar Liveness for Intelligent Virtual Empathy (ALIVE)"
}
}
84 changes: 84 additions & 0 deletions docs/alive/architecture.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,84 @@
---
sidebar_position: 0.0006
---
# Architecture

**ALIVE** is a modular system consisting of both frontend and backend subsystems, with clear separation of concerns, containerized deployment, and real-time multimodal interaction support.

---

## System Overview

ALIVE is split into two main subsystems:

### EVA – Empathetic Virtual Assistant (Backend)

- Built with **FastAPI** in **Python**
- Real-time WebSocket chat with LLM and RAG
- Emotional response tagging and summarization
- Handles user session, message history, and document retrieval

### AVATAR – Emotional Avatar Renderer (Frontend + API)

- Built with **ASP.NET Core** and **C#**
- Provides a REST API and WebRTC interface for avatar rendering
- Integrates with:
- **D-ID Clips API** for face animation
- **Azure Speech Services** for emotional TTS
- Supports dynamic rendering based on emotion-tagged content

---

## Subsystem Components

### EVA (FastAPI Application)

- `/eva/main.py` – Entrypoint with WebSocket routing and lifecycle hooks
- `/eva/llm/` – Interfaces for calling OpenAI/Anthropic APIs, emotion segmentation
- `/eva/rag/` – Local RAG document store using **ChromaDB**
- `/eva/db/` – Conversation storage and admin setting persistence (SQLite/PostgreSQL)
- `/eva/admin/` – Secure admin panel for prompt configuration, document uploads, and conversation management

### AVATAR (ASP.NET Core Service)

- `/avatar/` – Main backend orchestrating avatar generation
- `/avatar/wwwroot/` – Web frontend assets for avatar display
- `/avatar/Program.cs`, `/avatar/appsettings.json` – Service initialization and configuration

---

## Supporting Services

- **ChromaDB** (embedded) – RAG vector store for semantic search
- **Azure TTS** – Expressive speech synthesis based on emotion
- **D-ID API** – Avatar face animation based on transcript and voice
- **WebRTC** – Low-latency avatar playback

---

## Deployment Stack

| Component | Tech | Role |
|----------------|------------------|-------------------------------|
| EVA | Python + FastAPI | WebSocket chat, LLM, RAG |
| AVATAR | C# + ASP.NET | Audio/visual rendering |
| Chroma | Rust (DB engine) | Vector database for RAG |
| D-ID API | SaaS | Face animation |
| Azure Speech | SaaS | Emotion-aware TTS |
| Frontend | HTML/CSS/JS | User interface for avatar |
| DB | PostgreSQL/SQLite| Conversation persistence |

---

## Communication Flow

![Architecture](images/diagram.svg 'Architecture')

---

## Extensibility

- Swap model providers using `LLM_PROVIDER=openai|anthropic` in `.env`
- Extend document support by dropping files into `documents/` (or configuring another vector store)
- Customize the avatar renderer by modifying the AVATAR pipeline or using alternatives to D-ID
- Replace Azure TTS with ElevenLabs or local TTS via configurable adapters
149 changes: 149 additions & 0 deletions docs/alive/deployment.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,149 @@
---
sidebar_position: 0.0007
---

# Deployment

This guide outlines how to deploy the ALIVE system using the official Docker images. ALIVE consists of two main services:

- `sermas-eva`: Backend for chat, emotion tagging, RAG, and LLM integration
- `sermas-avatar`: Avatar rendering and emotional TTS using D-ID and Azure

---

## Prerequisites

Before you begin, ensure you have:

- Docker installed (v27+ recommended) including the compose plugin
- A valid OpenAI or Anthropic API key
- An Azure Speech key (for TTS)
- A D-ID API key (for the 3D avatar)

---

## Docker Images

The official images are hosted on Docker Hub:

| Component | Image |
|-----------|-------|
| EVA | [`thingenious/sermas-eva`](https://hub.docker.com/r/thingenious/sermas-eva) |
| AVATAR | [`thingenious/sermas-avatar`](https://hub.docker.com/r/thingenious/sermas-avatar) |

---

## Quick Start with `docker-compose`

1. Copy and rename the example configuration:

```shell
cp compose.example.yaml compose.yaml
# you might want to commment out / remove the "build" sections
cp .env.example .env
```

2. Edit the `.env` file and fill in:

```env
OPENAI_API_KEY=your-key
AZURE_SPEECH_API_KEY=your-key
DID_API_KEY=your-did-key
CHAT_API_KEY=choose-a-secret
# example generation with:
# openssl rand -hex 32
# or python -c "import secrets; print(secrets.token_hex(32))"
```

3. Start the stack:

```shell
docker compose -f compose.yaml up -d
```

Once running:

- EVA backend: [http://localhost:8000](http://localhost:8000), WebSocket at [ws://localhost:8000/ws](ws://localhost:8000/ws)
- AVATAR frontend: [http://localhost:3000](http://localhost:3000)

---

## Reverse Proxy with Nginx and HTTPS (Recommended)

To expose ALIVE services over the internet securely, you can place an Nginx reverse proxy in front of the EVA and AVATAR containers and enable HTTPS using Let's Encrypt and Certbot.

### Example Nginx Config (for `/etc/nginx/sites-available/alive`)

```nginx
server {
listen 80;
server_name your-domain.com;

location / {
return 301 https://$host$request_uri;
}
}

server {
listen 443 ssl;
server_name your-domain.com;

ssl_certificate /etc/letsencrypt/live/your-domain.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/your-domain.com/privkey.pem;

location / {
proxy_pass http://localhost:3000; # AVATAR frontend
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}

location /ws/ {
proxy_pass http://localhost:8000/ws/; # EVA WebSocket
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}

# optional (admin panel)
location /admin/ {
proxy_pass http://localhost:8000/admin/;
}

location /healthz {
proxy_pass http://localhost:8000/healthz;
}
}
```

## Advanced Configuration

You can configure behavior via environment variables. Some key options:

| Variable | Description |
|----------|-------------|
| `LLM_PROVIDER` | `openai` or `anthropic` |
| `MAX_HISTORY_MESSAGES` | Number of chat messages retained |
| `CHROMA_DB_DIR` | Directory for Chroma vector DB |
| `CHAT_WS_URL` | URL for WebSocket clients |
| `ADMIN_API_KEY` | Protects admin endpoints |

---

## Health & Debug

- EVA: `GET /healthz` → `{ "status": "ok" }`
- Admin Panel: `GET /admin/` (requires token)

You can inspect container logs with:

```shell
docker compose logs -f
```

---

## Source & Updates

For latest changes, visit the GitHub repository:

👉 [`thingenious/sermas`](https://github.com/thingenious/sermas)
1 change: 1 addition & 0 deletions docs/alive/images/diagram.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/alive/images/preview.webp
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading