AI Medical Consultation Platform

AI-powered platform for monitoring doctor-patient conversations and providing intelligent medical diagnosis assistance.

🚀 Quick Start

Prerequisites

• Node.js 18+ and npm
• Python 3.11.x (specifically 3.11, not 3.12+)
• LiveKit Cloud account
• OpenAI API key
• Supabase project

System Architecture

Testing Infrastructure

• AI Doctor Agent: Simulates a doctor (Dr. Aisha Bin Rashid) for testing
• Purpose: Allows testers to act as patients without needing real doctors
• Benefit: Rapid development and testing of conversation monitoring

Core Components

1. Voice Agent (app.py): AI doctor for realistic medical conversations
2. Conversation Monitor: Captures real-time doctor-patient dialogue
3. Medical Extractor (in development): LLM-based extraction of medical information
4. Diagnosis API: Llama3-OpenBioLLM-70B powered diagnosis service

Development Approach

Building incrementally with isolated components:

• ✅ Phase 1: Conversation monitoring
• 🔄 Phase 2: Medical information extraction agent (current)
• 🔮 Phase 3: Integration with conversation stream
• 🔮 Phase 4: Complete diagnosis pipeline

Starting the Application

Option 1: Using Docker (Recommended)

# From project root
make build    # Build both services
make up       # Start all services
make logs     # View logs
make down     # Stop services

Option 2: Manual Setup

Step 1: Start Backend Voice Agent

cd backend
python -m venv venv              # Create virtual environment (first time)
source venv/bin/activate         # Activate venv
pip install -r requirements.txt  # Install dependencies (first time)

# Option A: Use the start script (recommended)
./start_agent.sh                 # Kills old agents, starts new one

# Option B: Use make commands
make run                         # Stops old agents, starts new one

# Option C: Manual start
python app.py dev                # Start the agent

Step 2: Start Frontend Web App

cd frontend
npm install                      # Install dependencies (first time)
npm run dev                      # Start on http://localhost:3005

Step 3: Access the Application

1. Open http://localhost:3005
2. Log in with your credentials
3. Click "Connect" to start voice conversation

Repository Structure

This project consists of two main components:

📁 Frontend Repository

• Location: ./frontend/
• Technology: Next.js, TypeScript, Tailwind CSS
• Deployment: Vercel
• Port: 3005

📁 Backend Repository

• Location: ./backend/
• Technology: Python, LiveKit Agents, OpenAI
• Deployment: Railway/Render/AWS
• Port: LiveKit Cloud (no direct port)

Local Development

With Docker (Recommended)

# From project root
make build    # Build both services
make up       # Start all services
make logs     # View logs
make down     # Stop services

Without Docker

For detailed instructions:

• Frontend: See frontend/README.md
• Backend: See backend/README.md

Environment Setup

1. Frontend (.env.local):

   • LiveKit credentials
   • Supabase credentials
   • Site URL

2. Backend (.env):

   • LiveKit credentials
   • OpenAI API key

Docker Benefits

Using Docker with bind mounts allows you to:

• Edit code with your host tools
• See changes reflected immediately
• Keep consistent environments
• Use host CLI tools and MCP integrations

Deployment

• Frontend: Push to frontend repo → Auto-deploy to Vercel
• Backend: Push to backend repo → Deploy to your chosen platform

Monitoring & Logs

The backend agent creates detailed conversation logs for each session:

• Location: backend/logs/
• Format: conversation_{room}_{timestamp}.log
• Contents: Real-time transcripts, conversation flow, state changes
• Agent logs: backend/logs/agent.log for runtime debugging