RAG-based AI Chatbot

A production-ready Retrieval-Augmented Generation (RAG) chatbot built with FastAPI, Streamlit, AWS Bedrock, and ChromaDB. This enterprise-level application demonstrates modern software architecture, containerization, and cloud deployment practices.

🌟 Live Demo

Try the chatbot live: http://18.207.19.42:8501/

🏗️ Architecture

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────┐
│   Streamlit     │    │    FastAPI      │    │   AWS Bedrock   │
│   Frontend      │◄──►│    Backend      │◄──►│   (Claude AI)   │
│   (Port 8501)   │    │   (Port 8000)   │    │                 │
└─────────────────┘    └─────────────────┘    └─────────────────┘
                                │
                                ▼
                       ┌─────────────────┐
                       │    ChromaDB     │
                       │  Vector Store   │
                       └─────────────────┘

✨ Features

• Streaming Chat Interface: Real-time response streaming using Streamlit
• RAG Implementation: Retrieval-augmented generation with vector similarity search
• AWS Bedrock Integration: Claude 3 Haiku model for high-quality responses
• Vector Database: ChromaDB for efficient document storage and retrieval
• Containerized Deployment: Docker and Docker Compose for easy deployment
• CI/CD Pipeline: GitHub Actions for automated testing and deployment
• Health Monitoring: Comprehensive health checks and monitoring
• Production Ready: Proper logging, error handling, and security practices

🚀 Quick Start

Prerequisites

• Docker and Docker Compose
• AWS Account with Bedrock access
• Python 3.11+ (for local development)

1. Clone and Setup

git clone <your-repository-url>
cd genai-rag-chatbot
cp .env.example .env

2. Configure Environment

Edit .env file with your AWS credentials:

AWS_ACCESS_KEY_ID=your_access_key_here
AWS_SECRET_ACCESS_KEY=your_secret_key_here
AWS_DEFAULT_REGION=us-east-1
BACKEND_URL=http://localhost:8000

3. Deploy with Docker Compose

docker-compose up --build

4. Access the Application

• Frontend (Streamlit): http://localhost:8501
• Backend API: http://localhost:8000
• API Documentation: http://localhost:8000/docs

🛠️ Local Development

Backend Development

cd backend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
uvicorn app.main:app --reload

Frontend Development

cd frontend
python -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate
pip install -r requirements.txt
streamlit run app.py

📦 Project Structure

genai-rag-chatbot/
├── backend/
│   ├── app/
│   │   ├── __init__.py
│   │   ├── main.py              # FastAPI application
│   │   ├── models.py            # Pydantic models
│   │   ├── services/
│   │   │   ├── __init__.py
│   │   │   ├── bedrock_service.py    # AWS Bedrock integration
│   │   │   ├── embedding_service.py  # Text embedding service
│   │   │   └── vector_store_service.py # ChromaDB operations
│   │   └── routers/
│   │       ├── __init__.py
│   │       └── chat.py          # Chat endpoints
│   ├── Dockerfile
│   ├── requirements.txt
│   └── knowledge_base/
│       └── documents/           # Sample documents for RAG
├── frontend/
│   ├── app.py                   # Streamlit application
│   ├── Dockerfile
│   ├── requirements.txt
│   └── utils/
│       ├── __init__.py
│       └── api_client.py        # Backend API client
├── .github/
│   └── workflows/
│       └── ci-cd.yml            # GitHub Actions pipeline
├── docker-compose.yml
├── .env.example
├── .gitignore
└── README.md

🔧 API Endpoints

Chat Endpoints

• POST /chat/ - Send chat message (complete response)
• POST /chat/stream - Send chat message (streaming response)
• GET /chat/health - Chat service health check

System Endpoints

• GET / - API information
• GET /health - Main health check
• GET /docs - Interactive API documentation

Example Usage

import requests

# Send chat message
response = requests.post("http://localhost:8000/chat/", json={
    "message": "What is AWS Bedrock?",
    "max_tokens": 1000,
    "temperature": 0.7
})

print(response.json()["response"])

🌐 AWS EC2 Deployment

1. Launch EC2 Instance

• Choose Ubuntu 22.04 LTS
• Configure security groups (ports 22, 8000, 8501)
• Assign elastic IP (optional)

2. Setup EC2 Instance

# Update system
sudo apt update && sudo apt upgrade -y

# Install Docker
curl -fsSL https://get.docker.com -o get-docker.sh
sudo sh get-docker.sh
sudo usermod -aG docker ubuntu

# Install Docker Compose
sudo curl -L "https://github.com/docker/compose/releases/download/v2.20.0/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose
sudo chmod +x /usr/local/bin/docker-compose

# Reboot to apply changes
sudo reboot

3. Deploy Application

# Clone repository
git clone <your-repository-url>
cd genai-rag-chatbot

# Setup environment
cp .env.example .env
nano .env  # Edit with your AWS credentials

# Deploy
docker-compose up --build -d

# Check status
docker-compose logs -f

4. Setup GitHub Actions Runner (Optional)

# Create runner directory
mkdir actions-runner && cd actions-runner

# Download runner
curl -o actions-runner-linux-x64-2.311.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.311.0/actions-runner-linux-x64-2.311.0.tar.gz
tar xzf ./actions-runner-linux-x64-2.311.0.tar.gz

# Configure runner (follow GitHub instructions)
./config.sh --url https://github.com/your-username/genai-rag-chatbot --token YOUR_TOKEN

# Install as service
sudo ./svc.sh install
sudo ./svc.sh start

🧪 Testing

Run Tests

# Backend tests
cd backend
pytest tests/ -v

# Frontend tests (if implemented)
cd frontend
pytest tests/ -v

# Integration tests with Docker
docker-compose -f docker-compose.test.yml up --build

Health Checks

# Check backend health
curl http://localhost:8000/health

# Check frontend health
curl http://localhost:8501/_stcore/health

# Check chat service health
curl http://localhost:8000/chat/health

🔒 Security Considerations

• AWS credentials are stored in environment variables only
• No hardcoded secrets in the codebase
• CORS is configured for frontend-backend communication
• Input validation and sanitization implemented
• Proper error handling without exposing sensitive information

📊 Monitoring and Logging

• Structured logging throughout the application
• Health check endpoints for monitoring
• Docker container health checks
• Error tracking and reporting

🚨 Troubleshooting

Common Issues

Backend won't start:

• Check AWS credentials in .env file
• Ensure ChromaDB directory has proper permissions
• Verify Docker containers can access environment variables

Frontend can't connect to backend:

• Check BACKEND_URL in frontend environment
• Ensure backend is running and healthy
• Verify network connectivity between containers

AWS Bedrock errors:

• Verify AWS credentials and region
• Check Bedrock model availability in your region
• Ensure proper IAM permissions for Bedrock access

Vector store issues:

• Check ChromaDB persistence directory permissions
• Verify documents are properly processed and embedded
• Monitor ChromaDB logs for errors

📝 Environment Variables Reference

Variable	Description	Example
AWS_ACCESS_KEY_ID	AWS access key for Bedrock	AKIA...
AWS_SECRET_ACCESS_KEY	AWS secret key	wJalrXUt...
AWS_DEFAULT_REGION	AWS region for Bedrock	us-east-1
BACKEND_URL	Backend API URL for frontend	http://localhost:8000

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/new-feature)
3. Commit your changes (git commit -am 'Add new feature')
4. Push to the branch (git push origin feature/new-feature)
5. Create a Pull Request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

🙏 Acknowledgments

• AWS Bedrock for providing access to foundation models
• ChromaDB for the vector database solution
• FastAPI and Streamlit communities for excellent documentation
• The open-source community for various tools and libraries used

---

For questions or support, please contact me at fatulla@codage.az