Teacher SEL Coach

A production-ready, containerized RAG (Retrieval-Augmented Generation) service for generating structured Social-Emotional Learning (SEL) lesson plans. Built with FastAPI, PostgreSQL (with optional pgvector), and OpenAI embeddings.

🎯 Features

• Hybrid Retrieval: Combines BM25 (PostgreSQL tsvector) and vector similarity search with Reciprocal Rank Fusion (RRF)
• Structured Lesson Plans: Generates classroom-ready plans with 5 sections:
  • Objective
  • Activities (with time allocations)
  • Reflection
  • Assessment
  • Resources
• CASEL Alignment: Automatically aligns content to CASEL's five core competencies
• Grade Band Support: K-2, 3-5, 6-8, 9-12, or "all"
• Safety Validation: Rule-based + LLM safety checks with fail-open/fail-closed modes
• Timeboxed Activities: Explicit time allocations for all activities
• Differentiation Notes: Includes support strategies when applicable
• Citations: Source citations with metadata and snippets
• Works Without pgvector: Falls back to text search only if pgvector is not available

📋 Prerequisites

• Python 3.11+
• PostgreSQL 16+ (or Docker for database)
• OpenAI API key
• (Optional) Docker and Docker Compose for containerized setup

🚀 Quick Start

Option 1: Local Development (Recommended for Development)

1. Clone the Repository

git clone <your-repo-url>
cd SELCoach

2. Set Up Environment

# Create virtual environment
python3 -m venv venv
source venv/bin/activate  # On Windows: venv\Scripts\activate

# Install dependencies
pip install -e ".[dev]"

3. Configure Environment Variables

# Copy sample env file
cp .env.sample .env

# Edit .env and add your OpenAI API key
nano .env  # or use your preferred editor

Required variables in .env:

DB_DSN=postgresql+psycopg://seluser:selpass@localhost:5432/seldb
OPENAI_API_KEY=sk-your-actual-key-here
LLM_MODEL=gpt-4-turbo-preview
EMBED_MODEL=text-embedding-3-large
SAFETY_MODE=fail-open
LOG_LEVEL=INFO
ADMIN_ENABLED=false

4. Set Up PostgreSQL

Option A: Install PostgreSQL Locally

# Install PostgreSQL 16 via Homebrew (macOS)
brew install postgresql@16

# Start PostgreSQL service
brew services start postgresql@16

# Create database and user
createdb seldb
createuser seluser
psql -U $(whoami) -d seldb -c "ALTER USER seluser WITH PASSWORD 'selpass';"
psql -U $(whoami) -d seldb -c "GRANT ALL PRIVILEGES ON DATABASE seldb TO seluser;"
psql -U $(whoami) -d seldb -c "GRANT ALL ON SCHEMA public TO seluser;"

Option B: Use Docker for Database Only

# Start only the database container
docker compose -f docker/docker-compose.yml up -d db

5. Apply Database Migrations

# Using Python script (no psql required)
python scripts/apply_migrations.py

# Or manually with psql
export PATH="/opt/homebrew/opt/postgresql@16/bin:$PATH"  # Add to your shell config
psql -U seluser -d seldb -f db/migrations/0001_extensions.sql
psql -U seluser -d seldb -f db/migrations/0002_schema.sql
psql -U seluser -d seldb -f db/migrations/0003_indexes.sql

6. Ingest Seed Data

python -m rag.ingestion.ingest_cli --path data/teacher

7. Start the API Server

uvicorn api.main:app --reload --port 8001

The API will be available at:

• Main: http://localhost:8001
• API Docs: http://localhost:8001/docs
• Health Check: http://localhost:8001/healthz

Option 2: Docker Compose (Recommended for Production)

1. Clone and Configure

git clone <your-repo-url>
cd SELCoach
cp .env.sample .env
# Edit .env and add your OPENAI_API_KEY

2. Start Services

docker compose -f docker/docker-compose.yml up -d --build

3. Apply Migrations

# Using bootstrap script (requires local psql)
./scripts/bootstrap.sh

# Or using Docker exec
docker compose -f docker/docker-compose.yml exec db psql -U seluser -d seldb -f /app/db/migrations/0001_extensions.sql
docker compose -f docker/docker-compose.yml exec db psql -U seluser -d seldb -f /app/db/migrations/0002_schema.sql
docker compose -f docker/docker-compose.yml exec db psql -U seluser -d seldb -f /app/db/migrations/0003_indexes.sql

4. Ingest Data

docker compose -f docker/docker-compose.yml exec api python -m rag.ingestion.ingest_cli --path data/teacher

5. Access the API

• Main: http://localhost:8000
• API Docs: http://localhost:8000/docs

📖 API Usage

Health Check

curl http://localhost:8001/healthz

Generate Lesson Plan

curl -X POST http://localhost:8001/api/chat \
  -H "Content-Type: application/json" \
  -d '{
    "query": "Create a 30-minute lesson for 5th grade on coping with test anxiety.",
    "context": {
      "grade_band": "3-5",
      "casel": ["Self-Management"],
      "duration_minutes": 30
    }
  }'

Example Response

{
  "role_routed": "teacher",
  "safety": {
    "label": "safe",
    "notes": null
  },
  "answer": {
    "format": "lesson_plan",
    "content": "# Lesson Plan\n\n## Objective\n...",
    "disclaimer": "This is not medical or diagnostic advice."
  },
  "citations": [
    {
      "source_id": "seed_src_2.md-chunk-0",
      "source_title": "Managing Test Anxiety",
      "snippet": "Test anxiety is a common challenge..."
    }
  ],
  "why_this_answer": "Retrieved 3 relevant chunks. Aligned to CASEL: Self-Management. Grade band: 3-5. Duration: 30 minutes."
}

🏗️ Project Structure

SELCoach/
├── api/                    # FastAPI application
│   ├── main.py            # App entry point
│   ├── routers/           # API endpoints
│   │   ├── chat.py       # Chat endpoint
│   │   └── admin.py      # Admin endpoints
│   ├── schemas.py        # Pydantic schemas
│   ├── deps.py           # Dependency injection
│   └── config.py         # Configuration
├── rag/                   # RAG pipeline
│   ├── ingestion/        # Document ingestion
│   │   ├── loaders.py
│   │   ├── clean.py
│   │   ├── chunk.py
│   │   ├── labelers.py
│   │   ├── embed.py
│   │   └── ingest_cli.py
│   ├── retrieval/        # Hybrid retrieval
│   │   ├── filters.py
│   │   ├── hybrid.py
│   │   ├── rrf.py
│   │   └── store_pg.py
│   ├── generation/       # Lesson plan generation
│   │   ├── prompts_teacher.py
│   │   ├── planner.py
│   │   └── generator.py
│   └── safety/           # Safety validation
│       ├── rules.py
│       └── validator.py
├── db/
│   └── migrations/       # Database migrations
├── tests/                # Test suite
├── docker/               # Docker configuration
├── scripts/              # Utility scripts
├── data/
│   └── teacher/          # Seed data
└── pyproject.toml        # Dependencies

🔧 Configuration

Environment variables (see .env.sample):

Variable	Description	Default
DB_DSN	PostgreSQL connection string	postgresql+psycopg://seluser:selpass@localhost:5432/seldb
OPENAI_API_KEY	OpenAI API key (required)	-
LLM_MODEL	LLM model name	gpt-4-turbo-preview
EMBED_MODEL	Embedding model	text-embedding-3-large
SAFETY_MODE	Safety mode: fail-open or fail-closed	fail-open
LOG_LEVEL	Logging level	INFO
ADMIN_ENABLED	Enable admin endpoints	false

📝 Ingestion CLI

Ingest documents from a directory:

python -m rag.ingestion.ingest_cli \
  --path data/teacher \
  --default-grade "3-5" \
  --default-casel "Self-Awareness,Self-Management"

Options:

• --path: Path to directory containing markdown/text files
• --default-grade: Default grade band if inference fails
• --default-casel: Default CASEL competencies (comma-separated)

🧪 Testing

Run the test suite:

# Install dev dependencies
pip install -e ".[dev]"

# Run all tests
pytest

# Run with coverage
pytest --cov=rag --cov=api

# Run specific test file
pytest tests/test_ingestion.py

🐛 Troubleshooting

pgvector Extension Not Found

If you see errors about the vector extension:

Local PostgreSQL:

# Install pgvector
brew install pgvector

# Copy extension files (requires sudo)
sudo mkdir -p /opt/homebrew/opt/postgresql@16/share/postgresql@16/extension
sudo cp /opt/homebrew/Cellar/pgvector/*/share/postgresql@17/extension/* /opt/homebrew/opt/postgresql@16/share/postgresql@16/extension/
sudo cp /opt/homebrew/Cellar/pgvector/*/lib/postgresql@17/* /opt/homebrew/opt/postgresql@16/lib/postgresql@16/

# Create extension
psql -U seluser -d seldb -c "CREATE EXTENSION IF NOT EXISTS vector;"

Note: The application works without pgvector - it will use text search only.

Database Connection Issues

# Check PostgreSQL is running
brew services list | grep postgresql

# Test connection
psql -U seluser -d seldb -c "SELECT version();"

# Check database exists
psql -U $(whoami) -lqt | grep seldb

"No relevant content found" Error

This usually means:

1. Documents haven't been ingested - run ingestion CLI
2. Query doesn't match content - try broader queries like "emotions" or "test anxiety"
3. Filters too strict - the search will automatically relax filters if no results

Port Already in Use

If port 8000 is in use:

# Use a different port
uvicorn api.main:app --reload --port 8001

OpenAI API Errors

• Verify your API key is set correctly in .env
• Check you have sufficient API credits
• Verify the model names are correct

🏛️ Architecture

Retrieval Pipeline

1. Query Processing: Normalize filters (grade_band, CASEL)
2. Hybrid Search:
   • Lexical: PostgreSQL tsvector with plainto_tsquery
   • Vector: pgvector cosine similarity (if available)
3. Fusion: Reciprocal Rank Fusion (RRF) with k=60
4. Top-K: Return top 10 results with snippets

Generation Pipeline

1. Planner: LLM creates structured outline (JSON)
2. Generator: LLM expands outline into full lesson plan
3. Safety: Rule-based + LLM validation
4. Post-processing: Add citations, ensure disclaimer

Safety Validation

• Rules: Pattern matching for reject/needs_review keywords
• LLM Check: Classification prompt for final label
• Fail Modes: fail-open (allow on error) or fail-closed (reject on error)

📚 Database Schema

CREATE TABLE documents (
    id UUID PRIMARY KEY,
    source_id TEXT,
    source_title TEXT,
    publisher TEXT,
    published_at TIMESTAMPTZ,
    grade_band TEXT,                  -- 'K-2'|'3-5'|'6-8'|'9-12'|'all'
    casel_competencies TEXT[],        -- CASEL 5 subset
    tags TEXT[],
    content TEXT NOT NULL,
    content_tsv tsvector,             -- For text search
    embedding vector(1536),           -- Optional: only if pgvector available
    doc_hash TEXT UNIQUE,
    created_at TIMESTAMPTZ,
    updated_at TIMESTAMPTZ
);

🔐 Security Notes

• Never commit .env file (already in .gitignore)
• Admin endpoints are disabled by default (ADMIN_ENABLED=false)
• API keys should be kept secure
• In production, use proper authentication/authorization

📄 License

See LICENSE file.

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make changes with tests
4. Submit a pull request

📞 Support

For issues and questions, please open a GitHub issue.

🎓 CASEL Competencies

The system aligns to CASEL's five core competencies:

1. Self-Awareness: Understanding one's emotions, thoughts, and values
2. Self-Management: Managing emotions, behaviors, and goals
3. Social Awareness: Understanding others' perspectives and showing empathy
4. Relationship Skills: Building and maintaining healthy relationships
5. Responsible Decision-Making: Making ethical, constructive choices

🚧 Future Enhancements

• Support for PDF document ingestion
• Multi-language support
• User authentication and session management
• Lesson plan templates and customization
• Analytics and usage tracking
• Integration with learning management systems

---

Built with ❤️ for educators