Migrate from Gemini to OpenRouter free models + local embeddings

.github/workflows/ci.yml

@@ -0,0 +1,32 @@
+name: CI
+
+on:
+  push:
+  pull_request:
+
+jobs:
+  tests:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      - uses: actions/setup-python@v5
+        with:
+          python-version: "3.12"
+          cache: pip
+      - name: Install dependencies
+        run: pip install -r requirements.txt pytest httpx
+      - name: Unit and API tests (LLM mocked, embeddings real)
+        run: pytest -q
+      - name: Retrieval quality gate (offline eval)
+        run: python evals/run_eval.py
+
+  docker:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v4
+      # index/ is gitignored (private); build with an empty stand-in — the
+      # build-time index check treats a missing index as "ship without one"
+      - name: Create placeholder dir
+        run: mkdir -p index
+      - name: Docker build
+        run: docker build -f DOCKERFILE -t rag-resume-api .

.gitignore

@@ -23,4 +23,4 @@ scripts/publish-docker.ps1
 commands.txt
 docs/plans/
 *.json
-CLAUDE.md
+CLAUDE.md.pytest_cache/

DOCKERFILE

@@ -1,16 +1,24 @@
-FROM python:3.12-slim
-
-WORKDIR /app
-
-# Copy requirements and install dependencies
-COPY requirements.txt ./
-RUN pip install --no-cache-dir -r requirements.txt
-
-# Copy the application code
-COPY index/ ./index/
-COPY app.py .
-
-EXPOSE 8000
-
-# Run the server
-CMD ["python", "-m", "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]
+FROM python:3.12-slim
+
+WORKDIR /app
+
+# Copy requirements and install dependencies
+COPY requirements.txt ./
+RUN pip install --no-cache-dir -r requirements.txt
+
+# Pre-download the local embedding + reranker models so the container doesn't fetch them at startup
+RUN python -c "from fastembed import TextEmbedding; TextEmbedding('BAAI/bge-small-en-v1.5')" \
+    && python -c "from fastembed.rerank.cross_encoder import TextCrossEncoder; TextCrossEncoder('Xenova/ms-marco-MiniLM-L-6-v2')"
+
+# Copy the application code and the pre-built index (docs/ stays out of the
+# image — the index already holds the indexed text, raw files aren't needed)
+COPY index/ ./index/
+COPY app.py retrieval.py check_index.py ./
+
+# Fail the build if the shipped index was built with a different embedding model
+RUN python check_index.py
+
+EXPOSE 8000
+
+# Run the server (requires OPENROUTER_API_KEY at runtime)
+CMD ["python", "-m", "uvicorn", "app:app", "--host", "0.0.0.0", "--port", "8000"]

README.md

@@ -1,15 +1,29 @@
 ## RAG Resume API
 
+![CI](https://github.com/rahulk98/Resume-RAG-System/actions/workflows/ci.yml/badge.svg)
+
 ### Live
 Hosted and running at [rahul-krishnan.is-a.dev](https://rahul-krishnan.is-a.dev).
 
 ### Overview
-Retrieval-Augmented Generation (RAG) API that indexes documents in `docs/` and answers questions using LlamaIndex with Google Gemini (embeddings + LLM). FastAPI provides `/health` and `/query` endpoints. Containerized with Docker and deployed as an API on GCP Free tier.
+Retrieval-Augmented Generation (RAG) API that indexes documents in `docs/` and answers questions using LlamaIndex. The LLM runs on OpenRouter's free-model router (`openrouter/free`) and everything else runs locally — so there are no per-query API charges. FastAPI provides `/health`, `/query`, and `/query/stream` endpoints. Containerized with Docker and deployed as an API on GCP Free tier.
+
+**Pipeline highlights**
+- **Hybrid retrieval**: BM25 keyword search fused with vector search (reciprocal rank fusion), then reranked by a local ONNX cross-encoder (`Xenova/ms-marco-MiniLM-L-6-v2`) — better recall than embeddings alone, still zero API cost.
+- **Local embeddings**: `BAAI/bge-small-en-v1.5` via fastembed (ONNX, in-process).
+- **Multi-turn chat**: pass a `session_id` and follow-up questions are condensed into standalone queries using conversation history ("what did he do there?" → "what did Rahul do at SAP?").
+- **Degenerate-answer guard**: the free router occasionally returns a junk response; answers are validated and retried on a re-routed model.
+- **Evaluated**: a golden Q&A set gates retrieval quality in CI (offline) and scores answer faithfulness with an LLM judge (live) — see `evals/`.
+- **Production hardening**: per-IP rate limiting, daily quota, response caching, SSE streaming, query logging.
 
 ### Prerequisites
 - **Python**: 3.12+
-- **Environment**: `GOOGLE_API_KEY` set (in shell or `.env`)
-  - Optional Supabase logging: set `SUPABASE_URL` and `SUPABASE_SERVICE_ROLE_KEY` (or `SUPABASE_ANON_KEY`)
+- **Environment**: `OPENROUTER_API_KEY` set (in shell or `.env`) — get a free key at [openrouter.ai](https://openrouter.ai)
+  - Optional: `OPENROUTER_MODEL` to pin a specific model (default `openrouter/free`, which auto-routes to an available free model). Note: do **not** use `openrouter/auto:free` — it does not restrict routing to free models and can incur charges.
+  - Optional: `EMBED_MODEL_NAME` to change the local embedding model (default `BAAI/bge-small-en-v1.5`). Changing it requires rebuilding the index.
+  - Optional Sheets logging: set `SHEETS_JSON_KEY` (service-account JSON)
+
+> **Migrating from the Gemini version?** The embedding model changed (3072-dim Gemini → 384-dim local BGE), so the old persisted index is incompatible. Rebuild it once with `python build_index.py` (no API key needed — embeddings run locally). Safety nets: the Docker build fails fast if the shipped index doesn't match the embedding model (`check_index.py`), and at startup the app rebuilds a mismatched index from `docs/` when that folder is present. Raw documents are never copied into the Docker image.
 
 ### Quickstart
 ```bash
@@ -26,22 +40,35 @@ python -m uvicorn app:app --host 0.0.0.0 --port 8000
 ### API
 - **GET `/health`** → `{ "status": "ok", "model_loaded": boolean }`
 - **POST `/query`**
-  - Request body:
+  - Request body (`top_k` and `session_id` optional; send a stable `session_id` to enable follow-up questions):
 ```json
-{ "query": "Tell me about Rahul in 2 sentences.", "top_k": 5 }
+{ "query": "Tell me about Rahul in 2 sentences.", "top_k": 5, "session_id": "abc123" }
 ```
   - Response body:
 ```json
 { "query": "...", "answer": "..." }
 ```
+- **POST `/query/stream`** — same request body; responds with `text/event-stream` of `data: {"token": "..."}` events terminated by `data: [DONE]`.
 
 ### Docker
 ```bash
 # Build
 docker build -t rag-resume-api .
 
 # Run
-docker run -e GOOGLE_API_KEY=YOUR_KEY -p 8000:8000 rag-resume-api
+docker run -e OPENROUTER_API_KEY=YOUR_KEY -p 8000:8000 rag-resume-api
+```
+
+### Tests & evals
+```bash
+pip install pytest httpx
+pytest tests/ -v            # unit/API tests — LLM mocked, no key needed
+python evals/run_eval.py    # offline retrieval-quality eval (also runs in CI)
+```
+With `OPENROUTER_API_KEY` set:
+```bash
+pytest tests/ -v -m live         # live OpenRouter smoke tests
+python evals/run_eval.py --live  # full answer eval: keyword coverage, abstention, LLM-judged faithfulness
 ```
 
 ### Screenshots