RAG-Based Document Q&A System

An end-to-end retrieval-augmented generation application with:

• PDF ingestion pipeline with chunking and embedding
• Chroma vector database for semantic retrieval
• LLM-powered answer generation grounded in retrieved context
• React frontend for document upload and Q&A
• Latency benchmarking and basic accuracy evaluation scripts

Stack

• Backend: FastAPI, LangChain, OpenAI API, ChromaDB
• Frontend: React, Vite

Project Structure

rag-doc-qa/
  backend/
    app/
      main.py
      rag.py
      config.py
      schemas.py
    scripts/
      benchmark.py
      evaluate.py
      sample_qa.json
    data/
      uploads/
      chroma/
    requirements.txt
    .env.example
  frontend/
    src/
      components/
      App.jsx
      styles.css

Backend Setup

1. Go to backend directory.
2. Create and activate a virtual environment.
3. Install dependencies.
4. Copy .env.example to .env and set OPENAI_API_KEY.
5. Start API server.

cd backend
python -m venv .venv
.\.venv\Scripts\Activate.ps1
pip install -r requirements.txt
copy .env.example .env
uvicorn app.main:app --reload

API default: http://127.0.0.1:8000

Free Local Mode (No OpenAI API Key)

You can run this project fully local using Ollama.

1. Install Ollama and start it.
2. Pull required local models:

ollama pull llama3.2:3b
ollama pull nomic-embed-text

3. In backend/.env, set:

LLM_PROVIDER=ollama
EMBEDDING_PROVIDER=ollama
OLLAMA_BASE_URL=http://127.0.0.1:11434
OLLAMA_CHAT_MODEL=llama3.2:3b
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
OPENAI_API_KEY=

4. Restart backend server.

Frontend Setup

cd frontend
npm install
npm run dev

UI default: http://127.0.0.1:5173

API Endpoints

• GET /health
• POST /documents/upload with multipart file (PDF only)
• POST /qa/ask with JSON body:

{
  "question": "What does the report conclude?",
  "top_k": 4
}

Benchmarking

Run after indexing at least one document.

cd backend
python scripts/benchmark.py

Evaluation

1. Fill scripts/sample_qa.json with expected snippets.
2. Run:

cd backend
python scripts/evaluate.py

Notes

• Returned answers include citations with filename, page, and chunk ID.
• If retrieval does not contain enough context, the model is instructed to say it lacks enough information.