Hybrid RAG Engine

A from-scratch, production-style Retrieval-Augmented Generation (RAG) platform demonstrating hybrid retrieval, cross-encoder reranking, and citation-grounded LLM generation over your own documents.

This project implements patterns used in real enterprise RAG systems: combining dense vector search with keyword full-text search, reranking candidates with a cross-encoder, and generating grounded, cited answers with streaming output — all on top of a single Postgres database, no hosted vector-database dependency required.

Why this architecture?

Decision	Rationale
Postgres + pgvector instead of a dedicated vector DB (Pinecone/Weaviate)	Keeps operational footprint minimal — one database for both relational and vector data — while still supporting HNSW approximate nearest-neighbour search at the scale this project targets.
Hybrid search (cosine similarity + keyword full-text)	Dense embeddings alone miss exact-match terms (IDs, acronyms, product codes); keyword search alone misses semantic paraphrases. Combining both and fusing results gets the benefits of each.
Reciprocal Rank Fusion (RRF) for combining result lists	Cosine similarity and ts_rank_cd scores live on incompatible scales — RRF fuses ranked lists using rank position only, avoiding brittle score normalization.
Cross-encoder reranking (ms-marco-MiniLM-L-6-v2) with percentile-normalized scores	Bi-encoder (embedding) similarity is a fast but coarse filter. A cross-encoder that jointly attends to the query and passage gives a sharper relevance signal for the final top-k, at low latency since it only reranks a small candidate set.
Citation-grounded prompting + anti-hallucination instructions	The generation prompt requires every claim to cite a numbered context passage and explicitly instructs the model to say "I don't know" rather than fabricate an answer when context is insufficient.
SSE token streaming	Answers stream token-by-token to the client as they're generated, with a final citations event once generation completes.
Pluggable LLM provider (OpenAI or Azure OpenAI)	Swappable via a single config flag — the rest of the pipeline is provider-agnostic.

Architecture

flowchart LR
    subgraph Ingestion
        A[Upload PDF / TXT] --> B[Chunk text
token-aware, overlapping]
        B --> C[Embed chunks]
        C --> D[(Postgres + pgvector)]
    end

    subgraph QueryTime[Query time]
        Q[User query] --> E[Embed query]
        E --> F[Vector search
cosine / HNSW]
        Q --> G[Keyword search
tsvector / GIN]
        F --> H[Reciprocal Rank Fusion]
        G --> H
        H --> I[Cross-encoder rerank
percentile-normalized]
        I --> J[Prompt build
numbered citations]
        J --> K[LLM generation
SSE streaming]
        K --> L[Answer + citations]
    end

    D --> F
    D --> G

Loading

Features

• Token-aware chunking with configurable overlap (tiktoken)
• Hybrid retrieval: pgvector cosine similarity (HNSW index) + Postgres full-text search (GIN index)
• Reciprocal Rank Fusion for combining retrieval strategies without score-scale issues
• Cross-encoder reranking with percentile-normalized scoring
• Citation-grounded, anti-hallucination prompting
• Real-time SSE token streaming for the generation endpoint
• Pluggable LLM/embedding provider: OpenAI or Azure OpenAI
• Async FastAPI backend, SQLAlchemy 2.0 async ORM
• Dockerized Postgres + pgvector setup via docker-compose
• Unit-tested fusion, reranking, and chunking logic

Project structure

app/
  api/routes/       FastAPI routers (health, documents, query)
  db/                SQLAlchemy models + async session
  generation/        Prompting, LLM client, RAG orchestration pipeline
  ingestion/         Chunking, embeddings, document loader
  retrieval/         Vector search, keyword search, RRF fusion, reranking
  schemas/           Pydantic request/response models
  config.py          Centralized settings (pydantic-settings)
  main.py            FastAPI app assembly
scripts/init_db.sql  pgvector extension + schema + indexes
tests/               Unit tests for chunking, fusion, reranking

Getting started

1. Configure environment

cp .env.example .env
# fill in OPENAI_API_KEY (or the AZURE_OPENAI_* variables if LLM_PROVIDER=azure)

2. Start Postgres + pgvector and the API

docker compose up --build

The API is now available at http://localhost:8000 (interactive docs at /docs).

3. Ingest a document

curl -F "file=@your-document.pdf" http://localhost:8000/api/documents

4. Ask a question

curl -X POST http://localhost:8000/api/query \
  -H "Content-Type: application/json" \
  -d '{"query": "What does the document say about X?"}'

Or stream the answer token-by-token:

curl -N -X POST http://localhost:8000/api/query/stream \
  -H "Content-Type: application/json" \
  -d '{"query": "What does the document say about X?"}'

Running tests

pip install -r requirements.txt
pytest

Tests cover pure logic (chunking, RRF fusion, percentile normalization) and do not require a live database or LLM credentials.

Tech stack

Python · FastAPI · SQLAlchemy 2.0 (async) · PostgreSQL + pgvector · sentence-transformers (cross-encoder reranking) · OpenAI / Azure OpenAI · Server-Sent Events · Docker

License

MIT — see LICENSE.