Skip to content

RAG.md

Latest commit

 

History

History
634 lines (390 loc) · 8.05 KB

RAG.md

File metadata and controls

634 lines (390 loc) · 8.05 KB
Raw

RAG.md

Multilingual RAG Enhancement Plan

Project Vision

Transform the existing AI SQL Assistant into a production-ready Multilingual RAG-powered SQL Analytics Assistant capable of understanding user questions in multiple languages while maintaining accurate SQL generation and execution.

The enhancement must preserve the existing architecture and functionality while introducing a Retrieval-Augmented Generation (RAG) layer for schema-aware and business-aware query generation.

Project Goal

Current System:

Natural Language Question → LangChain Agent → NVIDIA NIM (Mistral Large 3) → SQL Generation → PostgreSQL Execution → Results

Target System:

User Question (Any Language) → Multilingual Embedding Generation → Vector Retrieval → Schema & Business Context Retrieval → LangChain SQL Agent → NVIDIA NIM (Mistral Large 3) → SQL Generation → PostgreSQL Execution → Results

Why This Upgrade

Current Limitations

  • Entire schema is injected into prompts.
  • Performance decreases as schema size grows.
  • No business glossary support.
  • No multilingual understanding.
  • Increased token consumption.
  • Limited scalability.

RAG Benefits

  • Retrieve only relevant schema information.
  • Support multiple languages.
  • Improve SQL accuracy.
  • Reduce prompt size.
  • Scale to large enterprise databases.
  • Support business definitions and KPI documentation.

Core Design Principle

Users should be able to ask database questions in different languages and receive the same SQL output.

Example:

English

Show top 10 customers by revenue

Hindi

राजस्व के अनुसार शीर्ष 10 ग्राहक दिखाओ

French

Montrez les 10 meilleurs clients par revenus

Spanish

Mostrar los 10 principales clientes por ingresos

All should retrieve the same schema context and generate equivalent SQL.

Technology Stack

LLM

Provider: NVIDIA NIM

Model:

mistralai/mistral-large-3-675b-instruct-2512

Purpose:

  • SQL generation
  • Query reasoning
  • Join inference
  • Aggregation logic
  • Response explanation

Reason:

Currently integrated into the project and provides strong SQL reasoning performance.

Embedding Model

Provider: NVIDIA NIM

Model:

BAAI/bge-m3

Purpose:

  • Multilingual embeddings
  • Cross-lingual semantic search
  • Schema retrieval
  • Business glossary retrieval

Reason:

  • Supports 100+ languages
  • Excellent retrieval quality
  • Optimized for multilingual RAG
  • Maps semantically equivalent questions into the same vector space

Vector Database

Primary:

ChromaDB

Reason:

  • Open source
  • Local deployment
  • Lightweight
  • Easy LangChain integration

Future Options:

  • Pinecone
  • Weaviate
  • Qdrant

Database

Primary Database:

PostgreSQL

PostgreSQL support must never be removed.

Framework

  • LangChain
  • Streamlit
  • SQLAlchemy
  • psycopg2
  • ChromaDB

Proposed Architecture

User Query ↓ Language Detection ↓ BGE-M3 Embeddings ↓ ChromaDB Vector Search ↓ Retrieve Schema Context ↓ Retrieve Business Definitions ↓ Context Assembly ↓ LangChain SQL Agent ↓ Mistral Large 3 ↓ Generate SQL ↓ SQL Validation ↓ PostgreSQL Execution ↓ Results

Project Structure Changes

Current Structure

app.py db.py chain.py prompts.py

New Structure

app.py db.py chain.py prompts.py

rag/ │ ├── embeddings.py ├── retriever.py ├── vector_store.py ├── indexing.py ├── language_detector.py ├── context_builder.py └── sql_guardrails.py

knowledge/ │ ├── schema_docs/ ├── business_rules/ ├── glossary/ └── kpi_definitions/

vector_db/

Knowledge Sources

The RAG system should index the following information.

Schema Metadata

Example

Table: customers

Description: Stores customer information.

Columns: customer_id name email created_at

Relationship Metadata

Example

orders.customer_id → customers.customer_id

Business Definitions

Example

Revenue

Definition: Total value of completed orders.

KPI Definitions

Example

Monthly Active Users

Definition: Distinct users active within a calendar month.

Data Dictionary

Business terms should be searchable.

Examples

Customer Revenue Sales Profit Retention Churn

Phase 1 - RAG Foundation

Goal

Build retrieval infrastructure.

Tasks

  • Install ChromaDB
  • Configure BGE-M3 embeddings
  • Create indexing pipeline
  • Create retrieval layer
  • Store schema metadata

Deliverables

  • Functional vector database
  • Successful retrieval pipeline

Success Criteria

Questions retrieve relevant schema information.

Phase 2 - Multilingual Retrieval

Goal

Enable language-independent retrieval.

Tasks

  • Integrate BGE-M3 embedding model
  • Add language detection
  • Test retrieval across languages

Example

Hindi Query

राजस्व के अनुसार शीर्ष ग्राहक

Should retrieve:

customers table orders table revenue definition

Success Criteria

Identical retrieval quality across supported languages.

Phase 3 - Dynamic Context Retrieval

Current State

Entire schema injected into prompt.

New State

Only relevant schema sections retrieved.

Prompt Structure

System Prompt

Retrieved Schema Context

Business Definitions

User Question

Benefits

  • Lower token usage
  • Faster responses
  • Better SQL quality

Phase 4 - SQL Accuracy Enhancement

Goal

Improve SQL correctness.

Retrieved Context Should Include

  • Relevant tables
  • Relevant columns
  • Foreign key relationships
  • Business definitions
  • KPI definitions

Expected Results

  • Better joins
  • Fewer hallucinations
  • More accurate aggregations

Phase 5 - Explainable SQL

Goal

Increase transparency.

Output Should Include

Generated SQL

Explanation

Tables Used

Columns Used

Retrieved Context

Example

SQL Generated

SELECT customer_name, SUM(order_amount) FROM orders GROUP BY customer_name

Explanation

This query calculates total revenue generated by each customer.

Phase 6 - Analytics Layer

Goal

Transform assistant into analytics platform.

Features

Automatic Chart Suggestions

Examples

Revenue Trends Sales Forecasts Customer Growth Top Products

Suggested Libraries

  • Plotly
  • Altair

SQL Safety Layer

All generated SQL must pass validation.

Block

DROP

DELETE

TRUNCATE

ALTER

UPDATE

unless explicitly enabled.

Allow

SELECT

WITH

Aggregate Queries

Analytical Queries

Read-only operations

Retrieval Strategy

Embedding Model

BAAI/bge-m3

Search Type

Vector Similarity Search

Similarity Metric

Cosine Similarity

Top K Retrieval

5

Future Enhancement

Hybrid Search

Vector Search + Keyword Search

Chunking Strategy

Chunk Size

300-500 Tokens

Overlap

50-100 Tokens

Chunk Metadata

Source Type

Schema

Business Rule

Glossary

KPI

Table Name

Column Name

Relationship Information

Performance Targets

Retrieval Time

Less than 500 ms

SQL Generation

Less than 5 seconds

Embedding Generation

Less than 1 second

Overall Response Time

Less than 8 seconds

Backward Compatibility Requirements

The existing AI SQL Assistant must remain fully operational throughout development.

The following features must never break:

  • PostgreSQL connectivity
  • SQL generation
  • SQL execution
  • Streamlit UI
  • Existing prompts
  • NVIDIA NIM integration

The RAG layer must enhance the system, not replace its core functionality.

Future Enhancements

Potential Phase 2 Features

  • Query history
  • Conversational memory
  • Multi-database support
  • Dashboard generation
  • Automated insights
  • LangGraph agents
  • Query optimization
  • Fine-tuned SQL models

Success Criteria

The project will be considered successful when:

  1. Users can query databases in multiple languages.
  2. Relevant schema information is retrieved automatically.
  3. SQL accuracy improves significantly.
  4. Token consumption decreases.
  5. Existing functionality remains intact.
  6. PostgreSQL integration remains stable.
  7. The assistant scales to enterprise-sized schemas.

The RAG implementation must augment and strengthen the existing AI SQL Assistant without introducing breaking changes to the current workflow.