NVIDIA NIM RAG Application

By BINATI AInalytics

A Retrieval Augmented Generation (RAG) application built with Streamlit, LangChain, and NVIDIA's NIM (NVIDIA Inference Microservices) API. This application allows you to query documents using natural language and get AI-powered responses based on the document content.

Demo

Features

• Document Processing: Automatically loads and processes PDF documents from a specified directory
• Vector Embeddings: Creates vector embeddings using NVIDIA's embedding models
• Semantic Search: Performs similarity search on document chunks using FAISS vector store
• AI-Powered Responses: Uses NVIDIA's Llama 3.1 Nemotron model for generating contextual answers
• Interactive UI: Clean Streamlit interface for easy interaction
• Document Context: Shows relevant document chunks used for generating responses

Prerequisites

• Python 3.8 or higher
• NVIDIA API Key (sign up at NVIDIA NGC)
• PDF documents to query (place in ./us_census directory)

Installation

1. Clone the repository (or download the script):

   git clone https://github.com/CyprianFusi/nvidia-nim-rag.git
   cd nvidia-nim-rag

2. Install required packages:

   pip install -r requirements.txt

3. Set up environment variables (optional): Create a .env file in the project root:

   NVIDIA_API_KEY=your_nvidia_api_key_here
   

4. Prepare your documents:

   • Create a directory named us_census in the project root
   • Place your PDF documents in this directory

Usage

1. Start the application:

   streamlit run rag_app_nvidia.py

2. Enter your NVIDIA API Key:

   • In the sidebar, enter your NVIDIA API Key
   • If you haven't set it in the .env file, you'll need to enter it manually

3. Embed your documents:

   • Click the "Embed Documents" button in the sidebar
   • Wait for the processing to complete (you'll see "Vector Store DB Is Ready")

4. Ask questions:

   • Type your question in the text input field
   • The AI will provide answers based on your document content
   • View the relevant document chunks in the expandable "Document Similarity Search" section

Configuration

Model Settings

• LLM Model: nvidia/llama-3.1-nemotron-nano-vl-8b-v1
• Chunk Size: 700 characters
• Chunk Overlap: 50 characters
• Document Limit: First 30 documents (configurable)

Customization Options

You can modify these parameters in the code:

# Text splitting parameters
chunk_size=700          # Size of each text chunk
chunk_overlap=50        # Overlap between chunks

# Document processing
st.session_state.docs[:30]  # Number of documents to process

# Model selection
model="nvidia/llama-3.1-nemotron-nano-vl-8b-v1"  # NVIDIA model to use

Project Structure

project-root/
│
├── rag_app_nvidia.py          # Main application file
├── .env                       # Environment variables (optional)
├── us_census/                 # Directory containing PDF documents
│   ├── document1.pdf
│   ├── document2.pdf
│   └── ...
└── README.md                  # This file

How It Works

1. Document Loading: The application loads PDF files from the us_census directory
2. Text Splitting: Documents are split into manageable chunks for processing
3. Vector Embedding: Text chunks are converted to vector embeddings using NVIDIA's embedding model
4. Vector Store: Embeddings are stored in a FAISS vector database for efficient similarity search
5. Query Processing: User questions are converted to embeddings and matched against document chunks
6. Response Generation: Relevant chunks are sent to the NVIDIA LLM along with the user question to generate contextual answers

API Dependencies

This application uses the following NVIDIA NIM services:

• Embeddings: NVIDIA embeddings model for text vectorization
• LLM: Llama 3.1 Nemotron model for text generation

Performance Notes

• Initial document embedding may take some time depending on document size and quantity
• Response times are logged in the console for performance monitoring
• Vector store is cached in Streamlit session state for efficiency

Troubleshooting

Common Issues

1. API Key Error: Ensure your NVIDIA API key is valid and has access to NIM services
2. No Documents Found: Check that PDF files are placed in the us_census directory
3. Memory Issues: Reduce the number of documents processed or chunk size if experiencing memory problems
4. Slow Performance: Consider reducing chunk size or number of documents for faster processing

Error Messages

• "Please enter your NVIDIA API Key to proceed further" - Add your API key in the sidebar
• Processing errors - Check document format and ensure PDFs are readable

Contributing

Feel free to submit issues, fork the repository, and create pull requests for any improvements.

License

This project is open source and available under the MIT License.

Acknowledgments

• Built with LangChain framework
• Powered by NVIDIA NIM APIs
• UI created with Streamlit