🚀 MCP RAG with Groq Llama 3.3 70B

An intelligent document retrieval and chat system that integrates Groq's free Llama 3.3 70B model with Model Context Protocol (MCP) tools for seamless document search and web browsing capabilities.

MCP.RAG.Video.Demo.mp4

✨ Features

• 🤖 Free Powerful LLM: Groq's Llama 3.3 70B model (completely free!)
• 📄 Smart Document Retrieval: Hybrid RAG with reranking from your Qdrant database
• 🌐 Real-time Web Search: Integrated Tavily API for current information
• 🔧 MCP Integration: Model Context Protocol for seamless tool calling
• 💬 Intelligent Chat: LLM automatically decides when to use tools
• 🎨 Beautiful UI: Modern Gradio interface with multiple tabs
• ⚡ Fast Performance: Sub-second document retrieval, 2-5s total response time

🎯 Demo

Ask questions like:

• "Who is Jawher Khalifa?" → Uses document retrieval
• "What are the latest AI trends in 2025?" → Uses web search
• "Tell me about the ML projects in the resume" → Uses both tools

The LLM intelligently chooses which tools to use based on your question!

📋 Prerequisites

Before deployment, ensure you have:

1. Python 3.8+ installed
2. Qdrant Database running (local or remote)
3. Groq API Key (free from console.groq.com)
4. Tavily API Key (optional, for web search)
5. Documents loaded in your Qdrant collection

🚀 Quick Start Deployment

Step 1: Clone and Setup Environment

# Navigate to your project directory
git clone https://github.com/jawherkh/MCP-RAG-Agent.git
cd "MCP-RAG-Agent"

# Install required packages
pip install groq gradio python-dotenv qdrant-client langchain-qdrant langchain-huggingface tavily-python sentence-transformers

Step 2: Get Your API Keys

Groq API Key (Free)

1. Visit console.groq.com
2. Sign up for a free account
3. Go to "API Keys" section
4. Create a new API key
5. Copy the key (starts with gsk_)

Tavily API Key (Optional)

1. Visit tavily.com
2. Sign up and get your API key
3. Copy the key (starts with tvly-)

Step 3: Configure Environment

manually create/edit .env file:

GROQ_API_KEY=gsk_your_groq_api_key_here
TAVILY_API_KEY=tvly_your_tavily_api_key_here

Step 4: Start Qdrant Database

Option A: Docker (Recommended)

docker run -p 6333:6333 -v ${pwd}/qdrant_data:/qdrant/storage qdrant/qdrant

Option B: Local Installation

# Download and run Qdrant locally
# Follow instructions at: https://qdrant.tech/documentation/quick-start/

Step 5: Load Your Documents

If you haven't loaded documents yet:

# Open and run the chunking notebook
jupyter notebook data/chunking.ipynb

Step 6: Deploy the Application

Option A: Web Interface (Recommended)

# Start the Gradio web app
python llm_app.py

Then open: http://localhost:7861

Option B: Command Line Interface

# Start the CLI chat
python llm_mcp_client.py

📚 API Reference

Main Functions

• GroqMCPClient.chat(message, system_prompt) - Main chat function
• retrieve_tool(query, k) - Document retrieval
• websearch_tool(query, k) - Web search

Environment Variables

Variable	Required	Description
GROQ_API_KEY	Yes	Groq API key for Llama access
TAVILY_API_KEY	No	Tavily API key for web search
QDRANT_URL	No	Qdrant URL (default: localhost:6333)
QDRANT_API_KEY	No	Qdrant API key if using cloud

🗂️ Project Structure

MCP RAG/
├── llm_app.py            # Gradio web interface for chat and tool testing
├── llm_mcp_client.py     # LLM client that integrates Groq with MCP tools (CLI)
├── mcp_tools.py          # Shared tools for both server and client (retrieval, websearch)
├── server.py             # MCP server exposing tools via FastMCP
├── requirements.txt      # Python dependencies
├── README.md             # Documentation and deployment instructions
├── .env                  # Environment variables (API keys, not committed)
├── .env.example          # Example environment file for deployment
├── deploy.py             # Automated deployment script (optional)
├── Dockerfile            # Docker container configuration (optional)
├── docker-compose.yml    # Docker Compose for app + Qdrant (optional)
├── utils/
│   ├── retrievers.py     # Document retrieval logic (hybrid, reranking)
│   └── ranker.py         # Reranking utilities
├── data/
│   ├── chunking.ipynb    # Notebook for document chunking and loading into Qdrant
│   └── docs/             # Folder for your PDF and other documents
├── __pycache__/          # Python cache files (ignored)
└── ...                   # Other supporting files

• llm_app.py: Main Gradio app for chatting with the LLM and using tools via UI.
• llm_mcp_client.py: Command-line client for LLM + MCP tools (for testing or automation).
• mcp_tools.py: Core logic for retrieval and websearch, shared by both server and client.
• server.py: Runs the MCP server, exposing tools for LLM or other clients.
• utils/: Custom retrieval and reranking logic for hybrid RAG.
• data/: Notebooks and document storage for chunking/loading into Qdrant.
• requirements.txt: All Python dependencies for the project.
• .env / .env.example: API keys and environment configuration.
• Dockerfile / docker-compose.yml: For containerized and production deployments.

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Add your improvements
4. Test thoroughly
5. Submit a pull request

📄 License

This project is for educational and research purposes. Please respect the terms of service for all APIs used.

---

🎉 You're Ready to Deploy!

Your MCP RAG system with Groq Llama 3.3 70B is now ready for deployment. The LLM will automatically use document retrieval and web search to provide intelligent, contextual responses.

Need help? Check the troubleshooting section or create an issue.

Enjoy your free, powerful AI assistant! 🚀