EduRAG — University Knowledge Base

A self-hostable RAG knowledge base for universities built with Next.js 16, Vercel AI SDK 6, MongoDB Atlas Vector Search, and Tavily.

Administrators crawl and index institutional content via a protected backend dashboard. Students interact with a public chat UI that surfaces cited answers and auto-generated FAQs.

Features

Knowledge Ingestion — Admin crawls institutional URLs via Tavily, chunks, embeds, and stores in MongoDB Atlas Vector Search
Conversational RAG — Students query an AI agent with semantic vector search, streamed cited answers
Voice Agent — High-fidelity voice interaction powered by Deepgram (Nova-3 STT & Aura-2 TTS)
Auto-FAQ Generation — Frequently asked questions are tracked and auto-synthesized
Automated Onboarding — Streamlined setup for branding, API configuration, and initial indexing
AI Elements UI — Production-grade chat components with citations
Admin Dashboard — Domain management, crawl progress, FAQ approval queue

Quick Start

1. Prerequisites

Node.js 18+
MongoDB Atlas account (for vector search)
Tavily API key
OpenAI-compatible LLM API (Cerebras, OpenAI, etc.)
Embedding API (Voyage AI recommended)

2. Install Dependencies

git clone <your-repo>
cd edurag
npm install

3. Environment Setup

Copy the example environment file:

cp .env.local.example .env.local

Edit .env.local with your credentials:

# Required
AUTH_SECRET=your-random-32-char-secret
BETTER_AUTH_URL=http://localhost:3000
CHAT_API_KEY=your-llm-api-key
CHAT_BASE_URL=https://api.cerebras.ai/v1
CHAT_MODEL=gpt-oss-120b

EMBEDDING_API_KEY=your-voyage-api-key
EMBEDDING_BASE_URL=https://api.voyageai.com/v1
EMBEDDING_MODEL=voyage-4-large
EMBEDDING_DIMENSIONS=2048

MONGODB_URI=mongodb+srv://user:pass@cluster.mongodb.net
TAVILY_API_KEY=tvly-your-key
ADMIN_SECRET=your-secure-admin-token-min-16-chars

# Voice Agent (Deepgram)
DEEPGRAM_API_KEY=your-deepgram-key
DEEPGRAM_STT_MODEL=nova-3
DEEPGRAM_TTS_MODEL=aura-2-thalia-en

# Optional
UNIVERSITY_URL=https://university.edu
UPLOADTHING_SECRET=sk_live_...
UPLOADTHING_APP_ID=...

4. MongoDB Atlas Vector Search Setup

Step 1: Create a MongoDB Atlas Cluster

Go to MongoDB Atlas
Create a free M0 cluster or use an existing one
Create a database user with read/write permissions
Whitelist your IP address (or 0.0.0.0/0 for development)

Step 2: Get Your Connection String

Click "Connect" on your cluster
Choose "Connect your application"
Copy the connection string and replace <password> with your database user password
Set this as MONGODB_URI in .env.local

Step 3: Create the Vector Search Index

Go to your cluster → Atlas Search → Create Search Index
Choose JSON Editor
Select the database (edurag) and collection (crawled_index)
Paste this index definition:

{
  "fields": [
    {
      "type": "vector",
      "path": "embedding",
      "numDimensions": 2048,
      "similarity": "cosine"
    }
  ]
}

Note: Thread ID filtering is done in code after vector search (post-filter) for maximum accuracy. No filter fields needed in the index.

Name the index index (matches VECTOR_INDEX_NAME in env)
Click Create Search Index

Note: numDimensions must match EMBEDDING_DIMENSIONS.

Voyage voyage-4-large = 2048

OpenAI text-embedding-3-large = 3072

OpenAI text-embedding-3-small = 1536

Step 4: Create Additional Indexes (Optional but Recommended)

In the Atlas UI, go to your database and create these indexes:

conversations collection:

{ "threadId": 1 }

faqs collection:

{ "normalized": 1 }  // unique
{ "public": 1, "count": -1 }

domains collection:

{ "url": 1 }  // unique

5. Run the Development Server

npm run dev

Open http://localhost:3000

Deploying to Netlify

1. Install the Netlify Next.js Plugin

npm install -D @netlify/plugin-nextjs

2. Set Environment Variables in Netlify Dashboard

Go to Site Settings → Environment Variables and add:

Variable	Value	Required
`AUTH_SECRET`	Random 32+ char secret (`npx auth secret`)	Yes
`BETTER_AUTH_URL`	Your full deploy URL e.g. `https://your-site.netlify.app`	Yes
`MONGODB_URI`	MongoDB Atlas connection string	Yes
`CHAT_API_KEY`	LLM API key	Yes
`CHAT_BASE_URL`	LLM base URL	Yes
`CHAT_MODEL`	`gpt-oss-120b`	Yes
`EMBEDDING_API_KEY`	Voyage AI key	Yes
`TAVILY_API_KEY`	Tavily key	Yes
`ADMIN_SECRET`	Min 16 chars	Yes
`DEEPGRAM_API_KEY`	Deepgram API key	For voice

Important: BETTER_AUTH_URL must exactly match your Netlify site URL (no trailing slash). Without it, auth redirects back to localhost after sign-in.

3. Deploy

# Push to your branch, or trigger a manual deploy in Netlify dashboard
git push origin main

Project Structure

edurag/
├── app/
│   ├── (public)/
│   │   ├── chat/page.tsx         # Student text chat UI
│   │   ├── (voice mode is integrated in chat/page.tsx via ?voice=1)
│   │   └── page.tsx              # Landing page
│   ├── admin/
│   │   ├── layout.tsx            # Auth guard
│   │   ├── login/page.tsx        # Token login
│   │   ├── page.tsx              # Dashboard
│   │   ├── domains/page.tsx      # Domain management
│   │   └── faqs/page.tsx         # FAQ approval
│   ├── setup/
│   │   └── page.tsx              # Automated onboarding flow
│   └── api/
│       ├── chat/route.ts         # Streaming text chat
│       ├── voice-token/route.ts  # Deepgram token auth
│       ├── voice-function/route.ts # Voice agent tool calling
│       ├── crawl/route.ts        # SSE crawl progress
│       ├── domains/route.ts      # Domain CRUD
│       ├── faqs/route.ts         # Public FAQs
│       ├── history/route.ts      # Conversation list
│       ├── history/[threadId]/route.ts # Conversation read/append/delete
│       └── threads/route.ts      # Thread deletion
├── lib/
│   ├── voice/                    # Deepgram & Voice logic
│   ├── agent/                    # App-specific agent wiring
│   ├── providers.ts              # LLM + Embedding factories
│   ├── vectorstore.ts            # MongoDB vector store
│   ├── crawl.ts                  # Tavily crawl pipeline
│   ├── auth.ts                   # Admin auth
│   ├── env.ts                    # Zod-validated env
│   └── errors.ts                 # Error handling
└── packages/
    └── agent/                    # Core Agent Orchestration logic

Usage

Admin Dashboard

Navigate to /admin/login
Enter your ADMIN_SECRET token
Add a domain URL to crawl (e.g., https://university.edu)
Configure crawl options (depth, breadth, limit, paths)
Click "Crawl & Index" — watch live SSE progress
Review and approve auto-generated FAQs

Student Chat

Navigate to /chat for text-based interaction.
Start voice by clicking the phone button in /chat (or open /chat?voice=1).
Ask questions about the university and view cited answers.
Session history is saved automatically in the sidebar.

API Reference

POST `/api/chat`

Streaming chat endpoint using Vercel AI SDK.

Request:

{
  "messages": [{ "id": "x", "role": "user", "content": "What are admission requirements?" }],
  "threadId": "session-123"
}

Response: SSE stream with toUIMessageStreamResponse()

POST `/api/crawl`

Admin-only crawl endpoint with SSE progress.

Request:

{
  "url": "https://university.edu",
  "threadId": "domain-123",
  "maxDepth": 2,
  "maxBreadth": 20,
  "limit": 100
}

Response: SSE events { type: 'status'|'progress'|'complete'|'error' }

GET `/api/faqs`

Public FAQ list (ISR cached).

GET|POST|DELETE `/api/domains`

Admin-only domain registry CRUD.

GET `/api/history`

Authenticated conversation list for the current user.

GET|POST|DELETE `/api/history/[threadId]`

Authenticated conversation read/append/delete for a single thread.

DELETE `/api/threads`

Thread clear endpoint implemented in app/api/threads/route.ts.

Auth: Requires an authenticated user session.

Request:

{
  "threadId": "session-123"
}

Success (200):

{
  "success": true
}

Error Responses:

401 Unauthorized

{
  "error": "Unauthorized",
  "code": "UNAUTHORIZED"
}

400 Bad Request

{
  "error": "Invalid request body",
  "code": "VALIDATION_ERROR"
}

500 Internal Server Error

{
  "error": "Failed to clear thread history",
  "code": "DB_ERROR"
}

In development, error responses can include an additional details field.

Deployment

Vercel (Recommended)

Push repo to GitHub
Import at vercel.com
Add all environment variables
Mark ADMIN_SECRET as sensitive
Deploy

Render

Create a Web Service
Build: npm run build
Start: node .next/standalone/server.js
Add output: 'standalone' to next.config.ts

Docker

FROM node:20-alpine
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build
CMD ["npm", "start"]

Environment Variables Reference

Variable	Description	Required
`AUTH_SECRET`	Random 32+ character secret, e.g., generated with `npx auth secret`	Yes
`BETTER_AUTH_URL`	Full deploy URL or localhost during development	Yes
`CHAT_API_KEY`	LLM API key	Yes
`CHAT_BASE_URL`	LLM API endpoint	Yes
`CHAT_MODEL`	Model name	Yes
`EMBEDDING_API_KEY`	Embedding API key	Yes
`EMBEDDING_BASE_URL`	Embedding endpoint	Yes
`EMBEDDING_MODEL`	Embedding model	Yes
`EMBEDDING_DIMENSIONS`	Vector dimensions (must match index)	Yes
`MONGODB_URI`	MongoDB Atlas connection string	Yes
`TAVILY_API_KEY`	Tavily crawl API key	Yes
`ADMIN_SECRET`	Admin auth token (min 16 chars)	Yes
`DEEPGRAM_API_KEY`	Deepgram API key	For voice
`DEEPGRAM_STT_MODEL`	Deepgram STT model	For voice
`DEEPGRAM_TTS_MODEL`	Deepgram TTS model	For voice
`FAQ_THRESHOLD`	Questions before FAQ synthesis	Default: 5
`CRAWL_*`	Crawl defaults	Optional

Tech Stack

Layer	Package
Framework	Next.js 16
AI SDK	Vercel AI SDK 6
Voice	Deepgram (v4 SDK)
Core Agent	@edurag/agent (Workspace)
Vector Store	@langchain/mongodb
LLM Provider	Cerebras (gpt-oss-120b)
Embeddings	Voyage AI (voyage-4-large)
Crawling	@tavily/core
Database	MongoDB Atlas
UI	shadcn/ui, AI Elements, Tailwind

License

MIT

Name		Name	Last commit message	Last commit date
Latest commit History 145 Commits
__tests__		__tests__
app		app
components		components
lib		lib
packages/agent		packages/agent
public		public
scripts		scripts
.env.example		.env.example
.gitignore		.gitignore
.npmrc		.npmrc
AGENTS.md		AGENTS.md
LICENSE		LICENSE
README.md		README.md
auth.ts		auth.ts
eslint.config.mjs		eslint.config.mjs
middleware.ts		middleware.ts
next.config.ts		next.config.ts
package-lock.json		package-lock.json
package.json		package.json
postcss.config.mjs		postcss.config.mjs
tsconfig.json		tsconfig.json
vitest.config.ts		vitest.config.ts

Folders and files

Latest commit

History

Repository files navigation

EduRAG — University Knowledge Base

Features

Quick Start

1. Prerequisites

2. Install Dependencies

3. Environment Setup

4. MongoDB Atlas Vector Search Setup

Step 1: Create a MongoDB Atlas Cluster

Step 2: Get Your Connection String

Step 3: Create the Vector Search Index

Step 4: Create Additional Indexes (Optional but Recommended)

5. Run the Development Server

Deploying to Netlify

1. Install the Netlify Next.js Plugin

2. Set Environment Variables in Netlify Dashboard

3. Deploy

Project Structure

Usage

Admin Dashboard

Student Chat

API Reference

POST /api/chat

POST /api/crawl

GET /api/faqs

GET|POST|DELETE /api/domains

GET /api/history

GET|POST|DELETE /api/history/[threadId]

DELETE /api/threads

Deployment

Vercel (Recommended)

Render

Docker

Environment Variables Reference

Tech Stack

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages

POST `/api/chat`

POST `/api/crawl`

GET `/api/faqs`

GET|POST|DELETE `/api/domains`

GET `/api/history`

GET|POST|DELETE `/api/history/[threadId]`

DELETE `/api/threads`