A multi-user PDF RAG (retrieval-augmented generation) application — and a portfolio piece whose purpose is to demonstrate retrieval quality and evaluation, not web plumbing. Users authenticate, upload PDFs, and chat with them: ask questions, summarize, or write in a document's style. Each user's documents are private (strict tenant isolation), every answer cites its sources, and the system says "I don't know" when the answer isn't in any retrieved chunk.
The thesis: auth, upload, and the job queue are commodity. The skill lives in the retrieval pipeline and the eval harness. No retrieval change ships without an eval number moving.
A whole PDF is never sent to the LLM per question. Instead each document is ingested once → chunked (page-aware) → embedded → stored. At query time only the relevant chunks are retrieved:
rewrite question (history-aware)
→ vector recall (top-12, pgvector HNSW)
→ identifier router (inject exact lexical matches for code-shaped tokens)
→ rerank (Voyage rerank-2.5, cross-encoder)
→ grounding prompt with citations
→ streamed answer
There's one deliberate, threshold-gated exception: a single small document (≤ 8000 tokens) takes a long-context path instead of RAG. It's explicit, never accidental.
Retrieval changes are validated with bun run eval, not vibes. Results live in
eval/: the harness, the labeled dataset (eval/dataset/questions.jsonl), and recorded
JSON snapshots (eval/results/) with a human-readable ablation log
(eval/results/ABLATION.md).
The recorded progression on the Layer-1 (retrieval-only, page-level) set:
| config | recall@1 | recall@5 | MRR | note |
|---|---|---|---|---|
| vector-only | 0.778 | 0.889 | 0.823 | baseline |
| hybrid-rrf | 0.583 | 0.778 | 0.669 | regressed — not shipped |
| rerank ← vector | 0.917 | 0.917 | 0.917 | +0.139 recall@1 over baseline |
| router + rerank | 1.000 | 1.000 | 1.000 | +0.083 — Layer-1 saturated on corpus |
1.000 is a ceiling on a small, curated 36-question set — the portfolio signal is the delta and the
mechanism (a measured recall failure traced to a root cause and fixed surgically), not the absolute
number.
The eval corpus PDFs are not committed (mixed/unclear licensing). The harness, dataset, and baselines are. To reproduce, drop equivalent PDFs per
eval/corpus/README.md, thenbun run eval:ingestandbun run eval.
| Concern | Choice |
|---|---|
| Framework | Next.js (App Router) — UI and API route handlers |
| Worker | Standalone tsx process consuming BullMQ (long-running ingestion) |
| Database | PostgreSQL + pgvector — vectors live with the data |
| ORM | Drizzle — SQL-first (hybrid search needs raw SQL: RRF, <=>, FTS) |
| Auth | Clerk |
| Queue | BullMQ + Redis (ioredis) |
| Embeddings | Voyage voyage-4-large (1024-d) + reranking rerank-2.5 |
| Generation | Anthropic Claude (claude-sonnet-4-6), Haiku for routing/judge — env-selectable to an OpenAI-compatible provider (Groq) for dev |
| PDF parsing | unpdf |
| Frontend state | Jotai |
| Lint + format | Biome |
| Package manager | bun (Node stays the runtime for Next and the worker) |
- bun (package manager) and Node
- Docker (for Postgres + Redis)
The project will not run without a .env.local file in the repo root. Every script loads it
(tsx --env-file=.env.local …) and Next reads it automatically. Create it with:
# --- infrastructure ---
DATABASE_URL=postgres://postgres:postgres@localhost:5432/pdf_rag
REDIS_URL=redis://localhost:6379
# --- auth (Clerk) ---
NEXT_PUBLIC_CLERK_PUBLISHABLE_KEY=pk_test_...
CLERK_SECRET_KEY=sk_test_...
# --- models ---
VOYAGE_API_KEY=... # embeddings + reranking
LLM_PROVIDER=anthropic # or "groq" for the free dev backend
GROQ_API_KEY=... # only when LLM_PROVIDER=groqdocker compose up -d # Postgres + pgvector and Redis
bun install # install dependencies
bun run db:migrate # apply db/migrations/0001_init.sql
bun run dev # Next app
bun run worker # ingestion worker (separate terminal)Then open the app, sign in via Clerk, upload a PDF, and chat once its status reaches ready.
bun run dev # Next app
bun run worker # ingestion worker
bun run eval # run the eval harness (snapshot + delta)
bun run eval:ingest # ingest the eval corpus
bun run lint && bun run typecheck
bun run test # unit tests