NoteMind — Master Architecture Blueprint

Status as of 2026-08-01. Everything except the AI features is built and tested. Features 7–9 (RAG, chat, summariser) are written but blocked on an OpenAI key + the Atlas vector index — see §9. 78 tests pass (57 backend, 8 frontend, 13 AI). New here? Read WALKTHROUGH.md first — it walks the code phase by phase. This document is the plan we build against, one feature at a time.

---

1. Requirements

1.1 What NoteMind is

A MERN SaaS where a user uploads PDFs, writes notes, and asks questions answered only from their own uploaded documents, via RAG, with citations.

1.2 Functional requirements

#	Requirement	Where it lives
F1	A visitor can register, log in, stay logged in across reloads, and log out	auth.*
F2	A user can create, read, update, delete, search and tag notes	note.*
F3	A user can upload a PDF and see its processing status	pdf.*
F4	A user can ask a question and get an answer with page citations	rag.*, ai/
F5	A user can scope a question to one document or search all of them	chat.ask
F6	A user sees a clear refusal when their documents do not cover the question	rag.py

1.3 Non-functional requirements

• Tenant isolation. No user can read, modify or search another user's data. This is the single most important property in the system; it is enforced in queries, not in if checks, and is covered by dedicated tests.
• No hallucination. Missing info → exactly "The uploaded document does not contain enough information."
• Session safety. Access tokens never touch localStorage; refresh tokens are httpOnly and rotated on every use.
• Responsive. Every page works at 375 / 768 / 1440 px.
• Explainable. Every answer cites the page it came from.

1.4 Out of scope (for now)

Real-time collaboration, sharing notes between users, OCR for scanned PDFs, mobile apps, billing. Deferred items are tracked in §10.

1.5 Process non-negotiables

• One feature at a time, one page at a time, one API at a time.
• Architecture → folders → API flow → schema → backend → frontend → testing. Never skipped.

---

2. Architecture (system level)

┌──────────────┐        ┌────────────────────────────┐        ┌──────────────────┐
│  React SPA   │ HTTPS  │  Express API  :5000        │        │  MongoDB Atlas   │
│  Vite  :5173 │──────► │  routes → controllers →    │──────► │  users, notes,   │
│  Tailwind    │ Axios  │  services → models         │Mongoose│  pdfs, chunks,   │
│  Framer      │        │  middleware: auth, upload, │        │  chats           │
│  Router      │◄────── │  validate, error           │◄────── │  + Vector Index  │
└──────────────┘  JSON  └────────────┬───────────────┘        └──────────────────┘
                                     │ HTTP (internal, shared-secret header)
                              ┌──────▼──────────────┐
                              │  Python AI  :8000   │  ai/ — FastAPI
                              │  parse → chunk →    │
                              │  embed → RAG query  │
                              └──────┬──────────────┘
                                     │
                              ┌──────▼───────┐
                              │  OpenAI API  │  embeddings + chat completion
                              └──────────────┘

Where Python lives — and why. All AI/ML code goes in a separate top-level ai/ service (FastAPI), never inside backend/. Node keeps what it is good at — auth, CRUD, ownership checks, the request/response cycle — and calls ai/ over HTTP for anything needing Python's ML ecosystem (pypdf, tokenizers, embedding libraries). Three reasons this beats embedding Python inside the Node repo or doing RAG in JS:

1. Ownership stays in one place. Every tenant check runs in Express before ai/ is ever called, so the AI service never has to re-implement "does this user own this PDF".
2. Slow work is isolated. PDF parsing and embedding are CPU-heavy; a separate process means they cannot block the Node event loop serving the rest of the app.
3. It scales and deploys independently — and it is the natural seam for the BullMQ queue in §8, since ai/ is already an out-of-process worker.

Three-layer backend, strictly enforced:

Layer	Responsibility	Must NOT do
Route	URL + HTTP verb + middleware chain	contain logic
Controller	read req, call service, shape res	talk to Mongoose
Service	business logic, DB access, OpenAI calls	know about req/res

This separation is what makes the RAG pipeline testable later — services are plain functions.

Auth model: JWT access token (15 min) + refresh token (7 days, httpOnly cookie). Access token held in React memory via Context, never localStorage (XSS surface).

RAG pipeline (feature 4, later):

PDF upload → Multer (disk/temp) → pdf-parse text extraction
  → chunk (≈800 tokens, 150 overlap, page number retained)
  → OpenAI text-embedding-3-small (1536 dims)
  → store chunks in `chunks` collection with embedding + pageNumber + pdfId
  → Atlas Vector Search index on `embedding`
Query → embed question → $vectorSearch topK=6 filtered by userId + pdfId
  → build context block with [page N] markers
  → GPT with strict system prompt: answer ONLY from context, else the fallback sentence
  → return { answer, citations: [{pdfId, page, snippet}] }

---

3. Folder Structure (target, production-ready)

NoteMind/
├── backend/
│   ├── src/
│   │   ├── config/          db.js, env.js, openai.js
│   │   ├── models/          User.js, Note.js, Pdf.js, Chunk.js, Chat.js
│   │   ├── routes/          auth.routes.js, note.routes.js, pdf.routes.js, chat.routes.js
│   │   ├── controllers/     auth.controller.js, ...
│   │   ├── services/        auth.service.js, rag.service.js, embedding.service.js, pdf.service.js
│   │   ├── middleware/      auth.middleware.js, upload.middleware.js,
│   │   │                    validate.middleware.js, error.middleware.js
│   │   ├── utils/           ApiError.js, asyncHandler.js, chunker.js, jwt.js
│   │   ├── validators/      auth.validator.js, ...
│   │   ├── app.js           express app (no listen)
│   │   └── server.js        db connect + listen
│   ├── uploads/             gitignored
│   ├── .env.example
│   └── package.json
│
├── ai/                      Python AI service (FastAPI) — see §2
│   ├── app/
│   │   ├── main.py          FastAPI app + /health
│   │   ├── routers/         ingest.py, query.py        (Feature 7-8)
│   │   ├── services/        pdf.py, chunker.py, embeddings.py, rag.py
│   │   └── schemas/         pydantic request/response models
│   ├── .venv/               gitignored
│   ├── .env.example
│   └── requirements.txt
│
└── frontend/
    ├── src/
    │   ├── api/             axiosInstance.js, auth.api.js, note.api.js, pdf.api.js
    │   ├── components/
    │   │   ├── ui/          Button, Card, Input, Modal, Spinner, Toast
    │   │   ├── layout/      Navbar, Sidebar, DashboardLayout
    │   │   └── feature/     NoteCard, PdfCard, ChatBubble, Citation
    │   ├── context/         AuthContext.jsx
    │   ├── hooks/           useAuth.js, useFetch.js, useDebounce.js
    │   ├── pages/           Landing, Login, Register, Dashboard/{Notes,Pdfs,Chat,Settings}
    │   ├── routes/          AppRoutes.jsx, ProtectedRoute.jsx
    │   ├── lib/             constants.js, formatters.js
    │   └── main.jsx
    ├── tailwind.config.js
    └── package.json

Why app.js separate from server.js: Supertest imports app without opening a port.

---

3A. Technology Stack — and why each piece

Layer	Choice	Why this and not the obvious alternative
UI	React 18 + Vite	Vite's dev server starts in <1s and does HMR over native ESM; CRA is unmaintained.
Routing	React Router 6	Nested routes let <ProtectedRoute> wrap a whole subtree, so a new dashboard page is guarded automatically.
Styling	Tailwind 3	Design tokens live in tailwind.config.js, so the palette is defined once. No naming bikeshed, no dead CSS.
Motion	Framer Motion	Declarative scroll-in animation; hand-rolled IntersectionObserver code would be more lines and less accessible.
HTTP	Axios	Interceptors are the whole reason — token injection and silent refresh live in one file (fetch has no equivalent hook).
API	Express 4	Smallest thing that supports the route→controller→service split cleanly.
DB	MongoDB + Mongoose	Documents match our shape (a chat is a nested message array), and Atlas Vector Search means no separate vector database.
Auth	JWT + bcryptjs	Stateless access tokens scale horizontally; the refresh token in Mongo gives us real revocation.
Uploads	Multer	Streams multipart to disk instead of buffering a 20 MB PDF in memory.
AI runtime	Python + FastAPI	The ML ecosystem (pypdf, tokenizers) is Python-first. FastAPI gives typed request/response models via Pydantic and free OpenAPI docs at /docs.
LLM	OpenAI text-embedding-3-small + gpt-4o-mini	1536 dims is the sweet spot for cost vs recall; gpt-4o-mini at temperature: 0 is cheap and follows the "refuse if unsupported" instruction reliably.
Backend tests	Jest + Supertest + mongodb-memory-server	Runs against a real MongoDB, so indexes and validators are actually exercised. A mocked Mongoose would pass while production rejected the same write.
Frontend tests	Vitest + React Testing Library	Shares Vite's config and transform pipeline — one build setup, not two.
AI tests	pytest	The chunker and citation builder are pure functions; they need no OpenAI key to test.

Why three processes instead of one? Node owns auth, ownership and CRUD. Python owns embeddings and parsing. Splitting them means CPU-heavy PDF work cannot block the Node event loop serving the rest of the app, and each scales independently. The cost is one network hop, which is negligible next to an OpenAI call.

---

3B. API Endpoints

Every response uses the same envelope, so the frontend never has to guess:

// success
{ "success": true,  "message": "Note created", "data": { /* ... */ } }
// failure
{ "success": false, "message": "Validation failed", "details": [ { "field": "title", "message": "..." } ] }

Auth — /api/auth

Method	Path	Auth	Body	Returns
POST	/register	—	{name,email,password}	201 {user, accessToken} + refresh cookie
POST	/login	—	{email,password}	200 {user, accessToken} + refresh cookie
POST	/refresh	cookie	—	200 {user, accessToken} + rotated cookie
POST	/logout	cookie	—	200, refresh token revoked server-side
GET	/me	Bearer	—	200 {user}

Notes — /api/notes (all require Bearer)

Method	Path	Body / Query	Returns
GET	/	?search=&tag=&page=&limit=	{items, total, page, limit}
POST	/	{title, content?, tags?, isPinned?}	201 {note}
GET	/:id	—	{note}
PATCH	/:id	any subset of the above	{note}
DELETE	/:id	—	200

PDFs — /api/pdfs (all require Bearer)

Method	Path	Body	Returns
GET	/	—	{pdfs}
POST	/	multipart/form-data, field file	201 {pdf} — status processing, ingestion continues in background
GET	/:id	—	{pdf} — poll this for status
GET	/:id/file	—	the raw PDF bytes (ownership-checked)
DELETE	/:id	—	200 — also deletes chunks + the file on disk

Chat — /api/chat (all require Bearer)

Method	Path	Body	Returns
POST	/ask	{question, pdfId?, chatId?}	{answer, citations[], chatId}
GET	/	—	{chats}
GET	/:id	—	{chat} with full message history

Users — /api/users (all require Bearer)

Method	Path	Body	Returns
PATCH	/me	{name?, email?}	{user} — 409 if the email belongs to someone else
PATCH	/me/password	{currentPassword, newPassword}	200, revokes every session

Health — GET /api/health → {status, environment, database, uptimeSeconds}

Internal (Python, not browser-reachable) — POST /ingest, POST /embed, POST /answer, GET /health. All but /health require the x-internal-key header.

Status codes

400 validation · 401 missing/invalid token · 403 authenticated but not allowed · 404 not found or not yours · 409 duplicate · 502/503/504 AI service failure.

Why 404 and not 403 for another user's data: returning 403 would confirm the resource exists. 404 leaks nothing — you cannot even probe which ids are real.

---

3C. Application Flow

Registration / login

Browser                    Express                     Mongo
   │  POST /auth/register     │                          │
   ├─────────────────────────►│ validate → hash (bcrypt) │
   │                          ├─────────────────────────►│ insert user
   │                          │ sign access + refresh    │
   │◄─────────────────────────┤ Set-Cookie: refreshToken │
   │  {user, accessToken}     │   (httpOnly, rotated)    │
   │                          │
   │  access token → memory (React state), never localStorage

Session restore on page reload

App mounts → AuthContext calls POST /auth/refresh (cookie sent automatically)
   ├─ 200 → setAccessToken(...) → user restored → ProtectedRoute renders the page
   └─ 401 → user stays null → ProtectedRoute redirects to /login
While this is in flight `loading === true`, and ProtectedRoute renders a spinner —
redirecting during that window would log out a perfectly valid user on every reload.

PDF upload → ingestion (async)

Browser ──multipart──► Express ──► Multer writes to backend/uploads/<uuid>.pdf
                          │        create Pdf{status:'uploaded'}
                          │◄── 201 returned IMMEDIATELY (does not wait)
                          │
                          └─ background: status='processing'
                             POST ai:8000/ingest {file_path,...}
                                Python: pypdf → page text
                                        chunk (≈800 tok, 150 overlap, page kept)
                                        embed batches of 96
                             ◄── {pages, chunks[{text,page_number,embedding}]}
                             insertMany(chunks) → status='ready'   (or 'failed' + error)

Browser polls GET /api/pdfs every 3s while anything is 'uploaded'|'processing',
and stops polling the moment nothing is in flight.

Asking a question (RAG)

POST /api/chat/ask {question, pdfId?}
   │
   ├─► ai:8000/embed        → question vector
   ├─► Mongo $vectorSearch  → top 6 chunks, filter {user, pdf?}   ◄── tenant boundary
   │      └─ no matches? → return the refusal sentence, never call the LLM
   ├─► ai:8000/answer       → gpt-4o-mini, temperature 0, strict system prompt
   │                          context passages carry [page N] markers
   ├─  build citations from the markers the model actually used
   └─► persist both messages to the Chat document
   ◄── {answer, citations:[{pdf,page,snippet}], chatId}

---

3D. How the Frontend and Backend Communicate

One origin in dev. Vite proxies /api → http://localhost:5000 (frontend/vite.config.js). The browser only ever sees localhost:5173, so there is no CORS preflight and no third-party-cookie problem during development.

One Axios instance (frontend/src/api/axiosInstance.js) owns every cross-cutting concern:

1. Request interceptor attaches Authorization: Bearer <token> from a module variable.
2. Response interceptor unwraps response.data, so components read res.data.note rather than res.data.data.note.
3. On 401 it performs one silent refresh and replays the original request. The user never sees the 15-minute token expiry.
4. Error normalisation turns every failure into {status, message, details} — components never dig through Axios error objects.

One API module per resource (auth.api.js, note.api.js, pdf.api.js, chat.api.js). Components import functions, never URLs, so an endpoint move is a one-line change.

Where the two tokens live, and why

Token	Stored	Lifetime	Reasoning
Access	JS module variable	15 min	localStorage is readable by any script, so one XSS hole = a stolen session. A module variable dies with the tab.
Refresh	httpOnly cookie, path=/api/auth	7 days	JavaScript cannot read it at all. The narrow path means it is not attached to ordinary API calls.

Role-based URLs

The dashboard lives under a role segment: /u/dashboard/... for users, /a/dashboard/... for admins. /dashboard/... still works and redirects to the right prefix.

The segment is navigation, not permission. It is derived from user.role, which comes from the database. RoleRoute.jsx only rewrites a URL that disagrees with your actual role — a normal user typing /a/dashboard/notes is sent back to /u/.... Even without that, every protected endpoint re-reads the role from the user document via requireAuth / requireRole, so a hand-typed URL grants nothing.

role cannot be self-assigned: validateRegister and validateUpdateProfile whitelist fields, so {"role":"admin"} in a request body is dropped. Tested in role.test.js.

Refresh is single-flight. The server rotates the refresh token on every use, so two overlapping refreshes would invalidate each other — the second sends a token the first just replaced, gets a 401, and logs out a valid user. React StrictMode double-invokes effects in dev and triggers this reliably, so refreshAccessToken() shares one in-flight promise between all callers.

---

4. Database Schema (all collections, final target)

users — { name, email (unique, lowercase), password (bcrypt, select:false), avatar, refreshToken, timestamps }

notes — { user (ref, indexed), title, content, tags:[String], isPinned, timestamps } Compound index { user: 1, updatedAt: -1 } for the dashboard list. Text index on { title, content } for search.

pdfs — { user (ref), originalName, storedName, size, pages, status: 'uploaded'|'processing'|'ready'|'failed', error, timestamps }

chunks — { user (ref), pdf (ref, indexed), text, pageNumber, chunkIndex, embedding: [Number] } Atlas Vector Search index vector_index on embedding, 1536 dims, cosine, with user and pdf as filter fields (critical — this is the tenant isolation boundary).

chats — { user (ref), pdf (ref, nullable), title, messages: [{ role, content, citations:[{pdf, page, snippet}], createdAt }], timestamps }

---

5. Build Order (one per session, each needs your confirmation)

#	Feature	Ships
#	Feature	Status
---	---	---
1	Project scaffold	✅ done — both apps boot, theme tokens, Axios instance, error middleware, health route, all 5 schemas, ai/ service
2	Auth	✅ done — register/login/refresh/logout/me, AuthContext, ProtectedRoute, Login + Register. 19 tests
3	Landing page	✅ done — sticky navbar, asymmetric hero, features, how-it-works, CTA, footer, Framer Motion
4	Dashboard shell	✅ done — responsive sidebar, nested routing, empty states
5	Notes CRUD	✅ done — full stack, debounced search, tags, pinning. 14 tests
6	PDF upload	✅ done — Multer, list, progress bar, status polling, delete. In-browser viewer deferred (§10)
7	RAG ingestion	⚠️ code complete, unverified — parse → chunk → embed → store. Needs OPENAI_API_KEY + Atlas Vector Search index
8	AI chat + citations	⚠️ code complete, unverified — $vectorSearch, strict prompt, citations, history. Same two prerequisites
9	Summarizer	❌ not started (AI — needs the §9 prerequisites)
10	Settings + polish	✅ done — editable profile, password change with session revocation, responsive audit. Theme switcher deferred
11	Deployment	✅ documented — see DEPLOYMENT.md

---

6. Design System (from the inspiration, NOT copied)

The reference is indigo-on-white, centered hero, pill badge, soft-bordered cards. NoteMind takes the feeling, not the layout — we differentiate with an asymmetric hero (copy left, live document-preview card right) and a violet→cyan gradient rather than indigo→sky.

// tailwind.config.js theme extension
colors: {
  brand: { 50:'#f5f3ff', 100:'#ede9fe', 500:'#8b5cf6', 600:'#7c3aed', 700:'#6d28d9' },
  accent:{ 400:'#22d3ee', 500:'#06b6d4' },
  ink:   { 900:'#0b1020', 700:'#232a3d', 500:'#5b6478', 300:'#9aa3b8' },
  surface:{ DEFAULT:'#ffffff', muted:'#f8fafc', border:'#e9edf5' },
},
borderRadius: { xl:'14px', '2xl':'20px', '3xl':'28px' },
boxShadow: {
  soft:  '0 1px 2px rgba(16,24,40,.04), 0 8px 24px rgba(16,24,40,.06)',
  lift:  '0 12px 32px rgba(109,40,217,.14)',
},
fontFamily: { sans:['Inter','system-ui','sans-serif'], display:['Sora','Inter','sans-serif'] }

Rules: 8px spacing scale, section padding py-24, max width max-w-6xl, cards rounded-2xl border border-surface-border shadow-soft, primary button bg-gradient-to-r from-brand-600 to-accent-500 with hover:shadow-lift hover:-translate-y-0.5 transition. Motion: fade+y:16 on scroll-in, duration .5 ease-out, stagger .08 — nothing bouncy.

---

7. Testing Strategy

54 tests currently pass.

• Backend (33): Jest + Supertest + mongodb-memory-server. Every endpoint gets happy path, validation failure (400), auth failure (401), and ownership failure. The ownership suite is the one that matters most in a multi-tenant RAG app — user B must not be able to read, update or delete user A's note even with a perfectly valid token.
• Frontend (8): Vitest + React Testing Library on AuthContext and ProtectedRoute — specifically the "still loading" case, which is what stops a valid session being redirected away on reload.
• AI (13): pytest on chunker.py and the citation builder. Both are pure functions, so they need no OpenAI key. This is where the mid-word chunk-splitting bug was caught.
• Manual: every page checked at 375 / 768 / 1440 px before a feature is called done.

Ownership failures return 404, not 403 — see §3B for why.

---

8. Running Locally

Three processes. The first two are all you need until you wire up OpenAI.

cd backend  && npm install && cp .env.example .env && npm run dev   # :5000
cd frontend && npm install && cp .env.example .env && npm run dev   # :5173
cd ai       && python -m venv .venv && .venv\Scripts\activate \
            && pip install -r requirements.txt && uvicorn app.main:app --reload --port 8000

Open http://localhost:5173.

MONGO_URI is now required — auth and notes cannot work without it. Set it to an Atlas connection string (or a local mongodb://127.0.0.1:27017/notemind). JWT_ACCESS_SECRET and JWT_REFRESH_SECRET are also required and must be different values; the test environment supplies its own throwaway pair, so npm test needs no .env.

Tests: npm test in backend/ and frontend/, pytest tests in ai/.

Python version note: ai/requirements.txt uses lower bounds rather than exact pins. This machine runs Python 3.14, and older exact pins had no 3.14 wheels — pip fell back to compiling pydantic-core from Rust source and failed.

---

9. Prerequisites Before Features 7 & 8 Work

The RAG code is written but cannot run until both of these exist:

1. OPENAI_API_KEY in ai/.env. Without it /ingest and /embed return 503 with a clear message rather than failing obscurely.
2. An Atlas Vector Search index named vector_index on the chunks collection. Mongoose cannot declare this — create it in the Atlas UI:

{
  "fields": [
    { "type": "vector", "path": "embedding", "numDimensions": 1536, "similarity": "cosine" },
    { "type": "filter", "path": "user" },
    { "type": "filter", "path": "pdf" }
  ]
}

The two filter entries are not optional — they are the tenant isolation boundary. Without them $vectorSearch would search across every user's chunks.

Also note $vectorSearch is an Atlas-only aggregation stage. It does not exist in a local mongod or in mongodb-memory-server, which is why Features 7–8 have no integration tests.

---

10. Future Improvements

Deferred deliberately: in-browser PDF viewer (react-pdf); summarizer (Feature 9); editable profile, password change and theme (Feature 10); hashing refresh tokens at rest.

Longer term: Redis-backed embedding cache; BullMQ queue for PDF ingestion so uploads survive a restart; streaming chat responses via SSE; hybrid search (vector + Atlas text) for better recall on acronyms; per-user rate limiting on the OpenAI routes; S3 instead of local disk; usage/cost dashboard per user.

---

11. Bugs Found and Fixed During Implementation

Recorded because each one is a trap worth not re-entering.

Bug	Consequence	Fix
jwt.js read env.JWT_SECRET, which did not exist	jwt.sign threw — auth could never have worked	Added the two secrets to env.js and made them required at boot
jsonwebtoken was imported but not in package.json	Module-not-found on any auth route	Installed it
One secret signed both token types	An access token would be accepted as a refresh token	Separate JWT_ACCESS_SECRET / JWT_REFRESH_SECRET; a test asserts the swap fails
Refresh cookie used sameSite:'none' without secure	Chrome silently drops the cookie on http://localhost — every reload logged the user out	'lax' in dev, 'none' + secure in prod
Refresh-token rotation raced React StrictMode's double-mounted effect	Two concurrent refreshes invalidated each other → random logouts on reload	Single-flight shared promise in refreshAccessToken()
AuthContext.logout used finally, not catch	A failed logout request rethrew, skipping navigate and stranding the user	Swallow the network error; always clear the session locally
Chunker snapped chunk ends to word boundaries but not starts	Overlapping chunks began mid-word ("ord24 word25…"), degrading embeddings	Advance the overlap start to the next whitespace

---

Next Step

Two things unblock the rest, in this order:

1. Add OPENAI_API_KEY to ai/.env and create the Atlas vector index from §9. That turns Features 7–8 from "written" into "working", and is the only thing standing between the current build and an end-to-end RAG demo.
2. Feature 9 (summarizer) and the rest of Feature 10 (editable profile, password change, theme) are the remaining unwritten features.

Smaller follow-ups worth doing at the same time: the in-browser PDF viewer (react-pdf), and hashing refresh tokens before storing them.