Noēsis

An iterative, confidence-gated research pipeline that plans queries, searches the web, extracts findings, and only writes a report once it has enough grounded evidence — with citations checked in code, not just prompted for.

🌐 Live Demo • 📸 Screenshots • 🏗 Architecture • 🧭 Architecture Decisions • 📖 Documentation

What this is

Noēsis takes a research topic, asks clarifying questions, then runs a loop: generate search queries → search the web (Exa) → extract findings from each result with an LLM → have a second LLM pass judge whether those findings are sufficient for a good report → if not, generate follow-up queries and repeat, up to 3 iterations. Once the findings pass a confidence gate, a final LLM call writes the report, and every claim it draws from a finding has to carry that finding's citation number — a validator checks that after the fact, it isn't just a prompt instruction taken on faith.

The interesting engineering problem here wasn't "wire up an LLM" — it's what happens when the LLM call fails, times out, or a free-tier model gets rate-limited mid-run. Every stage has a non-LLM fallback so the pipeline degrades instead of dying, and the whole thing streams its progress to the client as it happens.

I originally built this on a separate FastAPI + LangGraph + Neo4j + Redis backend to explore multi-agent orchestration and graph-based retrieval. I cut all of it before shipping: it added three more services to keep alive, was never actually reachable from the deployed frontend (the backend only ran locally), and none of the extra machinery was pulling weight the loop below doesn't already cover. What's described here is what's actually running at the live demo link above.

---

✨ Features

• Iterative research loop with a numeric confidence gate (stops early on strong findings, caps at 3 iterations otherwise)
• Citation validation in code — reports must cite findings by number; a validator flags out-of-range or missing citations instead of trusting the prompt
• Three-tier failure handling — per-stage model fallback chains → retry with backoff → deterministic local fallback (extractive summary, heuristic confidence score, findings-only report) if the LLM path is unavailable
• Live progress streaming over SSE (Vercel AI SDK data stream), not a post-hoc chunked replay
• Local-first session storage — runs with zero database via a JSON file store; switches to Neon Postgres transparently when DATABASE_URL is set, and falls back to local storage again if Postgres becomes unreachable mid-request
• Per-visitor rate limiting (5 research runs/hour) so a single user can't drive unbounded LLM spend
• Anonymous session ownership via HTTP-only visitor cookies, no signup required

---

🛠 Tech Stack

Model calls go through OpenRouter (Llama 3.3 70B for planning/report, Qwen 2.5 7B for extraction/analysis, each with its own fallback chain) via the Vercel AI SDK.

---

📸 Product Screenshots

Landing	Workspace

Pipeline	Report

History	Architecture

---

🏗 Architecture

📁 Repository Structure

src/
 ├── app/
 │   ├── api/deep-research/   # the pipeline: main loop, prompts, model calls, fallbacks
 │   ├── workspace/            # live-run views (search activity, extraction, confidence gate, sources, report)
 │   └── report/[id]/          # persisted report view
 ├── components/
 ├── lib/                      # session store, rate limiting, visitor cookies
 └── store/                    # client-side research state (Zustand)

🔄 Research Pipeline

       User Query + Clarifications
                  │
                  ▼
        Generate Search Queries
                  │
                  ▼
   ┌──────────────────────────────┐
   │  Search (Exa) → Extract (LLM) │◄─┐
   │  → Analyze / Confidence Gate  │  │ repeat with follow-up
   └──────────────────────────────┘  │ queries, up to 3 iterations
                  │                  │
        sufficient or confident? ────┘
                  │ yes
                  ▼
     Generate Report + Validate Citations
                  │
                  ▼
       Persist (Neon or local JSON)

🌐 Request Flow

Client (useChat, SSE)
        │
        ▼
Next.js API Route (/api/deep-research)
        │
        ▼
Research Loop (research-functions.ts)
   │              │
   ▼              ▼
OpenRouter      Exa Search
(LLM calls)     (web search)
        │
        ▼
Session Store (Neon Postgres → local JSON fallback)

---

🧭 Architecture Decisions

Why an iteration loop instead of a single prompt. A single "search and summarize" call produces a report even when the search results were thin — it has no way to know that. Splitting analysis into its own pass with a numeric confidence score gives the pipeline something concrete to check before committing to a final report, and a cheap way to ask for more evidence if the first pass falls short. Without it: faster, but silently worse reports on underspecified topics. Capped at 3 iterations because token cost grows with each round and returns diminish fast — this was a deliberate cost/quality tradeoff, not an arbitrary number.

Why citations are validated, not just prompted. Telling a model "always cite your sources" doesn't make it true — models still write ungrounded claims. validateCitations (in utils.ts) parses the generated report for [n] markers after the fact and checks each one against the findings that were actually given to the model, flagging out-of-range or missing citations as a warning activity. It's not full fact-checking — it can't tell you a citation is wrong, only that it's fabricated or absent — but that's a real, cheap, code-level check instead of a claim resting entirely on prompt compliance.

Why every stage has a non-LLM fallback. Free-tier OpenRouter models get rate-limited under load. Rather than let one throttled call fail the whole run, extractContent, analyzeFindings, and generateReport each catch model failures and fall back to a deterministic alternative: a keyword-ranked extractive summary, a source-count-based heuristic confidence score, or a plainly-labeled findings-only report. The tradeoff is a lower-quality result in the degraded case — but a visible, labeled one, never a crash.

Why local-first storage instead of requiring Postgres. Requiring a database to try the project locally is friction that doesn't teach anything about the research pipeline itself. session-store.ts writes to .noesis/sessions.json by default and switches to Neon Postgres only when DATABASE_URL is set — and drops back to local storage if a Postgres call fails mid-request, logging the fallback rather than 500ing. Without it: simpler code, but a database becomes a hard requirement for a project whose interesting part has nothing to do with persistence.

Why SSE data-stream events instead of raw token streaming. The client doesn't need token-by-token prose — it needs to know which pipeline stage is running, whether it's on iteration 2 of 3, and when the report is ready. createDataStreamResponse pushes structured activity/session/report events as the pipeline reaches each stage, which is what actually drives the workspace UI (search activity, extraction, confidence gate views). Streaming raw LLM tokens would look flashier but wouldn't carry the information the UI needs.

What I removed and why. An earlier version of this project had a separate FastAPI backend with LangGraph-orchestrated agents, Neo4j, Redis, and pgvector. I deleted it. It was never wired into the deployed frontend, none of the graph/vector features were read from in an actual request path, and the added operational surface (four more services to run and keep in sync) wasn't buying anything the loop above doesn't already do with one deployable Next.js app. Keeping it around would have been keeping nouns, not architecture.

---

⚡ Performance & Cost Controls

• 3-iteration cap on the research loop, with per-stage timeouts (20–45s depending on stage)
• Per-model fallback chains so a single throttled free-tier model doesn't fail the run
• Rate limiting: 5 research runs/hour per visitor+IP, enforced before any LLM or search call is made
• Route-safe streaming — the SSE connection survives Next.js client-side navigation
• Local-first fallback keeps read/write latency low in dev and avoids a hard DB dependency

---

✅ Test Coverage

14 tests across 3 suites (vitest), focused on the logic that actually has behavior worth verifying rather than imports or config:

• utils.test.ts — citation numbering and validateCitations (valid, out-of-range, uncited, and partially-grounded reports)
• research-functions.test.ts — local fallback behavior when the LLM call fails: extraction, confidence scoring, report generation
• deep-research-loop.test.ts — the confidence-gate loop itself: continues past low confidence, stops at the iteration cap, stops early on high confidence

npm test

This is not full coverage of the UI or the persistence layer — it's coverage of the parts of this project where a bug would silently produce a wrong research result.

---

💻 Quick Start & Setup

1. Clone the repository

git clone https://github.com/ayushgupta-15/Noesis.git
cd Noesis

2. Install dependencies

npm install

3. Configure environment variables

cp .env.local.example .env.local

Add your keys for OPENROUTER_API_KEY and EXA_SEARCH_API_KEY. DATABASE_URL is optional locally — leave it empty to use the local JSON session store.

4. Run the development server

npm run dev

5. Run the tests

npm test

---

Known limitations

• Rate limiting is in-memory per server instance, so it resets on cold start and doesn't share state across serverless instances — fine for a single-instance demo deployment, not for a multi-instance production one.
• Citation validation catches fabricated or missing citation numbers, not factually incorrect ones — it's a groundedness check, not a fact-checker.
• No automated test coverage yet for the session store's Postgres/local-fallback switching or the SSE client wiring.

---

🔮 Roadmap

• Persist and surface the citation groundedness score in the report UI, not just as a warning activity
• Move rate limiting to a shared store (Redis/Upstash) for multi-instance deployments
• PDF export
• Voice research input

---

⚖️ License

MIT License

---

🙏 Acknowledgements

• Vercel AI SDK
• OpenRouter
• Exa
• Next.js