OpenMerit

Merit, with receipts.

Built for Hack-Nation Challenge 02 — “The VC Brain” (Maschmeyer Group)

An AI-first VC operating system: signal-driven founder scoring, verifiable diligence, and honest backtests.

Three differentiators:

1. Founder Score from timestamped public signals (GitHub + Hacker News), always shown as an interval (68 ± 9), never a bare number.
2. Time-machine backtest — score cohort founders using ONLY signals from before their funding cutoff, rank against anonymized controls, report recall@k with limitations shown, never hidden.
3. Cold-start interviewer, self-service — when a founder has thin public signal, generate 3 sharp, checkable diligence questions and send the founder a public, tokenized apply link (/apply/[token], no login). They answer directly in a guided chat (at most one AI follow-up per question); each answer becomes a self-collected, timestamped signal and the confidence band visibly shrinks. Founder input is treated as untrusted data — it can never change a trust level, score, or recommendation.

Plus a per-claim Trust Score (VERIFIED / CORROBORATED / CLAIMED / CONTRADICTED / UNKNOWN) with clickable evidence, a 3-axis opportunity score that is never averaged (disagreement is surfaced), and a deterministic decision matrix in code — the LLM writes the memo, it never decides.

Guardrails (by design)

• Never fabricate. Missing data renders as an explicit gap (Cap table: not disclosed.).
• No demographic features or proxies anywhere in scoring — public, professional signals only.
• Control profiles are always anonymized (P1, P2, …); their real handles never reach the client.
• Every LLM call is logged (llm_trace: model, stage, truncated output).

Stack

Next.js 15 (App Router, TypeScript) · Supabase Postgres · OpenAI via the Vercel AI SDK (generateObject + zod) · Tailwind + shadcn-style UI · Recharts. Deploys to Vercel. An optional workspace access gate protects public demos.

Setup

pnpm install
cp .env.example .env.local   # then fill it in

.env.local:

SUPABASE_URL=
SUPABASE_SERVICE_ROLE_KEY=      # server-only; never exposed to the client
OPENAI_API_KEY=
LLM_MODEL_FAST=gpt-4o-mini
LLM_MODEL_STRONG=gpt-4o
GITHUB_TOKEN=                   # optional; raises GitHub rate limit to 5000/h
NEXT_PUBLIC_APP_URL=            # public site URL for OG/canonical, e.g. https://openmerit.vercel.app
APP_ACCESS_KEY=                 # recommended for public deployments
SEARXNG_URL=                    # optional self-hosted web evidence search
DOCLING_SERVE_URL=              # optional self-hosted OCR/document parser
DOCLING_SERVE_API_KEY=          # optional Docling service key
CRON_SECRET=                    # bearer token for /api/monitor/run
TRACE_STORE_OUTPUT=false        # keep false to avoid retaining sensitive prose

Apply all three migrations in order, or run supabase db push. Migration 0002 adds evidence provenance, analysis history, workflow, monitoring, access-rate buckets, and atomic publication; migration 0003 adds the founder self-service interview (interview_sessions, interview_turns).

pnpm dev        # http://localhost:3000

Local caveat — folder name: Next.js cannot build or dev-serve from a path containing a # (its RSC bundler builds module ids as path#export). If this project lives under a folder like HackNation #6 07_2026, rename it to drop the # (e.g. HackNation 6 07_2026) before running pnpm dev. Vercel is unaffected — it checks the repo out at a clean path. The tsx scripts (ingest:cohort, eval) run fine regardless.

Scripts

pnpm ingest:cohort   # ingest config/cohort.yaml (cohort + controls) into Supabase
pnpm eval            # run all 6 eval fixtures (incl. prompt-injection) through the pipeline, PASS/FAIL per check
pnpm typecheck       # tsc --noEmit
pnpm lint            # non-interactive ESLint
pnpm test            # deterministic unit tests
pnpm build           # next build

config/thesis.yaml (fund thesis) and config/cohort.yaml (backtest cohort + controls) are config, not code. Verify the cohort cutoff dates before demoing — a wrong cutoff silently invalidates the leakage guarantee. Add time-matched profiles under control_profiles so recall@k has a control group.

Deploy (Vercel)

1. Push to GitHub, import the repo in Vercel.
2. Set the env vars above in the Vercel project settings.
3. Ingest/verify/memo/backtest route handlers already set export const maxDuration = 60. Cohort ingestion can exceed serverless limits — run pnpm ingest:cohort locally against the production Supabase before the demo (the UI "Add founder" button handles single small ingests fine).
4. Click through the demo on the deployed URL.

Demo script

1. Meta-demo — on /, click Add founder, enter a real GitHub handle → a scored row with an interval appears in a few seconds.
2. Backtest — /backtest, toggle With cutoff / Without cutoff: the ranking visibly re-ranks as post-cutoff hype drops out. recall@k and the limitations card are shown.
3. Contradiction — /opportunities, paste eval/fixtures/contradiction.txt, Extract claims → Verify all → a red CONTRADICTED badge with an explanation popover.
4. Cold start (self-service) — open a thin founder, Generate apply link, scan the QR on a phone (or open the link), and answer the 3 questions as the founder in the guided chat. Back on the founder page, Refresh score → the confidence band tightens on screen (3 answers recorded · ±X → ±Y), and a linked opportunity's Founder axis leaves the “gap” state.
5. Memo — on an opportunity, Generate memo → recommendation banner with the matrix reason verbatim, three separate axis cards, and a memo with inline trust levels.

Architecture

app/                 pages (founders, opportunities, backtest) + API route handlers
lib/
  signals.ts         insert (dedupe) + query signals (occurred_at < cutoff = the backtest hinge)
  ingest/            github + hackernews signal fetchers
  scoring/
    founderScore.ts  decayed, weighted, log1p signal mass → percentile score + confidence band
    decision.ts      deterministic INVEST/PASS/REVIEW matrix (never the LLM)
  intelligence/
    prompts.ts       the 5 system prompts (verbatim)
    schemas.ts       zod schemas for every structured LLM output
    pipeline.ts      extract → verify → coldstart → axes → memo
  backtest.ts        time-machine ranking with per-founder cutoffs + anonymized controls
  opportunities.ts   claim/opportunity orchestration + full memo analysis
  interviews.ts      founder self-service interview: sessions, turns, one-shot interviewer step
app/apply/[token]/   public, standalone founder interview chat (no VC nav)
scripts/             ingest-cohort.ts, eval.ts (tsx)
supabase/migrations/ 0001_init.sql · 0002_production_foundations.sql · 0003_interviews.sql
eval/fixtures/       6 test decks (incl. injection.txt)