An evidence-backed institutional manager monitoring and investment-research system built around real public SEC 13F data — with deterministic Python analytics as the source of truth, machine learning as a screening signal, LLMs confined to evidence-backed synthesis, and every factual claim auditable to a source.
Workflow: Research → Quant → Risk → Memo → Evidence Audit.
Design principle: the LLM is never the source of truth. Python computes every number. Models screen; humans judge. The memo layer can only restate evidence that already exists in the workflow — and an audit agent blocks unsupported statements before delivery.
Status: live public demo. Browsing it uses a stored canonical workflow run and incurs no LLM calls. Pressing Re-run workflow executes the real agent workflow live, including Groq memo synthesis — the one path that reaches a provider, guarded by a per-client cooldown and a global hourly ceiling. Data ingestion stays disabled. The backend runs on a free instance that sleeps when idle, so the first request after a quiet period can take ~50 seconds to wake.
| Default view | A stored workflow run — the canonical Berkshire Hathaway 13F analysis, served from the database |
| LLM calls from browsing | Zero. Loading a manager, reading the memo, inspecting evidence and switching tabs are reads of persisted state |
| Live re-runs | Enabled, and the only path to a provider: one explicit click runs Research → Quant → Risk → ML → Groq memo → Evidence Audit. One run per client per 10 minutes, 3 per client per hour, 20 per hour across all visitors; a refusal returns 429 |
| Request surface | A manager id. No prompt, no model choice, no external URL, no manager creation — the analyzable universe is exactly what was seeded at boot |
| SEC ingestion & uploads | Disabled in the public demo — writes are off and the deployment holds no SEC credential |
| Provider credentials | Server-side only: LLM_API_KEY is set in the Render dashboard, never committed, never in the frontend bundle, never in an API response |
| Canonical analysis | A real SEC-derived run: memo synthesized by Groq (openai/gpt-oss-120b), Evidence Audit passed, 8 evidence objects, 8 citations all resolving |
Run it locally to execute the workflow yourself — see Run it.
Institutional investors — OCIOs, endowments, foundations, pensions — allocate to external managers and then must monitor them for years: Did concentration spike? Did a sector bet creep in? Is this manager's portfolio changing in a way that is unusual against peers? Each question triggers diligence, and every answer ends up in an investment-committee memo.
That monitoring work is exactly where AI helps and where AI must be constrained:
- The quantitative layer (concentration, turnover, sector drift, entries/exits) must be exact, reproducible, and testable — a hallucinated weight is worse than no analysis.
- The communication layer (what changed, what it might mean, what to ask the manager) is where LLMs shine — and where unsourced generation would be dangerous.
This project builds both layers, with a hard wall between them.
| Concern | Approach |
|---|---|
| Correctness of numbers | All metrics are pure Python functions over SQL-stored snapshots, unit-tested, never computed inside a prompt. |
| Hallucination control | The memo agent receives a structured evidence registry and must tag every factual statement [E-QUANT-CONC]-style. |
| Auditability | An evidence-audit agent verifies claim citations, memo citations, citation topical consistency, evidence-ID uniqueness, and the absence of trade-recommendation language. |
| Availability | With no LLM key configured, a deterministic memo generator produces the same memo structure. The workflow never depends on a generative model being reachable. |
| Honest ML | Unsupervised models (IsolationForest, PCA, K-Means, cosine similarity) are labeled everywhere as review-priority screens — never as recommendations, ratings, or forecasts. |
- Research Agent — builds the monitoring context and source registry (manager profile + snapshot provenance, with filing URLs for live 13F data).
- Quant Agent — deterministic calculations: top-5 concentration, one-way turnover proxy, sector shifts, entries/exits, position deltas.
- Risk Agent — applies documented monitoring thresholds and translates the ML anomaly screen into a review-priority alert.
- Similarity & style models — cosine-similarity peers, PCA style map, silhouette-gated K-Means style groups.
- Memo Agent — synthesizes an IC monitoring memo from the evidence registry (LLM when configured, deterministic otherwise).
- Evidence Audit Agent — the final control gate; blocks delivery on unsupported citations or recommendation language.
┌──────────────────────────────────────────┐
│ Data ingestion │
│ SEC EDGAR 13F-HR connector (live) │
│ CSV snapshot upload (validated) │
│ Seeded demo universe (6 managers) │
└───────────────────┬──────────────────────┘
│ SQLAlchemy / SQLite
┌───────────────────▼──────────────────────┐
│ Manager → Snapshot → Holding tables │
│ AnalysisRun audit trail │
└───────────────────┬──────────────────────┘
│
┌─────────────┐ ┌───────────────▼──┐ ┌──────────────────┐
│ Research │──▶│ Quant │──▶│ Risk │
│ Agent │ │ Agent (Python) │ │ Agent (rules+ML)│
└─────────────┘ └──────────────────┘ └────────┬─────────┘
│
┌──────────────────────┐ ┌───────────────▼──────────▼─────────┐
│ Similarity / PCA / │ │ Memo Agent │
│ K-Means (ML screen) │──▶│ LLM synthesis OR deterministic │
└──────────────────────┘ └───────────────┬────────────────────┘
│
┌───────────▼────────────┐
│ Evidence Audit │
│ errors block delivery │
└────────────────────────┘
Stack: Python 3.12 · FastAPI · SQLAlchemy 2 · SQLite · scikit-learn · React 19 · TypeScript · Vite · Docker Compose. Optional memo synthesis via any OpenAI-compatible endpoint (Groq preferred; deterministic fallback always available).
Every generated statement traces to an evidence object:
{
"id": "E-QUANT-CONC",
"label": "Top-five concentration",
"fact": "Top-five concentration moved from 65.0% to 72.0%.",
"source_type": "deterministic_calculation",
"manager_name": "Northstar Fundamental Equity",
"period": ["2026-03-31", "2026-06-30"],
"metric": "top5_concentration",
"value": 0.72,
"calculation": "Sum of the five largest position weights in the current snapshot; delta measured against the same calculation on the prior snapshot.",
"retrieved_at": "2026-08-24T19:41:03+00:00",
"source_url": null
}The audit agent fails a run for: missing claim citations, unknown memo citations (including comma-joined [E-A, E-B] groups), duplicate evidence IDs, and trade-recommendation language ("recommend buying", "buy rating", …). Topical mismatches — e.g. a concentration claim citing only the similarity model — are surfaced as review warnings.
| Model | Input | Output | What it is not |
|---|---|---|---|
| IsolationForest | 7 change features per snapshot transition (all transitions in the DB) | 0–1 unusualness score + peer-outlier flag for the latest transition | Not a risk rating, skill measure, or return forecast |
| Cosine similarity | Standardized style features (concentration, breadth, position size, cash, 7 sectors) | 0–1 peer similarity, top 3 shown | Not a statement about strategy comparability or quality |
| PCA | Same style matrix | 2-D style map + explained variance | Axes are statistical components, not named factors |
| K-Means | Same style matrix | Silhouette-selected style groups (k ≤ 3) | Groups describe positioning, never quality tiers |
Features and limitations are documented in-app (Methodology modal, served from /api/ml/methodology) so the model contract ships with the product rather than living only in this README. Small-sample behavior is explicit: fewer than four transitions → the anomaly model returns insufficient_history; silhouette < 0.2 → clustering is withheld rather than forced.
- Live SEC EDGAR 13F-HR (
POST /api/sec/ingest): submissions API → two latest filings → information-table XML → normalized weighted snapshots with filing-level source URLs. Compliant User-Agent required (SEC_USER_AGENT), sequential requests, retry with backoff. - CSV upload (
POST /api/managers/{id}/snapshots/upload): validated (ticker, security, sector, weight[, value_m], weights sum ≈ 1.0, size-capped, idempotent per report date). - Demo universe: six clearly-labeled synthetic managers so the app works with zero credentials.
13F limitations (surfaced in the product): quarter-end long U.S. equity positions only — no shorts, limited derivative detail, no non-reportable positions, no intraperiod activity, no sector field (positions default to "Other" until a security-master enrichment). A 13F is not a manager's complete portfolio or strategy.
 |
 |
 |
 |
 |
 |
Top row: the cinematic entry (/) — translucent filing planes resolve into an evidence lattice as you scroll, ending at "Every claim has a source." Below: the actual workspace at /universe, the Source → Evidence → Claim trace, the IC memo with hoverable [E-*] citations, and the live Berkshire 13F case (Alphabet CL A +3.5pp, Occidental −2.3pp, Chevron −2.0pp).
The product opens with an original scroll-driven narrative — the Intelligence Core: three translucent filing planes (evoking 13F information tables) carrying ~30 evidence nodes that converge from scattered raw data into an organized lattice across four pinned scenes (raw information → evidence emerges → deterministic analytics → every claim has a source), then recedes as a live preview of the real workspace rises to meet it. The object is pure SVG/DOM driven by scroll position — no animation library, no WebGL, only transform/opacity.
Deliberate constraints keep it institutional rather than decorative:
- The workspace stays restrained. Dense views drop the atmosphere to a quiet variant; tables keep tabular numerals and validated, CVD-checked chart colors; nothing animates in a way that competes with reading.
- Motion is earned. One dominant visual carries the whole narrative; charts reveal once; metric values resolve once on first view; route and tab transitions are 200–500 ms fades.
prefers-reduced-motionreplaces the pinned sequence with a static composed page and disables all ambient movement. - Evidence is spatial. The Evidence tab renders a Source → Evidence → Claim chain per selected object (hover any ID to trace it), and every
[E-*]citation inside the memo shows its underlying evidence on hover.
Testing against live EDGAR filings surfaced issues synthetic data never would: filers report the same CUSIP across multiple reporting units, so the parser aggregates rows by CUSIP before computing weights (otherwise concentration and position-level change detection double-count those names); CUSIPs are alphanumeric, so the UI labels them explicitly rather than mistaking them for tickers; and re-ingesting a CIK replaces only same-report-date snapshots, keeping user uploads intact.
Managers are tagged by data origin (DEMO / LIVE 13F / UPLOADED, derived from snapshot provenance — the SEC filing stays authoritative for holdings and values). Because 13F information tables carry no sector classifications, style models (cosine similarity, PCA, K-Means) require a sector-completeness gate: 13F-derived portfolios are excluded and reported rather than projected against sector-classified managers, which would visualize the metadata gap instead of style. (There is no reliable free CUSIP→sector source — CUSIP is proprietary — so no enrichment is fabricated.) Change-pattern anomaly screening remains available to all managers: those features are computable from any weighted snapshots. Peer-comparison and style-map exclusions are disclosed in the UI and in the memo's similarity evidence.
Requires Python 3.12 and Node 20+.
make setup # venv + pip install + npm install
make test # 163 backend tests
make api # FastAPI on :8000
make web # Vite dev server on :5173 (proxies /api)Open http://localhost:5173 — the landing narrative introduces the system; ENTER INTELLIGENCE WORKSPACE (or /universe directly) opens the live product with the seeded demo universe. Click a manager and Run monitoring workflow.
Docker Compose alternative:
docker compose up --build # api :8000, web :5173 (nginx)cp .env.example .env # set LLM_API_KEY (LLM_MODEL defaults to openai/gpt-oss-120b via Groq)Any OpenAI-compatible endpoint can be substituted through LLM_BASE_URL/LLM_MODEL. Configuration is explicit — the app reads only LLM_*, SEC_USER_AGENT, and its own override variables, never unrelated shell/provider environment. Without a key, the deterministic memo generator runs — by design. Each analysis response carries audit.memo_synthesis reporting the path actually taken (llm with provider/model, or deterministic after any fallback), persisted with the run for auditability; the methodology panel discloses the configured path. Keys are never logged or returned.
Set a compliant user agent, then use Import SEC 13F in the UI:
SEC_USER_AGENT=InstitutionalInvestmentIntelligence/0.2 you@example.com- Enter the workspace (
/universe): per-manager status, AUM, concentration Δ, turnover, rule flags, anomaly screen, last review. - Open a manager → Changes tab: deterministic position deltas, entries/exits, diverging sector bars (no LLM involved).
- Run monitoring workflow → Research → Quant → Risk → Memo → Audit execute; the IC Memo renders with inline
[E-*]citations. - Evidence tab: inspect every evidence object (metric, value, calculation, period, filing URL) and the audit verdict.
- Import SEC 13F with a real CIK → the same pipeline runs on live EDGAR filings.
| Route | Purpose |
|---|---|
GET /api/universe |
Dashboard rollup: latest metrics, rule flags, anomaly screen, last review |
GET /api/universe/style |
PCA style map + K-Means groups |
GET /api/managers/{id} |
Profile, snapshot summaries, latest holdings |
GET /api/managers/{id}/changes |
Deterministic change metrics (no ML/LLM) |
POST /api/analyze/{id} |
Full agent workflow → memo + evidence + audit, persisted. The only route that can reach a provider: gated by deployment mode and rate-limited per client |
GET /api/managers/{id}/runs/latest |
Most recent stored run — what a page load reads. No agents, no provider, no writes |
GET /api/managers/{id}/runs · GET /api/runs/{id} |
Review audit trail |
GET /api/config |
Deployment facts for the UI: mode, whether re-runs are enabled, provider/model labels |
GET /health · GET /api/health |
Liveness. Touches no database, no SEC, no provider |
POST /api/managers/{id}/snapshots/upload |
CSV snapshot ingestion (disabled in public-demo mode) |
POST /api/sec/ingest |
Live EDGAR 13F ingestion (disabled in public-demo mode) |
GET /api/ml/methodology |
Feature/model/threshold contract + configured synthesis path |
Interactive docs at http://localhost:8000/docs.
163 tests: analytics correctness (turnover, concentration, thresholds), evidence-audit edge cases (comma-joined citations, markdown-link false positives, recommendation-language ban, duplicate IDs), ML behavior (insufficient-history degradation, determinism, cluster suppression), 13F parsing/units/CUSIP aggregation/UA policy, API integration over an isolated database (uploads, error paths, full workflow, audit passage, run history), memo-synthesis fallback and sanitization, the public-demo safety contract (stored-run serving, gated re-runs, rate limiting, demo-seed integrity and idempotency), and the guarded public Groq contract (provider configuration read only from server environment; the key absent from every response, header and frontend input; zero provider calls from browsing; one click, one synthesis request; caller-supplied prompts and model names ignored; per-client cooldown, per-client hourly and global ceilings, with a blocked request charging none of them; ingest and uploads still refused; provider failure labelled deterministic; a stored Groq run still reporting Groq after the key is gone).
make test # backend suite
make lint # ruff
make scan # secret scan: tracked files, git history, frontend build outputNot deployed. This section documents how the application is intended to be deployed and what the demo does differently.
Stored runs by default. Opening a manager reads GET /api/managers/{id}/runs/latest — persisted state, no agents, no provider call, no writes. Browsing the demo (loading a manager, reading the memo, inspecting evidence, switching tabs, refreshing) costs nothing. A visitor sees a real workflow artifact, not a mock: the same memo, evidence registry, audit result and synthesis metadata the run produced.
Live re-runs are explicit and gated. The workflow executes only from a click on Run monitoring workflow. Nothing triggers it on mount, tab change, refresh, or navigation. POST /api/analyze/{id} takes a manager id and no body, so a caller cannot supply a prompt, pick a model, or redirect the workflow; it is gated by deployment mode and bounded by three limits — a per-client cooldown, a per-client hourly ceiling, and a global hourly ceiling (a per-client limit alone cannot bound a public URL, since every new address brings its own budget). Exceeding any of them returns 429 with Retry-After, which the UI reports as "Demo analysis was recently run. Please try again shortly."
| Variable | Local default | Public demo |
|---|---|---|
APP_ENV |
local |
public-demo |
ENABLE_PUBLIC_RERUN |
unset → enabled | true — the guarded live Groq workflow |
ENABLE_PUBLIC_WRITES |
unset → enabled | false (no 13F ingest, no uploads, no manager creation) |
ANALYZE_RATE_LIMIT_PER_HOUR |
5 |
3 |
ANALYZE_MIN_INTERVAL_S |
unset → none | 600 (unset would default to 600 outside local) |
ANALYZE_GLOBAL_LIMIT_PER_HOUR |
unset → none | 20 (unset would default to 20 outside local) |
LLM_BASE_URL / LLM_MODEL |
.env |
public config: Groq, openai/gpt-oss-120b |
LLM_API_KEY |
.env (gitignored) |
Render dashboard secret only |
CORS_ORIGINS |
localhost:5173 | exact frontend origin — never * |
VITE_API_BASE_URL (frontend build) |
empty → dev proxy | backend origin |
Unset gate variables follow APP_ENV, so a deployment that sets only APP_ENV=public-demo is safe by default: re-runs and writes off, cooldown and global ceiling on. The public demo opts in to re-runs explicitly and leaves writes off. When re-runs are disabled the UI says so on the button rather than failing on click.
Demo database. Built from committed seeds, never copied from a developer machine:
make demo-init # rebuild backend/data/demo.db: DEMO universe + one frozen live-13F run
make demo-serve # serve that database exactly as production would (stored-only, read-only)In a container the same initializer ships inside the application package, so a hosted demo rebuilds its database at boot and needs no mounted volume:
python -m app.demo_init --db $III_DB_PATH && uvicorn app.main:app --host 0.0.0.0 --port $PORTrender.yaml is a complete Render Blueprint for the backend — Docker runtime, /health check, ephemeral storage, and the public-demo environment already set. LLM_BASE_URL and LLM_MODEL are public configuration and live in the file; LLM_API_KEY is declared sync: false, so its value exists only in the Render dashboard and never in the repository, the logs, an API response, or the frontend bundle. SEC_USER_AGENT is deliberately not defined at all: the public service holds no SEC credential and cannot reach EDGAR even if a gate were misconfigured. CORS_ORIGINS is left unsynced, to be set to the exact frontend origin once that exists.
With the key removed the service still runs: every memo reports memo_synthesis.mode = deterministic, truthfully. A run's synthesis metadata records the path that run actually took, so history is never relabelled by a later configuration change.
The frontend deploys as a static Vite build; frontend/vercel.json pins the build and adds the SPA rewrite that keeps deep links such as /managers/7 working on refresh. Set VITE_API_BASE_URL to the backend origin at build time — it is public configuration, and no secret belongs in the frontend environment.
make demo-init is deterministic — it rebuilds rather than appends, so repeated runs produce identical state (7 managers, 14 snapshots, 171 holdings, exactly 1 analysis run). The seed at backend/app/demo_seed/live_13f_case.json is a real Berkshire Hathaway 13F workflow run frozen from a validated execution (audit passed, all citations resolving); make demo-freeze MANAGER=<id> RUN=<id> regenerates it. No .db file is ever committed.
- Demo data is synthetic and labeled as such.
- 13F-based analytics reflect disclosed long positions only (see above).
- ML outputs are relative to the small in-database universe and recalibrate as history grows.
- The system deliberately does not infer performance, personnel, fees, or process changes — those become diligence questions.
- No output anywhere is an investment recommendation; the audit layer blocks recommendation language mechanically.
- Security-master enrichment (CUSIP → ticker/sector) for 13F holdings
- Form ADV ingestion and manager-metadata retrieval agent
- User-defined monitoring thresholds per mandate
- Portfolio-level manager overlap analysis
- Memo export to PDF/DOCX; Postgres deployment; auth + roles
- LLM faithfulness/citation-coverage evaluation suite
AI was used substantially throughout this project — application code, the research workflow, tests, the synthesis system, and the interface. That is the point of the architecture rather than a caveat to it, and the constraints are explicit:
- Public source evidence is the factual basis. Every number in a memo is computed in Python from SQL-stored snapshots derived from SEC filings.
- LLM output is not itself evidence. The model receives a structured evidence registry and may only restate what it contains.
- Citations resolve to structured evidence objects. Each
[E-*]tag maps to a record with its metric, value, calculation, period and source URL. - A deterministic Evidence Audit checks the final memo — citation resolution, topical consistency, section-scoped evidence policy, and a ban on trade-recommendation language — and blocks delivery on failure.
- Analyst and investor judgment remains necessary. The system prioritizes review; it does not decide anything.
This is an independent research and engineering project built from public information. It is not affiliated with, endorsed by, or sponsored by Berkshire Hathaway, Commonfund, the SEC, Groq, or any manager or organization referenced in the analysis. It contains no proprietary or non-public information. Nothing here is investment advice, and the system produces no manager recommendation, rating, or return forecast: machine-learning outputs are review-priority screens, and memo language that would direct a trade is blocked mechanically by the audit layer.
Sector-level analysis is reported as unavailable wherever the underlying 13F filings carry no sector classifications — a data limitation, never an absence of change.