Add Upstream — equity research on first-hand signal (finance enterprise demo)

finance-equity-research/.gitignore

@@ -0,0 +1,42 @@
+.env.*
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.*
+.yarn/*
+!.yarn/patches
+!.yarn/plugins
+!.yarn/releases
+!.yarn/versions
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+.pnpm-debug.log*
+
+# env files (can opt-in for committing if needed)
+.env*
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts

finance-equity-research/AGENTS.md

@@ -0,0 +1,9 @@
+<!-- BEGIN:nextjs-agent-rules -->
+
+# This is NOT the Next.js you know
+
+This version has breaking changes — APIs, conventions, and file structure may all differ from your training data. Read the relevant guide in `node_modules/next/dist/docs/` (resolved from this file's directory; in monorepos the `next` package may not be visible from the repo root) before writing any code. Heed deprecation notices.
+
+This block is written and re-added by `next dev` — verify at `node_modules/next/dist/server/lib/generate-agent-files.js`. Removing it from a diff only re-creates the uncommitted change; committing it with your work keeps the tree clean.
+
+<!-- END:nextjs-agent-rules -->

finance-equity-research/CLAUDE.md

@@ -0,0 +1 @@
+@AGENTS.md

finance-equity-research/README.md

@@ -0,0 +1,54 @@
+# Upstream — equity research on first-hand signal
+
+Every research tool summarizes published coverage; by the time it's published, it's priced in. Upstream sends live TinyFish agents to the primary sources — Reddit, Trustpilot, app stores, careers pages, SEC EDGAR, Downdetector — and turns what customers and employees are doing *right now* into a directional read on a public company, with every claim citing a verbatim, timestamped source.
+
+**The signature:** the lead-time timeline. Cracker Barrel's complaint velocity turned on May 4, 2026; the CEO-departure 8-K reached EDGAR on July 27 — a measured **84-day lead**. Measured, not modeled.
+
+## Screens
+
+- **Live scan** — type any US-listed ticker; agents fan out (waves of 5), results stream in over SSE, the Direction Score assembles as each source lands.
+- **Company read** — Direction Score decomposed into Customer Sentiment (40%), Workforce (30%), Leadership (20%), Product/Ops (10%); signal tiles with baselines; an evidence table of verbatim quotes, each linked to its source and scrape time.
+- **Lead-time timeline** — the leading signal plotted against official filings on one axis.
+
+## How it uses TinyFish
+
+Escalation ladder per source: **fetch → stealth agent**. Plain fetch first (free); when a site blocks it (Reddit, Trustpilot 403 non-browser clients), the same source escalates to a stealth-profile agent with a US proxy:
+
+```ts
+// src/lib/tinyfish.ts
+const stream = await tf().agent.stream({
+  url: opts.url,
+  goal: opts.goal, // JSON shape embedded in the goal, cookbook-style
+  browser_profile: opts.stealth ? BrowserProfile.STEALTH : BrowserProfile.LITE,
+  proxy_config: { enabled: true, country_code: "US" },
+});
+for await (const event of stream) {
+  if (event.type === "STREAMING_URL") opts.onStreamingUrl?.(event.streaming_url, event.run_id);
+  else if (event.type === "PROGRESS") opts.onProgress?.(event.purpose);
+  else if (event.type === "COMPLETE") { complete = event; break; }
+}
+// COMPLETED only means the browser ran without crashing — validate content:
+const result = normalizeResult(complete.result);
+if (result != null) return { ok: true, ... };
+```
+
+Layoff intel uses **search → fetch** (targeted, ~10s) instead of browsing tracker UIs. SEC EDGAR is parsed deterministically in code — no LLM near structured data.
+
+## Anti-hallucination
+
+- Extracted quotes must appear **verbatim** in the fetched text (substring gate in code) or they're discarded.
+- Every source carries a `metricHint`; the normalizer must return `null` rather than substitute a different number.
+- Per-family scoring rubrics ("a layoff in the last 90 days never scores above 50") keep the read defensible.
+- Provenance (source URL, scraped-at) is stamped server-side, never trusted from the model.
+
+## Setup
+
+```bash
+npm install
+cp .env.example .env.local   # fill in keys
+node scripts/apply-schema.mjs   # idempotent; needs DATABASE_URL loaded
+node scripts/seed.mjs           # demo companies + CBRL lead-time backtest
+npm run dev
+```
+
+Postgres is raw (`postgres` driver over the Supabase session pooler) — no ORM, no supabase-js. Env vars are listed in `.env.example`. Design system: `docs/DESIGN.md` (locked from a Claude Design handoff in `docs/design-handoff/`).

finance-equity-research/db/schema.sql

@@ -0,0 +1,111 @@
+-- Upstream schema. Applied idempotently by scripts/apply-schema.mjs.
+-- Raw Postgres (Supabase session pooler). No supabase-js anywhere.
+
+create table if not exists companies (
+  id            bigint generated always as identity primary key,
+  ticker        text not null unique,
+  name          text not null,
+  exchange      text,
+  sector        text,
+  -- resolved source profile: subreddits, trustpilot domain, app ids, ats board,
+  -- edgar cik, newsroom url, downdetector slug… discovered via TinyFish search
+  source_profile jsonb not null default '{}'::jsonb,
+  created_at    timestamptz not null default now()
+);
+
+create table if not exists scans (
+  id              bigint generated always as identity primary key,
+  company_id      bigint not null references companies(id),
+  status          text not null default 'running',           -- running | complete | failed
+  direction_score numeric,                                   -- 0-100, null until enough families land
+  family_scores   jsonb not null default '{}'::jsonb,        -- {sentiment:{score,weight,baseline}, …}
+  provisional     boolean not null default true,
+  started_at      timestamptz not null default now(),
+  completed_at    timestamptz,
+  error           text
+);
+create index if not exists scans_company_started_idx on scans (company_id, started_at desc);
+
+-- one row per TinyFish source run inside a scan
+create table if not exists source_runs (
+  id            bigint generated always as identity primary key,
+  scan_id       bigint not null references scans(id),
+  source_key    text not null,               -- reddit | trustpilot | app_store | google_play | careers | layoffs | edgar | newsroom | downdetector
+  primitive     text not null,               -- search | fetch | agent
+  status        text not null default 'queued',  -- queued | running | complete | failed
+  tinyfish_run_id text,
+  streaming_url text,
+  started_at    timestamptz,
+  completed_at  timestamptz,
+  duration_ms   integer,
+  items_read    integer,
+  result        jsonb,
+  error         text
+);
+create index if not exists source_runs_scan_idx on source_runs (scan_id);
+
+-- normalized, citable rows: every claim in the UI is one of these
+create table if not exists evidence (
+  id           bigint generated always as identity primary key,
+  company_id   bigint not null references companies(id),
+  scan_id      bigint not null references scans(id),
+  family       text not null,               -- sentiment | workforce | leadership | ops
+  quote        text not null,               -- verbatim excerpt
+  source_key   text not null,
+  source_label text not null,               -- "r/CrackerBarrel", "Trustpilot · ★1 review"
+  source_url   text,
+  published_at date,
+  scraped_at   timestamptz not null default now(),
+  sentiment    numeric,                     -- -1..1 from the classifier
+  extra        jsonb not null default '{}'::jsonb
+);
+create index if not exists evidence_company_idx on evidence (company_id, scraped_at desc);
+create index if not exists evidence_scan_idx on evidence (scan_id);
+
+-- per-scan headline metrics that power the signal tiles
+create table if not exists signal_metrics (
+  id          bigint generated always as identity primary key,
+  company_id  bigint not null references companies(id),
+  scan_id     bigint not null references scans(id),
+  metric_key  text not null,                -- complaint_velocity | job_postings | app_rating | exec_events | …
+  family      text not null,
+  value       numeric,
+  unit        text,                         -- "/wk", "open", "★"
+  baseline    numeric,
+  baseline_label text,                      -- "vs prior week", "vs 90-day average", "was 3.8 on Jan 5"
+  delta_pct   numeric,
+  series      jsonb not null default '[]'::jsonb,  -- sparkline points [{t,v}]
+  sources     text,                         -- "Reddit · Trustpilot · X"
+  scraped_at  timestamptz not null default now()
+);
+create index if not exists signal_metrics_scan_idx on signal_metrics (scan_id);
+create index if not exists signal_metrics_history_idx on signal_metrics (company_id, metric_key, scraped_at desc);
+
+-- lagging official record: filings, press releases — the timeline's bottom track
+create table if not exists official_events (
+  id          bigint generated always as identity primary key,
+  company_id  bigint not null references companies(id),
+  event_type  text not null,               -- 8k_502 | press_release | earnings | other_filing
+  title       text not null,
+  occurred_on date not null,
+  url         text,
+  source      text not null default 'sec_edgar',
+  is_key      boolean not null default false,   -- highlighted (rust) on the timeline
+  created_at  timestamptz not null default now(),
+  unique (company_id, event_type, occurred_on, title)
+);
+
+-- measured lead-time reads: the product's headline claim, stored not derived-on-the-fly
+create table if not exists lead_time_reads (
+  id               bigint generated always as identity primary key,
+  company_id       bigint not null references companies(id),
+  signal_metric    text not null,           -- which series turned (complaint_velocity)
+  signal_start_on  date not null,           -- first sustained turn
+  signal_rule      text not null,           -- "+22% above trailing 4-week avg, 3 consecutive weeks"
+  event_id         bigint references official_events(id),
+  filed_on         date not null,
+  lead_days        integer not null,
+  narrative        text,                    -- "Customers turned 84 days before the filing."
+  series           jsonb not null default '[]'::jsonb,  -- indexed weekly points for the chart
+  created_at       timestamptz not null default now()
+);

finance-equity-research/docs/BRIEF.md

@@ -0,0 +1,82 @@
+# Upstream — Equity Research on First-Hand Signal
+
+> Working name: **Upstream** (signal before it's priced in). Angle brief, 2026-08-21. Grounded in `docs/research/`.
+
+## The one-liner
+
+An analyst picks a company. Upstream sends live web agents to the primary sources — customer complaints, app reviews, careers pages, exec announcements — and returns a **directional read with receipts**: a decomposed score, week-over-week velocity, and a timeline proving the signal led the filing.
+
+**The thesis on screen:** every other tool summarizes coverage; by the time it's published, it's priced in. Upstream watches the sources that *precede* coverage. The filing is the lagging indicator — we plot it as one.
+
+## What it is NOT (anti-generic guardrails)
+
+- Not a report generator. No prose walls. If a screen could have come out of Gemini deep research, it's wrong.
+- No number without a baseline. Every metric carries "vs. its own trailing history" — velocity over level, always.
+- No opaque score. The composite is always decomposed into its named inputs in the same view (Quiver DC-Insider pattern).
+- No claim without a receipt. Every evidence row links to the dated, scraped source — verbatim quote, timestamp, URL, scraped-at.
+- Honest about exclusions: LinkedIn is listed as "known, excluded (ToS)" — an engineering call shown in the UI, not a silent gap.
+
+## The three screens
+
+### 1. Live Scan (the TinyFish moment)
+User enters a ticker → source-resolution step (TinyFish **search**: find the subreddit, Trustpilot domain, app-store IDs, ATS job board, EDGAR CIK, newsroom URL) → fan-out of **fetch/agent** runs, one per source, streamed to the UI as a rail of source cards flipping from "agent browsing…" (with live `streaming_url` browser view available) → evidence rows land as each completes. The audience literally watches agents walk Reddit, Trustpilot, and a careers page. This screen is the demo's opening 60 seconds.
+
+### 2. Company Read (the spine — Quiver-style entity page)
+- Header: company, ticker, price context, **Direction Score** (0–100 with ▲/▼ trend) decomposed inline into its four families:
+  - **Customer Sentiment 40%** — Reddit post/comment sentiment + velocity, Trustpilot rating trend + review velocity, app-store rating deltas, BBB complaint velocity
+  - **Workforce 30%** — job-posting count by department (ATS API), posting velocity, layoffs.fyi/WARN events
+  - **Leadership 20%** — exec departures/appointments from newsroom + 8-K 5.02, tenure churn
+  - **Product/Ops 10%** — Downdetector incident volume, status-page incidents, pricing-page diffs (Wayback baseline)
+- Signal tiles: headline delta ("+38% complaint velocity, 7d") + sparkline + baseline note, one tile per family metric.
+- **"What changed this week"** module: ranked biggest movers across all signals — the analyst's first click.
+- Evidence tables per family: verbatim quotes/excerpts, dated, sourced, one hop from every number.
+
+### 3. Lead-Time Timeline (the killer visual, Bloomberg-ALTD pattern)
+One timeline: our leading signals (sentiment velocity, hiring velocity) plotted **against** official disclosure events (8-K filings, press releases) and price. The gap between "signal moved" and "filing landed" is the product, made visually self-evident. Pre-seeded calibration case: **Cracker Barrel** — complaint/sentiment collapse predating the 2026-07-27 CEO-departure 8-K by months. The pitch line: "here's the lead time on a known case; now run it live on any company."
+
+## Demo script (Glean GO)
+
+1. Open on Cracker Barrel's Lead-Time Timeline — the backtest. 30 seconds of "this is what leading actually means."
+2. Live scan on **Etsy** (or Starbucks — both have live 2026 stories): agents fan out on stage, score assembles piece by piece.
+3. Drill one evidence row from score → verbatim seller complaint, dated last week, linked to source.
+4. Closer: "What changed this week" across the tracked watchlist.
+
+Reliability plan: every scan persists; the stage demo can replay the latest stored scan instantly while a genuinely live scan streams in parallel. Conference wifi never gets to kill the demo.
+
+## Sources per scan (escalation ladder: search → fetch → agent)
+
+| Source | Primitive | Reliability |
+|---|---|---|
+| Reddit `.json` listings | fetch | High |
+| Trustpilot (`__NEXT_DATA__`) | fetch | High |
+| Apple/Google app store pages | fetch | High |
+| Greenhouse/Lever ATS APIs | fetch | Very high |
+| layoffs.fyi | fetch | High |
+| SEC EDGAR full-text (8-K 5.02) | fetch | Very high |
+| Company newsroom | agent (lite) | High |
+| Downdetector | agent (stealth) | Medium — cached fallback |
+| BBB complaints | agent (stealth) | Medium |
+| Pricing page + Wayback diff | fetch | Medium |
+| Glassdoor | excluded live; cached snapshot only | Low |
+| LinkedIn | excluded, stated in UI | — |
+
+Concurrency: waves of 5 agent runs (plan limits). Every run validated on content, not status.
+
+## Scoring (stated, not hidden)
+
+Per family: evidence normalized by LLM (GPT-5 for classification/sentiment; Fireworks open-source for bulk labeling) → family score = trend direction × velocity (magnitude × recency) vs own trailing baseline → composite = 40/30/20/10 weighted. Weights shown in the UI with a methodology popover: "starting weights, backtested against Cracker Barrel." Sub-scores always visible next to the composite.
+
+## Architecture
+
+- Next.js (App Router, TS, Tailwind + shadcn), deployed on Vercel.
+- TinyFish via `@tiny-fish/sdk`, server-side SSE routes (`runtime nodejs`, `maxDuration 800`), district-rent-shark pattern.
+- **Raw Postgres** (`postgres` driver, Supabase session pooler via `DATABASE_URL`) — no supabase-js.
+- Tables: `companies` (resolved source profile), `scans`, `source_runs` (per-source raw result + status + streaming url), `evidence` (normalized rows: quote, sentiment, date, url, family), `signal_scores` (per scan per family), `official_events` (8-Ks, press releases — the lagging track for the timeline).
+- Deltas computed between scans; Wayback + first-scan backfill seeds baselines for new companies.
+- Logging: every scan/source run logs id, source, duration, result size, reason on failure — success and failure both.
+
+## v1 cut (Friday EOD)
+
+- Live scan + Company Read + Lead-Time Timeline for a curated six: CBRL (pre-seeded backtest), ETSY, SBUX, TSLA, Z, UNH. Arbitrary-ticker entry works via source resolution but the six are guaranteed-good.
+- Score + evidence + "what changed" module. Watchlist page if time allows, else cut.
+- Not in v1: alerts, auth, sector-peer baselines (self-baseline only), Bluesky.