Cortex Frontend

cortex-next is the interactive Next.js client for Cortex. It does not run TRIBE v2 itself. Instead, it loads a reproducible runtime manifest, replays previously captured cortical event streams, renders them on a parcel-aware brain mesh, and presents both the TRIBE-derived summary and a separate emotion-analysis layer.

The current product path is backend-free recorded replay: the frontend reads a storage-hosted runtime manifest and fetches recorded stream files directly in the browser. The companion backend in ../cortex-fastapi is still used for fixture capture, publishing, and optional live/provider work.

How This Differs From Raw TRIBE v2

The upstream model predicts cortical activity. This frontend is the experimental replay and inspection layer around those predictions:

• raw TRIBE output is a dense numerical time series; the frontend consumes normalized event streams and summary artifacts
• raw TRIBE usage does not define a static library or deployment format; the frontend reads a manifest of fixed, reproducible examples
• raw TRIBE usage does not provide parcel-aware inspection; the frontend adds a 3D cortical viewer, hover labels, split-brain interaction, and region callouts
• raw TRIBE usage does not separate emotional interpretation from cortical evidence; the frontend displays the saved DeepSeek emotion layer alongside the preserved TRIBE summary

What This App Does

At a high level, the frontend:

1. lets the user browse and launch the recorded examples from a static manifest
2. fetches the selected example's recorded event stream directly from storage
3. applies captured cortical activity to the left and right hemisphere meshes
4. stores full frame history for post-analysis replay and scrubbing
5. renders hover inspection, split-brain viewing, and top-region callouts
6. shows the final TRIBE summary, recommendations, and technical evidence once replay completes
7. loads and presents a separate emotion-analysis artifact when one is available

Tech Stack

• Next.js 16 App Router
• React 19
• TypeScript
• Tailwind CSS 4
• Framer Motion
• Zustand
• Three.js
• three-mesh-bvh
• three-subdivide

Repo Structure

Important frontend entrypoints:

• app/page.tsx
  • mounts the application shell
• components/cortex-app.tsx
  • top-level client orchestrator for phase changes, prompts, loader state, split state, callouts, and panel visibility
• components/brain/brain-scene.tsx
  • native Three.js scene and animation runtime
• components/ui/*
  • prompt, feedback, hover tooltip, split switch, legend, and score panel
• lib/api.ts
  • static manifest loading plus the legacy backend session helpers
• lib/analysis-stream.ts
  • static recorded replay orchestration plus the legacy WebSocket path
• lib/store.ts
  • central Zustand state for phase, latest frame, summary, hover state, and errors
• lib/schemas.ts
  • runtime Zod validation for backend responses and stream events

End-To-End Flow

1. App shell and local UI state

app/page.tsx renders CortexApp.

CortexApp is the real UI controller. It owns:

• prompt submission
• loader progress and dismissal
• result-panel visibility
• split-brain toggle state
• region-label toggle state
• callout list state

It reads long-lived analysis state from Zustand and keeps transient UI choreography local.

2. Prompt submission

The input UI is components/ui/prompt-composer.tsx.

When the user submits:

1. the current stream is disconnected if one exists
2. Zustand is reset into a connecting phase
3. openAnalysisStream(request, handlers) is called

The composer now runs from the recorded example library only.

Recorded entries are loaded from NEXT_PUBLIC_RECORDED_MANIFEST_URL when that env var is set. Each selected example includes an events_url, so the browser can fetch the saved JSONL stream directly without creating a backend session. When present, the same manifest entry also carries an emotion_analysis_url, which is loaded separately so the emotional interpretation remains distinct from the recorded TRIBE summary.

3. Replay startup

Static replay mode is the default deployment path.

When NEXT_PUBLIC_RECORDED_MANIFEST_URL is configured, the frontend:

1. loads the runtime manifest through fetchFixtures()
2. submits the selected fixture with mode="replay" and its events_url
3. fetches that JSONL event stream from storage
4. schedules the recorded events locally in the browser

lib/analysis-stream.ts is responsible for:

• open timeout handling
• replay pacing for text, audio, and video
• parsing every incoming event through Zod
• forwarding valid events into the store
• surfacing stream errors cleanly

If no recorded manifest URL is configured, the legacy backend path can still create sessions and open the backend WebSocket.

4. Store updates

lib/store.ts is the single source of truth for streamed analysis state.

It tracks:

• phase
• statusText
• provider
• sessionId
• currentPrompt
• latestFrame
• frameHistory
• selectedFrameIndex
• replayMeta
• summary
• errorMessage
• hoverInfo

Incoming events update the store by type:

• session.accepted
• analysis.preparing_events
• analysis.started
• analysis.frame
• analysis.summary
• analysis.error
• analysis.done

This keeps the UI declarative while avoiding large prop chains.

The 3D Brain Scene

The center of the frontend is components/brain/brain-scene.tsx.

This file runs an imperative Three.js scene inside React. It does not rely on React Three Fiber. That is intentional: the scene has a lot of frame-by-frame mesh, color, hover, and split-state work, and the project keeps that work close to the renderer.

Scene setup

The scene boot sequence:

1. creates a WebGLRenderer, Scene, PerspectiveCamera, and OrbitControls
2. loads left and right hemisphere GLB assets
3. applies subdivision for the display shell
4. builds color buffers and parcel lookup data
5. loads hover guide / parcel mapping data from local JSON assets

Surface mapping

The backend sends sampled left/right activity arrays, not a full arbitrary mesh payload.

The frontend maps those arrays onto the display shell using:

• fsaverage5-aligned lookup data
• per-hemisphere sample-to-vertex mappings
• parcel labels and copy from the local hover guide

Animation and split behavior

The brain scene is responsible for:

• closed idle pose
• streamed activation coloring
• split-brain hinge motion
• panel-aware scaling and framing
• hover highlighting
• top-region callout placement

The split-brain effect is not a camera trick. Each hemisphere is moved through its own hinge group so the medial surfaces can be exposed.

Hover and callouts

Hover interaction is accelerated with three-mesh-bvh, which replaces default face raycasting with BVH-accelerated queries. That keeps parcel hover usable on the dense cortex mesh.

When inspection is enabled, the scene:

• resolves the hovered parcel
• emits hover info to the tooltip
• projects the top regions into screen space
• positions the callout cards on outer rails

The current callout system is layout-aware enough to keep the top cards separated and away from the center brain view better than the earlier fixed-position implementation.

Result UI

Once the backend emits analysis.summary, CortexApp waits briefly, then reveals the result panel.

components/ui/score-panel.tsx renders:

• the headline
• narrative summary
• a separate emotion estimate panel backed by the saved DeepSeek artifact
• audience-facing insight cards
• recommendations
• a collapsible technical evidence section
• top brain regions behind the response

The panel is intentionally split between:

• a product-facing top section
• a more technical evidence section below

Brain Guide and Parcel Metadata

The frontend now ships the parcel metadata needed for hover/callout rendering as local JSON assets.

That payload defines:

• parcel ids
• canonical parcel labels
• display names
• system ids and system labels
• role labels and role descriptions
• left/right vertex-to-parcel assignments

This is what lets the client show parcel-level copy instead of anonymous left/right heat.

Environment Variables

Main frontend variable for current recorded-only deployment:

• NEXT_PUBLIC_RECORDED_MANIFEST_URL
  • static runtime manifest URL used to load recorded examples and their event streams

Current Cloudinary deployment value:

NEXT_PUBLIC_RECORDED_MANIFEST_URL=https://res.cloudinary.com/dcelboq45/raw/upload/v1776287421/recorded-runtime/runtime-manifest.json

Legacy backend variable:

• NEXT_PUBLIC_API_BASE_URL
  • backend base URL used only when running the old backend session/WebSocket path

Do not set NEXT_PUBLIC_API_BASE_URL for frontend-only static replay mode.

Example legacy backend env:

NEXT_PUBLIC_API_BASE_URL=http://127.0.0.1:8000

Profile examples also live in:

• profiles/local-backend.env.example
• profiles/remote-do.env.example

Local Development

Frontend only

cd cortex-next
npm install
npm run dev

For recorded replay, set NEXT_PUBLIC_RECORDED_MANIFEST_URL in .env.local first.

Full local project via the root launcher

From the workspace root:

./start-both.sh local-static-recorded
./start-both.sh local-recorded
./start-both.sh local-tribe-cpu
./start-both.sh local-tribe-mps
./start-both.sh remote-do

The launcher defaults to local-static-recorded, kills stale local listeners by default, and only starts the backend for backend profiles.

Scripts

• npm run dev
• npm run build
• npm run start
• npm run lint

Important Current Limitations

The current product surface is intentionally replay-first.

That means:

• the UI currently submits recorded replay sessions only
• the example browser depends on NEXT_PUBLIC_RECORDED_MANIFEST_URL in static mode
• text examples intentionally do not expose media-style replay controls after completion
• live text/audio/video upload is still a separate future track and is not exposed in this interface

Relationship To The Backend

The frontend does not require the backend for the current static replay deployment.

In other words, this app is not a thin TRIBE v2 demo client. It is a replay, visualization, and inspection system built on top of precomputed TRIBE artifacts.

If you run the legacy backend path, the frontend expects:

• POST /api/v1/sessions
• GET /api/v1/brain/guide
• WS /api/v1/stream/{session_id}

The backend is responsible for:

• provider selection (tribe or recorded)
• timed-event preparation
• frame generation
• summary generation
• optional DeepSeek/OpenAI refinement

The frontend is responsible for:

• turning those events into a fluid interactive visualization
• managing the session lifecycle in the browser
• presenting both the audience summary and the technical evidence