zeapsu · spencerwozniak · May 12, 2026
diff --git a/docs/README.md b/docs/README.md
@@ -0,0 +1,33 @@
+# Sage docs
+
+Reference notes for changes made on this branch (`add-backend-requirements-txt`)
+since the last commit on `main` (`92b46c8 build: add backend/requirements.txt`).
+Each file documents one feature area: what was added, where it lives, and how
+to use it.
+
+## Contents
+
+- [agentation-install.md](./agentation-install.md) — installing the Agentation
+  dev panel in the Next.js app.
+- [chat-backend-integration.md](./chat-backend-integration.md) — wiring
+  `ChatWidget` to the live `/api/chat` endpoint (replaces the simulated reply).
+- [knowledge-base-ui.md](./knowledge-base-ui.md) — the upload modal next to the
+  command bar and the `KnowledgeBaseWidget` for browsing ingested documents.
+- [in-chat-command-routing.md](./in-chat-command-routing.md) — shared verb
+  detection so the command bar *and* the chat input both route "quiz",
+  "flashcards", "knowledge", etc. to the right view.
+- [generation-endpoints.md](./generation-endpoints.md) — backend
+  `/api/generate/flashcards` and `/api/generate/quiz` plus the
+  `FlashcardsView` / `QuizView` wrappers that consume them.
+- [audio-generation.md](./audio-generation.md) — `/api/generate/audio`
+  narration scripts, OpenAI TTS synthesis, and the dual-mode
+  `AudioPlayerWidget` (`<audio>` element when TTS is available, browser
+  `SpeechSynthesis` otherwise).
+
+## Conventions used here
+
+- File paths are relative to the repository root.
+- Code snippets are illustrative — read the actual files for current source of
+  truth; this folder documents intent and shape, not exhaustive APIs.
+- Each doc has a **Files touched** list at the bottom so reviewers can map
+  features to diffs quickly.
diff --git a/docs/agentation.md b/docs/agentation.md
@@ -0,0 +1,38 @@
+# Agentation dev panel
+
+Added the [Agentation](https://www.agentation.com/install) overlay so feedback
+posted from the running app surfaces in this Claude Code session via the
+`mcp__agentation__*` tool family.
+
+## What it does
+
+- `npm install agentation -D` was run inside `frontend/`.
+- A small client wrapper (`AgentationDev`) renders `<Agentation />` only when
+  `process.env.NODE_ENV === "development"`, so the overlay never ships in a
+  production build.
+- The wrapper is mounted at the end of `<body>` in the root layout, after
+  `{children}`, to keep its DOM out of the app tree.
+
+## Files touched
+
+- `frontend/package.json` — added `agentation` to `devDependencies`.
+- `frontend/src/components/AgentationDev.tsx` — new, dev-only wrapper.
+- `frontend/src/app/layout.tsx` — imports and renders `<AgentationDev />`.
+
+## Optional MCP server
+
+The npm package exposes a UI overlay only. To expose the agent feedback bridge
+to a CLI agent (Claude Code), the user runs the MCP install separately —
+this repo does not commit the server config:
+
+```
+npx agentation-mcp init      # interactive setup
+npx agentation-mcp doctor    # verify
+```
+
+The default MCP server listens on `http://localhost:4747`. If you want the
+React panel to talk to a non-default endpoint, pass it explicitly:
+
+```tsx
+<Agentation endpoint="http://localhost:4747" />
+```
diff --git a/docs/audio-generation.md b/docs/audio-generation.md
@@ -0,0 +1,167 @@
+# Audio (TTS) generation
+
+The audio view used to render a hard-coded `SAMPLE_TRACK` and fake playback
+with a `setInterval` timer. It now generates a real, KB-grounded narration
+script and plays actual audio.
+
+## End-to-end flow
+
+1. User types something containing `audio`, `listen`, or `podcast` (in the
+   command bar or in chat). The prompt is captured as `generationPrompt`.
+2. `<AudioView prompt={generationPrompt} />` renders a loading card and
+   `POST`s to `/api/generate/audio`.
+3. Backend retrieves KB chunks (semantic search if a topic is given, random
+   sample otherwise — same helper as flashcards/quiz), prompts the configured
+   provider for a spoken-style script, and splits it into sentence segments
+   with estimated timings (~2.5 words/sec).
+4. If an OpenAI API key is configured, the backend also synthesizes the
+   script to an MP3 via OpenAI TTS (`tts-1`) and caches it to disk; the
+   response includes `audio_url: "/api/audio/{id}.mp3"`.
+5. If no key is configured, `audio_url` is `null` and the frontend falls
+   back to the browser's `SpeechSynthesis` API for playback.
+6. `AudioPlayerWidget` plays the real `<audio>` element or speaks via
+   `SpeechSynthesis`, advances the transcript highlight in lockstep, and
+   updates the "Download" affordance accordingly.
+
+## Backend
+
+### `POST /api/generate/audio`
+
+Request:
+
+```json
+{
+  "topic": "string | null",
+  "tome_id": "string | null",
+  "voice": "alloy | echo | fable | onyx | nova | shimmer | null",
+  "provider": "string | null",
+  "model":    "string | null"
+}
+```
+
+Response:
+
+```json
+{
+  "id":       "<audio_id or local-…>",
+  "title":   "Audio: <topic>",
+  "voice":    "alloy" | "Browser",
+  "provider": "<resolved>",
+  "model":    "<resolved>",
+  "script":   "Full narration…",
+  "segments": [{"id":"s1","text":"…","startTime":0.0,"endTime":4.4}, …],
+  "duration": 217.8,
+  "audio_url": "/api/audio/<id>.mp3" | null,
+  "sources":  [{"document_id":"…","document_title":"…","chunk_index":3,
+                "similarity":0.81}, …],
+  "topic":    "<echoed>"
+}
+```
+
+- Segments are produced by a simple sentence regex; durations are
+  proportional to word count. They are not word-accurate but are good
+  enough for transcript scroll-lock.
+- `voice` is the OpenAI TTS voice that was used. Defaults to `alloy`.
+  Unknown voices are coerced to `alloy`. When no OpenAI key is present,
+  the response labels it `Browser` so the UI can show a "Browser TTS"
+  badge in place of the Download button.
+
+### `GET /api/audio/{audio_id}.mp3`
+
+Streams a previously-synthesized file from
+`SAGE_AUDIO_CACHE` (defaults to `/tmp/sage_audio`). The id is validated
+against `[a-f0-9-]{8,}` to keep the route from being abused as a
+directory traversal.
+
+### Empty KB
+
+Same behavior as the other generation endpoints: if the user's KB (or the
+scoped tome) has no chunks, the endpoint returns `400` with a pointer to
+the upload button.
+
+### Helper reuse
+
+`backend/api/audio.py` imports `_gather_context` and `_empty_kb_error`
+from `backend/api/generate.py` (the underscore is a hint, not a hard
+boundary). A new `llm_text(...)` helper in `generate.py` does a plain
+content-only completion (no JSON-mode), used by audio for the narration
+script.
+
+## Frontend
+
+### `AudioPlayerWidget`
+
+Two new props:
+
+- `audioUrl?: string | null` — when set, a hidden `<audio>` element is
+  rendered and used as the playback source of truth (`timeupdate` /
+  `ended` events drive `currentTime` and `isPlaying`).
+- `script?: string` — full narration text. Used by the
+  `SpeechSynthesis` fallback so it can speak the remaining text after a
+  seek (it can't restart mid-utterance).
+
+Playback semantics differ a bit between the two modes:
+
+|                   | `<audio>` mode                  | SpeechSynthesis mode              |
+|-------------------|---------------------------------|------------------------------------|
+| Play / pause      | `el.play()` / `el.pause()`      | `synth.speak()` / `synth.cancel()` |
+| Seek bar          | Sets `el.currentTime`           | Cancels and restarts speech from the nearest segment |
+| ±15s buttons      | Same as seek                    | Same as seek                       |
+| Transcript timing | Driven by `timeupdate`          | Driven by a 100ms interval and word-count-based segment times |
+| Footer affordance | `<a download>` for the MP3      | "Browser TTS" pill                 |
+
+### `AudioView`
+
+Same shape as the existing `FlashcardsView` / `QuizView` — loading,
+error-with-retry, and success states. The sources footer lists distinct
+source document titles.
+
+### API client
+
+`generateAudio({ topic, tomeId, voice, provider, model })` in
+`frontend/src/lib/sage-api.ts` returns an `AudioGeneration`. A small
+`resolveAudioUrl(path)` helper joins the relative audio path with the
+active API base (so it works both from the Next.js dev server hitting
+`http://localhost:8000` and the Tauri sidecar on `http://127.0.0.1:8080`).
+
+## Limitations
+
+- Word-level subtitle accuracy isn't possible from `/v1/audio/speech`;
+  segment timings are an estimate. They're tightly correlated to the
+  actual playback rate for `tts-1`, but a long sentence with technical
+  jargon will lag slightly.
+- The MP3 cache is on local disk (`/tmp/sage_audio` by default). Nothing
+  evicts it — fine for dev, but a long-running deployment will want a
+  TTL / size cap.
+- No streaming yet: the user waits for the script to finish generating
+  *and* the MP3 to finish synthesizing before playback can start. Could
+  be split into "play once script is ready (SpeechSynthesis)" while the
+  MP3 synthesizes in the background, but that's a follow-up.
+
+## Files touched
+
+### Backend
+
+- `backend/api/audio.py` (new) — narration + TTS endpoints.
+- `backend/api/generate.py` — added `llm_text(...)` helper.
+- `backend/api/chat.py` — updated the "audio" capability description in
+  the system prompt so the agent stops calling it a sample track.
+- `backend/main.py` — `include_router(audio.router)`.
+
+### Frontend
+
+- `frontend/src/components/AudioPlayerWidget.tsx` — added `audioUrl` and
+  `script` props, real `<audio>` mode, SpeechSynthesis fallback,
+  conditional Download / Browser-TTS badge.
+- `frontend/src/components/AudioView.tsx` (new) — fetch wrapper with
+  loading/error states and source footer.
+- `frontend/src/lib/sage-api.ts` — added `generateAudio`, `AudioGeneration`,
+  `AudioSegment`, `resolveAudioUrl`.
+- `frontend/src/app/page.tsx` — uses `<AudioView />` in the audio view-state;
+  audio prompts are stored in `generationPrompt` like flashcards/quiz.
+
+## How to enable server-side TTS
+
+Set `OPENAI_API_KEY` in the backend's environment (or `openai.api_key` in
+`SageConfig`). The endpoint will automatically synthesize MP3s; otherwise
+playback uses the browser voice with no extra setup.
diff --git a/docs/chat-backend-integration.md b/docs/chat-backend-integration.md
@@ -0,0 +1,47 @@
+# Chat backend integration
+
+`ChatWidget` previously faked replies with a `setTimeout` that echoed the
+user's message back. It now drives a real round-trip against the FastAPI
+`/api/chat` endpoint via the existing `sendChat()` helper in
+`frontend/src/lib/sage-api.ts`.
+
+## Behavior
+
+- The widget tracks a `sessionId` returned from the first request and reuses
+  it on follow-up turns, so the backend can persist the conversation in
+  `KnowledgeStore`.
+- The typing indicator now reflects actual backend latency instead of a
+  hard-coded 1.5s delay.
+- Backend errors are surfaced inline as an assistant-style message
+  (`**Error reaching backend:** …`) instead of being swallowed.
+- A new `initialQuery` prop lets the parent auto-send a prompt on mount —
+  this is how `page.tsx` forwards the user's first command-bar query into
+  the chat view.
+- Optional `tomeId` / `provider` / `model` props pass through to `sendChat`
+  for future tome-aware UI.
+
+## Provider/model badge
+
+`ChatWidget` and `CommandBar` both call `getRuntimeConfig()`
+(`GET /api/config`) on mount to label the current provider/model in the
+header pill, instead of the hard-coded "GPT-4o" string.
+
+## Files touched
+
+- `frontend/src/components/ChatWidget.tsx` — replaced simulated reply with
+  `sendChat()`, added session/loading/error state, `initialQuery`,
+  `onCommand` (see [in-chat-command-routing.md](./in-chat-command-routing.md)).
+- `frontend/src/components/CommandBar.tsx` — model label is now driven by
+  `getRuntimeConfig()`.
+- `frontend/src/app/page.tsx` — tracks the user's command-bar prompt and
+  forwards it to `ChatWidget` as `initialQuery`.
+
+## Running locally
+
+1. Backend: `cd backend && python main.py` (defaults to port 8000).
+2. Frontend: `cd frontend && npm run dev` (defaults to port 3000).
+
+CORS for `http://localhost:3000` is already allowed in `backend/main.py`.
+The `/api/chat` endpoint uses whichever provider is configured as
+`providers.default` in `SageConfig` (Ollama by default); see
+`backend/config.py` for the per-provider settings.