From 8fe7915677f6a2e039abfb428e48033879bf8e44 Mon Sep 17 00:00:00 2001 From: Ross Miller Date: Tue, 21 Jul 2026 10:44:05 -0500 Subject: [PATCH 01/10] docs(plans): commit the phase 4 plan with its post-flight corrections Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5 --- docs/plans/refactor/phase-4.md | 132 +++++++++++++++++++++++++++++++++ 1 file changed, 132 insertions(+) create mode 100644 docs/plans/refactor/phase-4.md diff --git a/docs/plans/refactor/phase-4.md b/docs/plans/refactor/phase-4.md new file mode 100644 index 0000000..1c1c4ae --- /dev/null +++ b/docs/plans/refactor/phase-4.md @@ -0,0 +1,132 @@ +# Phase 4 — Docs + AI Ergonomics + +> **ARCHITECT'S POST-FLIGHT CORRECTIONS (written mid-effort after Phases 0-3 landed; these override the assumptions below):** +> 1. `buildServer()` has existed since Phase 0 and Phase 2 evolves it to accept `overrides?: Partial`. S4 consumes it; never re-extract. Crash recovery is now `services/recover-from-crash.ts`, a composition-level service over BookRepository + ArtifactStore, wired in `startServer` only — the server README's buildServer/startServer point stands with that wording. +> 2. `.github/workflows/ci.yml` exists since Phase 0 and runs on **macos-14** (Linux fails on platform-coupled audiobook tests, reasoning commented in the workflow). S5 appends the drift job to it. +> 3. `shared/` is FLAT modules, not directories: `domain.ts`, `contracts.ts`, `book-status.ts` (zod-free literal tuple + SIX predicates including `isGeneratingToc`, param currently `BookStatus` post-P3 tightening), `provider.ts`, `responses.ts` (includes `ProfileResponse`), `events.ts` (five per-stream SSE unions), `sanitize-mermaid.ts`, `mermaid-theme.ts`, `node/data-dir.ts`. Document reality, not the planned directory names. +> 4. The port catalog is **13**: TextGeneration, KeyVault, ImageGeneration, BookRepository, ArtifactStore, SpeechSynthesis, AudioAssembly, DiagramRenderer, EpubImport, EpubExport, BackgroundTasks, Clock, OsFileManager. Each ships `.ts` + `.fake.ts` + `.contract.ts` + `.fake.test.ts`. Contract conventions worth documenting in `server/ports/README.md`: missing entities are `code === 'ENOENT'` errors (never instanceof), deliberately synchronous methods stay synchronous, DiagramRenderer returns escaped-source fallback on failure and never `''`. +> 5. `mcp-config.ts` lives at `client/lib/mcp-config.ts`. The client tree for READMEs is `client/{app,api,features/*,components/ui,hooks,lib,store}` with the dialog state machine at `features/library/dialogs/`. +> 6. CLAUDE.md rewrite: the stale "Zod schemas live in server/schemas.ts" convention line becomes "Zod domain schemas live in shared/, the single source of truth for both sides." Keep the TDD conventions line from consolidation delta 9. +> 7. Seven sanctioned behavior changes exist by now (see follow-ups ledger); ADR content is unaffected but do not describe pre-refactor behavior as current. +> 8. Verify counts fresh: tests were 458+ client-side at P3 merge and Phase 2 adds contract/service tests; the routes doc still expects ≥58 paths across ~12 route files post-split. +> 9. **OWNER ADDITION — the architecture hub (new task S6b, same priority as S6).** Create root `ARCHITECTURE.md` as THE entry point for a cold principal-engineer/CTO reader. Content: five small single-purpose mermaid diagrams, each with at most a short paragraph of prose, links doing the rest. (1) Context view: Electron shell, renderer, embedded Fastify server, the on-disk library, external AI providers, local Kokoro TTS. (2) The hexagon: server core with its 13 ports and their adapters. (3) Client data flow: features → hooks → typed api client → HTTP/SSE → thin routes → services → ports. (4) Sequence diagram of the adaptive loop, create → TOC approve → chapter 1 → read → feedback + quiz → next chapter generation. (5) The dependency rule: client → shared ← server, lint-enforced, with electron allowed shared only. Diagram nodes use real folder names at package granularity so the existing path-exists acceptance greps cover them and they only change when the top-level shape changes. Style rule for the WHOLE docs package: diagrams first, prose second, links third, never wordy, no doc over roughly one screen except CONTEXT.md and the generated routes doc. Hyperlinking discipline: every folder README starts with an "Up: ARCHITECTURE.md" breadcrumb and ends with "Related:" links, ADR index and CONTEXT.md link back to the hub, CLAUDE.md and README.md point at ARCHITECTURE.md as the reading entry point. S9's CLAUDE.md rewrite and the READMEs must conform. Also note: Phases 6 and 7 were approved after this plan was written, E2E-on-fakes and durability (schema versioning, persisted generation jobs, AI error taxonomy). They land BEFORE Phase 4, so the ADR set grows to EIGHT (add 0007 schema versioning + migrations, 0008 persisted generation jobs), the hexagon diagram must show the final port list, and the routes/test counts will have moved again. Phases 8 and 9 (observability, security, release engineering) were explicitly declined by the owner because the app is local-only with no cloud component, record that rationale in ADR 0004's consequences or a short note in ARCHITECTURE.md, not as a gap. + +Assumes P2 (`server/{domain,ports,adapters,services,routes}`) and P3 (`client/{api,features,components/ui,hooks,store}`, `shared/`) have merged. Every task below is docs-or-tooling only: no runtime behavior changes except the one small, behavior-preserving server extraction in S4 (consolidation delta 4: buildServer already exists from Phase 0 — consume it, do NOT re-extract). + +Consolidation deltas that modify this plan: S1 must NOT delete `docs/plans/refactor/` (each phase's reconciled plan lives there); S5 appends the drift job to the existing P0-created `.github/workflows/ci.yml`; port naming follows P2's shipped catalog (EpubImport/EpubExport etc., not EpubIO); add a TDD line to the CLAUDE.md conventions and encode test-first ordering in the add-feature skill steps. + +Verified recon deltas vs the decision record (use these, they correct stale CLAUDE.md): +- 58 unique `/api/*` paths across 9 route files (CLAUDE.md documents 14). Routes register inside `startServer()` (post-P0: inside `buildServer()`). +- `server/schemas.ts` was 378 lines, pure Zod, 60+ exports (post-P1 it lives in `shared/`). +- **No `job` term exists in code.** `task-manager.ts` uses `BackgroundTask` / `TaskType` / `TaskStatus` consistently. Only CLAUDE.md prose invents `GenerationJob` and `generation-queue.ts`. Delete the prose, keep the code names. +- `.claude/skills/` already holds `generate-book` and `rca`. Two frontmatter shapes exist; new skills follow `generate-book` (name + description). +- Existing JSDoc is sparse (≈28 blocks total) but the style is already correct: constraint- and rationale-stating. `book-store.ts:658` and `kokoro-service.ts:21` are the reference exemplars (post-P2 these live in adapters). +- MCP command comes from `src/lib/mcp-config.ts` (post-P3: `client/lib/mcp-config.ts`): `pnpm mcp:dev` with `TUTOR_API_URL=http://127.0.0.1:3147`. + +## Ordered implementer tasks + +**S1 — Docs surface prep.** Create `docs/adr/`. Confirm the real post-P2/P3 folder names before writing any path into a doc. Keep `docs/screenshots/` and `docs/plans/refactor/`. +*Accept:* `ls docs/` shows `adr/ plans/ screenshots/`; no stale plans outside `plans/refactor/`. + +**S2 — `CONTEXT.md` glossary** (repo root). Terms below, one to three sentences each, each naming the type or module that owns it. Include the "Overloaded terms" section verbatim from this plan. +*Accept:* every term listed appears; every domain term cites a real symbol (`rg -q '' shared/ server/` passes for each). + +**S3 — Six ADRs** from the bullet drafts below. Filenames `0001-…` to `0006-…`, one format: `# NNNN. Title` / `Status: Accepted` / `Date:` / `## Context` / `## Decision` / `## Consequences` / `## Revisit when`. Prose from bullets, no invented facts. `docs/adr/README.md` = index table. +*Accept:* six files + index; index rows match filenames; each ADR ≤400 words; no ADR references a nonexistent path. + +**S4 — Routes-doc generator.** buildServer() exists from Phase 0 (no listen, no recoverFromCrash — that exclusion is the guard against mutating the real library). Add `scripts/generate-routes-doc.ts`: register an `onRoute` hook (captures method, url, prefix), `await app.ready()`, `await app.close()`, sort by path then method, emit `docs/api-routes.md` with a "generated by `pnpm docs:routes` — do not edit" banner, a grouped table (Method | Path | Source file), and a total count. Add `"docs:routes": "tsx scripts/generate-routes-doc.ts"`. +*Accept:* `pnpm docs:routes` emits ≥58 paths; running it twice is byte-identical; no book directory is created or modified; `pnpm test` and `tsc --noEmit` still pass. + +**S5 — Staleness gate.** Append to the existing `.github/workflows/ci.yml`: `pnpm docs:routes && git diff --exit-code docs/api-routes.md` with an actionable failure message ("run `pnpm docs:routes` and commit"). Add the same regenerate-and-diff to `lefthook.yml` `pre-commit` only if it runs in <2s; otherwise CI only. +*Accept:* deleting a line from `docs/api-routes.md` makes the step fail locally when replayed; restoring it passes. + +**S6 — Fractal READMEs** (7 files, outlines below). Each ≤250 words, present tense, links to ADRs by number, no file-by-file inventory (that rots). +*Accept:* all 7 exist; each links at least one ADR or CONTEXT.md term; relative link targets resolve. + +**S7 — JSDoc, server.** Exported symbols in `server/ports/*`, `server/adapters/*`, `server/services/*`. Rules: state the constraint, invariant, failure mode, or ordering requirement the signature cannot show; never restate parameter names or types; skip anything genuinely self-evident. Every port interface gets a file-level block naming its adapters and its fake. Estimate ~95 blocks. +*Accept:* `pnpm lint` clean; spot-check 10 random blocks — zero restate a signature; port files each name their fake. + +**S8 — JSDoc, client + shared.** `client/hooks/*` + feature hooks (~10), `client/api/*` client functions (document auth/streaming/abort semantics, ~15), `shared/*` predicates and constants (~20). Total ~45. Combined estimate: ~140 blocks. +*Accept:* same as S7; every SSE/streaming hook documents its cleanup and abort behavior. + +**S9 — `CLAUDE.md` rewrite.** Keep: project one-liner, How It Works, Tech Stack table, Key Design Decisions, Domain & Architecture principles (aspirational), **all Electron conventions verbatim** (127.0.0.1, `Origin: null`, dual CSP lists, `node-linker=hoisted`, `vite.config.ts` `external()` bundle list, never repoint source `index.html`), Development commands, Conventions (add the TDD line: tests land before or with implementation; contract test before adapter, service test before service). Delete: the per-file File Structure tree, the API routes table, every reference to `book-generator.ts`, `generation-queue.ts`, `chapters.ts`, `prompts/*.md`, `src/pages/LibraryPage.tsx`. Replace with: a 12-line top-level tree (`client/ server/ shared/ electron/ docs/ scripts/`) each line pointing at its folder README; "API routes: see `docs/api-routes.md` (generated, run `pnpm docs:routes`)"; "Domain language: see `CONTEXT.md`"; "Decisions: see `docs/adr/`". Target ≤200 lines. +*Accept:* every path mentioned in CLAUDE.md exists; Electron constraint list unchanged; ≤200 lines. + +**S10 — `.mcp.json`** at repo root: +```json +{"mcpServers":{"tutor":{"command":"pnpm","args":["mcp:dev"],"env":{"TUTOR_API_URL":"http://127.0.0.1:3147"}}}} +``` +Note in `server/README.md` that the packaged Electron app binds a random port, so MCP use requires `pnpm dev:server` on 3147. +*Accept:* `pnpm mcp:dev` starts and lists tools; `mcp__tutor__list_books` succeeds with `dev:server` running; `.mcp.json` is not gitignored. + +**S11 — `.claude/skills/verify/SKILL.md`.** Frontmatter `name: verify`, description covering "verify this repo is green before commit/PR/handoff" (explicitly project-scoped to avoid collision with global verification skills). Sections: When to use → Steps (`pnpm test`, `pnpm typecheck`, `pnpm lint` expecting zero warnings, `pnpm docs:routes && git diff --exit-code docs/api-routes.md`, boot check = `pnpm dev:server` + `curl -s 127.0.0.1:3147/api/health`, kill it) → Failure handling (fix and rerun the full set, never report the first error) → Report format (table: check, command, result, evidence line). +*Accept:* running the skill on a clean tree reports all-pass with real command output pasted. + +**S12 — `.claude/skills/add-feature/SKILL.md`.** Frontmatter `name: add-feature`, description "add a capability end to end following the test-first port → adapter → service → route → client api → feature slice recipe". Sections: When to use / when not (pure UI tweak, no new I/O) → Decide whether a new port is needed (only for a new external dependency) → Steps 1-8 in TDD order (1. define domain types in `shared/`; 2. declare the port interface; 3. write the contract test + in-memory fake — red; 4. write the adapter — contract green; 5. write the service unit test against the fake — red; 6. write the service — green; 7. add the thin route + Zod body schema, regenerate `pnpm docs:routes`; 8. add one function to `client/api` with its mocked-fetch test, then the feature slice hook + component) → Conventions checklist (no raw `fetch` outside `client/api`, no SDK import outside adapters, no magic strings, JSDoc on new exports, tests colocated, test commits precede implementation commits) → Finish with `verify`. +*Accept:* a dry run on a hypothetical capability names real files at each step and produces a plan an implementer could execute. + +**S13 — Phase gate.** Run the checklist below; open the PR. + +Parallelization: S2→S3; S4→S5; S6, S7, S8, S10, S11, S12 independent after S1; S9 needs S2+S4+S6. + +## ADR bullet drafts + +**0001 — Filesystem as the database.** *Context:* single-user desktop app; content is Markdown chapters plus YAML metadata; users want to read, diff, back up, and hand-edit their books; SQLite was the obvious alternative. *Decision:* `books/{id}/{meta.yml,toc.yml,chapters/NN.md,progress.yml,feedback/NN.yml}` under the OS data dir; Zod validates on every read; writes are write-tmp-then-`rename`; `recoverFromCrash()` at boot sweeps `.tmp` leftovers and resets interrupted books. *Consequences:* + human-readable, git-friendly, trivially backed up, EPUB import/export is a file operation, zero migration tooling; − no transactions across files, no query layer so listing the library reads every `meta.yml`, invariants live in code not the schema, crash safety is our own responsibility; ok because there is exactly one writer. *Revisit when:* multi-user or sync arrives, or library listing exceeds a few hundred books. + +**0002 — Just-in-time chapter generation, quiz masks the latency.** *Context:* a chapter takes tens of seconds to generate; pre-generating the whole book wastes tokens because feedback and quiz results change later chapters, and it delays first read. *Decision:* generate chapter 1 on TOC approval, then one chapter at a time; submitting feedback fires the next generation in the background while the reader answers a 3-question quiz over the chapter just read; progress streams over SSE; a separate opt-in `generate-all` BackgroundTask exists. *Consequences:* + perceived latency near zero, each chapter adapts to the previous chapter's quiz and feedback, tokens spent only on what is read; − generation state is in-memory so a restart mid-generation loses the stream (book stays valid, reader retriggers), SSE plumbing on both sides, the quiz is load-bearing UX not just pedagogy. *Revisit when:* generation drops below a few seconds. + +**0003 — Local Kokoro TTS over cloud TTS.** *Context:* audiobooks need per-chapter narration of an entire book; cloud TTS is metered per character and would dominate cost, and BYOK users have no shared billing relationship with us. *Decision:* Kokoro-82M ONNX (q8) via `kokoro-js` + `onnxruntime-node`, model downloaded on first use with progress reported as a BackgroundTask; ffmpeg (also downloaded on demand) muxes chapter WAVs into an M4B with chapter markers; male-first voice ordering with `am_michael` pinned first. *Consequences:* + zero marginal cost, fully offline, no third-party gets book content, speed and voice are ours to control; − ~100MB first-run download plus an ffmpeg fetch, native ONNX binaries must stay `asarUnpack`ed and external in the Vite build (ADR 0006), quality ceiling below premium cloud voices, CPU-bound generation needs worker tuning. *Revisit when:* a cloud voice is dramatically better and users ask for it as an opt-in. + +**0004 — Single package, monorepo-shaped.** *Context:* one `package.json` served `src/` + `server/` + `electron/` + `lib/` with tangled imports; real workspaces would give hard boundaries but add resolution complexity on top of an Electron build that already fights pnpm. *Decision:* top-level `client/ server/ shared/ electron/` with `@client/* @server/* @shared/*` aliases and ESLint-enforced import boundaries (server → server+shared, client → client+shared, shared → neither); one `package.json`, one lockfile, one `node_modules`. *Consequences:* + boundaries are enforced today with zero build risk, Electron packaging untouched, one install and one test command; − dependencies are not partitioned per package so nothing stops a client-only library being imported server-side except lint and review, and versions cannot diverge. *Extraction path:* the folders are already package-shaped; adding `pnpm-workspace.yaml` plus per-folder `package.json` files is mechanical once boundaries have held, but `node-linker=hoisted` and the `external()` list must be re-verified in all three Electron modes first. *Revisit when:* a second app consumes `shared/`, or CI time justifies per-package caching. + +**0005 — AI SDK behind a TextGeneration port.** *Context:* `generateObject`/`streamText` were called inline in five files; three providers are user-selectable per call; timeouts were redefined in four files; nothing that touched AI could be unit-tested. *Decision:* one `TextGeneration` port (`generateObject`, `streamText`, `runToolConversation`) with a Vercel-AI-SDK adapter wrapping the model-resolution logic, plus an in-memory fake and a contract test; services depend on the port, routes stay thin; prompts remain typed TypeScript modules (not runtime `.md` files) so they survive Electron bundling and type-check. *Consequences:* + services are unit-testable without network or keys, timeout/retry/abort policy lives in one place, adding a provider is an adapter concern; − one indirection between a service and the SDK, and the port must expose streaming honestly (chunks + abort) rather than the SDK's richer surface. *Revisit when:* a provider needs a capability the port cannot express without leaking SDK types. + +**0006 — Electron packaging constraints.** *Context:* pnpm's symlinked store, ESM-only unified/remark packages, and native ONNX binaries each break electron-builder differently; three run modes load the renderer from different origins. *Decision:* `.npmrc` `node-linker=hoisted`; `vite.config.ts` `external()` bundles the unified/remark/rehype ecosystem into `dist-electron/` and keeps CJS and native packages external; native audiobook deps are listed in `build.files` and `asarUnpack`; the embedded server binds `127.0.0.1` on a random free port; CORS accepts `Origin: null` and any localhost origin; `connect-src` lists both `127.0.0.1:*` and `localhost:*` in **both** `index.html` and `electron/main.ts`; source `index.html` always points at the source entry, never at build output; dynamically imported CJS modules need `mod.default?.default ?? mod.default`. *Consequences:* + all three modes work and audiobooks work in the packaged DMG; − the bundle list is manual and grows when a new ESM markdown dep appears, hoisting weakens pnpm's isolation, and every packaging change must be checked in all three modes, not just dev. *Revisit when:* electron-builder resolves pnpm symlinks natively. + +## CONTEXT.md glossary + +**Core domain** — **Book**: a generated learning artifact, one directory, owns `BookMeta` (`BookMetaSchema`). **Chapter**: one ~1,500-word Markdown unit, addressed by 1-based `chapterNum`, stored `chapters/NN.md`. **TOC**: the approved chapter plan (`TocSchema`), reviewable and revisable before generation. **Feedback**: the reader's liked/disliked response to a chapter (`FeedbackSchema`); submitting it triggers the next chapter. **Quiz**: 3 questions after a chapter, or a longer final quiz across the book (`QuizSchema`); masks generation latency. **Progress**: per-chapter scroll fraction plus completed flag (`ProgressSchema`), completed at ≥90%. **LearningProfile**: global reader model — identity, style, `Preferences`, `Skill[]`, with optional per-book `profileOverrides`. **Audiobook**: an M4B with chapter markers synthesized locally from a book's chapters, tracked by `AudiobookManifest`. **Series**: a named ordering of related Books (`series` + `seriesOrder` on `BookMeta`), no entity of its own. **Interview**: the conversational flow that fills in a LearningProfile by asking the reader questions. **BackgroundTask**: a cancellable long-running server operation (`generate-all`, `generate-epub`, `generate-cover`, `install-audiobook`, `generate-audiobook`) with progress streamed over SSE. + +**The Skill collision** — **Skill** (domain): a named competency with a 1-10 level in the LearningProfile, and its per-chapter `subskill` weightings in the TOC. **Agent Skill**: a `SKILL.md` under `.claude/skills/` that instructs Claude Code. Both keep their names; always qualify in prose ("learner Skill" vs "Agent Skill") and never rename the domain type. + +**Other overloaded terms to define and use consistently** — **Section**: an intra-chapter slice used for pagination and navigation (`split-sections.ts`, `useSectionNavigation`), never a synonym for Chapter. **Status**: three unrelated enums — `BookStatus` (`generating_toc`…`failed`), `TaskStatus` (`running`/`done`/`error`/`cancelled`), `GenerationStage` (`streaming`/`saving`/`quiz`/`done`/`error`); always qualify. **Review**: `toc_review` (a BookStatus awaiting approval) vs Smart Review (spaced-repetition re-quizzing); unrelated. **Prompt** vs **Brief**: `prompt` is the reader's short topic request on `BookMeta`; `brief` is the long agentic-generation spec stored per book. **Summary**: `ChapterSummary` (cross-chapter context for generation) vs the book-completion summary UI. **Reference**: source material saved for agentic generation (`ReferenceEntry`), not a citation. **Narration**: chapter Markdown transformed for speech (`markdown-to-narration.ts`), distinct from the chapter text. **Progress** appears as scroll progress, `TaskProgress`, and skill progress — qualify each. **Numbering**: `chapterNum` is 1-based and canonical (179 uses); `chapterIndex` is 0-based and array-local only (12 uses). **No `job`**: background work is a `BackgroundTask`; the term "job" appears only in stale docs and must not return. + +## Fractal README outlines + +**`server/README.md`** — what the server is (embedded Fastify, `127.0.0.1`, random port under Electron, 3147 standalone); the hexagonal layering rule (routes parse → services decide → ports do I/O) with the one-line dependency direction; where each layer lives, linking the three sub-READMEs; how to add an endpoint (pointer to the `add-feature` Agent Skill) and the reminder to run `pnpm docs:routes`; the `buildServer` vs `startServer` split and why crash recovery lives only in the latter; pointer to ADRs 0001, 0002, 0005. + +**`server/ports/README.md`** — a port is an interface owned by the domain, named for the capability not the vendor; the current port list, one line each; the rule that every port ships an in-memory fake plus a contract test both adapter and fake must pass; what may not appear in this folder (SDK imports, `fs`, `fetch`, env vars); how to add a port; link ADR 0005. + +**`server/adapters/README.md`** — adapters are the only place real I/O happens, one per port per technology; naming convention `-.ts`; each adapter's external constraints (filesystem atomic writes, Kokoro model download, ffmpeg presence, keychain); adapters are tested by the shared contract test plus their own integration tests; Electron packaging gotchas that bite adapters (native deps external, dynamic CJS double-default), link ADR 0006. + +**`server/services/README.md`** — services hold application logic and orchestrate ports; they never import an SDK or `fs`, and never touch Fastify request/reply; the generation flow as the worked example (approve TOC → chapter 1 → feedback → next chapter, streamed); BackgroundTask ownership and cancellation; unit-tested against fakes, no network in tests; link ADR 0002. + +**`client/features/README.md`** — one folder per user-facing capability, each owning its components, hooks, and slice-local state; logic in hooks, components render; all server access goes through `client/api` — zero raw `fetch` in a feature; cross-feature sharing goes to `components/ui` or `shared/`, never feature-to-feature imports; the dialog state machine replaces ad-hoc dialog booleans; where Redux slices live and why RTK Query was not adopted. + +**`shared/README.md`** — the only code both sides import; contents (Zod domain schemas, status unions with predicates, API contract types, named constants); the hard rule that `shared/` imports neither `client/` nor `server/` and contains no I/O; schemas are the single source of truth — validate at every boundary; changing a schema is a contract change touching both sides plus `docs/api-routes.md`; link ADR 0004 and CONTEXT.md. + +**`electron/README.md`** — `main.ts` and `preload.ts` roles; the three modes table (dev/preview/build) with renderer origin and API routing; the non-negotiable constraints (127.0.0.1, `Origin: null` CORS, dual CSP lists, hoisted linker, `external()` bundle list, never repoint source `index.html`); how to verify a packaging change in all three modes; link ADR 0006. + +## Risks + +1. **Docs describe a tree that P2/P3 renamed.** Mitigation: S1 is a fresh `ls` of the merged branch, and every doc path is asserted by an acceptance grep. Do not start S6 or S9 from this plan's assumed folder names. +2. **Routes-doc generator mutates real user data.** `startServer()` runs `recoverFromCrash()`. The generator must boot `buildServer()` only. The acceptance check (data-dir mtimes unchanged) is the guard. +3. **Route registry ≠ hand-written table.** The generated doc will surface paths CLAUDE.md never listed and may expose internal authoring endpoints. That is correct and desirable, but review the first output before committing. +4. **Stale-doc CI gate creates commit friction.** Keep the failure message actionable. +5. **ADR drift into fiction.** Each ADR must be written from code that exists, with a cited symbol or file. Reject any consequence bullet the implementer cannot point at. +6. **JSDoc bloat.** The failure mode is 140 blocks that restate signatures, which is worse than none. Enforce the spot-check; deleting a weak block is always allowed. +7. **CLAUDE.md loses a load-bearing Electron constraint** during the trim. Diff the Electron section explicitly and require it to be unchanged. +8. **Agent Skill name collisions.** Make the `verify` description explicitly project-scoped. +9. **MCP port assumption.** `.mcp.json` hardcodes 3147; document the `pnpm dev:server` prerequisite. + +## Phase-gate checklist + +- [ ] `pnpm test` green; `pnpm typecheck` clean; `pnpm lint` zero warnings. +- [ ] `pnpm docs:routes` is idempotent and `git diff --exit-code docs/api-routes.md` passes. +- [ ] Boot check: `pnpm dev:server` + `curl 127.0.0.1:3147/api/health` returns ok; `pnpm electron:preview` renders the library. +- [ ] Generating routes docs left the real book library untouched (mtimes unchanged). +- [ ] Every path referenced in CLAUDE.md, CONTEXT.md, the 7 READMEs, and the 6 ADRs exists. +- [ ] Six ADRs plus index; CONTEXT.md covers all core terms, the Skill collision, and every flagged overload. +- [ ] `.mcp.json` connects and `mcp__tutor__list_books` returns. +- [ ] Both Agent Skills execute end to end on a dry run. +- [ ] No `.DS_Store` added; conventional commits; one PR for the phase. From 8a1120a4de1de557e5d3234e855dd4c77096b981 Mon Sep 17 00:00:00 2001 From: Ross Miller Date: Tue, 21 Jul 2026 10:44:08 -0500 Subject: [PATCH 02/10] feat(scripts): generate the API routes doc from the Fastify registry The hand-written route table in CLAUDE.md listed 14 paths against the 59 the server actually serves, which is the failure mode any hand-maintained table reaches eventually. docs/api-routes.md is now a projection of the route registry, regenerated by pnpm docs:routes. The generator boots buildServer, never startServer, so it cannot run migration, crash recovery or job resume against a real library, and it points TUTOR_DATA_DIR at a temp directory it asserts is still empty afterwards. Route attribution is checked back against app.hasRoute and fails loudly rather than publishing a plausible wrong path. Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5 --- docs/api-routes.md | 162 ++++++++++++++++++++++++++++ package.json | 7 +- scripts/generate-routes-doc.ts | 186 +++++++++++++++++++++++++++++++++ tsconfig.json | 2 +- 4 files changed, 353 insertions(+), 4 deletions(-) create mode 100644 docs/api-routes.md create mode 100644 scripts/generate-routes-doc.ts diff --git a/docs/api-routes.md b/docs/api-routes.md new file mode 100644 index 0000000..6e5437f --- /dev/null +++ b/docs/api-routes.md @@ -0,0 +1,162 @@ +# API Routes + + + +70 routes over 59 paths, registered by 17 modules under [`server/routes/`](../server/README.md). The embedded Fastify server serves all of them on `127.0.0.1`, on port 3147 when run standalone with `pnpm dev:server` and on a free port chosen at launch under Electron. + +Routes are grouped by the module that registers them. The `HEAD` route Fastify pairs with each `GET` is omitted. Request and response shapes live in [`shared/contracts.ts`](../shared/contracts.ts) and [`shared/responses.ts`](../shared/responses.ts), and the streaming routes emit the SSE unions in [`shared/events.ts`](../shared/events.ts). + +### `server/index.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/health` | + +### `server/routes/assessment.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/books/:id/chapters/:num/feedback` | +| `GET` | `/api/books/:id/chapters/:num/quiz` | +| `POST` | `/api/books/:id/final-quiz` | + +### `server/routes/audiobook-generation.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/books/:id/audiobook` | +| `POST` | `/api/books/:id/audiobook` | +| `GET` | `/api/books/:id/audiobook/file` | +| `POST` | `/api/books/:id/audiobook/reveal` | +| `GET` | `/api/books/:id/chapters/:num/audio` | +| `GET` | `/api/books/:id/chapters/:num/audio/status` | + +### `server/routes/audiobook.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/audiobook/install` | +| `GET` | `/api/audiobook/status` | +| `GET` | `/api/audiobook/voices` | +| `GET` | `/api/audiobook/voices/:voiceId/preview` | + +### `server/routes/authoring.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/books/:id/brief` | +| `PUT` | `/api/books/:id/brief` | +| `PUT` | `/api/books/:id/chapters/:num/content` | +| `GET` | `/api/books/:id/feedback` | +| `PATCH` | `/api/books/:id/meta` | +| `PUT` | `/api/books/:id/quiz/:num` | +| `GET` | `/api/books/:id/references` | +| `GET` | `/api/books/:id/references/:name` | +| `PUT` | `/api/books/:id/references/:name` | +| `GET` | `/api/books/:id/summaries` | +| `PUT` | `/api/books/:id/summaries/:num` | +| `PUT` | `/api/books/:id/toc` | +| `POST` | `/api/books/create-skeleton` | + +### `server/routes/chat.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/chat` | + +### `server/routes/covers.ts` + +| Method | Path | +|---|---| +| `DELETE` | `/api/books/:id/cover` | +| `GET` | `/api/books/:id/cover` | +| `POST` | `/api/books/:id/cover/generate` | +| `POST` | `/api/books/:id/cover/suggest-prompt` | +| `POST` | `/api/books/:id/cover/upload` | + +### `server/routes/epub.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/books/:id/export-epub` | +| `POST` | `/api/books/:id/export-epub` | + +### `server/routes/generation.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/books` | +| `POST` | `/api/books/:id/chapters/:num/regenerate` | +| `POST` | `/api/books/:id/generate-all` | +| `POST` | `/api/books/:id/generate-next` | +| `GET` | `/api/books/:id/generation-status` | +| `GET` | `/api/books/:id/generation-stream` | +| `POST` | `/api/books/:id/start` | +| `POST` | `/api/books/:id/toc/revise` | + +### `server/routes/import.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/books/import/confirm` | +| `POST` | `/api/books/import/preview` | + +### `server/routes/library.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/books` | +| `DELETE` | `/api/books/:id` | +| `GET` | `/api/books/:id` | +| `PATCH` | `/api/books/:id` | +| `PUT` | `/api/books/:id/rating` | +| `POST` | `/api/books/:id/reset` | +| `GET` | `/api/books/:id/toc` | +| `GET` | `/api/books/search` | +| `GET` | `/api/progress/skills` | + +### `server/routes/models.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/providers/:provider/models` | + +### `server/routes/profile.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/profile` | +| `PUT` | `/api/profile` | +| `POST` | `/api/profile/interview` | +| `POST` | `/api/profile/suggest-skills` | + +### `server/routes/reading.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/books/:id/chapters/:num` | +| `PUT` | `/api/books/:id/progress/:num` | + +### `server/routes/settings.ts` + +| Method | Path | +|---|---| +| `DELETE` | `/api/settings/api-key` | +| `POST` | `/api/settings/api-key` | +| `GET` | `/api/settings/api-key-status` | + +### `server/routes/suggestions.ts` + +| Method | Path | +|---|---| +| `POST` | `/api/books/:id/profile-suggestions` | +| `POST` | `/api/books/suggest` | +| `POST` | `/api/books/suggest-details` | + +### `server/routes/tasks.ts` + +| Method | Path | +|---|---| +| `GET` | `/api/tasks` | +| `DELETE` | `/api/tasks/:taskId` | +| `GET` | `/api/tasks/stream` | diff --git a/package.json b/package.json index 63af194..c8f6a46 100644 --- a/package.json +++ b/package.json @@ -16,13 +16,14 @@ "test": "vitest run", "test:watch": "vitest", "e2e": "playwright test", - "lint": "eslint --max-warnings 0 client/ server/ electron/ shared/ e2e/ playwright.config.ts scripts/build-e2e-epub-fixture.ts", - "lint:fix": "eslint --fix --max-warnings 0 client/ server/ electron/ shared/ e2e/ playwright.config.ts scripts/build-e2e-epub-fixture.ts", + "lint": "eslint --max-warnings 0 client/ server/ electron/ shared/ e2e/ playwright.config.ts scripts/build-e2e-epub-fixture.ts scripts/generate-routes-doc.ts", + "lint:fix": "eslint --fix --max-warnings 0 client/ server/ electron/ shared/ e2e/ playwright.config.ts scripts/build-e2e-epub-fixture.ts scripts/generate-routes-doc.ts", "typecheck": "tsc --noEmit", "release": "scripts/release.sh", "mcp:dev": "tsx server/mcp-server.ts", "diagnose:quiz": "tsx scripts/diagnose-quiz.ts", - "e2e:fixture:epub": "tsx scripts/build-e2e-epub-fixture.ts" + "e2e:fixture:epub": "tsx scripts/build-e2e-epub-fixture.ts", + "docs:routes": "tsx scripts/generate-routes-doc.ts" }, "license": "GPL-3.0", "packageManager": "pnpm@10.30.1", diff --git a/scripts/generate-routes-doc.ts b/scripts/generate-routes-doc.ts new file mode 100644 index 0000000..393c7b2 --- /dev/null +++ b/scripts/generate-routes-doc.ts @@ -0,0 +1,186 @@ +/** + * Generates `docs/api-routes.md` from the Fastify route registry, so the API + * reference is a projection of the code rather than a hand-maintained table + * that silently rots. `pnpm docs:routes` runs it and CI fails when the + * committed file differs from a fresh run. + * + * It boots `buildServer()`, never `startServer()`. That distinction is the + * whole safety story. `startServer` runs library migration, crash recovery + * and interrupted-job resume, all of which write to the reader's on-disk + * library, and none of which a docs generator has any business triggering. + * `buildServer` registers routes and stops. As a second, independent guard + * this script points `TUTOR_DATA_DIR` at a throwaway temp directory before + * importing any server module, then asserts that directory is still empty, + * so an adapter that starts writing eagerly fails the generator instead of + * quietly touching real books. + * + * The route list comes from `printRoutes`, which walks Fastify's own radix + * tree, rather than from an `onRoute` hook. A hook only fires for routes + * registered after it is added, and by the time `buildServer` returns every + * plugin has already booted, so a hook added here sees nothing. Re- + * registering the plugins one at a time to catch them would mean copying + * `buildServer`'s registration list into this file, which is exactly the + * kind of hand-maintained duplicate the generator exists to eliminate. + * Every path this script parses out of the tree is checked back against + * `app.hasRoute`, so a parsing mistake fails loudly rather than publishing + * a plausible wrong path. + * + * Source-file attribution is a literal search of `server/routes/*.ts` for + * the path string, because Fastify keeps no record of which module + * registered a route. Any path that does not resolve to exactly one module + * throws, which is what stops the attribution column from drifting into + * guesswork. + */ +import { mkdtempSync, readdirSync, readFileSync, rmSync } from 'node:fs' +import { writeFile } from 'node:fs/promises' +import { tmpdir } from 'node:os' +import { basename, join } from 'node:path' +import { fileURLToPath } from 'node:url' +import { glob } from 'node:fs/promises' + +const repoRoot = fileURLToPath(new URL('..', import.meta.url)) + +// Set before the first server import: composition-root.ts resolves the data +// directory while constructing adapters, which happens on import. +const scratchDataDir = mkdtempSync(join(tmpdir(), 'tutor-routes-doc-')) +process.env.TUTOR_DATA_DIR = scratchDataDir + +const { buildServer } = await import('../server/index.js') + +interface RouteRow { + method: string + path: string + sourceFile: string +} + +/** + * Turns `printRoutes` output back into absolute paths. + * + * The tree is a radix tree, so a node holds only the segment that its + * parent did not already cover and a full path is the concatenation of the + * segments down to it. Indentation is a fixed four columns per level, and + * a node's own methods, when it has any, trail in parentheses. Nodes with + * no methods are pure branch points and contribute a segment but no route. + */ +function parseRouteTree(tree: string): Array<{ method: string; path: string }> { + const segments: string[] = [] + const parsed: Array<{ method: string; path: string }> = [] + + for (const line of tree.split('\n')) { + const markerIndex = Math.max(line.indexOf('├──'), line.indexOf('└──')) + if (markerIndex < 0) continue + + const depth = markerIndex / 4 + const label = line.slice(markerIndex + 4) + const methodMatch = /^(.*) \(([A-Z, ]+)\)$/.exec(label) + const segment = methodMatch ? methodMatch[1] : label + + segments.length = depth + segments[depth] = segment + if (!methodMatch) continue + + const path = segments.join('') + for (const method of methodMatch[2].split(', ')) { + // Fastify pairs a HEAD route with every GET via exposeHeadRoutes. That + // is a transport detail, not part of the surface anyone writes a + // client against. + if (method === 'HEAD') continue + parsed.push({ method, path }) + } + } + + return parsed +} + +const routeSources = new Map() +for await (const file of glob(join(repoRoot, 'server', 'routes', '*.ts'))) { + if (basename(file).includes('.test.')) continue + routeSources.set(`server/routes/${basename(file)}`, readFileSync(file, 'utf8')) +} +// The health route is registered on the root instance in buildServer rather +// than in a route plugin, so index.ts has to be searched as well. +routeSources.set('server/index.ts', readFileSync(join(repoRoot, 'server', 'index.ts'), 'utf8')) + +/** + * Method as well as path, because they are not enough on their own: `GET + * /api/books` is registered in `library.ts` and `POST /api/books` in + * `generation.ts`, so a path-only search matches both. + */ +function attribute(method: string, path: string): string { + const escaped = path.replace(/[.*+?^${}()|[\]\\]/g, '\\$&') + // Tolerates the type parameters and the newline that a multi-line + // registration puts between the call and its path argument. + const registration = new RegExp(`\\.${method.toLowerCase()}\\b[^(]*\\(\\s*['"\`]${escaped}['"\`]`) + const owners = [...routeSources.entries()] + .filter(([, source]) => registration.test(source)) + .map(([file]) => file) + if (owners.length === 1) return owners[0] + throw new Error( + `Could not attribute ${method} ${path} to a single route module (matched ${owners.length}: ${owners.join(', ') || 'none'}). ` + + 'A route registered from a computed path cannot be found by literal search. ' + + 'Give it a literal path, or teach this script how to find it.', + ) +} + +// Every failure below is reported as a plain message rather than a thrown +// stack. kokoro-js pulls in an espeak WASM bundle that installs a +// process-level unhandledRejection handler which rethrows, so an +// unhandled rejection here surfaces from inside a single-line minified +// bundle and Node prints that entire line, burying the real message in a +// megabyte of WASM glue. +try { +const app = await buildServer() +await app.ready() +const rows: RouteRow[] = parseRouteTree(app.printRoutes({ commonPrefix: false })).map(({ method, path }) => { + // Guards the tree parser: a mis-assembled path fails here rather than + // shipping into the doc as a plausible-looking lie. + if (!app.hasRoute({ method: method as 'GET', url: path })) { + throw new Error(`Parsed ${method} ${path} out of printRoutes, but Fastify has no such route. The tree parser is wrong.`) + } + return { method, path, sourceFile: attribute(method, path) } +}) +await app.close() + +const leftovers = readdirSync(scratchDataDir) +rmSync(scratchDataDir, { recursive: true, force: true }) +if (leftovers.length > 0) { + throw new Error( + `buildServer() wrote to the data directory: ${leftovers.join(', ')}. ` + + 'Generating the routes doc must not touch the library. Find the eager write and make it lazy.', + ) +} + +rows.sort((a, b) => a.path.localeCompare(b.path) || a.method.localeCompare(b.method)) + +const uniquePaths = new Set(rows.map((row) => row.path)) +const byFile = new Map() +for (const row of rows) { + const group = byFile.get(row.sourceFile) + if (group) group.push(row) + else byFile.set(row.sourceFile, [row]) +} + +const sections = [...byFile.entries()] + .sort(([a], [b]) => a.localeCompare(b)) + .map(([sourceFile, group]) => { + const body = group.map((row) => `| \`${row.method}\` | \`${row.path}\` |`).join('\n') + return `### \`${sourceFile}\`\n\n| Method | Path |\n|---|---|\n${body}\n` + }) + +const doc = `# API Routes + + + +${rows.length} routes over ${uniquePaths.size} paths, registered by ${byFile.size} modules under [\`server/routes/\`](../server/README.md). The embedded Fastify server serves all of them on \`127.0.0.1\`, on port 3147 when run standalone with \`pnpm dev:server\` and on a free port chosen at launch under Electron. + +Routes are grouped by the module that registers them. The \`HEAD\` route Fastify pairs with each \`GET\` is omitted. Request and response shapes live in [\`shared/contracts.ts\`](../shared/contracts.ts) and [\`shared/responses.ts\`](../shared/responses.ts), and the streaming routes emit the SSE unions in [\`shared/events.ts\`](../shared/events.ts). + +${sections.join('\n')}` + +await writeFile(join(repoRoot, 'docs', 'api-routes.md'), doc, 'utf8') +console.log(`Wrote docs/api-routes.md: ${rows.length} routes, ${uniquePaths.size} paths, ${byFile.size} modules`) +} catch (error) { + rmSync(scratchDataDir, { recursive: true, force: true }) + console.error(error instanceof Error ? error.message : String(error)) + process.exit(1) +} diff --git a/tsconfig.json b/tsconfig.json index da186b1..18a718f 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -21,6 +21,6 @@ "@shared/*": ["./shared/*"] } }, - "include": ["shared/**/*.ts", "server/**/*.ts", "client/**/*.ts", "client/**/*.tsx", "electron/**/*.ts", "e2e/**/*.ts", "playwright.config.ts", "scripts/build-e2e-epub-fixture.ts"], + "include": ["shared/**/*.ts", "server/**/*.ts", "client/**/*.ts", "client/**/*.tsx", "electron/**/*.ts", "e2e/**/*.ts", "playwright.config.ts", "scripts/build-e2e-epub-fixture.ts", "scripts/generate-routes-doc.ts"], "exclude": ["node_modules", "dist", "dist-electron"] } From 0077c1d729e78f6d6ba6eb84d869c406a26a145f Mon Sep 17 00:00:00 2001 From: Ross Miller Date: Tue, 21 Jul 2026 10:53:51 -0500 Subject: [PATCH 03/10] docs: add the architecture hub, the domain glossary, and the routes drift gate ARCHITECTURE.md is the entry point for a reader who has never seen the repo. Five diagrams carry it, one per question a cold reader asks, and the prose between them only says what a diagram cannot. Every folder README links back to it. CONTEXT.md records the ubiquitous language and, more usefully, the words this domain overloads. Status is three unrelated enums, review is two unrelated features, and job and task stopped being synonyms when the job journal landed. CI and lefthook now regenerate docs/api-routes.md and fail on a diff, so the generated doc cannot drift from the routes it describes. The lefthook copy is scoped to server changes and runs in under a second. Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5 --- .github/workflows/ci.yml | 13 ++++ .mcp.json | 11 +++ ARCHITECTURE.md | 146 +++++++++++++++++++++++++++++++++++++++ CONTEXT.md | 67 ++++++++++++++++++ lefthook.yml | 12 ++++ 5 files changed, 249 insertions(+) create mode 100644 .mcp.json create mode 100644 ARCHITECTURE.md create mode 100644 CONTEXT.md diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index bd28061..055c116 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -41,6 +41,19 @@ jobs: - run: pnpm test + # docs/api-routes.md is generated from the Fastify route registry, so it + # can only be wrong if someone changed a route and did not regenerate it. + # This step is what makes that impossible to merge. It runs last because + # it boots the server, and a failure here is a stale file rather than + # broken code. + - name: Check the generated API routes doc is current + run: | + pnpm docs:routes + git diff --exit-code docs/api-routes.md || { + echo "::error file=docs/api-routes.md::docs/api-routes.md is stale. Run 'pnpm docs:routes' and commit the result." + exit 1 + } + # The end-to-end journey suite, in its own job so it runs alongside verify # rather than behind it. `pnpm test` stays the fast one and never waits on a # browser. See e2e/README.md for what the journeys cover and why the server diff --git a/.mcp.json b/.mcp.json new file mode 100644 index 0000000..1d4eac6 --- /dev/null +++ b/.mcp.json @@ -0,0 +1,11 @@ +{ + "mcpServers": { + "tutor": { + "command": "pnpm", + "args": ["mcp:dev"], + "env": { + "TUTOR_API_URL": "http://127.0.0.1:3147" + } + } + } +} diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..c915720 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,146 @@ +# Architecture + +Tutor is one Electron desktop app for one reader on one machine. It generates a book chapter by chapter, and each chapter is shaped by the feedback and quiz results of the one before it. Everything below follows from that, and from there being exactly one writer and no cloud. + +Start here, then follow the links. Domain words are defined in [CONTEXT.md](CONTEXT.md), decisions and their costs in [docs/adr/](docs/adr/README.md), and the HTTP surface in [docs/api-routes.md](docs/api-routes.md), which is generated from the route registry rather than written by hand. + +## 1. What talks to what + +```mermaid +flowchart LR + reader([Reader]) + subgraph app["Tutor.app"] + electron["electron/
main and preload"] + client["client/
React renderer"] + server["server/
embedded Fastify"] + end + library[("On-disk library
Markdown and YAML")] + providers["AI providers
Anthropic, OpenAI, Google"] + local["Kokoro TTS and ffmpeg
on this machine"] + + reader --> client + electron --> client + electron --> server + client -->|"HTTP and SSE on 127.0.0.1"| server + server --> library + server -->|"the reader's own API key"| providers + server --> local +``` + +The Fastify server is embedded in the Electron main process rather than deployed anywhere. It binds `127.0.0.1` on a free port at launch, or 3147 when run standalone with `pnpm dev:server`. The library is plain Markdown and YAML under the OS data directory, which is [ADR 0001](docs/adr/0001-filesystem-as-the-database.md). Narration is synthesized locally instead of by a metered cloud service, which is [ADR 0003](docs/adr/0003-local-kokoro-tts.md). + +## 2. The server hexagon + +```mermaid +flowchart LR + subgraph core["server core"] + direction TB + routes["routes/
parse and delegate"] + services["services/
application logic"] + domain["domain/
pure rules"] + routes --> services + services --> domain + end + + subgraph ports["ports/ — 15 interfaces"] + direction TB + pStore["BookRepository
ArtifactStore
LibraryMigrator
JobJournal
KeyVault"] + pAi["TextGeneration
ImageGeneration"] + pMedia["SpeechSynthesis
AudioAssembly
DiagramRenderer
EpubImport
EpubExport"] + pSys["BackgroundTasks
Clock
OsFileManager"] + end + + subgraph adapters["adapters/ — the only I/O"] + direction TB + aStore["fs-*.ts
file-key-vault.ts"] + aAi["ai-sdk-text-generation.ts
http-image-generation.ts"] + aMedia["kokoro-speech-synthesis.ts
ffmpeg-audio-assembly.ts
kroki- and electron-diagram-renderer.ts
epub2-import.ts, epub-gen-export.ts"] + aSys["in-memory- and journalled-background-tasks.ts
system-clock.ts, os-file-manager.ts"] + end + + services --> pStore + services --> pAi + services --> pMedia + services --> pSys + pStore --> aStore + pAi --> aAi + pMedia --> aMedia + pSys --> aSys +``` + +Nothing in the core names an adapter. [`server/composition-root.ts`](server/composition-root.ts) is the one place a real adapter is chosen, and `buildServer(overrides)` lets a test or the Electron shell substitute one. Every port ships an in-memory fake and a contract test that the fake and every adapter must both pass, which is what stops a fake from drifting into a convenient fiction. See [`server/ports/README.md`](server/ports/README.md) and [`server/adapters/README.md`](server/adapters/README.md) for the full mapping, and [ADR 0005](docs/adr/0005-ai-sdk-behind-a-port.md) for why the AI SDK sits behind one. + +## 3. How a request travels + +```mermaid +flowchart LR + components["features/
components"] --> hooks["features/
hooks"] + hooks --> store["store/
Redux slices"] + hooks --> api["api/
the one HTTP client"] + api -->|"HTTP and SSE"| routes["routes/"] + routes --> services["services/"] + services --> ports["ports/"] + ports --> adapters["adapters/"] +``` + +Components render and hooks decide. Every call to the server goes through [`client/api/`](client/README.md), and a raw `fetch` or `new EventSource` anywhere else is an ESLint error rather than a convention, because the client previously held eighty four scattered fetch calls and two competing reconnect policies. + +## 4. The adaptive loop + +```mermaid +sequenceDiagram + actor Reader + participant Client as client/ + participant Server as server/ + participant AI as TextGeneration + participant Disk as library + + Reader->>Client: topic and prompt + Client->>Server: POST /api/books + Server->>AI: draft the table of contents + Server->>Disk: meta.yml, toc.yml + Server-->>Client: SSE, status toc_review + Reader->>Client: approve the TOC + Client->>Server: PUT /api/books/:id/toc + Server->>AI: generate chapter 1 + Server-->>Client: SSE chunks as they stream + Server->>Disk: chapters/01.md + Reader->>Client: read, then submit feedback + Client->>Server: POST /api/books/:id/chapters/1/feedback + Server->>Disk: feedback/01.yml + Server->>AI: generate chapter 2 in the background + Reader->>Client: answer the quiz while it generates + Note over Server,AI: chapter 2 is shaped by chapter 1's feedback and quiz result +``` + +Chapters are generated one at a time rather than up front, and the quiz exists partly to cover the generation latency. That is [ADR 0002](docs/adr/0002-just-in-time-chapter-generation.md). If the app is closed mid-generation the work is not lost, because jobs are journalled to disk and resumed at the next boot, which is [ADR 0008](docs/adr/0008-persisted-job-journal.md). + +## 5. The dependency rule + +```mermaid +flowchart TD + client["client/"] --> shared["shared/"] + server["server/"] --> shared + electron["electron/"] --> shared + client -. "ESLint error" .-> server + shared -. "ESLint error" .-> client +``` + +`shared/` is the dependency root and imports neither side. It holds the Zod schemas, the status predicates, the HTTP contract types, and the SSE event unions, so the two halves of the app validate against the same definitions. This is one package shaped like a monorepo rather than real workspaces, which is [ADR 0004](docs/adr/0004-single-package-monorepo-shaped.md), and the folders are already package-shaped if that ever needs to change. + +## Deliberately out of scope + +Observability and telemetry, a security-hardening pass, and release engineering were all considered and declined. The app runs locally on one machine, holds one reader's data, has no cloud component, and has no multi-user surface, so each of those would add machinery with nothing to protect or measure. The reasoning is recorded in [ADR 0004](docs/adr/0004-single-package-monorepo-shaped.md) rather than left as an unexplained gap. + +## Where to read next + +| Area | Start at | +|---|---| +| Server, routes, services, ports, adapters | [`server/README.md`](server/README.md) | +| React renderer and feature slices | [`client/README.md`](client/README.md) | +| Types both sides depend on | [`shared/README.md`](shared/README.md) | +| Electron shell and packaging | [`electron/README.md`](electron/README.md) | +| End-to-end journeys | [`e2e/README.md`](e2e/README.md) | +| Every decision and what it cost | [`docs/adr/`](docs/adr/README.md) | +| Domain vocabulary | [`CONTEXT.md`](CONTEXT.md) | +| Generated HTTP surface | [`docs/api-routes.md`](docs/api-routes.md) | diff --git a/CONTEXT.md b/CONTEXT.md new file mode 100644 index 0000000..9d9fac6 --- /dev/null +++ b/CONTEXT.md @@ -0,0 +1,67 @@ +Up: [ARCHITECTURE.md](ARCHITECTURE.md) + +# Domain language + +The words this codebase uses, and the type or module that owns each one. Names here are the ubiquitous language. They appear unchanged in schemas, services, components, prompts, and UI copy, and a synonym for one of them is a bug in the writing rather than a stylistic choice. + +Several words in this domain are overloaded, some of them badly. The later sections exist because those collisions are real and permanent, so the fix is to qualify them consistently rather than to rename around them. + +## Core domain + +| Term | Meaning | Owned by | +|---|---|---| +| **Book** | A generated learning artifact. One book is one directory on disk and one `BookMeta` record. | `BookMetaSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Chapter** | One roughly 1,500-word Markdown unit, addressed by a 1-based `chapterNum`, stored as `chapters/NN.md`. | [`server/adapters/fs-book-repository.ts`](server/adapters/fs-book-repository.ts) | +| **TOC** | The chapter plan the reader reviews and revises before any chapter is generated. | `TocSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Feedback** | The reader's response to a chapter just read. Submitting it is what triggers the next chapter. | `FeedbackSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Quiz** | Three questions after a chapter, or a longer final quiz across the whole book. | `QuizSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Progress** | Per-chapter scroll fraction plus a completed flag, where completed means at least 90 percent. | `ProgressSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **LearningProfile** | The global reader model, meaning identity, `aboutMe`, `Preferences`, and `Skill[]`, with optional per-book `profileOverrides`. | `LearningProfileSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Audiobook** | An M4B with chapter markers, synthesized locally from a book's chapters. | `AudiobookManifest` in [`shared/domain.ts`](shared/domain.ts) | +| **Series** | A named ordering of related books. It is two fields on `BookMeta`, `series` and `seriesOrder`, and has no entity of its own. | `BookMetaSchema` in [`shared/domain.ts`](shared/domain.ts) | +| **Interview** | The conversational flow that fills in a LearningProfile by asking the reader questions. | [`server/services/interview-profile.ts`](server/services/interview-profile.ts) | +| **Background task** | A cancellable long-running server operation with progress streamed over SSE. | `BackgroundTasks` port in [`server/ports/background-tasks.ts`](server/ports/background-tasks.ts) | + +A background task is one of five `TaskType` values, `generate-all`, `generate-epub`, `generate-cover`, `install-audiobook`, and `generate-audiobook`. The wire shape the client receives is `ClientTask` in [`shared/responses.ts`](shared/responses.ts). + +## The Skill collision + +**Skill** in the domain is a named competency with a 1 to 10 level on the LearningProfile, plus its per-chapter `subskill` weightings on the TOC. **Agent Skill** is a `SKILL.md` under [`.claude/skills/`](.claude/skills) that instructs Claude Code. + +Both keep their names. The domain type is never renamed to avoid the collision, and prose always qualifies which one it means by writing "learner Skill" or "Agent Skill". + +## Overloaded terms + +**Status** is three unrelated enums and they are never interchangeable. + +| Enum | Values | Lives in | +|---|---|---| +| `BookStatus` | `generating_toc`, `toc_review`, `generating`, `reading`, `complete`, `failed` | [`shared/book-status.ts`](shared/book-status.ts) | +| `TaskStatus` | `running`, `done`, `error`, `cancelled` | [`shared/responses.ts`](shared/responses.ts) | +| `GenerationStage` | `streaming`, `saving`, `quiz`, `done`, `error` | [`shared/responses.ts`](shared/responses.ts) | + +`BOOK_STATUSES` is the single source of truth for the first of those, and the six predicates beside it (`isGenerating`, `isGeneratingToc`, `isAwaitingTocApproval`, `isReadable`, `isComplete`, `isFailed`) are the only sanctioned way to ask about a book's status. The literals themselves appear nowhere else. + +**Job** and **task** are not synonyms. A background task is the live, in-memory unit the tray shows. A job is its persisted record on disk, written by the `JobJournal` port so an interrupted task can be resumed after a restart. The schemas are `GenerationJobSchema` and friends in [`shared/domain.ts`](shared/domain.ts), and `GenerationJobType` covers every `TaskType` plus `generate-chapter`, which is the just-in-time single chapter path that never went through background tasks. Use "job" only when you mean the durable record. + +**Section** is an intra-chapter slice used for pagination and reader navigation, in `client/lib/split-sections.ts` and `client/features/reader/hooks/useSectionNavigation.ts`. It is never a synonym for Chapter. + +**Review** is two unrelated things. `toc_review` is the BookStatus in which a generated TOC is waiting for the reader to approve it. Smart Review is the spaced-repetition flow that re-quizzes a reader on questions they missed, in `client/features/quiz/components/SmartReviewFlow.tsx`. + +**Prompt** and **brief** are different lengths of the same idea. A `prompt` is the reader's short topic request stored on `BookMeta`. A `brief` is the long agentic-generation specification stored per book and consumed by the MCP authoring services. + +**Summary** is either a `ChapterSummary`, which is cross-chapter context fed into generation, or the book-completion summary the reader sees at the end. Qualify which. + +**Reference** is source material saved for agentic generation, as `ReferenceEntry` in [`shared/domain.ts`](shared/domain.ts). It is not a citation. + +**Narration** is chapter Markdown transformed for speech, in [`server/services/markdown-to-narration.ts`](server/services/markdown-to-narration.ts). It is a derived form of the chapter text and not the chapter text itself. + +**Progress** appears three ways, as reading scroll progress, as `TaskProgress` on a background task, and as skill progress on the profile. Always qualify it. + +**Error kind** is `AiErrorKind` in [`shared/responses.ts`](shared/responses.ts), one of `auth-failed`, `rate-limited`, `overloaded`, `timed-out`, `network-failed`, `content-refused`, or `unknown`. It classifies why an AI call failed, which is what decides whether the adapter retries. It is not a status. + +## Numbering + +`chapterNum` is 1-based and canonical, and it is what appears in file names, routes, and every persisted record. `chapterIndex` is 0-based, array-local, and never crosses a module boundary. A function that takes one and returns the other is doing a conversion and should say so in its name. + +Related: [ARCHITECTURE.md](ARCHITECTURE.md), [docs/adr/](docs/adr/README.md), [shared/README.md](shared/README.md), [docs/api-routes.md](docs/api-routes.md) diff --git a/lefthook.yml b/lefthook.yml index f9d0a7c..002b941 100644 --- a/lefthook.yml +++ b/lefthook.yml @@ -8,6 +8,18 @@ pre-commit: run: pnpm exec eslint --max-warnings 0 {staged_files} typecheck: run: pnpm typecheck + # Runs on every commit rather than only in CI because it takes under a + # second and because the failure it catches, a route change committed + # without regenerating docs/api-routes.md, is cheapest to fix in the + # commit that caused it. + docs-routes: + glob: "server/**/*.ts" + run: | + pnpm docs:routes + git diff --exit-code docs/api-routes.md || { + echo "docs/api-routes.md is stale. It has been regenerated for you. Run 'git add docs/api-routes.md' and commit again." + exit 1 + } pre-push: commands: From 1296eb485cdedff9cc2b32dae514db2c844fca06 Mon Sep 17 00:00:00 2001 From: Ross Miller Date: Tue, 21 Jul 2026 10:58:11 -0500 Subject: [PATCH 04/10] docs(adr): record the eight decisions this architecture rests on Each ADR was written from code that exists, with a citable symbol or file, and any draft claim that could not be found in the tree was dropped rather than smoothed over. Two drafts were wrong and are corrected here. Chapter generation is no longer lost on restart, so 0002 describes the journal and resume path instead of the in-memory one, and prompt text lives in typed template literals inside each service rather than in runtime markdown files, so 0005 says that instead. Also adds the two Agent Skills the repo needs to be worked on by an agent, verify for the green-before-commit check and add-feature for the test-first port to adapter to service to route to client recipe, plus .mcp.json for the built-in MCP server. Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5 --- .claude/skills/add-feature/SKILL.md | 49 ++++++++++++++++ .claude/skills/verify/SKILL.md | 57 +++++++++++++++++++ ARCHITECTURE.md | 6 +- docs/adr/0001-filesystem-as-the-database.md | 30 ++++++++++ .../0002-just-in-time-chapter-generation.md | 29 ++++++++++ docs/adr/0003-local-kokoro-tts.md | 27 +++++++++ .../0004-single-package-monorepo-shaped.md | 28 +++++++++ docs/adr/0005-ai-sdk-behind-a-port.md | 29 ++++++++++ .../0006-electron-packaging-constraints.md | 28 +++++++++ ...ed-library-with-forward-only-migrations.md | 28 +++++++++ docs/adr/0008-persisted-job-journal.md | 28 +++++++++ docs/adr/README.md | 16 ++++++ 12 files changed, 352 insertions(+), 3 deletions(-) create mode 100644 .claude/skills/add-feature/SKILL.md create mode 100644 .claude/skills/verify/SKILL.md create mode 100644 docs/adr/0001-filesystem-as-the-database.md create mode 100644 docs/adr/0002-just-in-time-chapter-generation.md create mode 100644 docs/adr/0003-local-kokoro-tts.md create mode 100644 docs/adr/0004-single-package-monorepo-shaped.md create mode 100644 docs/adr/0005-ai-sdk-behind-a-port.md create mode 100644 docs/adr/0006-electron-packaging-constraints.md create mode 100644 docs/adr/0007-versioned-library-with-forward-only-migrations.md create mode 100644 docs/adr/0008-persisted-job-journal.md create mode 100644 docs/adr/README.md diff --git a/.claude/skills/add-feature/SKILL.md b/.claude/skills/add-feature/SKILL.md new file mode 100644 index 0000000..dbb6d2d --- /dev/null +++ b/.claude/skills/add-feature/SKILL.md @@ -0,0 +1,49 @@ +--- +name: add-feature +description: Add a capability to Tutor end to end, following the test-first port, adapter, service, route, api client, feature slice recipe. +--- + +# Add a Feature to Tutor + +The recipe for adding a capability that needs new server behavior, from domain types down to the UI. Test commits precede or accompany implementation commits at every step. That ordering is deliberate and should stay visible in git history. + +## When to use, when not + +Use this when the feature reaches new server behavior or a new external dependency. Skip it for a pure UI tweak with no new I/O. That is just a component edit, no port, service, or route involved. + +## Decide whether a new port is needed + +Only add a port when the feature reaches a genuinely new external dependency, meaning a new service, a new binary, or a new file format. Reusing an existing port is the common case. There are 15 today (`text-generation`, `key-vault`, `image-generation`, `book-repository`, `artifact-store`, `speech-synthesis`, `audio-assembly`, `diagram-renderer`, `epub-import`, `epub-export`, `background-tasks`, `job-journal`, `library-migrator`, `clock`, `os-file-manager`). Check `server/ports/README.md` for the current list before assuming you need a new one. + +## Steps, in TDD order + +1. Define or extend the domain types in `shared/`. Persisted entities are Zod schemas in `shared/domain.ts`. Request bodies go in `shared/contracts.ts`. Response shapes go in `shared/responses.ts`. SSE event unions go in `shared/events.ts`. `shared/` is flat, there is no subfolder per feature. + +2. Declare the port interface in `server/ports/.ts`, named for the capability and never for the vendor. + +3. Write the contract test `server/ports/.contract.ts` and the in-memory fake `server/ports/.fake.ts`, then `server/ports/.fake.test.ts`, which runs the contract against the fake. This is RED first, there is no adapter yet. + +4. Write the adapter `server/adapters/-.ts` and run the same contract against it. That turns the contract GREEN and is what stops the fake drifting from the real thing. `fs-book-repository.ts`, `kokoro-speech-synthesis.ts`, and `system-clock.ts` are worked examples of this naming. When the technology name already implies the capability's own prefix, the capability half gets shortened instead of duplicated, as in `epub2-import.ts` for the `epub-import` port. A couple of adapters, such as `os-file-manager.ts`, carry no technology prefix at all because there is only one plausible implementation. Match the port's name unless one of those two cases applies. + +5. Write the service unit test `server/services/.test.ts` against the fake. RED. + +6. Write the service `server/services/.ts` as a `createX(deps)` factory, where `deps` is an object of ports by name, for example `{ ai, books, clock }`. GREEN. See `server/services/create-book.ts` for the shape. + +7. Add the thin route in the matching `server/routes/*.ts` module (`library.ts`, `reading.ts`, `assessment.ts`, `authoring.ts`, and others by domain area, not one file per entity). Parse the body with the Zod schema from `shared/contracts.ts` through `parseBody()` in `server/http/parse.ts`, then delegate to the service. Register any new port and service through `server/composition-root.ts`, adding a field to the `Ports` interface and wiring it in `createPorts()`. Then run `pnpm docs:routes` and commit the regenerated `docs/api-routes.md`. + +8. Add one function to the matching `client/api/*.ts` module with its mocked-fetch test, then the feature hook and component under `client/features//` (see `audiobook`, `chat`, `creation`, `library`, `markdown`, `profile`, `progress`, `quiz`, `reader`, `settings` for the existing slices, each with its own `components/` and `hooks/`). + +## Conventions checklist + +- No raw `fetch` or `new EventSource` outside `client/api/`. Both are lint errors, enforced in `eslint.config.mjs`, not just conventions. +- No SDK or vendor import outside `server/adapters/`. +- No `fs` in `server/services/` or any domain module. +- No magic strings where a named constant already exists. +- JSDoc on new exported symbols, stating a constraint the signature cannot show on its own. +- Tests colocated beside the file they cover, as `*.test.ts`. +- Test commits before implementation commits. +- Domain names taken from `CONTEXT.md`, never invented fresh. + +## Finish + +Run the `verify` skill before calling the feature done. diff --git a/.claude/skills/verify/SKILL.md b/.claude/skills/verify/SKILL.md new file mode 100644 index 0000000..915436d --- /dev/null +++ b/.claude/skills/verify/SKILL.md @@ -0,0 +1,57 @@ +--- +name: verify +description: Verify the Tutor repo is green before a commit, a PR, or a handoff. Runs this project's test, typecheck, lint, generated-docs drift, and server boot checks, then reports a pass or fail table. +--- + +# Verify Tutor + +Runs the checks that gate a commit, a PR, or a handoff to another agent in this repo. This is not a general test runner. It is the specific set of commands this project uses to call a change safe. + +## When to use + +- Before committing. +- Before opening a PR. +- Before handing work to another agent. +- After any change that touches the server routes. + +## Steps + +Run these in order. Each names the exact command and what to expect. + +1. `pnpm test`. Runs the full Vitest suite. Expect every test to pass, with no failures reported. + +2. `pnpm typecheck`. Runs `tsc --noEmit`. Expect no output. + +3. `pnpm lint`. Runs ESLint across the project with `--max-warnings 0`. Expect no output. Zero warnings is the bar, not zero errors. + +4. `pnpm docs:routes && git diff --exit-code docs/api-routes.md`. Regenerates `docs/api-routes.md` from the live route definitions, then fails if the regenerated file differs from what is committed. If it fails, the fix is to commit the regenerated file, not to hand-edit the doc. + +5. Boot check. Starts the real server, hits the health endpoint, then stops the server. Run as separate commands so the server can be killed cleanly: + + ``` + pnpm dev:server & + SERVER_PID=$! + sleep 2 + curl -s http://127.0.0.1:3147/api/health + kill $SERVER_PID + ``` + + Expect the curl output to be `{"status":"ok"}`. This binds port 3147, so if an already-running dev server is holding that port, this step fails with `EADDRINUSE`. That is a real signal to go check for a stray server, not noise to ignore or retry past. + +Optionally, `pnpm e2e` is a heavier Playwright suite that builds the app and drives a real browser. It is not part of the default set above because of that cost, but run it when a change touches end-to-end reader or generation flows. + +## Failure handling + +Fix every failure, then rerun the full set from step 1. Never report the first failure and stop partway. If a failing check is out of scope for the current change, say so explicitly in the report, with the actual output as evidence, rather than silently skipping it. + +## Report format + +Report a markdown table, one row per step, Evidence always a real line copied from the command's actual output, never a paraphrase or a guess. + +| Check | Command | Result | Evidence | +|-------|---------|--------|----------| +| Tests | `pnpm test` | Pass | `Test Files 136 passed (136)` | +| Typecheck | `pnpm typecheck` | Pass | (no output) | +| Lint | `pnpm lint` | Pass | (no output) | +| Docs drift | `pnpm docs:routes && git diff --exit-code docs/api-routes.md` | Pass | (no diff) | +| Boot check | `pnpm dev:server` + `curl .../api/health` | Pass | `{"status":"ok"}` | diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index c915720..2cbbb7a 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -42,7 +42,7 @@ flowchart LR services --> domain end - subgraph ports["ports/ — 15 interfaces"] + subgraph ports["ports/ (15 interfaces)"] direction TB pStore["BookRepository
ArtifactStore
LibraryMigrator
JobJournal
KeyVault"] pAi["TextGeneration
ImageGeneration"] @@ -50,7 +50,7 @@ flowchart LR pSys["BackgroundTasks
Clock
OsFileManager"] end - subgraph adapters["adapters/ — the only I/O"] + subgraph adapters["adapters/ (the only I/O)"] direction TB aStore["fs-*.ts
file-key-vault.ts"] aAi["ai-sdk-text-generation.ts
http-image-generation.ts"] @@ -68,7 +68,7 @@ flowchart LR pSys --> aSys ``` -Nothing in the core names an adapter. [`server/composition-root.ts`](server/composition-root.ts) is the one place a real adapter is chosen, and `buildServer(overrides)` lets a test or the Electron shell substitute one. Every port ships an in-memory fake and a contract test that the fake and every adapter must both pass, which is what stops a fake from drifting into a convenient fiction. See [`server/ports/README.md`](server/ports/README.md) and [`server/adapters/README.md`](server/adapters/README.md) for the full mapping, and [ADR 0005](docs/adr/0005-ai-sdk-behind-a-port.md) for why the AI SDK sits behind one. +Nothing in the core names an adapter. [`server/composition-root.ts`](server/composition-root.ts) is the one place a real adapter is chosen, and `buildServer(overrides)` lets a test or the Electron shell substitute one. Every port ships an in-memory fake and a shared contract test, and every adapter that can be exercised without spending money or downloading a model runs that same contract, which is what stops a fake from drifting into a convenient fiction. See [`server/ports/README.md`](server/ports/README.md) and [`server/adapters/README.md`](server/adapters/README.md) for the full mapping, and [ADR 0005](docs/adr/0005-ai-sdk-behind-a-port.md) for why the AI SDK sits behind one. ## 3. How a request travels diff --git a/docs/adr/0001-filesystem-as-the-database.md b/docs/adr/0001-filesystem-as-the-database.md new file mode 100644 index 0000000..45da1c2 --- /dev/null +++ b/docs/adr/0001-filesystem-as-the-database.md @@ -0,0 +1,30 @@ +# 0001. Filesystem as the database + +Status: Accepted +Date: 2026-07-21 + +## Context + +Tutor is a single-user desktop app. A book's content is Markdown chapters plus YAML metadata, and readers expect to read, diff, back up, and hand-edit their own books. SQLite was the obvious alternative to a plain filesystem layout. + +## Decision + +Each book lives at `books/{id}/` under the OS data directory, holding `meta.yml`, `toc.yml`, `chapters/NN.md`, `progress.yml`, and `feedback/NN.yml`. [`createFsBookRepository`](../../server/adapters/fs-book-repository.ts) is the only adapter that touches these files, and every read is validated against a Zod schema before the caller sees it. Every write goes through [`fs-paths.ts`](../../server/adapters/fs-paths.ts)'s `writeYaml` helper, or an equivalent inline pattern for chapter Markdown, which writes a `.tmp` file first and renames it into place, so a reader never observes a half-written file. [`createRecoverFromCrash`](../../server/services/recover-from-crash.ts) runs once at boot. It sweeps any `.tmp` leftovers from an interrupted write and resets a book's status when the process died mid-generation. + +`meta.yml` and `learning-profile.yml` also carry a schema version stamp that is checked on every read. That stamp was added after this decision. See [ADR 0007](0007-versioned-library-with-forward-only-migrations.md) for the migration path it enables. + +## Consequences + +**What this buys** +- Chapters and metadata stay human-readable and git-friendly, and a book directory is trivially backed up by copying it. +- EPUB import and export become plain file operations against the same layout. + +**What this costs** +- There is no transaction across files, so a multi-file update can partially land if the process dies mid-write. Boot-time recovery narrows this risk, but it does not eliminate it. +- There is no query layer, so listing the library means reading every book's `meta.yml`. +- Invariants live in application code rather than in a schema the filesystem can enforce. +- Crash safety is this app's own responsibility, which is acceptable because exactly one process holds the data directory at a time. + +## Revisit when + +Tutor gains multiple users or any kind of sync, or the library grows past a few hundred books and reading every `meta.yml` on each list call becomes noticeable. diff --git a/docs/adr/0002-just-in-time-chapter-generation.md b/docs/adr/0002-just-in-time-chapter-generation.md new file mode 100644 index 0000000..59bc965 --- /dev/null +++ b/docs/adr/0002-just-in-time-chapter-generation.md @@ -0,0 +1,29 @@ +# 0002. Just-in-time chapter generation, quiz masks the latency + +Status: Accepted +Date: 2026-07-21 + +## Context + +A chapter takes tens of seconds to generate. Pre-generating the whole book up front wastes tokens, because feedback and quiz results from earlier chapters are meant to change how later chapters get written, and it delays the first chapter the reader actually wants. + +## Decision + +TOC approval triggers chapter 1's generation. Chapters then generate one at a time. [`generate-next-chapter.ts`](../../server/services/generate-next-chapter.ts) generates one chapter's text and quiz. Both flows below share it. [`chapter-generation-stream.ts`](../../server/services/chapter-generation-stream.ts) is the in-memory hub behind the single-chapter SSE routes, and it streams chapter text to a subscriber as the model generates it. Submitting feedback through [`submit-feedback.ts`](../../server/services/submit-feedback.ts) scores the reader's quiz answers and fires the next chapter's generation. Meanwhile, [`generate-quiz.ts`](../../server/services/generate-quiz.ts) produces a 3-question quiz over the chapter just read, for the reader to answer while that runs. A separate, opt-in `generate-all` background task in [`generate-all-chapters.ts`](../../server/services/generate-all-chapters.ts) generates every remaining chapter without the per-chapter quiz pause, for a reader who wants it all at once. + +Each single-chapter generation is journalled while it streams and cleared once it settles, so a process that dies mid-stream leaves a record behind. There is no way to resume a partially streamed chapter with any provider, so the next boot seeds that chapter's generation with an explicit interrupted error the reader's error panel already shows. The opt-in `generate-all` task differs. It checkpoints progress and resumes on the next boot by re-reading how many chapters exist on disk. See [ADR 0008](0008-persisted-job-journal.md) for the persisted job journal behind both. + +## Consequences + +**What this buys** +- Perceived latency drops to near zero, because the reader takes a quiz while the next chapter generates instead of watching a spinner. +- Each chapter can adapt to the previous chapter's quiz results and feedback before it gets written. +- Tokens get spent only on chapters the reader actually reaches. + +**What this costs** +- The single-chapter flow needs SSE plumbing on both the server and the client, plus reconnect handling. +- The quiz is now load-bearing UX rather than only a pedagogical add-on, so a reader who skips it loses the latency-masking the design depends on. + +## Revisit when + +Chapter generation drops to a few seconds. At that point, masking the latency with a quiz stops paying for the complexity it costs. diff --git a/docs/adr/0003-local-kokoro-tts.md b/docs/adr/0003-local-kokoro-tts.md new file mode 100644 index 0000000..2a25382 --- /dev/null +++ b/docs/adr/0003-local-kokoro-tts.md @@ -0,0 +1,27 @@ +# 0003. Local Kokoro TTS over cloud TTS + +Status: Accepted +Date: 2026-07-21 + +## Context + +Audiobooks need narration for an entire book, chapter by chapter. Cloud text-to-speech is metered per character and would dominate the cost of narrating a whole book, and bring-your-own-key users have no billing relationship with Tutor to route that cost through. + +## Decision + +Narration runs locally through [`kokoro-speech-synthesis.ts`](../../server/adapters/kokoro-speech-synthesis.ts), which loads the `onnx-community/Kokoro-82M-v1.0-ONNX` model at `q8` quantization using `kokoro-js` and `onnxruntime-node`. [`audiobook-installer.ts`](../../server/services/audiobook-installer.ts) downloads the model and a separate ffmpeg binary on first use, and it reports progress as a background task. Together those downloads total approximately 195 MB, 115 MB for the model and 80 MB for ffmpeg. [`ffmpeg-audio-assembly.ts`](../../server/adapters/ffmpeg-audio-assembly.ts) muxes the per-chapter WAV files into a single M4B with chapter markers once [`generate-audiobook.ts`](../../server/services/generate-audiobook.ts) has narrated every chapter. The voice list orders male voices first, and within Kokoro's American-male group, `am_michael` is pinned ahead of every other voice regardless of its position in Kokoro's own catalogue. + +## Consequences + +**What this buys** +- Narrating a book costs nothing beyond the one-time download, no matter how many books get narrated or how long they run. +- Narration runs fully offline, and no book content ever reaches a third party. + +**What this costs** +- The first use of the audiobook feature pays for a large one-time download of the model and ffmpeg. +- The native ONNX binaries must stay unpacked from the asar archive and external in the Vite build. [ADR 0006](0006-electron-packaging-constraints.md) covers that constraint in full. +- Kokoro's voice quality sits below premium cloud voices. + +## Revisit when + +A cloud voice is clearly better and users ask for it as an opt-in alongside the local engine, not as a replacement for it. diff --git a/docs/adr/0004-single-package-monorepo-shaped.md b/docs/adr/0004-single-package-monorepo-shaped.md new file mode 100644 index 0000000..4aed241 --- /dev/null +++ b/docs/adr/0004-single-package-monorepo-shaped.md @@ -0,0 +1,28 @@ +# 0004. Single package, monorepo-shaped + +Status: Accepted +Date: 2026-07-21 + +## Context + +One `package.json` used to serve the React app, the Fastify server, the Electron shell, and a shared library, and their imports were tangled across all four. Real pnpm workspaces would add hard boundaries, but they would also add resolution complexity on top of an Electron build that already fights pnpm. + +## Decision + +The repo is split into top-level `client/`, `server/`, `shared/`, and `electron/` folders, reached through `@client/*`, `@server/*`, and `@shared/*` aliases declared in both [`tsconfig.json`](../../tsconfig.json) and [`vite.config.ts`](../../vite.config.ts). [`eslint.config.mjs`](../../eslint.config.mjs) enforces the boundary between them with the rule `@typescript-eslint/no-restricted-imports`, which forbids server code from importing client code and client code from importing server code. A second rule, `no-restricted-syntax`, forces every client-to-server call through `client/api/` rather than a raw `fetch` or `EventSource` constructed elsewhere. `shared/` may import neither zone. All of it still runs from one `package.json`, one lockfile, and one `node_modules`, with `node-linker=hoisted` set in `.npmrc`. + +## Consequences + +**What this buys** +- It enforces the same import boundaries a real workspace would give, starting today, without adding the build risk a workspace migration would bring to the Electron packaging that [ADR 0006](0006-electron-packaging-constraints.md) describes. +- It keeps one install and one test command, `pnpm test`, for the whole repo. + +**What this costs** +- Dependencies stay unpartitioned, so nothing but lint and code review stops a client-only library from being imported on the server, or the reverse. +- The four zones cannot version independently. + +**Deliberately out of scope.** Observability and telemetry, a security-hardening pass, and release engineering were all considered and declined, because Tutor runs locally on one machine with no cloud component and no multi-user surface for any of them to protect. + +## Revisit when + +A second application wants to consume `shared/`, or CI time justifies per-package caching. The folders are already package-shaped for that move. Adding `pnpm-workspace.yaml` and a `package.json` per folder is mechanical once the import boundaries above have held for a while, but `node-linker=hoisted` and the `external()` allow-list in `vite.config.ts` need re-verifying across all three Electron modes first. diff --git a/docs/adr/0005-ai-sdk-behind-a-port.md b/docs/adr/0005-ai-sdk-behind-a-port.md new file mode 100644 index 0000000..1697399 --- /dev/null +++ b/docs/adr/0005-ai-sdk-behind-a-port.md @@ -0,0 +1,29 @@ +# 0005. AI SDK behind a TextGeneration port + +Status: Accepted +Date: 2026-07-21 + +## Context + +Calls to the Vercel AI SDK used to sit directly inside route and service modules, five files calling `generateObject` and `streamText` against the SDK itself, per the account in [`../../server/ports/text-generation.ts`](../../server/ports/text-generation.ts). Anthropic, OpenAI, and Google are all user-selectable providers on a per-call basis, so each of those call sites also resolved a provider and model on its own. Each caller hand-rolled its own timeout and abort handling rather than sharing one policy. Nothing that touched an AI model could be exercised in a test without a network call and a live key. + +## Decision + +One `TextGeneration` port now covers every shape a caller needs, `streamText`, `generateObject`, and `runToolConversation`, the last running a short tool-calling conversation and yielding only its text output. [`../../server/adapters/ai-sdk-text-generation.ts`](../../server/adapters/ai-sdk-text-generation.ts) is the only module allowed to import the `ai` package. It resolves provider and model, combines a caller's cancellation signal with a five-minute generation timeout, and retries through the policy in [`../../server/adapters/retry-policy.ts`](../../server/adapters/retry-policy.ts), explicitly setting `maxRetries: 0` on every SDK call so the SDK's own retry can never run underneath this adapter's. A failure maps to a typed `TextGenerationError` carrying one of seven `kind` values, `auth-failed`, `rate-limited`, `overloaded`, `timed-out`, `network-failed`, `content-refused`, or `unknown`. [`../../server/ports/text-generation.fake.ts`](../../server/ports/text-generation.fake.ts) scripts responses in memory, and [`../../server/ports/text-generation.contract.ts`](../../server/ports/text-generation.contract.ts) pins the fake's behavior. That contract is deliberately fake-only, since a real subject would spend money against a live provider on every run. Services depend on the port and routes stay thin. Prompt text lives in typed TypeScript template literals inside each service module rather than runtime markdown files, plus one shared fragment, [`../../server/prompts/formatting-rules.ts`](../../server/prompts/formatting-rules.ts), so prompts survive Electron bundling and type-check like any other code. + +## Consequences + +**What this buys** + +- Services are unit-testable against the fake, with no network call and no provider key. +- Timeout, retry, and abort policy live in one place instead of being redefined per caller. +- Adding a provider is an adapter-only change. + +**What this costs** + +- Every service call now passes through one extra indirection before it reaches the SDK. +- The port can only expose streaming as plain text chunks plus abort, not the SDK's fuller stream. A tool call's own arguments and result are never visible on the returned iterable, only their side effect. + +## Revisit when + +A provider needs a capability the port cannot express without leaking SDK types into a service. diff --git a/docs/adr/0006-electron-packaging-constraints.md b/docs/adr/0006-electron-packaging-constraints.md new file mode 100644 index 0000000..440a616 --- /dev/null +++ b/docs/adr/0006-electron-packaging-constraints.md @@ -0,0 +1,28 @@ +# 0006. Electron packaging constraints + +Status: Accepted +Date: 2026-07-21 + +## Context + +Electron packaging breaks in several independent ways. pnpm's symlinked store confuses electron-builder's dependency resolution. The unified and remark ecosystem ships ESM-only with deep transitive dependencies electron-builder cannot follow. The audiobook feature's ONNX runtime is a native addon, and `kokoro-js` pulls in a phonemizer that bundles the espeak-ng WASM build, neither of which bundles like ordinary JavaScript. On top of that, the three run modes, dev, preview, and a packaged build, load the renderer from three different origins. + +## Decision + +[`../../.npmrc`](../../.npmrc) sets `node-linker=hoisted` so electron-builder sees a flat `node_modules` instead of pnpm's symlinks. In [`../../vite.config.ts`](../../vite.config.ts), the Electron main process build's `external()` function bundles the unified, remark, rehype, mdast, and hast packages straight into `dist-electron/`, while CJS packages and the native audiobook dependencies (`kokoro-js`, `onnxruntime-node`, `onnxruntime-common`, `fluent-ffmpeg`, `phonemizer`, and the Hugging Face packages) stay external. [`../../package.json`](../../package.json)'s `build.files` lists those same native packages so electron-builder copies them into the packaged app, and `build.asarUnpack` unpacks `onnxruntime-node` from the asar archive. The embedded server always binds `127.0.0.1` on a free port, chosen by passing port `0` to `startServer` in [`../../electron/main.ts`](../../electron/main.ts). CORS in [`../../server/index.ts`](../../server/index.ts)'s `isAllowedOrigin` accepts a `null` origin and any `localhost` or `127.0.0.1` origin. `connect-src` in both [`../../index.html`](../../index.html) and `electron/main.ts` lists `http://127.0.0.1:*` and `http://localhost:*`. The source `index.html` always points at its source entry, never at build output. A dynamically imported CJS module, `epub-gen-memory` in [`../../server/adapters/epub-gen-export.ts`](../../server/adapters/epub-gen-export.ts), handles being double-wrapped under Node's ESM loader. [`../../scripts/bundle-fingerprint.sh`](../../scripts/bundle-fingerprint.sh) prints every bare specifier that survives into the built main bundle, run as a no-op check that a change to the bundle list has not let a new specifier leak through unbundled. + +## Consequences + +**What this buys** + +- All three run modes load the renderer correctly, and audiobook generation works in the packaged DMG, not only in dev. + +**What this costs** + +- The bundle list in `vite.config.ts` is manual and grows whenever a new ESM-only markdown dependency is added. +- Hoisting the pnpm store weakens the isolation pnpm otherwise gives each dependency. +- A packaging change has to be checked in all three modes, not only `electron:dev`, since dev never loads from `dist-electron/` or an asar archive at all. + +## Revisit when + +electron-builder resolves pnpm's symlinked store natively. diff --git a/docs/adr/0007-versioned-library-with-forward-only-migrations.md b/docs/adr/0007-versioned-library-with-forward-only-migrations.md new file mode 100644 index 0000000..c0ec0bc --- /dev/null +++ b/docs/adr/0007-versioned-library-with-forward-only-migrations.md @@ -0,0 +1,28 @@ +# 0007. Versioned on-disk library with forward-only migrations + +Status: Accepted +Date: 2026-07-21 + +## Context + +The library is plain YAML on the filesystem, and its shape used to be implied by whichever Zod `.default()` happened to be in the running build. An older library meeting a newer schema failed at read time with no diagnosis of what was wrong or how old the data was. + +## Decision + +`meta.yml` on each book and `learning-profile.yml` globally now carry a `schemaVersion` field, defined in [`../../shared/schema-version.ts`](../../shared/schema-version.ts), with an absent field read as version 1, since every build before this one wrote neither file with a version at all. Each of the two gets its own ordered chain of pure forward-only steps, walked by `migrateForward` in [`../../server/migrations/migrate.ts`](../../server/migrations/migrate.ts) and documented in [`../../server/migrations/README.md`](../../server/migrations/README.md). A `LibraryMigrator` port, defined in [`../../server/ports/library-migrator.ts`](../../server/ports/library-migrator.ts) and implemented by [`../../server/adapters/fs-library-migrator.ts`](../../server/adapters/fs-library-migrator.ts), runs `migrate()` once inside `runStartupTasks` in [`../../server/index.ts`](../../server/index.ts), before crash recovery, because recovery reads and writes `meta.yml` through the current schema and would silently skip a book still at an old version. `buildServer` itself stays mutation-free, only `runStartupTasks` touches disk. The first time a document is migrated, the migrator writes a one-time backup of its exact original bytes alongside it, named `meta.yml.bak-v1` for every migration written so far, since version 1 is the only version any released build has ever produced. A document newer than the running build supports is never written to and is reported failed, raising `SchemaTooNewError` on any later direct read. Committed fixture libraries at old schema versions, under `server/migrations/__fixtures__/`, are the test corpus proving each step round-trips. + +## Consequences + +**What this buys** + +- One auditable migration pass at boot, logged as a single line, instead of read-time coercion scattered across whichever code happens to read a file first. +- Schemas can stop leaning on `.default()` to paper over an old shape. + +**What this costs** + +- There is no backward migration, so downgrading the app fails loudly with `SchemaTooNewError` rather than risk silently corrupting a newer library. +- A book added to disk while the app is already running waits for the next boot before it is migrated. + +## Revisit when + +The library outgrows the filesystem, or a migration ever needs to be reversible. diff --git a/docs/adr/0008-persisted-job-journal.md b/docs/adr/0008-persisted-job-journal.md new file mode 100644 index 0000000..fb4803f --- /dev/null +++ b/docs/adr/0008-persisted-job-journal.md @@ -0,0 +1,28 @@ +# 0008. Persisted job journal with disk-truth resume + +Status: Accepted +Date: 2026-07-21 + +## Context + +Background tasks, EPUB export, cover generation, audiobook install and generation, and generating every remaining chapter, lived only in memory, tracked by `BackgroundTasks`. A restart stranded any job still running, and crash recovery could only reset a book's status rather than pick a job back up. + +## Decision + +[`../../server/ports/job-journal.ts`](../../server/ports/job-journal.ts) defines a `JobJournal` port, one YAML file per job under `{dataDir}/jobs/`, written through the same temp-then-rename `writeYaml` helper every other adapter uses. [`../../server/adapters/fs-job-journal.ts`](../../server/adapters/fs-job-journal.ts) is the real adapter. [`../../server/adapters/journalled-background-tasks.ts`](../../server/adapters/journalled-background-tasks.ts) composes a `JobJournal` onto an existing `BackgroundTasks` as a decorator, calling `record()` on `start()` and `clear()` on `succeed()`, `fail()`, and `cancel()`, rather than making `BackgroundTasks` itself asynchronous. The same `BackgroundTasks` contract test, unchanged, passes against the decorated adapter, which is the proof that adding persistence changed no observable behavior. At boot, [`../../server/services/resume-interrupted-jobs.ts`](../../server/services/resume-interrupted-jobs.ts) reads every surviving record after crash recovery finishes, from `runStartupTasks` in [`../../server/index.ts`](../../server/index.ts). Only `generate-all` and `generate-audiobook` auto-resume. Every other job type is marked cleanly retriable instead, either in the existing task tray or, for a chapter already mid-stream, as an interrupted generation the reader already knows how to surface. Resume never trusts the journalled checkpoint to decide what to redo. It recomputes the real start point from the book's own metadata on disk, so the checkpoint is advisory only, a progress label rather than a decision. Chapter generation restarts a whole chapter rather than resuming mid-stream, because, per [`../../server/services/chapter-generation-stream.ts`](../../server/services/chapter-generation-stream.ts), there is no way to resume a partially streamed chapter with any provider. `TUTOR_NO_AUTO_RESUME=1` is a debugging escape hatch that leaves every journal record untouched. + +## Consequences + +**What this buys** + +- No already-saved chapter or narrated audio file is ever regenerated on resume. +- The existing `BackgroundTasks` contract test is the proof the decorator is transparent, with no test loosened to make it pass. + +**What this costs** + +- Resume can only restart a whole step, a chapter or an entire narration pass, never a partial one. +- A `JobJournal` instance can only checkpoint or clear a job it recorded itself, so two processes pointed at the same data directory can discover each other's interrupted jobs but not jointly manage one still running. + +## Revisit when + +A job type appears whose steps are not idempotent from disk, or auto-resume is observed spending money the user did not intend. diff --git a/docs/adr/README.md b/docs/adr/README.md new file mode 100644 index 0000000..4bd4737 --- /dev/null +++ b/docs/adr/README.md @@ -0,0 +1,16 @@ +# Architecture Decision Records + +These are the decisions that shaped Tutor's architecture, in the order they were made. + +| Number | Decision | Status | +|--------|----------|--------| +| 0001 | [Filesystem as the database](0001-filesystem-as-the-database.md) | Accepted | +| 0002 | [Just-in-time chapter generation, quiz masks the latency](0002-just-in-time-chapter-generation.md) | Accepted | +| 0003 | [Local Kokoro TTS over cloud TTS](0003-local-kokoro-tts.md) | Accepted | +| 0004 | [Single package, monorepo-shaped](0004-single-package-monorepo-shaped.md) | Accepted | +| 0005 | [AI SDK behind a port](0005-ai-sdk-behind-a-port.md) | Accepted | +| 0006 | [Electron packaging constraints](0006-electron-packaging-constraints.md) | Accepted | +| 0007 | [Versioned library with forward-only migrations](0007-versioned-library-with-forward-only-migrations.md) | Accepted | +| 0008 | [Persisted job journal](0008-persisted-job-journal.md) | Accepted | + +Up: [ARCHITECTURE.md](../../ARCHITECTURE.md) From 1e6cd587835f4653afd7b23509ffc5fb2b2274ce Mon Sep 17 00:00:00 2001 From: Ross Miller Date: Tue, 21 Jul 2026 11:01:21 -0500 Subject: [PATCH 05/10] docs: give every folder a README and point CLAUDE.md at them Ten folder READMEs, each opening with an Up breadcrumb to the hub and closing with Related links, so a reader who lands anywhere in the tree can walk up or sideways without grepping. Three of them state a rule the outline had wrong. Services are not free of node:fs, five of them read a file behind a path-returning ArtifactStore and the real rule is that fs must not appear in a route. Features do import each other, so the README describes the pattern that exists rather than one that does not. The port list is fifteen, not thirteen. CLAUDE.md drops the per-file tree and the fourteen-row API table it had grown wrong, and points at ARCHITECTURE.md, CONTEXT.md, docs/adr/ and the generated docs/api-routes.md instead. Every Electron constraint is byte-identical; the only two edits inside that section are stale file references, epub-gen-memory moved to server/adapters/epub-gen-export.ts and the renderer entry is client/app/main.tsx, which is what index.html already says. Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5 --- CLAUDE.md | 160 +++++++++++++----------------------- client/README.md | 27 ++++++ client/features/README.md | 30 +++++++ e2e/README.md | 1 + electron/README.md | 28 +++++++ server/README.md | 33 ++++++++ server/adapters/README.md | 37 +++++++++ server/migrations/README.md | 1 + server/ports/README.md | 35 ++++++++ server/services/README.md | 15 ++++ shared/README.md | 2 + 11 files changed, 265 insertions(+), 104 deletions(-) create mode 100644 client/README.md create mode 100644 client/features/README.md create mode 100644 electron/README.md create mode 100644 server/README.md create mode 100644 server/adapters/README.md create mode 100644 server/ports/README.md create mode 100644 server/services/README.md diff --git a/CLAUDE.md b/CLAUDE.md index 077b456..60a2486 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -2,24 +2,46 @@ AI-generated books tailored to your learning style. Books are generated chapter-by-chapter with a feedback loop: after each chapter, quiz questions and feedback shape how subsequent chapters are generated. The book literally rewrites itself based on how you're learning. +## Start here + +| Question | Answer lives in | +|---|---| +| How does this fit together? | [`ARCHITECTURE.md`](ARCHITECTURE.md), the entry point, five diagrams | +| What does this word mean? | [`CONTEXT.md`](CONTEXT.md), the domain glossary | +| Why is it built this way? | [`docs/adr/`](docs/adr/README.md), eight decisions and what each cost | +| What endpoints exist? | [`docs/api-routes.md`](docs/api-routes.md), generated, run `pnpm docs:routes` | +| How do I add a capability? | the `add-feature` Agent Skill | +| Is the repo green? | the `verify` Agent Skill | + ## How It Works 1. **Create a book** — Enter a topic + prompt, AI generates a table of contents 2. **Approve the TOC** — Review, edit, reorder chapters, then approve 3. **Read chapter-by-chapter** — Quick, digestible chapters (~1,500 words, 5-10 min read) teaching specific concepts -4. **Inline chat** — Click any sentence to slide out a chat panel for deeper AI explanation, then return to where you left off +4. **Inline chat** — Select any text to slide out a chat panel for deeper AI explanation, then return to where you left off 5. **Feedback** — After finishing a chapter, give feedback on what resonated/didn't 6. **Generation triggered** — Submitting feedback triggers next chapter generation in the background 7. **Quiz while waiting** — Optional 3-question quiz to test retention and aid memory while next chapter generates 8. **Adaptive** — Next chapter incorporates feedback + quiz results (wrong answers trigger brief recap at start) -## Architecture +## Repo layout + +Each line links to the README that owns that folder. There is no file-by-file tree here on purpose, because it rots. -- **Storage:** Filesystem — Markdown chapters + YAML metadata in `books/` -- **Backend:** Fastify server (`server/`) -- **Frontend:** React 19 + Vite (`src/`) -- **AI:** Vercel AI SDK (`ai` + `@ai-sdk/anthropic`) with structured output via `generateObject()` -- **Learning profile:** Global defaults in `books/learning-profile.yml` with per-book overrides +``` +client/ React 19 renderer → client/README.md + api/ the only code that talks to server → client/README.md + features/ one folder per capability → client/features/README.md +server/ embedded Fastify, hexagonal → server/README.md + ports/ 15 interfaces, fakes, contracts → server/ports/README.md + adapters/ the only place real I/O happens → server/adapters/README.md + services/ application logic over ports → server/services/README.md + migrations/ forward-only schema steps → server/migrations/README.md +shared/ types both sides import → shared/README.md +electron/ main and preload, packaging → electron/README.md +e2e/ Playwright journeys on fakes → e2e/README.md +docs/ ADRs, generated routes, plans → docs/adr/README.md +``` ## Tech Stack @@ -36,63 +58,7 @@ AI-generated books tailored to your learning style. Books are generated chapter- | AI | Vercel AI SDK (`ai` + `@ai-sdk/anthropic`) | | Validation | Zod | | Config | YAML (`yaml` package) | -| Testing | Vitest | - -## File Structure - -``` -tutor/ -├── books/ # Generated content (gitignored except learning-profile) -│ ├── learning-profile.yml # Global learning style config -│ └── {book-id}/ -│ ├── meta.yml # Status, title, prompt, overrides -│ ├── toc.yml # Approved table of contents -│ ├── chapters/ -│ │ └── 01.md ... NN.md # Chapter content (markdown) -│ ├── progress.yml # Per-chapter scroll progress -│ └── feedback/ -│ └── 01.yml ... NN.yml # Feedback + quiz per chapter -├── server/ # Backend (Fastify) -│ ├── index.ts # Server entry point -│ ├── schemas.ts # Zod schemas for all YAML metadata -│ ├── routes/ -│ │ ├── books.ts # CRUD, generation triggers, progress -│ │ ├── chapters.ts # Chapter content, status, quiz -│ │ └── profile.ts # Learning profile management -│ ├── services/ -│ │ ├── book-generator.ts # AI generation (TOC, chapters, quiz) -│ │ ├── book-store.ts # Filesystem read/write -│ │ └── generation-queue.ts # In-memory background generation tracking -│ └── prompts/ -│ ├── generate-toc.md -│ ├── generate-chapter.md -│ └── generate-quiz.md -├── src/ # Frontend (React + Vite) -│ ├── main.tsx -│ ├── App.tsx -│ ├── pages/ -│ │ ├── LibraryPage.tsx # Book grid with progress bars -│ │ └── ReaderPage.tsx # Chapter reader + feedback + quiz -│ ├── components/ -│ │ ├── BookCard.tsx -│ │ ├── BookGrid.tsx -│ │ ├── WizardModal.tsx # 3-step: prompt → TOC → generating -│ │ ├── MarkdownReader.tsx # Renders chapter markdown, clickable sentences -│ │ ├── InlineChatPanel.tsx # Slide-out AI chat for explaining selected text -│ │ ├── FeedbackForm.tsx -│ │ ├── QuizPanel.tsx -│ │ └── ProgressBar.tsx -│ ├── hooks/ -│ │ └── useScrollProgress.ts -│ ├── lib/ -│ │ ├── api.ts # Fetch wrapper for backend -│ │ └── utils.ts # cn() helper (shadcn) -│ └── store.ts # Redux Toolkit store -├── components.json # shadcn/ui config -├── index.html # Vite entry -├── vite.config.ts -└── vitest.config.ts -``` +| Testing | Vitest, Playwright for journeys | ## Key Design Decisions @@ -100,31 +66,10 @@ tutor/ - **Chapter length** — ~1,500 words (5-10 min), flex longer when content demands - **TOC approval** — Step-by-step wizard before generation begins - **Progress tracking** — Scroll-based auto-tracking (completed at ≥90%) -- **Generation flow** — Just-in-time: one chapter at a time, quiz masks latency -- **Background generation** — In-memory `Map`, fire-and-forget on quiz submit -- **If server restarts mid-generation** — Book stays valid, user can retrigger from the reader -- **Inline chat** — Click any sentence to open a slide-out panel for AI-powered deeper explanation; dismissing returns to reading position - -## API Routes - -| Method | Path | Purpose | -|--------|------|---------| -| `GET` | `/api/books` | List all books | -| `POST` | `/api/books` | Start new book (generates TOC) | -| `GET` | `/api/books/:id` | Get book metadata + progress | -| `DELETE` | `/api/books/:id` | Delete a book | -| `POST` | `/api/books/:id/reset` | Reset reader interaction (progress, rating, feedback, quiz answers) | -| `GET` | `/api/books/:id/toc` | Get table of contents | -| `PUT` | `/api/books/:id/toc` | Approve TOC, triggers Ch.1 generation | -| `GET` | `/api/books/:id/chapters/:num` | Get chapter markdown content | -| `GET` | `/api/books/:id/chapters/:num/status` | Check generation status | -| `PUT` | `/api/books/:id/progress/:num` | Update scroll progress | -| `POST` | `/api/books/:id/chapters/:num/feedback` | Submit chapter feedback | -| `GET` | `/api/books/:id/chapters/:num/quiz` | Get quiz questions | -| `POST` | `/api/books/:id/chapters/:num/quiz` | Submit quiz, triggers next chapter | -| `POST` | `/api/books/:id/chapters/:num/chat` | Inline chat about a sentence/passage | -| `GET` | `/api/profile` | Get learning profile | -| `PUT` | `/api/profile` | Update learning profile | +- **Generation flow** — Just-in-time: one chapter at a time, quiz masks latency ([ADR 0002](docs/adr/0002-just-in-time-chapter-generation.md)) +- **Background work** — A `BackgroundTask` in memory, journalled to disk by the `JobJournal` port so an interrupted one can resume at the next boot ([ADR 0008](docs/adr/0008-persisted-job-journal.md)) +- **If the app restarts mid-generation** — `generate-all` and audiobook jobs resume from disk without redoing finished work; a single interrupted chapter surfaces in the reader's retry panel +- **Inline chat** — Select any text to open a slide-out panel for AI-powered deeper explanation; dismissing returns to reading position ## Electron Packaging @@ -141,38 +86,45 @@ This is an Electron app using `vite-plugin-electron`. Three modes exist with dif - **Address**: Always use `127.0.0.1` (not `localhost`) for server communication — avoids IPv6 mismatch on macOS - **CORS**: Server must accept `Origin: null` (file:// protocol) and any `localhost`/`127.0.0.1` origin — enforced in `server/index.ts:isAllowedOrigin()` - **CSP**: Both `index.html` meta tag and `electron/main.ts` header must allow `http://localhost:*` AND `http://127.0.0.1:*` in `connect-src` -- **pnpm + electron-builder**: `.npmrc` requires `node-linker=hoisted`. The electron build bundles the unified/remark/rehype ecosystem into `dist-electron/` (via rollup) since electron-builder can't resolve their deep transitive deps. The bundle list is in `vite.config.ts` `external()`. CJS packages (fastify, etc.) stay external — if a CJS transitive dep is missing, add it to `package.json` `dependencies` (e.g., `json-schema-ref-resolver` for fastify). For CJS packages imported dynamically, handle the double-default: `mod.default?.default ?? mod.default` (see `epub-gen-memory` import in `books.ts`). -- **Never modify `index.html` or `package.json` to match build output** — `dist/` is the build target, source files must keep source references (`/src/main.tsx`) +- **pnpm + electron-builder**: `.npmrc` requires `node-linker=hoisted`. The electron build bundles the unified/remark/rehype ecosystem into `dist-electron/` (via rollup) since electron-builder can't resolve their deep transitive deps. The bundle list is in `vite.config.ts` `external()`. CJS packages (fastify, etc.) stay external — if a CJS transitive dep is missing, add it to `package.json` `dependencies` (e.g., `json-schema-ref-resolver` for fastify). For CJS packages imported dynamically, handle the double-default: `mod.default?.default ?? mod.default` (see `epub-gen-memory` import in `server/adapters/epub-gen-export.ts`). +- **Never modify `index.html` or `package.json` to match build output** — `dist/` is the build target, source files must keep source references (`/client/app/main.tsx`) ## Development ```bash pnpm test # Run all tests +pnpm typecheck # tsc --noEmit +pnpm lint # ESLint, zero warnings is the bar +pnpm e2e # Playwright journeys against the fake AI adapter +pnpm docs:routes # Regenerate docs/api-routes.md, CI fails on drift pnpm electron:dev # Dev mode (Vite + Electron + HMR) pnpm electron:preview # Build then run (test production rendering) pnpm electron:build # Build + package DMG pnpm dev:server # Fastify standalone on port 3147 +pnpm mcp:dev # MCP server, needs dev:server on 3147 (see .mcp.json) ``` ## Conventions -- Zod schemas live in `server/schemas.ts` — single source of truth for all data shapes +- Zod domain schemas live in `shared/`, the single source of truth for both sides - YAML for all metadata, Markdown for chapter content -- Vercel AI SDK (`ai` package) for all AI calls — prefer `generateObject()` for structured output +- Vercel AI SDK is reached only through the `TextGeneration` port, never imported outside `server/adapters/` - Tests colocated with source files (`*.test.ts`) -- Path aliases: `@client/*` → `client/*`, `@server/*` → `server/*` +- **TDD** — tests land before or with implementation, visible in commit order. Contract test before adapter, service test before service, api-client test before the client function +- Path aliases: `@client/*` → `client/*`, `@server/*` → `server/*`, `@shared/*` → `shared/*` +- Domain names come from `CONTEXT.md`. Do not invent a synonym for a word that already has an owner -## Domain & Architecture (Aspirational) +## Domain & Architecture -The codebase is moving toward domain-driven design with ports-and-adapters separation. Apply these when adding or refactoring code; existing code does not all conform yet. +These are the rules new and refactored code follows. The server conforms today. The client conforms on the api boundary and is still converging elsewhere. -- **Ubiquitous domain language**: `Book`, `Chapter`, `TOC`, `Feedback`, `Quiz`, `Progress`, `LearningProfile`, `Audiobook`. Use these names everywhere — schemas, services, components, prompts, UI copy. Don't invent synonyms. -- **Pure domain core** in `server/schemas.ts` and any future domain modules — Zod types and pure functions only. No `fs`, `fetch`, AI SDK imports, or env vars inside the domain. -- **Ports for every external dependency**: AI providers, filesystem persistence, audio synthesis, EPUB tooling, image generation, API key storage, background queues, Electron IPC, frontend → backend HTTP. Each gets a single named module that the rest of the app depends on by shape, not by SDK. -- **Adapters do the I/O**: only the adapter touches the SDK, library, child process, or filesystem. Swappable and testable in isolation. -- **Routes are thin**: parse input → call a port → return result. No business logic, no direct `fs` or SDK calls in `server/routes/*.ts`. -- **Frontend goes through one client**: components import from `src/lib/api.ts`, not raw `fetch`. New endpoints get a function in the client. -- **No new SDK sprinkling**: when adding a third-party SDK, wrap it behind a port first, then consume the port from services. +- **Ubiquitous domain language**: `Book`, `Chapter`, `TOC`, `Feedback`, `Quiz`, `Progress`, `LearningProfile`, `Audiobook`, `BackgroundTask`. Use these names everywhere — schemas, services, components, prompts, UI copy. `CONTEXT.md` is the register +- **Pure domain core** in `shared/` and `server/domain/` — Zod types and pure functions only. No `fs`, `fetch`, AI SDK imports, or env vars inside the domain +- **Ports for every external dependency**: each gets a single named module the rest of the app depends on by shape, not by SDK. Every port ships an in-memory fake and a contract test +- **Adapters do the I/O**: only the adapter touches the SDK, library, child process, or filesystem. Swappable and testable in isolation +- **Routes are thin**: parse input → call a service → return result. No business logic, no direct `fs` or SDK calls in `server/routes/*.ts` +- **Frontend goes through one client**: components import from `client/api/`, not raw `fetch`. A raw `fetch` or `new EventSource` outside `client/api/` is an ESLint error. New endpoints get a function in the client +- **No new SDK sprinkling**: when adding a third-party SDK, wrap it behind a port first, then consume the port from services ## UI / Frontend Design @@ -192,7 +144,7 @@ The codebase is moving toward domain-driven design with ports-and-adapters separ ### Page Layout Patterns - **Header**: Centered title only, draggable region (`-webkit-app-region: drag`), no navigation buttons inside header -- **Back button**: Absolute-positioned overlay on the content area below the header — `absolute left-6 top-3 z-20` on a plain `