Skip to content

Phase 6 — Committed E2E journey suite on fake adapters - #53

Merged
rsml merged 18 commits into
masterfrom
phase-6-e2e-journeys
Jul 21, 2026
Merged

Phase 6 — Committed E2E journey suite on fake adapters#53
rsml merged 18 commits into
masterfrom
phase-6-e2e-journeys

Conversation

@rsml

@rsml rsml commented Jul 21, 2026

Copy link
Copy Markdown
Owner

Phase 6 — Committed E2E journey suite on fake adapters

Objective

Prove the whole app works end to end, deterministically, with zero live-provider traffic and zero API keys, by driving the real browser UI against the real Fastify server whose edges are fakes.

This is a pure test addition. Not one line under client/, server/, shared/, or electron/ changed, and the gate below proves that with a command rather than a claim. The suite is the regression net Phase 7's durability work leans on, and it is what a cold reader opens to find out whether Phase 2's hexagon is load bearing or decorative. It is load bearing. Every journey boots the production buildServer and hands it fakes through the same overrides argument Electron already uses to hand it a diagram renderer, unchanged.

It also earned its keep before it merged. See "What it caught" below.

Journeys

# Journey File
a Create a book, watch the table of contents stream, revise it, approve it, generate chapter 1 create-book.spec.ts
a Open a book on disk and page between chapters reading.spec.ts
b Quiz, wrong answer, feedback, and the chapter-2 prompt the model actually receives adaptive-loop.spec.ts
c Import an EPUB, preview it, land it in the library epub-import.spec.ts
d Export a finished book to a real EPUB and download it epub-export.spec.ts
e Rename, tag, search, and delete in the library library-crud.spec.ts
f The audiobook install gate, and generation once the engine is installed audiobook.spec.ts
g The packaged app rehydrates the reading position, two runs, one file different electron-rehydration.spec.ts
h A provider failure's real message reaches the reader, not boilerplate generation-failure.spec.ts

Harness

e2e/support/app.ts boots the real server in process, registers @fastify/static over the real production bundle, and listens on an ephemeral port. Serving the client from the same origin that answers its API calls is what makes client/api/http.ts resolve its base to empty and issue relative /api/... requests exactly as it does in the packaged app, with no proxy, no CORS, and no Electron stub. The client under test is the production bundle, unmodified.

In process rather than a child process, because a journey needs to reach into the fake model mid flight, to add a failure rule or to read back the prompt the server actually sent, and a serialisation boundary would be in the way of both.

e2e/support/scripted-text-generation.ts answers by intent rather than by call order. Phase 2's createFakeTextGeneration answers first in first out, which is right for a unit test that knows how many model calls the code under test makes. A journey does not know that. Approving a table of contents fires a skill classification, a chapter stream, and a quiz, in an order the UI decides and may reorder tomorrow, and a fixture list coupled to that order is the largest available source of flake. Rules match on the prompt or the schema and are consulted newest first, which makes failure injection a one line addition that shadows a default rather than a rewrite that deletes it.

A call that matches neither a queued script nor a rule throws. There is no default answer on purpose. Zero live-provider traffic is then a property of the design, not a promise, because the only way to get a response out of that object is to have written the response down first.

e2e/README.md covers how to run it, how to add a journey, why fakes rather than mocks, why there is not one data-testid in the codebase, and the one product-side source of nondeterminism a journey has to respect.

What it caught

Three things, before it merged.

A real generation bug, #50. POST /api/books/:id/generate-next answers 200 with an empty body. pipeHubToSse attaches request.raw.on('close', ...) and ends the reply from it, and on Node 16 and later that event fires when the request stream finishes rather than when the client disconnects, so for a POST whose body Fastify already read it fires immediately. Generation still runs and the chapter still lands on disk, but the reader hangs in its generating state forever. It affects the three routes sharing that helper, generate-next, regenerate, and generation-stream, and nothing else, because openSseStream attaches no close listener. Confirmed four independent ways, the last against the production startServer path over a real socket with no browser, no fakes, and no static plugin.

The part worth dwelling on: the identical request through fastify.inject returns the complete event stream, because light-my-request does not model the socket lifecycle. Every inject test in the repository passes straight over this. That is the clearest argument this phase can make for its own existence.

Not fixed here. Phase 7 owns server/ this window and this phase changes no production file. Two assertions are quarantined with test.fixme naming the issue, and both flip green when it lands.

Seven accessibility gaps, #51. The suite forbids data-testid, so a control a locator cannot name is a control a screen reader cannot name either. The worst is a book card, which is a div with an onClick and therefore has no role, no accessible name, and no tab stop, so a keyboard user cannot open a book at all. Every journey works around its gap with a legitimate text or role locator, so fixing these will not break the suite, several locators simply get shorter.

A dead script, #52. scripts/diagnose-quiz.ts imports a module the hexagonal restructure deleted, and scripts/ sits outside both pnpm typecheck and pnpm lint, which is why nobody noticed. This phase added its own fixture builder to both gates rather than widening them, because widening them would have meant either fixing an unrelated file or leaving the gate off.

Gate evidence

$ pnpm e2e --project=web          # run 1     14 passed, 3 skipped   9.6s
$ pnpm e2e --project=web          # run 2     14 passed, 3 skipped   9.2s
$ pnpm e2e --project=electron     # run 1     2 passed              16.6s
$ pnpm e2e --project=electron     # run 2     2 passed              16.5s
$ pnpm test                       1131 passed across 119 files      2.41s
$ pnpm typecheck                  clean
$ pnpm lint                       zero warnings
$ git diff --stat master -- client/ server/ shared/ electron/
                                  (empty)
$ rg "from '(ai|@ai-sdk/[a-z-]+)'" e2e/     none
$ rg "waitForTimeout" e2e/                  none

Green twice consecutively on both projects with retries: 0. The web number includes the vite build that global setup runs, so the suite itself is well inside the 120 second budget and both CI jobs are far inside four minutes. The three skips are the quarantined assertions on #50, not flake. Nothing in this suite has flaked once, across roughly forty runs during development.

Mutation evidence. A test suite that has never failed is not yet evidence, so two deliberate breaks were run and reverted.

The first is the one the plan names, the scripted model throwing on the chapter stream:

1) [web] > create-book.spec.ts:79 > revises the table of contents, persists it, and generates chapter 1 on approval
   Error: expect(locator).toBeVisible() failed
1 failed, 13 passed

The second removes the provider key from the fake vault, simulating a regression in key handling rather than in generation, to show the suite fails across a different axis and each failure names itself:

1) [web] > create-book.spec.ts:38 > creates a book and streams the table of contents into the wizard
2) [web] > create-book.spec.ts:79 > revises the table of contents, persists it, and generates chapter 1 on approval
3) [web] > generation-failure.spec.ts:66 > chapter 1 generation failure reaches the creation wizard intact: a plain error with a distinctive message
4) [web] > generation-failure.spec.ts:66 > chapter 1 generation failure reaches the creation wizard intact: a typed error subclass
4 failed, 10 passed

Both reverted, tree clean, suite green again.

Runner decision

@playwright/test, standalone, e2e/ at the top level, chosen over extending Phase 3's hand-rolled CDP harness and over vitest browser mode. Phase 3's harness was right for what it did, a one-off pixel measurement where a screenshot and a byte compare were the whole job, but a committed journey suite needs auto-waiting locators, per-test isolation, traces, and reporting, and hand-rolling those is exactly the clever-over-simple trap this effort's principles reject. Web-first assertions retry to a deadline, which is why not one journey contains a sleep and why the plan's SSE timing risk never materialised. And one tool covers Electron through _electron.launch(), so the packaged-app journey needed no second harness, which is the fact that actually decided it, because the alternative was CDP-attaching to Electron by hand. Vitest browser mode was rejected as the newest and least battle-tested option, and folding a browser suite into pnpm test would have put a fast unit run behind a slow one. pnpm test stays at 2.4 seconds. Journeys are pnpm e2e.

The honest cost is two devDependencies, one cached Chromium download in CI, and a second test runner in the repo.

Deviations from the plan

  1. Five config touches, not three. The plan named the vitest exclude, the tsconfig include, and the ESLint boundary. pnpm lint globs its four zones explicitly, so the boundary would never have run without adding e2e/ to that script, and the EPUB fixture builder needed the tsconfig include and the lint glob too. The gate's real invariant is untouched.
  2. Vitest excludes e2e/**/*.spec.ts, not e2e/**. The scripted adapter's contract test lives under e2e/support/ as a *.test.ts, needs no browser, and belongs in the fast suite.
  3. Neither CI job builds the client as a separate step. e2e/global-setup.ts runs vite build itself, so there is one code path and no way for a job, or a developer, to run the journeys against a bundle it did not build. E2E_SKIP_BUILD=1 skips it while iterating.
  4. The Electron project runs serially. Two Electron applications launching at once each start their own server and load the audiobook stack, which turned a six second test into a thirty four second one on a loaded machine. There are two tests and nothing to gain from overlapping them.
  5. Journey (f)'s second half seeds the engine through the install route, not the filesystem. The plan said to seed bin/ffmpeg and the model directory. That would be inert: createPorts spreads overrides last, so the fake SpeechSynthesis replaces the real adapter outright, and the fake's installed state is two in-memory booleans that never read the filesystem. The journey flips them through POST /api/audiobook/install in process, which is what the service's own unit test does. Zero filesystem writes, zero child processes, zero network.
  6. The plan's note about routes/audiobook.ts:18 is stale. That handler calls only ports.speechSynthesis, never the installer service. Phase 2 fixed it in b77e9bd.
  7. Journey (g) persists only the reading position, and opens the book by clicking it. redux-persist's default reconciler merges one level deep, so a persisted slice replaces that slice's initial state wholesale. A fixture that wrote a partial settings slice deleted the provider configuration the client reads on its first render and the renderer died before painting. Persisting only readingProgress is both sufficient and safe, and it is the slice the journey is actually about.
  8. Journey (a) is two files. Creating a book and reading one are separate subjects, and making every other journey drive the whole wizard first would mean one wizard regression failed six journeys that have nothing to do with the wizard.
  9. Two assertions are quarantined, per the plan's own quarantine policy, on Chapter generation SSE closes before writing a byte, so the reader hangs forever #50 rather than on flake. Nothing in this suite has flaked.

Findings reported, not patched

Everything above under "What it caught", filed as #50, #51, and #52. No production file was touched to accommodate any of them.

For Phase 7

Issue #50 is a prerequisite, not a nice to have. Phase 7's persisted generation jobs and clean restart recovery lean on GET /api/books/:id/generation-stream, which is one of the three broken routes. Any fix needs a real-socket test, because fastify.inject structurally cannot see this failure mode.

The failure-injection point is live and parameterized. Adding a per-error-class variant is one object literal in CASES in e2e/journeys/generation-failure.spec.ts, with no new locator and no new navigation:

{ name: 'rate limited', thrown: new RateLimitError('...'), expected: /rate limit/ }

respond: { throws: <value> } rethrows the value unchanged, so a typed error subclass survives all the way to the screen, which the suite already asserts. isChapterStream, isChapterStreamFor(n), isTocStream, and chapterNumberFrom(prompt) are exported from e2e/support/default-script.ts for matching.

Unquarantining is deleting two test.fixme markers. Both quarantined tests are written and correct.

https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5

rsml added 18 commits July 21, 2026 08:04
The phase plan lands before the work it describes, so the branch reads in
the order it was executed. Section 0 records the five facts found in the
code that contradict what the plan was written against, rather than
letting the plan and the tree drift apart silently.

The largest of those is the precondition. Phase 2 was expected to leave
four modules importing model-client directly, which would have blocked
three journeys. The grep is empty on master, so every journey is in scope.

Two smaller corrections matter for implementation. key-store.ts is gone,
so the data directory env var is no longer load bearing, and the scripted
adapter has to satisfy FakeTextGeneration rather than the bare port,
because the contract suite types its subject to the fake's scripting and
recording surface.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
One Playwright test creates a book and watches the fixture table of contents
stream into the wizard. It looks small and it proves the entire harness. The
built client is served from the same Fastify instance that answers its API
calls, so client/api/http.ts resolves its base to empty and issues relative
requests exactly as it does in production. The server-sent-event stream
reaches the browser and reassembles. The scripted model answers the
create-book call site. Everything writes into a throwaway data directory
rather than the reader's real library.

The biggest unknown going in was whether Playwright's loader could resolve
the server's TypeScript, since every server module imports with a .js
specifier through an @server or @shared alias. It can, unmodified. The
tsx child process fallback the plan held in reserve is not needed.

The scripted adapter answers by intent rather than by call order. A journey
makes model calls in whatever order the UI decides, and coupling fixtures to
that order is the largest available source of flake. Rules match on the
prompt or the schema and are consulted newest first, which makes failure
injection a one line addition rather than a rewrite. Phase 2's first-in
first-out scripting surface is kept on top of the rules, both so a journey
can force a one-off answer and so the adapter can satisfy the port's own
contract suite, which types its subject to that surface.

A call matching neither a queued script nor a rule throws. There is no
default answer on purpose, so zero live-provider traffic is a property of
the design rather than a promise.

Four config lines carry it. Vitest excludes the journeys only, not all of
e2e, because the scripted adapter's contract test belongs to vitest.
Tsconfig and the lint glob take the new zone in. The ESLint boundary lets a
journey reach the server it boots and forbids it reaching the client it is
supposed to drive from the outside.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The journeys are only worth anything if the object standing in for the AI
provider behaves the way the port says a TextGeneration behaves. So Phase 2's
own contract suite runs against this adapter with no exemptions, covering
cancellation mid stream, schema validation of every generated object, request
recording, and the tool conversation's step limit. That is the fidelity
guarantee the plan asks for, and it is why the adapter kept the fake's
scripting surface rather than inventing its own.

Ten further tests cover the rule surface the contract does not know about.
Rules answer by intent regardless of call order, a later rule shadows an
earlier one, a failure rule throws the exact value it names so a typed error
class survives to the caller, a queued script beats a matching rule, and an
unmatched call throws while naming every rule that was on offer. That last
one is what makes zero live provider traffic a property of the design.

Writing the allOf test found two real defects in what shipped with the
skeleton. The default script's comment claimed precedence reads top to
bottom, when rules are pushed onto the front of the list and precedence
therefore reads bottom to top. The default rules are disjoint so nothing
behaved wrongly, but anyone adding a narrow rule would have put it in the
wrong place. And the two matcher helpers took different request shapes, so
combining a system matcher with a prompt matcher failed to compile. Both now
read one shared shape, which is what allOf needs to infer.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
Journey (a)'s second half, plus the two shared helpers the rest of the
suite needs before it can fan out.

Seeding writes a book straight to the data directory through the same
filesystem repository production uses, never through raw file writes, so a
fixture takes the same validation and atomic write path a real save takes
and a book the app cannot read fails at the seed rather than three
assertions later. Most journeys are not about creating a book, they are
about exporting one or renaming one or listening to one, and making each of
them drive the whole wizard first would mean one wizard regression failed
six journeys that have nothing to do with the wizard.

The reader page object hides the fact that a chapter is paged one section at
a time, so a journey asks to finish a chapter rather than clicking Next in a
loop of its own. It also offers the chapter tab strip as a separate way to
move, because the end of chapter control opens the quiz and a journey about
reading should not have to answer questions to turn a page.

Three locator facts were found by running this rather than by reading the
markup, and each is written down where it was needed. The section controls
are named Previous section and Next section, so a loose match on Next would
take the end of chapter control instead. The chapter tab strip holds a
Chapter N button per chapter, so the article's own Chapter N label has to be
scoped to the article. And a book card is a div with an onClick, carrying no
role and no accessible name, so a journey reaches it by its title text.

That last one is a real accessibility gap rather than a test inconvenience,
since a keyboard user cannot reach a book card at all. It is reported to the
architect rather than patched here, because this phase changes no production
file.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
Journey (g) plus the two CI jobs and the suite's README, which is the last
of the plan's tasks that does not belong to a single journey.

The Electron project exists for exactly one thing the web project cannot
reach. In the browser redux-persist writes to localStorage, and in the
packaged app it writes through IPC to a file the main process owns, so the
rehydration path is only observable with a real main process running. Two
runs over the same seeded library, differing only in redux-state.json. With
a remembered position the reader opens there, and without one it derives a
position from the server's own progress record. Both are correct and getting
them backwards is the bug a smoke test should catch.

Writing it surfaced a redux-persist behaviour worth recording rather than
working around. The default reconciler merges one level deep, so a persisted
slice replaces that slice's initial state wholesale instead of being folded
into it. A fixture that wrote a partial settings slice therefore deleted the
provider configuration the client reads on its first render, and the
renderer died before painting anything. The fixture builder now persists
only the reading position and says why in a comment, since that is the slice
this journey is actually about.

The Electron project runs its two tests serially. Two Electron applications
launching at once each start their own server and load the audiobook stack,
and on a loaded machine that turned a six second test into a thirty four
second one.

Neither CI job builds the client as a separate step. Global setup builds it,
so there is one code path and no way for a job to run the journeys against a
bundle it did not build itself. The web job keeps the skip flag for the
Electron binary and the Electron job deliberately drops it, which is the
whole reason the two are separate jobs.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The first half of journey a proved the wizard and the toc stream, this half
proves the rest of the acceptance, that editing the table of contents
actually persists, that approving it calls the real start route, and that
chapter 1 lands on disk with generatedUpTo at 1.

A revised toc fixture with visibly different chapter titles gives the
revision something to prove, since streaming the same titles back would
pass even if persistence were broken. default-script.ts now points the
revise-toc rule at that fixture instead of at the original, so revising
demonstrably changes the table of contents rather than replaying it.

The revise panel's submit control has no shared page object yet, since
wizard.ts is owned by another concurrent phase-6 agent, so this adds a
small helper local to the spec file rather than editing that file.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
This seeds a finished three chapter book, right clicks its card, and clicks Export EPUB, then waits for the browser download event the client's auto download anchor fires once the background task completes.

The export port is deliberately not faked here. Every other journey stubs the AI port with a scripted model, but an epub export makes no AI call at all, it only converts markdown to HTML and hands it to the real epub gen memory library. Faking that library would only prove the test calls a fake correctly, running it for real is what proves the double default handling in server/adapters/epub-gen-export.ts still works end to end, since that handling exists specifically for the Electron production build.

The test asserts the downloaded file is a non empty real zip by checking its PK header, that book.epub landed on disk under the book's own directory before the download fired, and that the background task reached status done through GET /api/tasks.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
This adds journey e, library crud and search, as its own spec file owned by
this task. Each operation gets its own test so a failure names the specific
operation that broke, rather than an assertion partway through one long
scenario.

Every mutation asserts on screen and on disk. A card can show a change that
a failed request never actually persisted, and the server can also persist a
change the grid fails to re render because fetchBooks did not run. Checking
only the screen would miss the first case, checking only disk would miss
the second, so rename, tag add, and delete each read the book back off disk
through readBook or bookRepository after the on screen assertion passes.

Delete opens an in app confirmation dialog that requires typing delete,
not a native window.confirm, so no dialog auto accept handler was needed.

Search reaches the toolbar's search box through the documented Cmd or Ctrl
plus F shortcut rather than clicking its own toggle button, because that
button is icon only with no aria label, so no role or text locator can name
it. The Rename dialog's Title field has the same kind of gap, its label is
plain text with no htmlFor or id pairing to the input, so the test reaches
it by position among the dialog's textboxes instead of by label.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
Adds journey (c) from the phase 6 plan. Importing an EPUB shows a preview
dialog with the file's title and chapter count, and confirming it returns
to the library with the book present.

This journey drives the real epub2 parser against a real file rather than
the scripted model, since the e2e app fixture deliberately never overrides
epubImport. The fixture EPUB is built by a committed script,
scripts/build-e2e-epub-fixture.ts, using epub-gen-memory, the same library
the production export adapter wraps, instead of being checked in from an
unknown source. A binary nobody can regenerate or diff is a black box,
while a script anyone can read and rerun proves what produced it and lets
it be regenerated on demand. The script and the journey share one source
of truth for the fixture's title and chapter titles in
e2e/fixtures/sample-book.ts, so the two cannot drift apart.

Confirming the import only closes the dialog once the request round trips
successfully, and leaves it open on failure, so the journey waits for the
dialog to close before checking the library and the disk, rather than
racing ahead of a click that has only just been dispatched.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The README claimed two findings when writing the journeys produced five, and
a vague count is the kind of thing that quietly becomes zero. Each is now
named with the component it lives in, so the follow up issue has a work list
rather than a sentiment.

The point of the no test id rule is that it turns a locator problem into an
accessibility report, and the list is the evidence that it worked.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The fixture builder landed in scripts/, which sits outside both gates, so a
drift in the epub-gen-memory API or in the fixture's own types would have
gone unnoticed until the import journey failed for a reason that looked
unrelated. One include entry and one lint path fix that.

Only that one file is added rather than all of scripts/, because
scripts/diagnose-quiz.ts still imports server/services/generation-manager.js,
a module the hexagonal restructure deleted. That script is dead and belongs
to the final sweep's dead code pass, and widening the gate to cover it here
would mean either fixing an unrelated file or turning the gate off again.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
This adds journey (f) as e2e/journeys/audiobook.spec.ts, with its own page object helpers in e2e/support/journeys/audiobook.ts.

The first test seeds a fully generated book, asks the library's context menu to generate an audiobook, and asserts the missing components install gate appears, since nothing in the test has installed the model or ffmpeg. The test never clicks the gate's Download button, because in the real app that button downloads the Kokoro model and an ffmpeg binary, roughly 195 MB combined, from huggingface and evermeet.cx over the public internet. The test only asserts the button is present and named correctly, so it never risks starting that download.

The second test flips the fake speech synthesis port's installed state through fastify.inject rather than through the UI. Filesystem seeding of the paths the real installer service checks would not work in this harness, because the fake port never reads the filesystem and only flips its installed state inside its own install method. This mirrors the same operation an existing unit test performs by calling install directly on the fake object. The test then drives the real UI flow to start generation, and confirms the reader's chapter listen control and its player panel appear once the fake engine has narrated the book.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
Two controls carry the accessible name "Next section" once a chapter has
more than one, the chapter rail's mini next and the edge tap zone, and both
are wired to the same goNext callback. Playwright's strict mode rejects a
click on a name that resolves to two elements, so paging through a chapter
threw rather than advancing. The reading journey never hit it because that
journey moves between chapters through the tab strip instead.

The duplicate name is a real accessibility defect and it is reported rather
than patched, because this phase changes no production file. Taking the
first match is safe here in a way it never is for a quiz option, since the
two are accidental copies of one control rather than alternatives that mean
different things, so either one advances the reader by exactly one section.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The plan promised the suite would be a regression net for Phase 7. It
turned out to be one before Phase 7 started, so the two facts that changes
are written into the plan's reconciliation section rather than living only
in an issue.

Chapter generation's server sent event stream ends before it writes a byte,
because its close handler treats a finished request body as a client
disconnect. The reader hangs in its generating state forever while the
chapter lands correctly on disk. It is filed rather than fixed, because
Phase 7 owns the server this window and this phase changes no production
file, and two assertions are quarantined against the issue so they flip
green when the fix arrives.

The second fact is the more useful one to keep. The identical request
through fastify.inject returns the complete event stream, because
light-my-request does not model the socket lifecycle, so every inject test
in the repository passes straight over this bug. That is the clearest
argument this phase can make for why it exists.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
The accessibility list grew from five findings to seven as the last two
journeys landed, and both additions came from controls a journey could not
address, the reader's duplicated next section name and the feedback form's
unassociated textareas. All seven are now filed as one issue, so the readme
names the issue rather than carrying a list that will drift.

The flake section also needed a paragraph. Two assertions are quarantined
right now and neither is flake, so leaving the section saying only that
flake gets quarantined would have read as if the suite already had some.
Both are blocked on a real generation bug this suite found, and each
quarantine names it.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
This proves the app's adaptive loop rather than its plumbing. A reader answers the chapter 1 quiz with one deliberate wrong answer, submits feedback with two distinctive marker sentences, and the test asserts that the scripted model's recorded chapter 2 prompt actually contains those markers and the wrongly answered question. That single assertion is what tells the adaptive loop apart from a button that merely looks like it worked, since a click through the quiz and feedback screens would look identical even if the server silently discarded every answer.

generate-quiz.ts shuffles each question's options with Math.random() before saving, so the on screen option order is neither the fixture's authored order nor stable across runs. Every option in quiz.ts and in this journey is located by its exact text through correctOptionFor and wrongOptionFor, never by position or letter, and the next person to touch this file should keep it that way rather than reintroducing a positional locator that passes locally and flakes in CI.

The journey is split into two tests on purpose. Generation is genuinely server side truth, the model is called and the chapter is saved to disk, but server/routes/generation.ts's pipeHubToSse closes the SSE reply with zero bytes almost immediately because it listens on the incoming request's own close event rather than the response's, so the browser never learns the chapter finished. This was confirmed as a real production bug against the actual server with a plain fetch and no browser involved, filed as github.com//issues/50 and out of scope for this phase. The first test proves the full adaptive loop through the model's recorded prompt and never waits on anything the page renders. The second test is quarantined with test.fixme and asserts only that chapter 2's prose becomes visible, ready to flip back to a normal test once issue 50 is fixed.

e2e/support/journeys/quiz.ts is a new page object for the quiz panel and the feedback form. Its two feedback textareas are located by placeholder rather than by label, because FeedbackForm.tsx's labels sit as plain siblings of their textareas with no for, id, or aria label connecting them, which is a separate accessibility gap reported here rather than patched, since this phase changes no production file.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
…r intact

Journey (h) adds e2e/support/journeys/generation-failure.ts and
e2e/journeys/generation-failure.spec.ts. The support module exports one
case shape, GenerationFailureCase, with a name, the value the scripted
model throws, and the text the reader must see, plus two runners that
each drive a full journey against a fixed screen and fixed navigation.
Phase 7 adds coverage for its typed AI error taxonomy by appending an
object literal to the CASES array in the spec file, and nothing in the
support module needs to change for that.

The negative assertion, that the surface's own generic fallback text is
not visible, is the load-bearing half of this journey. A test that only
checks that some error appeared would still pass if the app flattened
every thrown error into the same boilerplate line, so the case only
proves the message survives intact once it also proves the fallback
never rendered in its place.

The reader path, runGenerationFailureCase, drives a seeded book through
its quiz and feedback screens to fail chapter 3 and land on
GenerationPanel. While building this journey I found and lead-p6
independently confirmed that POST /api/books/:id/generate-next never
delivers any SSE event to a real browser, success or failure, because
pipeHubToSse in server/routes/generation.ts closes the reply from a
request.raw close listener that fires as soon as the request body is
read, not only on client disconnect. That is filed as issue #50 and is
out of scope for this phase, so its test case is test.fixme rather than
test, with the runner left intact and correct for a one line flip back
once the fix lands.

The wizard path, runWizardChapterOneFailureCase, is unaffected because
POST /api/books/:id/start answers through a different helper that never
attaches that listener, and it is what proves journey (h)'s claim today.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
Journey (h) ran only its first case through the creation wizard, and left
the typed error subclass to the reader surface, which is quarantined behind
issue 50. That meant the phase never actually demonstrated the one property
Phase 7 most needs from this harness, that a typed error subclass reaches
the screen with its own message intact rather than flattened into
boilerplate, because the only test making that claim was skipped.

Both cases now run through the wizard, which uses openSseStream and is
unaffected by the bug. The quarantined reader cases stay exactly as they
are, so nothing is lost when they come back.

Claude-Session: https://claude.ai/code/session_01FSkESkJTRpt3Ye2cM7c5n5
@rsml
rsml merged commit 12247a8 into master Jul 21, 2026
3 checks passed
@rsml
rsml deleted the phase-6-e2e-journeys branch July 21, 2026 14:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant