feat: read a PDF as its own pages, with the narration painted on

[NeoVand]

What changed

A PDF can now be read as its own pages, with the narration painted onto the paper where it actually sits — down to the word.

The markdown stays underneath, unchanged. It is still what the voice, the assistant and the study layer read, so this is a choice of presentation, not a different import: every PDF has both views and switching is instant. A separate import path would have meant a PDF you can see but cannot listen to.

In the page view you get: the passage being spoken highlighted where it was printed, with the current word lit inside it; double-click any sentence to play from there, with a hover wash showing what a click will take; follow-along scrolling that yields the moment you scroll yourself; your highlights and margin notes; the assistant's pointer; the contents panel; and zoom.

How the two are reconciled

The reader's passages come from LiteParse's markdown; the page view draws the PDF. Joining them is the whole problem, and the bridge is that LiteParse can emit a box for every word it read (emitWordBoxes) from the same parse that produced the markdown — so both sides descend from the same glyphs.

Placement is a monotonic affine-gap alignment of a page's words against the passages anchored to it, plus a second pass inside whatever region a run of failures was bracketed into (that pass is what places table rows). Geometry is recomputed per page from the stored bytes with targetPages, ~16ms a page, rather than stored — exactly what the DocumentPageInfo comment anticipated.

Measured on real papers: 91% of passages placed on "Attention Is All You Need", 92% on "Deep Residual Learning" (two-column), with 80–83% of words boxed individually. The remainder is almost entirely synthesized narration for degenerate tables, which never appears on the page in that form.

Two findings worth recording, both from real papers rather than intuition:

• textItems come back in raster order while the markdown is in reading order. On a two-column page the boxes interleave the columns line by line, and a monotonic alignment finds almost nothing — 57% placed. A recursive XY-cut that splits at vertical gutters before horizontal ones recovers it: 57% → 92%.
• pdf.js display renders never finish in a hidden tab (they continue via requestAnimationFrame). Anything pdf.js-rendered has to be verified under Playwright, not a preview pane.

Themes

The original pages are printed onto the reader's own paper. A PDF page being white with black ink on it is a fact about the paper it was made for, not the room it is read in — and the reading view has always drawn the same document on the theme's paper in the theme's ink, so the page view now uses those very colours, read off the stack as --reader and --reader-ink-strong.

It is one duotone ramp: white → paper, black → ink, greys proportional. Under Sunny that is a barely visible warming; under Cocoa the page is warm dark brown with cream type; under Midnight a full inversion. Light and dark are the ramp's endpoints rather than separate code paths.

Colour is left alone — remapping it would be recolouring the author's work rather than the paper under it — and photographs and rendered figures are lifted out by their operator-list rects and put back whole, so a figure keeps its own background on a dark page. 41ms for a retina page.

Drawing, which is where this actually breaks

A paper with a 163,000-operator figure on one page taught most of these. pdf.js renders one page at a time, so a slow page can take the view down with it:

• A page already being drawn is never queued again; a page the reader has left is abandoned mid-draw (RenderTask.cancel); draws go nearest-the-reader-first.
• Pages are painted off-screen and handed over whole, so a canvas keeps showing what it has until the next picture is finished — no clear-then-fill flash on redraw, and no strobe while a zoom slider is dragged.
• Bitmaps are released against a pixel budget rather than the moment a page leaves view. A canvas is sized when its draw begins, so an abandoned draw leaves a full-size bitmap with nothing recorded against it; budgeting by the record let those reach ~300 MB after a few sweeps at retina scale. Budgeting by the pixels holds it at ~134 MB.

Verification

• npm run lint
• npm run check
• npm run test:coverage — 522 unit tests; domain/pdf-layout.ts and domain/page-tone.ts added to the coverage include list at 96% and 98%
• Relevant Chromium/visual journey — 46 e2e including all 12 committed visual baselines (unchanged)

New e2e coverage, each checked to fail against the code it guards: placement lands on the printed line rather than where reflowed text would put it; reading ahead survives pages placing underneath it; no bitmap is released across three sweeps of a six-page document; the drawing tracks its sheet through a zoom drag (360px of lag before, ≤2px after); the drawn paper samples equal to the theme's --reader.

Review checklist

• Privacy and network behavior are unchanged. Nothing new leaves the device; the page geometry and the pixels both come from the PDF already in local storage.
• Storage/migration. No schema change and no migration: geometry is recomputed per page on demand and never persisted, which is what DocumentPageInfo's comment reserved. Documents written before this read identically.
• Keyboard, screen-reader, contrast, reduced-motion. The stack takes focus so a keyboard can scroll it (WCAG 2.1.1), with an axe pass over the view in the suite. Follow-scrolling and mark transitions honour prefers-reduced-motion. Contrast is the theme's own ink-on-paper by construction.
• Visual changes include desktop screenshots — captured during development (page view with a passage highlighted, two-column placement, a figure at 150%, the page under Cocoa/Sakura/Midnight) but not yet attached here; the CLI cannot upload them. Happy to drop them in.

Known limits, stated plainly

• No text selection on the page yet, so select-to-play and select-to-annotate remain reading-view only. Existing annotations display in the page view; they just cannot be created there. A pdf.js text layer is the fix.
• ~9% of passages do not place — degenerate tables and one- or two-token fragments. Those simply do not highlight; everything else on the page still does.
• Two pre-existing quality problems were measured while working on this and are not addressed here: LiteParse downsamples embedded images about 7× linearly (the Transformer figure is 1520×2239 in the PDF and arrives as 219×323), which affects the reading view only; and multi-row table headers flatten badly. Both deserve their own change.

🤖 Generated with Claude Code