Skip to content

feat: select text on an original page, and act on it - #150

Merged
NeoVand merged 3 commits into
mainfrom
feat/pdf-text-selection
Aug 27, 2026
Merged

feat: select text on an original page, and act on it#150
NeoVand merged 3 commits into
mainfrom
feat/pdf-text-selection

Conversation

@NeoVand

@NeoVand NeoVand commented Aug 27, 2026

Copy link
Copy Markdown
Owner

What changed

Text on an original page can be selected now, and everything that begins with a selection works from it: play, explain, highlight, note.

Before this the page was a picture. Highlights and notes made in the reflowed view displayed on it, but nothing could be created there, because both explain and annotation start from a selection and there was nothing to select.

How

Rather than laying pdf.js's own text layer over the canvas and then working out what a selection means, the layer is built from the boxes the placement already used — one invisible span per word, positioned in PDF points and scaled with the sheet, so a zoom moves it instead of rebuilding it.

The payoff is that each word arrives already knowing its passage and its position within it. Reading a selection is then a matter of asking its endpoints what they are, and the result is the same shape the reflowed view produces — so playSelectedPassage, annotateSelection and openExplainBox take it unchanged.

Words a passage claimed only as a region are attributed to it too. A table row is placed by its cells rather than by the words of its spoken reading, so it carries rectangles but no per-word boxes; without this the title page of a paper — which is mostly tables — was the one part that could not be selected into anything. On "Attention Is All You Need" page 1 that is 351 of 391 words knowing their passage, now 389.

The selection actions and the explain panel became snippets rendered inside whichever view is showing, so they sit in that view's scroll and travel with the page.

Two small things worth knowing, both caught by testing rather than reasoning: the highlight overlay had to stop taking pointer events, or it stood between the words and the pointer trying to select them; and the spaces and newlines that make a copied selection read properly are the only children in normal flow, so at any inherited size they stacked up as a visible column of blank lines down the corner of the page.

Verification

  • npm run lint
  • npm run check
  • npm run test:coverage — 529 unit tests; domain/pdf-layout.ts at 96.5%
  • Relevant Chromium/visual journey — 47 e2e including the 12 committed visual baselines, unchanged

The new e2e drags across two words on a rendered page and asserts: the selection contains the expected word, the actions appear inside the page's own scroll, highlighting leaves a mark on the paper, and playing the selection starts playback.

Review checklist

  • Privacy and network behavior are unchanged. No new data leaves the device; the word layer is built from geometry already computed locally.
  • Storage/migration. No schema change. Annotations created from the page use the existing anchors and are indistinguishable from ones made in the reflowed view — they display in both.
  • Keyboard, screen-reader, contrast, reduced-motion. The word layer is inert to assistive tech beyond being selectable text; the actions are the same buttons, with the same labels, as the reflowed view. Contrast and motion are unchanged.
  • Visual changes include desktop screenshots — captured (selection with the action bar, a highlight created from the page) but not attached; the CLI cannot upload them.

Known limits

  • A word selected inside a region-placed passage resolves to that passage's start rather than to the word — right for playing and annotating a table row, coarser than the per-word precision prose gets.
  • Annotation markers (the margin buttons) still render only in the reflowed view; the highlight itself shows in both.

🤖 Generated with Claude Code

NeoVand and others added 3 commits August 27, 2026 02:27
The page was a picture, so nothing on it could be selected — which left
explain and note-taking as reflowed-view-only, since both begin with a
selection.

There is a layer of invisible words over the paper now, one span per word,
positioned in PDF points and scaled with the sheet so a zoom moves it
rather than rebuilding it. It is built from the boxes the placement
already used rather than from pdf.js's own text layer, which means each
word arrives knowing its passage and its position within it — so a
selection is read by asking its endpoints what they are, and the reader's
existing "play from here", "highlight", "note" and "explain" paths take it
exactly as they take a selection made in the reflowed view.

Words a passage claimed only as a region — a table row, placed by its
cells rather than by the words of its reading — are attributed to it too.
Without that the title page of a paper, which is mostly tables, was the
one part that could not be selected into anything: 351 of 391 words on it
knew their passage, now 389.

The selection actions and the explain panel are snippets rendered inside
whichever view is showing, so they sit in that view's scroll and travel
with it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…a click put it away

The invisible text layer is drawn transparent so the page's own picture of
the words is what the reader sees. A selection overrides that: the browser
paints selected text in its own colour, so every selected word appeared
twice — once as printed, once as a slightly-off ghost beside it. The ghost
was beside rather than on top because the spans were sized by their boxes
but rendered at whatever width the fallback font gave them, so `overflow:
hidden` clipped some and let others run long. Both are now fixed at once:
each word is measured in the family the layer actually uses and squeezed
onto its box with `scaleX`, and `::selection` keeps the highlight while
holding the text transparent.

Putting a selection away was also close to impossible. Pressing on a blank
part of a page — a margin, a figure, the space between paragraphs — places
no caret, because there is no text node under the pointer, so the browser
leaves the old selection standing and the actions with it. And a selection
made anywhere else on the screen was ignored rather than treated as the
page's selection ending, which stranded the actions on the paper. The
selection is now watched on the document, an outside selection clears it,
and a press on blank paper drops it by hand.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The browser reports a new selection on every frame of a drag, and each one
was read back and offered: the actions re-anchored to wherever the pointer
had reached, flashing along under it for the length of the gesture. A drag
is one gesture, so it is now read once. Nothing is offered while a pointer
is down, the press itself takes the previous selection's actions away, and
the release is what brings them back — at the place the reader stopped.

A press on the actions is left alone, or taking them away underneath the
pointer would mean the button was never clicked. Keyboard selection is
unaffected: with no pointer down, each change is still read as it happens.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@NeoVand
NeoVand merged commit 260faa1 into main Aug 27, 2026
5 checks passed
@NeoVand
NeoVand deleted the feat/pdf-text-selection branch August 27, 2026 21:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant