feat: select text on an original page, and act on it - #150
Merged
Conversation
The page was a picture, so nothing on it could be selected — which left explain and note-taking as reflowed-view-only, since both begin with a selection. There is a layer of invisible words over the paper now, one span per word, positioned in PDF points and scaled with the sheet so a zoom moves it rather than rebuilding it. It is built from the boxes the placement already used rather than from pdf.js's own text layer, which means each word arrives knowing its passage and its position within it — so a selection is read by asking its endpoints what they are, and the reader's existing "play from here", "highlight", "note" and "explain" paths take it exactly as they take a selection made in the reflowed view. Words a passage claimed only as a region — a table row, placed by its cells rather than by the words of its reading — are attributed to it too. Without that the title page of a paper, which is mostly tables, was the one part that could not be selected into anything: 351 of 391 words on it knew their passage, now 389. The selection actions and the explain panel are snippets rendered inside whichever view is showing, so they sit in that view's scroll and travel with it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…a click put it away The invisible text layer is drawn transparent so the page's own picture of the words is what the reader sees. A selection overrides that: the browser paints selected text in its own colour, so every selected word appeared twice — once as printed, once as a slightly-off ghost beside it. The ghost was beside rather than on top because the spans were sized by their boxes but rendered at whatever width the fallback font gave them, so `overflow: hidden` clipped some and let others run long. Both are now fixed at once: each word is measured in the family the layer actually uses and squeezed onto its box with `scaleX`, and `::selection` keeps the highlight while holding the text transparent. Putting a selection away was also close to impossible. Pressing on a blank part of a page — a margin, a figure, the space between paragraphs — places no caret, because there is no text node under the pointer, so the browser leaves the old selection standing and the actions with it. And a selection made anywhere else on the screen was ignored rather than treated as the page's selection ending, which stranded the actions on the paper. The selection is now watched on the document, an outside selection clears it, and a press on blank paper drops it by hand. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The browser reports a new selection on every frame of a drag, and each one was read back and offered: the actions re-anchored to wherever the pointer had reached, flashing along under it for the length of the gesture. A drag is one gesture, so it is now read once. Nothing is offered while a pointer is down, the press itself takes the previous selection's actions away, and the release is what brings them back — at the place the reader stopped. A press on the actions is left alone, or taking them away underneath the pointer would mean the button was never clicked. Keyboard selection is unaffected: with no pointer down, each change is still read as it happens. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What changed
Text on an original page can be selected now, and everything that begins with a selection works from it: play, explain, highlight, note.
Before this the page was a picture. Highlights and notes made in the reflowed view displayed on it, but nothing could be created there, because both explain and annotation start from a selection and there was nothing to select.
How
Rather than laying pdf.js's own text layer over the canvas and then working out what a selection means, the layer is built from the boxes the placement already used — one invisible span per word, positioned in PDF points and scaled with the sheet, so a zoom moves it instead of rebuilding it.
The payoff is that each word arrives already knowing its passage and its position within it. Reading a selection is then a matter of asking its endpoints what they are, and the result is the same shape the reflowed view produces — so
playSelectedPassage,annotateSelectionandopenExplainBoxtake it unchanged.Words a passage claimed only as a region are attributed to it too. A table row is placed by its cells rather than by the words of its spoken reading, so it carries rectangles but no per-word boxes; without this the title page of a paper — which is mostly tables — was the one part that could not be selected into anything. On "Attention Is All You Need" page 1 that is 351 of 391 words knowing their passage, now 389.
The selection actions and the explain panel became snippets rendered inside whichever view is showing, so they sit in that view's scroll and travel with the page.
Two small things worth knowing, both caught by testing rather than reasoning: the highlight overlay had to stop taking pointer events, or it stood between the words and the pointer trying to select them; and the spaces and newlines that make a copied selection read properly are the only children in normal flow, so at any inherited size they stacked up as a visible column of blank lines down the corner of the page.
Verification
npm run lintnpm run checknpm run test:coverage— 529 unit tests;domain/pdf-layout.tsat 96.5%The new e2e drags across two words on a rendered page and asserts: the selection contains the expected word, the actions appear inside the page's own scroll, highlighting leaves a mark on the paper, and playing the selection starts playback.
Review checklist
Known limits
🤖 Generated with Claude Code