Conversation
The renderer took each DOM text node through `DOMText.text()`, which collapses whitespace for *extraction*: it trims the node and then restores a boundary space only when the original began or ended with a literal space. A Markdown soft break reaches the renderer as a bare newline, so `<p>Wort A\n<strong>fett</strong>` handed layout "Wort A" and the word separator was gone — "Wort Afett". Between two plain-text runs nothing showed, because the newline sat inside one node and collapsed normally; only a node boundary lost it, which is why every inline type was affected. Give layout the node's source text instead and let the existing CSS whitespace handling do the work: `collectInline` collapses per `white-space` and emits one space token per run, and the line box already ignores a space at a line's start and drops a pending one at its end. A hard break is a `<br>` element, not whitespace, so it is untouched. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three more instances of the defect behind the soft-break fix, each found by comparing the swift engine against webkit on the same input. A whitespace-only text node was dropped by the parser inside a fixed list of containers. Between block children that run is insignificant, but between two inline siblings — `<a>x</a>\n<a>y</a>` in a `<div>` — it is the only thing separating the words, and dropping it at parse time left no consumer able to recover it: a footer link row rendered "ImpressumDatenschutzAGB". The node is now kept, and the decision moves to the two places that reassemble the nodes and can see both sides of it. The box tree already trimmed an inline run's edges; plain-text assembly now drops a whitespace-only child only where a block boundary is adjacent, which also stops `<section>`, `<article>`, `<main>` and friends from leaking a space that `<div>` never leaked. `DOMText.text()` restored a boundary space only when the source spelled it with a literal space, so extraction lost the same separator the renderer did: `swifttext html` printed "Wort Afett danach.". It now collapses each run of whitespace to one space and keeps the boundaries, with the block trimming above taking care of the edges. `collectInline` preserved spaces verbatim only for `white-space: pre`. Under `pre-wrap` they became `.space` tokens, of which a line keeps at most one — and `pre-wrap` is exactly what the Markdown stylesheet gives a fenced code block, so every code block in a Markdown-to-PDF render came out flattened against the left margin. `pre-wrap` now carries its spaces inside word tokens, split at each space/non-space transition so lines can still wrap. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`swifttext ocr --markdown` wrote every bullet twice — `- • Punkt eins`. The markdown path composes blocks from the page's own text lines, keeping only the regions from Vision's segmentation. A segmented list item arrives with its marker already stripped, but `TextLineSemanticComposer` then replaces that text with lines read from the page, where the marker is a painted glyph like any other character — and Markdown adds a marker of its own on top. The composer now strips the marker from the first line of each item it re-sources, sharing one implementation with the extraction path's `cleanListItemText`. Recognition stays narrow, because a list item may well begin with something marker-shaped: a bullet glyph is unambiguous, but a hyphen, dash, asterisk or ordinal counts only when a space follows, which keeps "E-Mail schreiben" and "1.5 Millionen Euro" intact. That also fixes the extraction path's old `^[0-9]+[.)\s]+`, which ate the opening of any item starting with a bare number — "2026 war das Jahr" became "war das Jahr". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
`swifttext ocr --markdown` flattened every document it read: `# Titel` came back as body text and `**fett**` as `fett`. The information was never missing, only dropped — the reader took `page.string`, a plain String. `page.string` and `page.attributedString` are indexed alike, so the fonts were one lookup away from the ranges the fragment reader already had. A text fragment now carries the runs its characters are set in — size, bold, italic, monospaced — read through the font rather than parsed out of its name, since a family like `Helvetica-Bold` is one member, not Helvetica with a weight. The runs travel with the text through the composer and into the blocks. A page never says "this is a heading", it says "this is 22pt", so the levels are read from the document as a whole: body text is whichever size sets the most characters — on the sample, 327 against 17, 13 and 13 — and each distinctly larger size is a level, deepest-first. Levels therefore follow a document's own scale rather than a table of point sizes this code cannot know, and a single large word inside a sentence stays emphasis, because a block counts as a heading only when all of it is set that way. A stylesheet often stops scaling before `h4`, leaving it bold body text and, in the page, identical to a bold sentence. Those are separated by shape, on Oliver's rule: a heading is a label, not a statement, so a short standalone bold line that does not end in sentence punctuation becomes one. "Vierte Ebene" is a heading; "Ein komplett fetter Absatz." is not. Emphasis moves whitespace outside its delimiters — a page hands over "wichtig ", and `**wichtig **` is not emphasis at all — and a heading's own weight is written as the level, not doubled as bold. OCR reports characters and geometry but no font, so a page read that way has no runs and comes out exactly as before. Table cells are still written plain. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A line is split into fragments for two unrelated reasons, and `combinedText` rejoined both with a tab. A gap wide enough to be structural — between two table columns — is worth a tab. But the OCR reader also splits a line into its individual words, so each gets its own bounds, and those pieces meet exactly where one space stood. Every OCR'd heading came out as `Zweite\tEbene`. The gap itself tells them apart, since the word splitter cuts at a gap's midpoint and leaves the pieces touching. A gap past roughly half the type size is one someone put there on purpose and keeps its tab; anything narrower was a word space and rejoins as one. The threshold is relative to the type size, so a layout reads the same at any scale, and it is the same order of magnitude as the one the PDF reader already uses to decide where to split at all — the two existing column tests still get their tab. Also removes leftover debugging that wrote to stdout in debug builds: the two prints in the OCR word splitter, and a dump in the PDF reader guarded by a hardcoded supermarket receipt string, along with the character-bounds logger that only it called. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Preserving spaces under `pre-wrap` pushed Layout.swift to 1208 lines, past SwiftLint's 1200-line limit, so CI failed on the one file-length violation. Raising the limit would have been the wrong way to green it. The file had a seam already: block and table layout, then a `// MARK: - Inline layout` extension holding the whole inline pass — the token stream, the line packing, `white-space`, font fallback, bidi reordering and text decoration. That extension moves to InlineLayout.swift, leaving 768 lines behind and 452 in the new file. Members keep the access they had. Only what block layout genuinely reaches for widens from private to internal: `InlineToken`, because sizing a shrink-to-fit box measures a token stream; `layoutInline` and `collectInline`, which block layout calls; and the engine's `fonts`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Started from one reported bug — a Markdown soft break losing its space next to an inline element — and followed the same defect class through the codebase. Each finding was confirmed by comparing the
swiftengine againstwebkiton identical input, or by round-tripping Markdown through PDF and back.Rendering: whitespace that separates words
Soft break next to an inline element dropped its space (the reported bug).
StyledElementfed layoutDOMText.text(), which restores a boundary space only when the source spells it with a literal" "— never\n. So<p>Wort A\n<strong>fett</strong>reached layout as"Wort A"and renderedWort Afett. Layout now gets the node's source text and the existing CSS whitespace handling does the work. A real document line renderedFN 477766yStand: 16.09.2026.Whitespace-only nodes between inline siblings were dropped by the parser. Inside a fixed container list (
div,blockquote,body, …) the parser discarded whitespace-only text. Between block children that's right; between two inline siblings it destroys the only word separator, so a footer link row renderedImpressumDatenschutzAGBagainst webkit'sImpressum Datenschutz AGB. The node is kept and the decision moved to the two places that reassemble the nodes and can see both sides of it.white-space: pre-wrapcollapsed runs of spaces. Spaces were preserved verbatim only forpre; underpre-wrapthey became collapsible separators, of which a line keeps at most one. The Markdown stylesheet setspre { white-space: pre-wrap }, so every fenced code block in a Markdown→PDF render was flattened against the left margin. Pre-existing, not a regression.Extraction lost the same separator —
swifttext htmlprintedWort Afett danach.— and<section>/<article>/<main>leaked a spurious space that<div>did not. Both came from the same expression; they now follow one rule.PDF → Markdown: recovering structure
Double bullets.
swifttext ocr --markdownwrote- • Punkt eins. The segmenter's items were already clean, but the composer re-sources item text from the page's own text layer, where the bullet is a painted glyph — and Markdown adds a marker of its own. Marker recognition is deliberately narrow, soE-Mail schreibenand1.5 Millionen Eurosurvive; that also fixes the old^[0-9]+[.)\s]+, which turned2026 war das Jahrintowar das Jahr.Headings and emphasis were dropped entirely. The information was never missing — the reader took
page.string, a plainString.page.stringandpage.attributedStringare index-aligned, so the fonts were one lookup away from ranges the fragment reader already had.A page never says "this is a heading", it says "this is 22pt". So body text is whichever size sets the most characters (327 against 17/13/13 on the sample), and each distinctly larger size becomes a level, deepest-first — levels follow a document's own scale rather than a table of point sizes. A block is a heading only when all of it is set that way, so one large word in a sentence stays emphasis.
Where a stylesheet stops scaling —
h4is bold body text, identical in the page to a bold sentence — the two are separated by shape, per @Cocoanetics' rule: a heading is a label, not a statement. A short standalone bold line that doesn't end in sentence punctuation becomes one.Vierte Ebeneis a heading;Ein komplett fetter Absatz.is not.A realistic document now round-trips identical to its source, headings, bold, italic, inline code and lists included.
Tabs inside OCR'd text.
combinedTextrejoined every fragment pair with a tab, but fragments are split for two unrelated reasons — a structural column gap, and the words the OCR reader separates to give each its own bounds. Every OCR'd heading came outZweite\tEbene. The gap width now tells them apart.Also removes leftover debugging that wrote to stdout in debug builds, including a dump gated on a hardcoded supermarket receipt string.
Scope and limits
h4+ set at body size rely on the shape heuristic — nothing in the page distinguishes them from a bold sentence.Verification
pdftotextnormalises runs of spaces and can't answer these questions.🤖 Generated with Claude Code