Skip to content

feat(layout): break lines within CJK runs and support word-break - #134

Open
danperks wants to merge 1 commit into
lucid-softworks:mainfrom
danperks:feat/cjk-line-breaking
Open

danperks wants to merge 1 commit into
lucid-softworks:mainfrom
danperks:feat/cjk-line-breaking

Conversation

@danperks

Copy link
Copy Markdown

What & why

Inline layout treated the text between breaking spaces as one atomic unit, so a line could only ever
be broken where the author wrote a space. That is fine for Latin prose and wrong for scripts that
don't use spaces: a Japanese or Chinese paragraph was a single unbreakable "word", so it never
wrapped, overflowed its container, and laid out one line tall where it should have been many.

This adds a UAX #14 subset covering the CJK case:

  • a break is allowed between two ideographic characters (LB18/LB999),
  • never before a closing bracket, sentence-ending mark, or non-starter such as a small kana or a
    prolonged sound mark (LB13/LB16/LB19) — those may not begin a line,
  • never after an opening bracket (LB14) — it may not end a line.

Hangul is deliberately excluded: Korean is space-separated and UAX #14 only permits breaking between
syllables under an explicit line-break value, so treating it like Han would wrap Korean mid-word.

How it fits the existing line-filling loop

Rather than reworking the loop, a break opportunity inside a word is expressed by emitting several
InlineItem::Words from one source word, which the existing wrapping logic already handles. Because
an ideographic break is an opportunity and not a separator, the pieces are rejoined with nothing
between them — leads_space is set on the first piece only, and it drives both the width measured
during line filling and the rejoin, so a run that stays on one line paints byte-identically to
before. The same segmentation is applied in collect_inline_words, so min-content width agrees with
what can actually be placed on a line (grid, flex and table sizing depend on that).

word-break

The property was previously unparsed, and it has to be honoured here because normal, break-all
and keep-all select between three different sets of opportunities. The pair prohibitions apply to
all three — no value of word-break may strand a full stop at the start of a line.

keep-all suppresses the letter-to-letter opportunities but not the one after
U+3000 IDEOGRAPHIC SPACE: that character is a space rather than a typographic letter unit despite
sitting in the CJK punctuation block, and it is the only thing that lets keep-all text wrap at all.
WPT word-break-keep-all-005 covers exactly this.

Measured impact

Matched A/B over css/css-text (1,964 tests), same harness and same machine, only the code differing:

tests
Baseline PASS 353 (18.0%)
With this change 456 (23.2%)
Newly passing 105
Newly failing 2
Net +103

I also ran the same directory twice on the identical post-change build to establish a noise floor:
±1 test, and the one unstable test is in the shaping family below. So +103 is signal rather
than run-to-run variance.

The two newly-failing tests

shaping/shaping-024 and shaping/shaping-025 are Mongolian shaping tests that download
NotoSansMongolian-regular.woff2. They are flaky independently of this change: running
css/css-text/shaping three times on one unchanged build gives

shaping-023.html  FAIL, FAIL, PASS
shaping-025.html  FAIL, PASS, FAIL

shaping-024 fails in all three of those runs and passed once pre-change. Mongolian (U+1800–U+18AF)
is not in the ideographic set, and with default word-break this change is behaviour-identical for
non-CJK text, which neither test declares.

Breadth check, and a pre-existing bug this exposes

I also ran a matched A/B over css/css-writing-modes css/css-text-decor css/css-flexbox css/css-grid
(5,295 tests) because the change touches the shared inline path including vertical writing mode. Net
−3, all three in css-grid/grid-items/grid-item-block-axis-content-contribution-00{1,2,3}.

Those are worth explaining, because they are not a fault in this change. They style the item
font: 10px/1 Ahem and rely on 15 X glyphs measuring exactly 150px in a 150px box. The CSS
font shorthand is not implemented
, so neither font-size nor font-family is applied and the
text is measured at the default 16px in the fallback face. Measured advance per character:

declaration advance
font-family: Ahem; font-size: 10px 10.00px correct
font: 10px Ahem 10.70px wrong
font: 10px/1 Ahem 10.70px wrong
font: 10px/1 "Ahem" 10.70px wrong

At 10.70px the 15 glyphs need 160px and genuinely do not fit the 150px box. Previously break-all
was ignored, so the run could not wrap and stayed one line, coincidentally matching the reference.
Now that the opportunities exist the engine correctly wraps overflowing text — the new behaviour is
right and the wrong input is the font metrics. Respelling that one rule with longhands makes the item
lay out on a single line as the test intends.

I've left that out of this PR to keep it focused, but it looks high leverage: 6,573 of 49,011 files
under css/ (13.4%) use the font shorthand, most of them precisely to pin down Ahem metrics. Happy
to send it as a follow-up if you'd like it.

🤖 How this was built

  • Authored primarily with an LLM (which: Claude Opus 5, driven via Cursor)

Checklist

  • PR title is a Conventional Commit
  • cargo test --workspace passes — adds linebreak unit tests (pair prohibitions, keep-all,
    break-all, the U+3000 case), ideographic_segments_are_placed_without_gaps covering the
    rejoin, a_run_that_exactly_fills_the_line_does_not_wrap, and
    word_break_parses_and_inherits
  • cargo fmt --all + cargo clippy clean for this change (clippy reports pre-existing warnings
    in wurl, which this PR does not touch)
  • WPT report shows no conformance regressions — +103 net on css/css-text; the 2 remaining
    shaping failures are demonstrated flakiness and the 3 css-grid ones are the pre-existing
    font shorthand gap, both analysed above

Made with Cursor

Inline layout treated the text between breaking spaces as one atomic unit, so a
line could only be broken where the author wrote a space. Japanese and Chinese
are written without spaces, which made a whole paragraph a single unbreakable
word: it never wrapped, overflowed its container, and laid out one line tall
where it should have been many.

Add a UAX lucid-softworks#14 subset covering the CJK case. A break is allowed between two
ideographic characters, but never before a closing bracket, sentence-ending
mark, or non-starter such as a small kana, and never after an opening bracket,
so a line cannot begin or end on a character that may not sit there. Hangul is
excluded: Korean is space-separated and breaking between syllables needs an
explicit line-break value.

The break opportunities are expressed by emitting several words from one source
word, so the existing line-filling loop wraps at them with no changes. Because
an ideographic break is an opportunity rather than a separator, the words are
rejoined without one — the run paints exactly as if it were never split. The
same segmentation is applied when collecting words for intrinsic sizing, so
min-content width agrees with what can actually be placed on a line.

This needs word-break, which was previously unparsed, since normal, break-all
and keep-all select between three different sets of opportunities. The pair
prohibitions apply to all three: no value may strand a full stop at the start
of a line. keep-all suppresses the letter-to-letter opportunities but not the
one after U+3000 IDEOGRAPHIC SPACE, which is a space rather than a typographic
letter unit and is the only thing that lets keep-all text wrap at all.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant