Repository navigation
Bound the shaping caches' work and memory on hostile fonts - #301
Merged
Merged
Conversation
LazySubtables::has_cache counted the subtables that parse before an index by scanning from 0 on every call, and the walk asks for it at every class-based context subtable past the eighth for every glyph, so a lookup of N such subtables cost O(N^2) per glyph. A GSUB of 32,000 offsets to one ChainContextFormat2 took 7 s to shape "aaaa". The count now advances from where the last call left it and stops at the eighth subtable that parses, which fixes the cutoff for the rest of the application: each subtable is counted at most once per lookup application. A unit test counts subtable reads per visit. The walk also reads each subtable's slot once per visit (the accelerator check now goes through LazySubtables::get_admitted), a subtable that fails to parse takes no room in the parsed list, the list reserves room for every subtable once a walk has parsed eight (as parsing them all up front did, instead of copying at each doubling), and parsing goes through a Subtable trait instead of function pointers.
LookupAccel::build kept a 24-byte digest per subtable for every lookup index a run applied, even after both reading budgets were spent, so what a Font kept grew with lookup indices times subtables rather than with the table: a 66 KB GSUB of 500 indices naming one lookup of 32,000 subtables kept 375 MB, and lookups that overlap in the table can make a 96 KB GSUB keep 1.5 GB. - Each digest kept costs one entry of the table's budget (twice the table's length plus 64 Ki), taken before any coverage is read. A lookup the budget cannot pay for keeps no digest and admits every glyph, which only means less skipping. - Lookup indices with the same LookupList offset share one accelerator. Fonts compilers write list each lookup once at increasing offsets, which needs no sorting to tell. - A lookup whose subtables all have the lookup's digest (one subtable, or many sharing a coverage) keeps only that: the walk checks the lookup's digest before any subtable, so theirs would rule nothing more out. tests/hostile_layout_caches.rs builds such tables on Open Sans and checks the bytes a used Font keeps and that it shapes as a fresh Font.
Once a table's 16 map slots held other (script, language, FeatureVariations record) keys, every feature query resolved the whole language system into a map and dropped it, more work per query than the walk the cache replaces, and a font shaped in many languages did that for most of its queries. Past the slots, a query now walks the language system, as it does without a cache. The first 16 keys a font is shaped with stay kept; a map and the walk answer alike, so which keys those are never shows in the output. The walk-equivalence test now gives each language its own cache, so every key it checks is a kept map.
STABILITY.md says Font::coords is no longer a const fn, next to the note that its slice now borrows the font. The crate docs, the README and a new "Supported targets" section of STABILITY.md say the core crate needs pointer-width atomics with compare-and-swap: the Font caches are built through them, and OwnedFace and Language already needed them through alloc::sync::Arc, so thumbv6m-none-eabi did not build before 0.24.0 either.
shape_samples shapes every sample text with one Font, whose caches are built from its second call on. Each text is now shaped with a fresh Font too, run by run for bidi text, and the two outputs must be equal, so a fuzzed font that makes the caches change the output crashes the shape and shape_text targets.
A vertical run moves every glyph by half its horizontal advance, and at non-default coordinates that advance comes from HVAR, whose delta walks the variation store's regions per glyph. That walk ran for every glyph of every call: about half of a 100-character Hahmlet[wght] TTB call at wght 700, which made such runs slower than 0.23.0 although the vertical origins themselves were cached. The varied horizontal and vertical advances, from HVAR and VVAR as from phantom points, now share the font's two advance caches (a font has HVAR or not, so one instance only ever stores one kind), as HarfBuzz keeps an advance cache per font. The vertical origins and phantom-point advances now look in the font's cache before the call's own maps, so a warm call adds no map entries. Horizontal runs of fonts with HVAR get the cached advances too. The glyph caches stay at three of at most 4096 entries each.
A lookup whose subtables all have the lookup's digest keeps none of theirs, but its subtables were charged before that was known; the docs now say so.
A lookup whose subtables keep no digests of their own (all of them have the lookup's) admits every subtable at a glyph the walk stopped at, so the walk no longer asks the accelerator per subtable there. An inline slot past the last subtable reads as a subtable that does not parse, so a slot lookup needs no bounds check besides the array's own, and the cache rank check is inlined.
A Font's first call reads each applied lookup's subtable coverages up front, so a hostile lookup of 32,000 subtables read 32,000 coverages per application on top of parsing them. The first 1,024 are read now; the subtables past them admit every glyph and are parsed when a glyph reaches them, as every subtable was before the accelerators. Real lookups stay far below the cap (Noto Sans KR's ccmp has 268).
Once eight subtables of a lookup had parsed, the list reserved room for every subtable of the lookup. Each nested lookup application holds its own list, so a lookup of 32,000 subtables that applies itself again held room for all of them at every level: 88 MiB on the #301 review's font, where the walk parsed ten per level. The reservation is now at most 56 more entries; past it the list grows by doubling. Also say in the OnceBox docs that the lookup accelerators can keep different digests depending on which thread builds them first, while shaping output does not change.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #297
Bounds the work and memory of the per-
Fontshaping caches that #295 added, on the hostile tables the #295 review built and five more, and finds the per-glyph work that kept long vertical runs slower than 0.23.0. Shaping output is byte-identical to main everywhere (see Verification).What changed
1. Subtable cache rank (the blocker),
src/shape/lazy.rsLazySubtables::has_cachecounted the subtables that parse before an index by scanning from 0, at every class-based context subtable past the eighth, for every glyph: O(N²) per glyph in the lookup's subtable count. It now keeps its count across calls, advancing from where the last call stopped and stopping at the eighth subtable that parses, which fixes the cutoff for the rest of the lookup application. Each subtable is counted at most once per application, so a visit costs O(1) amortized.A unit test counts subtable reads: four glyphs over 6,000 class-based subtables now take at most 30,000 reads; the old scan took 48,039,904 (checked by reverting the fix). Another test checks the rank against a fresh scan in three visiting orders.
The walk also got cheaper per subtable visit, which the 32,000-subtable fonts below need to stay near 0.23.1:
get_admitted), and a slot past the last subtable reads as one that does not parse, so a visit does one bounds check;Subtabletrait instead of function pointers;2. Accelerator memory,
src/tables/layout/accel.rsu16map per index.DirectAccel); subtables past them admit every glyph and are parsed when reached, as every subtable was before Cut per-call shaping cost with per-Font accelerators, plans and metric caches #295. Real lookups stay far below (Noto Sans KR'sccmphas 268).What a
Fontkeeps per table is now one slot pointer per lookup, a 40-byte header per lookup applied, and 24 bytes per kept digest, for at most 2 x table length + 64 Ki digests.tests/hostile_layout_caches.rsbuilds hostile GSUB and GPOS tables on Open Sans and checks the bytes a usedFontkeeps (with a per-thread counting allocator) and that it shapes as a freshFont; on main its two memory tests fail (3.1 MB kept for a 4.3 KB GSUB; 19.2 MB kept against a bound of 2.0 MB).3. Vertical long runs,
src/shape/position.rs,src/font/cache.rsProfiled with in-process laps around the pipeline stages (a copy of the crate with
Instantlaps, one sigilbuzz per binary), Hahmlet[wght] at wght 700, 20 texts of 100 Hangul syllables, TTB, warmFont:subtract_vertical_origins)h_advance, per glyphv_origin, per glyph(The per-glyph laps are a separate build; each includes about 40 ns of lap overhead.) A vertical run moves every glyph by half its horizontal advance, and at non-default coordinates that advance comes from
HVAR, whose delta walks the variation store's regions. That walk ran for every glyph of every call; the vertical origins were cached, but each glyph's first lookup in a call also inserted it into the call's ownBTreeMap.HVARandVVARas from phantom points, share the font's two existing advance caches (a font hasHVARor not, so one instance only ever stores one kind). Still three caches of at most 4096 four-byte entries,Send + Sync, single-word atomic entries.HVARfonts gain too (Hahmlet LTR 100: 39.8 to 22.3 us; Source Serif 4 VF wght 700 LTR 100: 94.1 to 62.2 us).src/face/outlines.rsneeded changing.4. Feature maps past the cap,
src/ot/layout_select.rsPast 16 (script, language, FeatureVariations record) keys,
FeatureMaps::getreturnsNoneand the query walks the language system, as without a cache, instead of resolving the whole language system into a map per query. The first 16 keys a font is shaped with stay kept; a map and the walk answer alike, so which keys those are never shows in the output.5 to 7
docs/STABILITY.md:Font::coordsis no longer aconst fn, next to the lifetime note.docs/STABILITY.md: the core crate needs pointer-width atomics with compare-and-swap (cfg(target_has_atomic = "ptr")). Checked:cargo build -p sigilbuzz --target thumbv6m-none-eabi --no-default-featuresfails on this branch, and on 0.23.1 (fce2d02) throughalloc::syncinlanguage/mod.rsandowned.rs.fuzz/src/lib.rs:shape_samples(theshapeandshape_texttargets) shapes every text with the reusedFontand with a freshFont, bidi runs included, and asserts the glyphs are equal.Before and after
Measurement
One sigilbuzz per binary, as a user links it (crates.io 0.23.0, 0.23.1 at fce2d02, main at 8b04190, this branch),
lto = "thin",codegen-units = 1. Each run reports the median thread-cycle time of 11 batches of about 10 ms; the versions of a row run in alternation, and a table cell is the median over 5 runs (3 for first-call rows). Windows 11, i9-11900H, shared with other jobs at about 25% background load, so differences under about 4% are noise; rows that came out more than 3% slower than main were re-measured with 7 to 15 runs and were within noise. Times are microseconds per call unless marked.The #295 tables came from one binary that linked 0.23.0, 0.23.1 and the branch together. In such a binary main's Hahmlet TTB 100 run measured 40 us; in single-version binaries it measures 43 to 50 us with this harness and 88 to 107 us with the review's harness (wall clock over every call, the first ones included). This branch measures 20 to 31 us in all three, since warm calls no longer walk
HVARper glyph. The tables below are single-version only.Hostile layout tables (items 1 and 2)
Open Sans with its GSUB or GPOS replaced. The first four are the review's (
make_hostile.py,make_hostile2.py); the others are new (make_hostile3.py): format 1 contexts, GPOS chained context positioning in formats 2 and 3, 8,000 lookups at distinct offsets that overlap in a 96 KB GSUB (index sharing cannot help there, only the table budget), and 16 distinct lookups of 2,000 subtables whose digests differ (all kept, as they should be). Output is identical on 0.23.1, main and this branch for every font.Time per call, warm
Font(first call in parentheses). For the millisecond rows, 0.23.1 and the branch are the median of 9 alternating runs, and main is the range of single calls over several runs, which vary by 30% or more under the machine's load. The rows in seconds are the range of four single calls per version in one session.Heap, measured with a counting allocator over three calls on one
Font: the peak during the calls, and what theFontkeeps after them.The review's 65,535-index variant would have kept about 50 GB on main; on this branch its indices share one accelerator. The overlapping-lookups table is the case only the budget bounds: 6.3 MB kept for a 96 KB table, which is what the budget allows (24 bytes per digest for 2 x 96,058 + 64 Ki digests is 5.9 MiB, plus the 8,000 lookups' slots and headers).
Vertical runs against 0.23.0 (item 3)
Warm
Font, TTB, 20 texts per row. Hahmlet hasglyf,gvarandHVARbut novmtx; Noto Sans KR VF ttf hasvmtxandHVAR; the otf hasVORG,VVARandHVAR; Source Serif 4 VF is CFF2 withHVARand noVORG; Source Code Pro is static CFF withoutVORG.With the review's harness (wall clock, one font per spec built by
root.clone().with_coords, first calls included, median of 7 alternating processes), Hahmlet wght 700 TTB: 100 characters, 0.23.0 80.9 us, main 103.3 us, this branch 31.1 us; 8 characters, 14.0, 7.8 and 5.8 us. The review measured 64.8 and 88.7 us (100) and 11.28 and 6.71 us (8) for 0.23.0 and main.A font's first call is still slower than 0.23.0 for fonts without
VORG(Hahmlet TTB 100: 69.2 us on 0.23.0, 503.7 on main, 510.9 here; Source Serif 4 VF: 50.0, 153.4 and 160.6; Source Code Pro: 9.7, 23.4 and 24.4), because 0.24.0 takes HarfBuzz's vertical origins from glyph extents, which a first call computes per new glyph. This PR does not change that.40 languages on one Font (item 4)
The review's jobs: "office AVAT" in 12 or 40 languages, one
Font, per call.(A quieter earlier run gave the same ratios: Calibri 40 languages 13.84 to 7.98 us, Segoe UI 10.43 to 5.13 us.)
The #295 rows, main against this branch
No row is slower than main beyond noise. The tables were measured at 37108e3; 375ee8b changes only first calls of lookups with more than 1,024 subtables, and the first-call rows re-measured on it were within noise of main.
Repeated calls on one Font
A fresh Face and Font for every call
Verification
shape_cmp.pyfrom the Close variable-font metric parity gaps with HarfBuzz 14.5.0 #283 review as jobs files: 22 fonts, every axis setting, both directions), seeds 99 and 7: 190,070 jobs each. Each job shaped by main with a freshFont, and by this branch both with a freshFontand with oneFontper setting rebound from one root and shared by 4 threads in shuffled orders: 0 differ, 0 differences between threads.gen_jobs.py, seed 1717), the same three ways: 0 differ.Font, and the two agree.lazy.rs(the cache rank against a fresh scan in three orders; reads per visit),accel.rs(digests charged to the budget, shared indices, lookups that keep only their own digest, the first-call cap),layout_select.rs(keys past the slots walk and are never resolved),position.rs(variedHVAR/VVARadvances kept and read back per direction),tests/hostile_layout_caches.rs(bytes a usedFontkeeps on three hostile tables, output against a freshFont).cargo fmt --all -- --check;cargo +1.99.0 clippy --workspace --all-targets --all-features -- -D warnings; the same with--no-default-features;cargo +1.99.0 clippy -p sigilbuzz --no-default-features -- -D warnings;cargo test --workspace --all-features(201 test binaries);cargo build --no-default-features;cargo build -p sigilbuzz --target wasm32-unknown-unknown --no-default-features; MSRV afterCARGO_RESOLVER_INCOMPATIBLE_RUST_VERSIONS=fallback cargo +stable generate-lockfile:cargo +1.81 check --workspace --lib --no-default-featuresandcargo +1.81 check --workspace --lib --bins --all-features. All pass.rustlang/rust:nightly, cargo-fuzz 0.13.2,-O -a,-max_len=65536 -rss_limit_mb=2048, seeds fromtests/fixturesandtests/fonts, no--privileged), 10 minutes per run:face: 432,408 runs, cov 6,705, peak RSS 670 MB, no findings.shapeon the final code with-timeout=10: 81,121 runs, cov 15,832, peak RSS 636 MB, no findings; on 37108e3: 99,602 runs, none.shaperun (at the vertical fix, before the walk trimming) stopped after 143,129 runs on a timeout: a fuzzed font whose GSUB turns a 30-character Devanagari sample into 3,079 glyphs. That input is slow on main too: one fuzz-build run of it takes 23.5 to 24.7 s on main and 32.0 to 35.3 s on this branch, which now shapes every text twice (once perFont); natively 0.72 to 0.81 s on main against 0.45 to 0.57 s here for both shapings. A run with-timeout=60on the same code: 46,012 runs, no findings.Not done
VORG(table above).docs/PERFORMANCE.mdstill holds the 0.6.0 Criterion numbers.CHANGELOG lines (0.24.0)
Replace #295's vertical-speed line with:
Fontare faster than in 0.23.0: a CFF run about as fast (Source Code Pro, 100 characters: 9.5 to 8.6 us), CFF2 and variedglyfruns two to three times as fast (Source Serif 4 VF at wght 700: 77 to 24 us; Hahmlet at wght 700: 67 to 20 us) (Cut the per-call shaping cost and the vertical-run slowdown #289, Cut per-call shaping cost with per-Font accelerators, plans and metric caches #295, Bound the shaping caches' work and memory on hostile fonts #297). A font's first top-to-bottom call is slower than in 0.23.0 for fonts withoutVORG, whose vertical origins come from glyph extents since 0.23.1.Add to Changed:
Fontkeeps each glyph's varied advance, fromHVAR,VVARor phantom points, from its second call on, as HarfBuzz keeps an advance cache, so a run no longer walks the variation store for every glyph of every call (Bound the shaping caches' work and memory on hostile fonts #297). 100 characters of Source Serif 4 VF at wght 700 shape left to right in 62 us instead of 119 us in 0.23.0, and Hahmlet at wght 700 in 22 us instead of 90 us.Fontkeeps for its GSUB and GPOS is bounded by the tables' size, however many lookups a table lists: per table, at most twice its length plus 64 Ki subtable digests of 24 bytes, with the lookup indices that name one lookup sharing its digests (Bound the shaping caches' work and memory on hostile fonts #297). A lookup of thousands of subtables costs about what it did in 0.23.1, within about 10%.Fontkeeps the language systems of at most 16 combinations of script, language and FeatureVariations record per table; a run with another combination looks its features up in the font as before (Cut per-call shaping cost with per-Font accelerators, plans and metric caches #295, Bound the shaping caches' work and memory on hostile fonts #297).cfg(target_has_atomic = "ptr")), whichalloc::sync::Arcalready needed;thumbv6m-none-eabidoes not build, as in 0.23 (Bound the shaping caches' work and memory on hostile fonts #297).Amend #295's
Font::coordsline to:Font::coordsreturns a slice that borrows the font, since a font built withwith_variationsowns its coords, and is no longer aconst fn(Cut the per-call shaping cost and the vertical-run slowdown #289, Cut per-call shaping cost with per-Font accelerators, plans and metric caches #295, Bound the shaping caches' work and memory on hostile fonts #297).Review fixes (coordinator)
The independent review of this PR found no blockers. Two items fixed by the coordinator in
61cc790:src/shape/lazy.rs: once eight subtables had parsed, the list reserved room for every subtable of the lookup, in every nested lookup application. A 64 KB GSUB whose ChainContextFormat3 lookup of 32,000 offsets applies itself again from its tenth subtable peaked at 88.4 MiB (main 16.65 MiB, 0.23.1 103.5 MiB) and took 10.95 ms warm (main 5.65 ms). The reservation is now capped at 56 more entries; the reviewer measured 2.5 MiB with no reservation. New testa_walk_that_parses_a_few_subtables_holds_room_for_a_few.src/sync.rs: the docs said every cache is a pure function of the font. The lookup accelerators' digests depend on the shared budget when they are built, so racing threads can keep different digests (memory and speed can follow call order); output does not change, since a lookup without a digest admits every glyph.The review verified: 8-thread shaping on one Font (107,136 and 164,736 shapes, 0 differences from a fresh Font), every glyph's advances of 9 variable fonts from the caches (1,013,508 checks, 0 mismatches), accelerator supersets and the 1,024-coverage cap, memory proportional to table size, and a clean
shapefuzz run (247,076 runs).