Repository navigation
whatif: measured speed-ups, and benchmarks on ten documents - #2
Merged
Merged
Conversation
`prumo whatif` rebuilds the document with one change at a time and times it against the plain build. The changes are the preamble precompiled with mylatexformat, draft mode, and tikz's external library (experimental, and only with --shell-escape). The builds run in a temp copy without the probe, one at a time: a warm-up each, then the baseline and the variants in turns. The table compares medians, calls anything inside the spread (2 ms at least) noise, puts the probe's bound beside each saving, and compares the output by page count, the words on each page and a 24 dpi render. The externalize line hands the hook to tikz's picture builds too. Without it each picture build typesets the whole document, which is where a page count going from 2 to 3 comes from. XeTeX and LuaTeX cannot precompile a preamble, and the row says why. doctor has a mylatexformat row. --check-log now tells a file opened only to test that it exists from one read outside the hooks (siunitx with tikz opens color.sty and never reads it). The new fixture has eight claims, all tested. The TeX image installs mylatexformat, and make_artifacts.sh takes fixture names.
Seven public documents join the three that live here: the ACM sigconf sample with BibTeX and with biblatex, the metropolis demo, florilegium's manual, the HoTT book, Open Logic and the Stacks Project. corpus.toml pins each one to a commit and fetch.py takes it from its own repository, so nothing under share-alike or GFDL terms is stored here. kaobook is left out, because thmtools' sibling= fails on the 2026-11-01 kernel. bench.py gains --overhead (builds with and without the probe, in turns) and --whatif. charts.py draws four charts, light and dark, from the JSON: where the time goes, what the probe costs, package times on pdfLaTeX against LuaLaTeX, and what whatif saves. Color means phase in all of them, and every pair of colors is checked for colorblind readers. The benchmark README has the charts with their tables. The top-level README links it and says where the precompiled preamble fails. Findings 60 to 64.
Rebuilt alone, the fixture compiled Latin Modern 12 into luaotfload's cache and its .fls listed that. In the full run CI makes, the biblatex fixture's latexmk passes have compiled those fonts already, so CI's .fls has no such lines and the shape check failed. make_artifacts.sh says so now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
For 0.2.0, before the tag.
prumo whatifprumo whatif main.tex [--try preamble,draft,externalize] [--runs 5] [--shell-escape]rebuilds the document with one change at a time and times it against the plain build:preambleprecompiles the preamble with mylatexformat. pdfTeX only: XeTeX cannot dump the class's OpenType font, and LuaTeX's dump fails or the build on it does. The row says which, with the error.draftbuilds without a PDF (-draftmode,-no-pdfon XeTeX), which is what writing the PDF and the images cost.externalizeturns on tikz's external library from the command line. Experimental, and refused without--shell-escape. The hook goes to the picture builds as well; without it each picture build typesets the whole document, and that is where 2 pages becoming 3 came from.The builds run in a temp copy without the probe, one at a time (parallel builds slow each other down), with a warm-up each and then the baseline and the variants in turns. The table compares medians and calls anything inside the spread (2 ms at least) noise. The probe's bound sits beside each saving, and one-time costs (the format build, the first externalized run) are their own column. The output is compared by page count, the words on each page and a 24 dpi render.
pdftotext -layoutalone flagged an externalized picture whose text moved a column, so it is not enough.--jsonis schema 1.Also in here: a mylatexformat row in
doctor,doctor --check-logtelling an existence check from a read outside the hooks, awhatiffixture with eight claims (all tested), mylatexformat in the TeX image, andmake_artifacts.shtaking fixture names.Benchmarks
Ten documents now. Seven of them are fetched at a pinned commit and never stored here: the ACM sample (BibTeX and biblatex), the metropolis demo, florilegium's manual, the HoTT book, Open Logic and the 7,710-page Stacks Project.
bench.pymeasures the probe's cost (builds with and without it, in turns) and runswhatifon all of them.charts.pydraws four charts from the JSON, light and dark. kaobook is left out because thmtools'sibling=fails on the 2026-11-01 kernel.What came out of it:
Tested
cargo test, clippy and fmt. The TeX tests in the image, the new fixture's byte-identical PDF included.whatifon the fixture on TeX Live 2021 too.