Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 52 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,8 @@ I had no way to see why. Commenting packages out one at a time is not insight. p
is one instrumented build that shows where the time goes, which package loaded which, and what you can
drop.

It has been run on my own corpus and on a few public documents (a thesis template, a beamer
deck, a 76-page arXiv paper; see [the benchmarks](docs/benchmarks/README.md)). Please report
what breaks!
It has been run on my own corpus and on public documents from an ACM paper to the 7,710-page
Stacks Project (see [the benchmarks](docs/benchmarks/README.md)). Please report what breaks!

This project was made with the assistance of AI (Claude).

Expand Down Expand Up @@ -63,6 +62,7 @@ prumo prune docs/ --engines pdflatex # corpus-wide keep list and tlmgr remove pl
prumo prune docs/ --apply --yes # run the plan; every tlmgr command is logged
prumo unused main.tex # which \usepackage lines can go: remove the line, rebuild, compare
prumo unused obra.cls --corpus 'docs/**/*.tex' # a class judged against every document that uses it
prumo whatif main.tex --shell-escape # measured speed-ups: precompiled preamble, draft mode, externalized tikz pictures
prumo time main.tex --env tblr,minted # which environments to time (default: tikzpicture, axis, tcolorbox, tblr, longtable, tabularx, minted, lstlisting, forest)
prumo time output.prumo # records from a build prumo did not run (Overleaf: docs/overleaf.md)
prumo doctor --check-log main.tex # files that bypassed the hooks; log vs records
Expand Down Expand Up @@ -148,6 +148,54 @@ row. `prumo compare main.tex --runs 4` on the same paper:
note: xelatex's first run took 4050 ms (font cache build); the reported run is warm.
```

## What `whatif` measures

`prumo whatif` rebuilds the document with one change at a time and times it against the
plain build. Here it is on the whatif fixture:

```
$ prumo whatif fixtures/whatif/main.tex --shell-escape
== prumo whatif: main (pdflatex, median of 5 runs after a warm-up, 1 page, -shell-escape)
experiment baseline variant saved bound one-time output
preamble 247 ms 90 ms 157 ms 64% 207 ms 319 ms identical
draft 247 ms 244 ms within noise - - n/a (no PDF)
externalize* 247 ms 245 ms within noise 3 ms 503 ms identical
```

The three experiments:

- `preamble` precompiles everything before `\begin{document}` into a format with
mylatexformat and builds on that format. The one-time column is the format build. Only
pdfTeX can do it at all. XeTeX cannot write a format once the class has loaded its
OpenType font, and on LuaTeX either the format build or the build on it fails. Even on
pdfTeX it fails on preambles that open files or play with catcodes, as it did on three of
the six pdfLaTeX documents in the [benchmarks](docs/benchmarks/README.md). The row says
which step failed, with the error.
- `draft` builds without writing a PDF (`-draftmode`, or `-no-pdf` on XeTeX). What it saves
is what writing the PDF and including images cost. There is no PDF to compare.
- `externalize` is experimental. It turns on tikz's external library from the command line,
so each picture is built once into its own PDF and included from then on. It needs
`--shell-escape`, because tikz builds each picture with a shell command. The one-time
column is the first run, which builds every picture. The external library breaks
`remember picture`, overlays and references between pictures, so look at the PDF too.

Every build runs once to warm up, then five times (`--runs`), one build at a time and without
the probe. The baseline and the variants take turns, so drift hits them all alike. The table
compares medians. A difference no larger than the spread, the slowest run minus the fastest,
is "within noise". The output column compares the page count, the words on each page
(`pdftotext`) and a coarse render of each page with the baseline's. When they differ, the
report says the saving is not a clean one.

The bound column is a cross-check from one probed build: the preamble phase for `preamble`,
and the time inside `tikzpicture` environments for `externalize`. The fixture's one picture
takes 3 ms, so externalizing it saves nothing. The [benchmarks](docs/benchmarks/README.md)
run all three experiments on ten documents and three engines. Watch the one-time cost.
Externalizing Open Logic's pictures saves 4 s of a 31 s build, after a first run of 93
minutes.

Each experiment runs in its own copy of your directory, and the formats and picture files
stay there. Under the table, `whatif` says how to make each change yourself.

## Running from Docker, nothing installed

No TeX Live? No Rust? `docker/Dockerfile.user` builds an image with a full TeX Live, Poppler,
Expand Down Expand Up @@ -327,6 +375,7 @@ know which of its macros, that is texprof's question.
`--out` says), nothing next to your document. `prune --apply` is the one destructive command, it
re-prints its plan first and needs `--yes`.
- Fetch anything. It reads the local `texlive.tlpdb`; no network, ever.
- Make a speed-up for you. `whatif` measures a change on a copy and says how to make it.
- Prove a package unused. "No observable effect" means this document, this build, these
comparisons; a package can matter for input the document does not exercise, and the report
says so every time.
Expand Down
3 changes: 3 additions & 0 deletions docker/Dockerfile.tex
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,9 @@
RUN tlmgr install cm-super courier fontname siunitx translations csquotes \
&& tlmgr path add

# mylatexformat: `whatif` precompiles the whatif fixture's preamble with it.
RUN tlmgr install mylatexformat

# Fixture builds run as the invoking user with HOME under /tmp; keep tlmgr quiet about it.
ENV HOME=/tmp/home

Expand Down
7 changes: 5 additions & 2 deletions docs/03-architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,8 +24,9 @@

`prumo run` produces the model and caches it under `<out>/<job>/`, where `<out>` defaults to a
per-document directory in the user's cache (`run::default_out`; `prumo where`). Every report command either
uses the cache or runs the build. `prune` merges models across a corpus. `unused` is the only
command that runs more than one build.
uses the cache or runs the build. `prune` merges models across a corpus. `compare` builds once
per engine. `unused` and `whatif` rebuild the document with one change at a time; `whatif`'s
timed builds run without the probe.

## BuildModel

Expand Down Expand Up @@ -108,6 +109,7 @@ prumo fonts <main.tex> [--json]
prumo packages <main.tex> [--tlpdb PATH] [--json] # tlmgr package set, with the file that caused each
prumo prune <paths...> [--tlpdb PATH] [--keep LIST] [--keep-hyphenation all|used] [--apply --yes]
prumo unused <main.tex> [--build <cmd>] [--jobs N] [--compare text|render] # phase 2
prumo whatif <main.tex> [--try preamble,draft,externalize] [--runs N] [--shell-escape] [--engine E] [--json]
prumo doctor [--check-log] # engines, kernel date, pdffonts, tlpdb, recorder
prumo where [<main.tex>] [--out DIR] # the cache root, or a document's artifact directory
prumo clean [<main.tex>] [--stale] [--yes] [--json] # list the cache (sizes, documents); delete with --yes
Expand Down Expand Up @@ -163,6 +165,7 @@ prumo/
report/ text, json, dot renderers
prune.rs corpus merge, keep set, plan
unused.rs phase 2
whatif.rs measured speed-ups: precompiled preamble, draft mode, externalized pictures
tex/prumo.sty embedded via include_str!
scripts/prumo_report.py reference, frozen
fixtures/ documents + checked-in build artifacts for no-TeX tests
Expand Down
39 changes: 39 additions & 0 deletions docs/10-verified-findings.md
Original file line number Diff line number Diff line change
Expand Up @@ -254,3 +254,42 @@ Items 1–18 hold, with these corrections and additions:
inside page 1's brackets (`[1{.../pdftex.map}]`). On a one-line document the shipout phase
is 45 ms of 132 ms on pdfLaTeX, all of it page 1; how much of that is the map was not
isolated.
60. **A precompiled preamble works on pdfTeX only.** mylatexformat
(`pdftex -ini -jobname=main-fmt "&pdflatex" mylatexformat.ltx main.tex`, then
`pdflatex -fmt=main-fmt main.tex`) takes the whatif fixture from 244 ms to 91 ms on TeX
Live 2026 and from 137 ms to 58 ms on TeX Live 2021, with the same text. XeTeX stops with
`! Can't \dump a format with native fonts or font-mappings.` even without fontspec: the
class's `\normalsize` loads Latin Modern as an OpenType font for the TU encoding. On TeX
Live 2026, LuaHBTeX writes the format but exits 1 after a Lua error in lua-uni-algos
(`lua-uni-stage-tables.lua:141: attempt to perform arithmetic on a string value`), and a
build on that format dies with `(pdf backend): referenced object has wrong type`. On TeX
Live 2021 the dump succeeds and the build on it dies the same way.
61. **tikz's external library from the command line.** With
`\AddToHook{package/tikz/after}{\usetikzlibrary{external}\tikzexternalize}\input{main}`
on the command line, each picture build runs `\def\tikzexternalrealjob{main}\input{main}`,
which never sees the hook. It typesets the whole document as "the picture", and a full
letter page goes where the picture was: 1 page became 2 on the test document. That is
most likely the 2 → 3 pages in the brief, whose document was not run here. Passing `system call=` with the hook in the picture's
command line fixes it. The text and page count are then identical on all three engines,
on TeX Live 2026 and 2021. tikz's default system call is pdflatex unless the driver says
otherwise, so the system call also names the document's engine. Ten pictures of 150 lines
each went from 944 ms to 138 ms, against the probe's 828 ms in `tikzpicture`.
62. **An existence check is not a read.** siunitx runs `\RequirePackage{color}` at
`begindocument/before`. When tikz has already loaded xcolor, which claims `color.sty`,
the kernel opens `color.sty` and does not read it. pdfTeX's and XeTeX's `.fls` list it,
LuaTeX's does not, and neither the log nor the records show it. `doctor --check-log`
called it "read outside the LaTeX file hooks". With a log it now lists such files as
opened but never read.
63. **`pdftotext -layout` moves an externalized picture's text.** On the benchmark paper,
externalize was reported as changing the output: line 64 of the text differed. The pages
render the same (78 of 240,000 pixels differ at 50 dpi, antialiasing) and every page has
the same words. The picture is an included PDF now, and `-layout` places its text a column
or a line away. `whatif` compares the words on each page and a 24 dpi render instead
(the render check is `unused`'s), and then catches a real one: one page of the metropolis
deck on XeLaTeX and LuaLaTeX has the same words and 0.9% of its pixels changed.
64. **thmtools' `sibling=` fails on the 2026-11-01 kernel.** `\usepackage{amsthm,thmtools}`,
`\declaretheorem[numberwithin=section]{theorem}` and
`\declaretheorem[sibling=theorem]{proposition}` stop with `Command \c@proposition already
defined` (thmtools 0.76, TeX Live 2026). The same file builds on TeX Live 2021 (thmtools
0.72, kernel 2021-11-15). Every kaobook example fails this way, so kaobook is not in the
benchmarks.
67 changes: 53 additions & 14 deletions docs/STATUS.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,17 +44,26 @@ can check it.

### Benchmarks (`docs/benchmarks/`)

- `run.sh` runs `bench.py` in the full TeX Live 2026 image pinned to one P-core of the
13900K: four documents (one line; a five-page paper with biber, pgfplots and siunitx;
florilegium's manual; a 3,991-page generated book), three engines, a cold run with fresh
font caches and five warm runs (three for the book), medians per phase. Results and
method in `docs/benchmarks/README.md`; the widest spread is 4%.
- On the one-line document LuaLaTeX spends 235 of 315 ms starting up, 22 ms on the preamble
and 54 ms writing the PDF. On the paper it is 236, 763 and 135 of 1,392 ms (pdfLaTeX 855
ms in all). On the book LuaTeX's shipout is 2.7 ms a page against pdfTeX's 0.7. A cold
LuaLaTeX run is 4.4 s instead of 0.3 s.
- Scale: the book's 8,669 records are read in 3 ms, and `--html` writes 1.3 MB in 8 ms.
Plain output now lists the slowest `--top` pages instead of all 3,991.
- Ten documents now, three engines. Three live here (one line, the five-page paper, the
3,991-page generated book). Seven are fetched at a pinned commit and never stored: the
ACM sigconf sample with BibTeX and with biblatex, the metropolis demo deck, florilegium's
manual, the HoTT book, Open Logic and the 7,710-page Stacks Project. `corpus.toml` has
each one's commit, license, engines and prepare step, `fetch.py` lays them out. kaobook
was left out, because thmtools' `sibling=` fails on the 2026-11-01 kernel, in five lines
too.
- `bench.py` has three modes: the phases (as before), `--overhead` (builds with and without
the probe in turns, plus per-package times from the probed records) and `--whatif`.
`charts.py` draws four SVG charts, light and dark, from the JSON. Color means phase in
all of them, the palette checked with the dataviz validator, the gray included after you
could not tell startup from body.
- The probe costs 7 ms on the one-line document and under 6% everywhere. On the paper,
LuaLaTeX's extra preamble time is biblatex (+151 ms) and fontspec's LuaTeX code (67 ms).
- On the corpus, the precompiled preamble works on two of the six pdfLaTeX
documents (paper 49%, deck 25%). Draft pays on XeLaTeX (13 to 38%). Externalize pays on
the deck and florilegium (15 to 32%), but its first run on a long book is a pass per
picture: 93 minutes on Open Logic for 4 s a build.
- The run had seven P-cores busy at once, and its phase numbers came out 0.3 to 5% slower
than the quieter run before it. The README says so.

### Public documents

Expand All @@ -81,6 +90,36 @@ and a paragraph on texprof. texprof is not a pdfTeX: it is TeX with e-TeX and Pr
DVI, and could not build a LaTeX format against the 2026-11-01 kernel. `docs/02-prior-art.md`
is corrected; CLAUDE.md still says "pdfTeX only".

### whatif

- `prumo whatif main.tex [--try preamble,draft,externalize] [--runs 5] [--shell-escape]`,
from the brief. Each experiment rebuilds its own copy of the directory with one change.
The timed builds run without the probe: a warm-up each, then the baseline and the variants
in turns. Medians are compared. A difference no larger than the larger spread (2 ms at
least, the polling resolution) is "within noise". The output is compared by page count,
the words on each page and a 24 dpi render (finding 63). `--json` has the same fields
under schema 1.
- The builds run one at a time, not on `unused`'s parallel pool, because parallel builds
slow each other down. The temp copy, the timeout and the text comparison come from `run`
and `unused`.
- `preamble` works on pdfTeX (the fixture goes from 244 to 91 ms). XeTeX and LuaTeX fail,
and the row gives the error (finding 60). `doctor` has a mylatexformat row and the TeX
image installs it.
- `externalize`: the brief's command line made every picture build typeset the whole
document (finding 61). The system call now carries the hook and names the engine. The text
is identical on all three engines, and ten pictures went from 944 to 138 ms. Without
`--shell-escape` it is refused when asked for and not run by default. The table and the
README mark it experimental.
- New fixture `whatif`, eight claims, all tested. `make_artifacts.sh whatif` regenerates one
fixture now. The fixture turned up finding 62: `doctor --check-log` now tells an
existence check from a read.
- My choices: `--runs` is at least 2. The comparison uses the last run's PDF. Logs go to
`<cache>/<job>/whatif/<experiment>/`. With `--shell-escape` the probed run is
`--build "<engine> -shell-escape"`, so `time` rebuilds afterwards. No `--latexmk` or
`--build`, since their wall time is several passes.
- `cargo test`, clippy and fmt pass, and so do the 19 TeX tests in the image. TeX Live 2021
runs it too (on a copy of the fixture without siunitx, which that image lacks).

### Only if cheap

- `prumo time output.prumo` and `prumo graph output.prumo` read records from a build prumo
Expand All @@ -89,9 +128,9 @@ is corrected; CLAUDE.md still says "pdfTeX only".
- LuaLaTeX font definitions per page: the `luaotfload.patch_font` hypothesis is wrong for
`\addfontfeature`, so nothing shipped (finding 57).

**Blocked:** nothing. **Needs me:** the v0.2.0 tag. **Open questions:** should the
LuaTeX font count go in after all, through `font.nextid()`? Should `compare` default to three
runs, so it reports a median?
**Blocked:** nothing. **Needs me:** review and merge the `whatif` PR, then the v0.2.0 tag.
**Open questions:** should the LuaTeX font count go in after all, through `font.nextid()`?
Should `compare` default to three runs, so it reports a median?

## Earlier milestones, one line each

Expand Down
Loading
Loading