Record the commit a package was built from, and document the build caches - #3
Merged
Merged
Conversation
Jertlok
force-pushed
the
build/provenance-and-cache-docs
branch
2 times, most recently
from
September 16, 2026 06:26
7da4a5b to
39a63aa
Compare
abuild puts the last commit that touched an aport in the apk's .PKGINFO, and
uses that commit's date as SOURCE_DATE_EPOCH so a rebuild is reproducible. It
runs git in $startdir, which under pmbootstrap is the copy of the aport in the
chroot, and link_to_git_dir() points that copy's .git at pmaports so it can:
but then the APKBUILD is not the one git knows at that path ("main/foo/APKBUILD"
against "APKBUILD"), so git_last_commit() finds nothing, git_dirty() sees a tree
full of changes, and the aport being a git worktree -- which pmaports is in the
porthole workspace -- breaks it a second way, since the .git file it links to
names a path that does not exist in the chroot.
Every package pmbootstrap has ever built therefore says
commit = -dirty
and falls back to the mtime of the copied APKBUILD, i.e. the time of the copy,
as its date: the build is not reproducible and the apk does not say what it was
built from.
Ask the aport's own checkout instead, with the same three git commands abuild
would run, and pass ABUILD_LAST_COMMIT and SOURCE_DATE_EPOCH in the build
environment. A dirty aport keeps abuild's meaning -- "<commit>-dirty" and no
date from git -- and an aport outside a git checkout is left to abuild.
Verified: hello-world built twice in a row is byte-identical as an apk, and
its .PKGINFO names the commit that last touched main/hello-world. Before, the
two apks differed and both said "-dirty".
Assisted-by: Claude
Signed-off-by: Giuseppe Maggio <jertlok@proton.me>
Two things were undocumented. crossdirect 5.3.1-r6 (porthole-dev pmaports) runs GCC's link steps natively, including lto-wrapper and lto1, and leaves qemu only what has to answer for the target's GCC; cross_compiling.md still said that everything but the compiler goes through qemu. And nothing described the caches at all: which directory holds what, that a crossdirect build and a QEMU-only build of the same package write to the same cache but cannot share entries, how to point two work directories at one cache (measured: 268 of 274 compiles served across work directories), and what CI has to restore, with the two things that silently make a restored cache useless -- the chroot's uid 12345 owning it, and a 5G per-arch limit against a 10G cache budget. Assisted-by: Claude Signed-off-by: Giuseppe Maggio <jertlok@proton.me>
Jertlok
force-pushed
the
build/provenance-and-cache-docs
branch
from
September 16, 2026 06:37
39a63aa to
86b311e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things around making builds fast and their output trustworthy.
Every apk pmbootstrap has ever built says
commit = -dirty. abuild takesthe commit and
SOURCE_DATE_EPOCHfrom git in$startdir, which here is thecopy of the aport inside the chroot; the
.gitlink put there for this makespmaports look like it was moved to the build directory, and a pmaports git
worktree (what the porthole workspace uses) breaks it a second way. So the
apk does not say what it was built from, and its date is the time of the copy,
which makes the build non-reproducible. pmbootstrap now runs abuild's own
three git commands in the aport's checkout and passes the answers in.
Verified: hello-world built twice is byte-identical as an apk, and names the
commit that last touched
main/hello-world.Docs.
docs/build_caches.mdis new: which cache directory holds what,which chroot's cache a crossdirect / QEMU-only / cross-native build writes to
(they are not all the target's), how to point two work directories at one
cache -- measured: 268 of 274 compiles served across work directories, 2m17s
against 5m00s cold -- and what CI should restore, with the two things that
silently make a restored cache useless: the chroot's uid 12345 owning it, and
a 5G per-arch ccache against a 10G cache budget.
docs/cross_compiling.mdgets what crossdirect 5.3.1-r6 (porthole-dev pmaports, separate PR) now links
natively and what still has to go through QEMU.
Checks: pytest (the whole suite,
test_bootimgneeds mount privileges thiscontainer does not have), ruff 0.15.22 check and format, mypy, codespell,
vermin, shellcheck, shfmt and markdownlint all pass.