A cross toolchain is never rebuilt over another one's sysroot: the sysroot records which toolchain built it - #37
Merged
Merged
Conversation
…me out of a cache Run 35492373903 (794c481) failed in stage 02, in gcc pass 2's libstdc++: "cannot convert '<brace-enclosed initializer list>' to 'unsigned int'" at __GTHREAD_COND_INIT. It was called a from-scratch rebuild and was not one. The workflow's last restore key is a bare "work02-", so with every input changed it still restored the previous run's sysroot, and the toolchain was rebuilt on top of it. gcc's fixincludes keeps a private copy of every header it "fixes", and pthread.h is one (pthread_incomplete_struct_argument). Pass 1 normally runs before any C library exists and finds nothing. Here it found the old sysroot's glibc 2.40 headers and kept that pthread.h. glibc was then rebuilt from the release branch, which changes struct __pthread_cond_s and PTHREAD_COND_INITIALIZER together. The pass-1 compiler went on reading its private, old pthread.h beside the sysroot's new thread-shared-types.h, and the old initializer does not fit the new struct. The build log says all of it: "Fixing directory .../sysroot/usr/include", "Applying pthread_incomplete_struct_argument to pthread.h" in gcc-pass1.log, and the cache line "restored from key: work02-...-35482105602". So pass 1 removes usr/include when a C library's headers are there. The step only runs when the toolchain is being rebuilt, and linux-headers (which already clears the directory) and glibc put the headers back right after it. An incremental rebuild is then the same as a from-scratch one in the respect that broke, and a later glibc header change cannot meet a stale private copy, because pass 1 no longer has one. Neither the new gcc nor the glibc branch was at fault by itself; the order they were built in, over an old tree, was.
…sroot records which toolchain built it The first version of this fix removed usr/include before gcc pass 1 when glibc's headers were there. Two readers' worth of looking said that was one header too narrow. A tree with the kernel's headers and no glibc would keep them, and fixincludes freezes private copies of dozens of linux/*.h too. And the headers are only what was shown to matter: a restored sysroot also keeps the old compiler's directory under tools and superseded libraries beside their replacements. So the rule is the general one and sits before the first step. The sysroot holds a marker: a hash of the binutils, gcc, glibc, kernel-header, mpfr, gmp and mpc versions and of glibc's patch set. A populated sysroot whose marker is missing or different was built by another toolchain: every stamp is archived, the sysroot is removed, and stage 01 starts from layout. It refuses if anything is mounted under the sysroot, and removes with --one-file-system, because stage 03 binds /dev, /proc and this repository in there. A sysroot with no headers gets this toolchain's marker. Stamps could not have caught this: the header that went stale is not an input of any step. It is the third time something outside a step's inputs has decided a result (the microcode beside the kernel tree, the release keys outside the cache), and the first where the cure is a record of what the tree was built by. The workflow keeps its bare work02- restore key on purpose. build/config/** is in that cache's key, so without the fallback every kernel fragment edit would rebuild the toolchain and the base system from nothing; with this rule the fallback is safe, because a restored tree from another toolchain clears itself. Exercised outside the build against six fake sysroots: fresh (marker written), same toolchain (left alone), gcc moved (cleared, two stamps archived), a cached tree with no marker (cleared), kernel headers and no glibc (cleared), something mounted under it (refused, nothing deleted).
DevomB
changed the title
…cord lives with the stamps
Review of the rule found that its removal could reach the source tree. Stage
03 binds this repository into the sysroot. A bind mount from the same
filesystem has the sysroot's own device number, so rm --one-file-system
walks into it: that guard does not exist for exactly the mount that matters.
That left `grep " ${LFS}/" /proc/mounts` as the only thing between a stale
record and an rm -rf of the checkout, and it is blind three ways, each
failing open: /proc/mounts writes a space as \040 (this checkout is called
"Linux Distro"), it names the resolved path where $LFS may hold a symlink,
and grep reads $LFS as a pattern, so a bracket in it misses or errors.
The test now resolves the path, escapes it the way /proc/mounts does, and
compares each mountpoint as a fixed string from its start; awk gets the
string through ENVIRON, because -v would turn \040 back into a space. A
mounts file that cannot be read counts as mounted. And the question is asked
a second way that does not depend on parsing anything: the four directories
stage 03 binds the repository, the sources and the work tree onto are empty
when nothing is mounted, so any of them holding a file is a refusal.
The record moves from the top of the sysroot to the stamps directory. Stage
06's root image is a copy of the sysroot with an exclude list that did not
name it, so every verified root would have carried a build-host file at /.
The cache restores the stamps and the sysroot together, a lost stamps
directory reads as no record and clears, and the archive step moves the old
record aside with the other stamps. The identity also hashes gcc and
binutils patch sets, for the day the tree has any.
Exercised outside the build: the six sysroots again, the sixth now with a
file standing in for the repository (refused, and the file survived), and
the mount test against a mounts file holding a plain path, a path with a
space, one with brackets, one reached through a symlink, and a sibling
whose name merely starts the same; all eight answers right, and an
unreadable mounts file answers mounted.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes the stage 02 failure on main (run 35492373903, 794c481).
What failed. gcc pass 2's libstdc++:
cannot convert '<brace-enclosed initializer list>' to 'unsigned int'at__GTHREAD_COND_INIT.Why. It was not a from-scratch rebuild. The workflow's last restore key is a bare
work02-, so it restored the previous run's sysroot and rebuilt the toolchain on top of it. gcc's fixincludes keeps a private copy of headers it "fixes", andpthread.his one. Pass 1 found the old glibc'spthread.hand kept it; glibc was then rebuilt from the release branch, which changesstruct __pthread_cond_sand its initializer together; the pass-1 compiler read its old private copy beside the new struct.Evidence in the logs.
gcc-pass1.log: "Fixing directory .../sysroot/usr/include", "Applying pthread_incomplete_struct_argument to pthread.h". Job log: "Cache restored from key: work02-...-35482105602".Fix (second commit, the general rule). The sysroot records which toolchain built it: a hash of the binutils, gcc, glibc, kernel-header, mpfr, gmp and mpc versions and glibc's patch set. A populated sysroot with a missing or different marker has every stamp archived and is removed before stage 01's first step. It refuses if anything is mounted under the sysroot and removes with
--one-file-system. The first commit's narrower header removal is replaced by it.Why the workflow's fallback key stays.
build/config/**is in that cache key. Without the fallback, every kernel fragment edit would rebuild the toolchain and base system from nothing. With this rule the fallback is safe.Checked outside the build against six fake sysroots: fresh, same toolchain, gcc moved, a cached tree with no marker, kernel headers without glibc, and something mounted under it (refused, nothing deleted).
Not proven until a Distro run gets through stage 02.