[8.8.0] Preserve order of recorded inputs#29068
Merged
Merged
Conversation
iancha1992
added
the
soft-release-blocker
iancha1992
changed the title
|
Thank you for contributing to the Bazel repository! This pull request has been marked as stale since it has not had any activity in the last 30 days. It will be closed in the next 30 days unless any other activity occurs. If you think this PR is still relevant and should stay open, please post any comment here and the PR will no longer be marked as stale. |
github-actions
Bot
added
the
stale
fmeum
changed the title
github-actions
Bot
removed
the
stale
I haven't been able to reproduce this in a test, but this should fix the following crash observed while running `bazel info`: ``` FATAL: bazel crashed due to an internal error. Printing stack trace: java.lang.NullPointerException: Cannot invoke "java.util.concurrent.ExecutorService.shutdownNow()" because "this.materializationExecutor" is null at com.google.devtools.build.lib.remote.RemoteExternalOverlayFileSystem.afterCommand(RemoteExternalOverlayFileSystem.java:145) at com.google.devtools.build.lib.remote.RemoteModule.afterCommand(RemoteModule.java:1034) at com.google.devtools.build.lib.runtime.BlazeRuntime.afterCommand(BlazeRuntime.java:787) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:807) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:266) at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:608) at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$0(GrpcServerImpl.java:679) at io.grpc.Context$1.run(Context.java:566) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) ``` Closes bazelbuild#27690. PiperOrigin-RevId: 833722608 Change-Id: I88c485a01e5967657ec3b5529a47639b743b18e6 (cherry picked from commit a7d0e91)
Don't print a message when it's successful. Users can always look under `external` to verify which repo came from the cache. Closes bazelbuild#27699. PiperOrigin-RevId: 834096735 Change-Id: I3916fb240218a6b68ecf48417142b998ca281598 (cherry picked from commit 3ca9ce1)
Fixes the creation of empty directories and also contains a speculative fix for the following issue observed during a sequence of real builds: ``` Error in path: Failed to materialize remote repo @@protoc-gen-validate+: [unix_jni.cc:302] /home/ubuntu/.cache/bazel/_bazel_ubuntu/123/external/protoc-gen-validate+/example-workspace/.bazelrc (File exists) ERROR: //:foo :: Error loading option //:foo: error evaluating module extension @@gazelle+//:extensions.bzl%go_deps ``` The mentioned file is a symlink. Closes bazelbuild#27711. PiperOrigin-RevId: 836122472 Change-Id: I8becd8c3640a659d28dc433340db962c18563d9f (cherry picked from commit b27ea05)
Ensures that the returned `Path` is still in the overlay file system.
Also make the error message emitted by `Path#checkSameFileSystem` more informative. This is motivated by and helped discover the above as the fix for the following crash observed when using the remote repo contents cache with an explicit `--sandbox_base`:
```
Caused by: java.lang.IllegalArgumentException: Files are on different filesystems: /dev/shm/bazel-sandbox.b10976335efa519b0184f3091ac8e21f7beefb92142303f9ab2c3341f45a2f28/linux-sandbox/18/execroot/_main/external/c-ares+/configs/ares_build.h (on com.google.devtools.build.lib.unix.UnixFileSystem@5e0a8154), /home/ubuntu/.cache/bazel/_bazel_ubuntu/123/execroot/_main/external/c-ares+/configs/ares_build.h (on com.google.devtools.build.lib.remote.RemoteExternalOverlayFileSystem@6cd9bfda)
at com.google.devtools.build.lib.vfs.Path.checkSameFileSystem(Path.java:964)
at com.google.devtools.build.lib.vfs.Path.createSymbolicLink(Path.java:523)
at com.google.devtools.build.lib.vfs.Path.createSymbolicLink(Path.java:535)
at com.google.devtools.build.lib.sandbox.SymlinkedSandboxedSpawn.copyFile(SymlinkedSandboxedSpawn.java:129)
```
Alternative to bazelbuild#27721
Closes bazelbuild#27802.
PiperOrigin-RevId: 837832265
Change-Id: I3b73167496b011aef66954d59ca3804b4b64996f
(cherry picked from commit 8eaf6a9)
Fixes bazelbuild#27981 Fixes the following type of crash and, incidentally, a remote repo contents cache test that resulted in a related crash: ``` FATAL: bazel crashed due to an internal error. Printing stack trace: java.lang.IllegalStateException: Unknown error during configuration creation evaluation at com.google.devtools.build.lib.skyframe.SkyframeExecutor.getConfiguration(SkyframeExecutor.java:2143) at com.google.devtools.build.lib.skyframe.SkyframeExecutor.createConfiguration(SkyframeExecutor.java:1876) at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:281) at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.runAnalysisPhase(AnalysisPhaseRunner.java:399) at com.google.devtools.build.lib.buildtool.AnalysisPhaseRunner.execute(AnalysisPhaseRunner.java:144) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargetsWithoutMergedAnalysisExecution(BuildTool.java:512) at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:414) at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:907) at com.google.devtools.build.lib.runtime.commands.CqueryCommand.exec(CqueryCommand.java:197) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:783) at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:266) at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:608) at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$0(GrpcServerImpl.java:679) at io.grpc.Context$1.run(Context.java:566) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) at java.base/java.lang.Thread.run(Unknown Source) Caused by: com.google.devtools.build.lib.skyframe.toolchains.PlatformLookupUtil$InvalidPlatformException: com.google.devtools.build.lib.packages.BuildFileNotFoundException: no such package '@@[unknown repo 'toolchains_llvm_boostrapped' requested from @@ (did you mean 'toolchains_llvm_bootstrapped'?)]//platforms': The repository '@@[unknown repo 'toolchains_llvm_boostrapped' requested from @@ (did you mean 'toolchains_llvm_bootstrapped'?)]' could not be resolved: No repository visible as '@toolchains_llvm_boostrapped' from main repository at com.google.devtools.build.lib.analysis.platform.PlatformFunction.compute(PlatformFunction.java:75) at com.google.devtools.build.lib.analysis.platform.PlatformFunction.compute(PlatformFunction.java:43) at com.google.devtools.build.skyframe.ParallelEvaluator.bubbleErrorUp(ParallelEvaluator.java:414) at com.google.devtools.build.skyframe.ParallelEvaluator.waitForCompletionAndConstructResult(ParallelEvaluator.java:207) at com.google.devtools.build.skyframe.ParallelEvaluator.doMutatingEvaluation(ParallelEvaluator.java:173) at com.google.devtools.build.skyframe.ParallelEvaluator.eval(ParallelEvaluator.java:672) at com.google.devtools.build.skyframe.AbstractInMemoryMemoizingEvaluator.evaluate(AbstractInMemoryMemoizingEvaluator.java:182) at com.google.devtools.build.lib.skyframe.SkyframeExecutor.evaluate(SkyframeExecutor.java:4279) at com.google.devtools.build.lib.skyframe.SkyframeExecutor.lambda$evaluateSkyKeys$0(SkyframeExecutor.java:2278) at com.google.devtools.build.lib.concurrent.Uninterruptibles.callUninterruptibly(Uninterruptibles.java:35) at com.google.devtools.build.lib.skyframe.SkyframeExecutor.evaluateSkyKeys(SkyframeExecutor.java:2274) at com.google.devtools.build.lib.skyframe.SkyframeExecutor.getConfiguration(SkyframeExecutor.java:2126) ... 16 more ``` Closes bazelbuild#28004. PiperOrigin-RevId: 845941915 Change-Id: I6ead8dd1662efe90f529a6e21041a225882415dc (cherry picked from commit d6dc631)
Recorded inputs for repo rules and module extensions are no longer separated by type, but recorded in the order they are encountered. This is necessary preparatory work to avoid Skyframe cycles that could otherwise occur while checking cache, lockfile or marker file entries for up-to-dateness. This requires a breaking change to the lockfile structure, but has the positive side effect that each recorded input is now represented as a single line in the lockfile and thus more compact. Along the way, the error message for an out-of-date lockfile entry with `--lockfile_mode=errors` are improved and now include the precise reason for the invalidation. For repository rules, the repo mapping entries recorded while loading the repo rule definition are now included in the predeclared inputs hash, which makes cache entries more specific and avoids unnecessary Skyframe lookups. Work towards bazelbuild#27517 Closes bazelbuild#27603. PiperOrigin-RevId: 844721202 Change-Id: Id063a88508591086b07bd89a822cf22b2d90610d (cherry picked from commit 41ccfef)
fmeum
commented
Jun 22, 2026
github-actions
Bot
added
team-ExternalDeps
Wyverald
approved these changes
Jun 22, 2026
Merged
via the queue into
bazelbuild:release-8.8.0
with commit Jun 22, 2026
30685f4
47 checks passed
github-actions
Bot
removed
the
awaiting-review
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Recorded inputs for repo rules and module extensions are no longer separated by type, but recorded in the order they are encountered. This is necessary preparatory work to avoid Skyframe cycles that could otherwise occur while checking cache, lockfile or marker file entries for up-to-dateness.
This requires a breaking change to the lockfile structure, but has the positive side effect that each recorded input is now represented as a single line in the lockfile and thus more compact.
Along the way, the error message for an out-of-date lockfile entry with
--lockfile_mode=errorsare improved and now include the precise reason for the invalidation.For repository rules, the repo mapping entries recorded while loading the repo rule definition are now included in the predeclared inputs hash, which makes cache entries more specific and avoids unnecessary Skyframe lookups.
Work towards #27517
Closes #27603.
PiperOrigin-RevId: 844721202
Change-Id: Id063a88508591086b07bd89a822cf22b2d90610d
(cherry picked from commit 41ccfef)