[8.8.0] Add a remote repo contents cache#29075
Merged
Merged
Conversation
b055d93 to
967de9a
Compare
c845c2e to
635532b
Compare
Makes it possible to build `FileSystem`s composed out of existing `FileSystem`s outside the `vfs` package. Some methods already had their visibility upgraded for this purpose. Closes bazelbuild#26796. PiperOrigin-RevId: 798226663 Change-Id: Ic9e4cdefadc7ad8d50a5b69df91837c99c5f6c64 (cherry picked from commit 4f46e82)
Collaborator
Author
|
I realized that this lacks one more cherry-pick that simplifies it, will send it inline and mark as RFR when done. |
auto-merge was automatically disabled
June 15, 2026 13:19
Pull request was converted to draft
5b14e05 to
9aad6ba
Compare
Collaborator
Author
|
Ready again, I included 4f46e82 |
Collaborator
Author
Wyverald
reviewed
Jun 16, 2026
auto-merge was automatically disabled
June 17, 2026 08:52
Head branch was pushed to by a user without write access
Wyverald
approved these changes
Jun 18, 2026
Merged
via the queue into
bazelbuild:release-8.8.0
with commit Jun 18, 2026
3e991ca
46 checks passed
fmeum
added a commit
to fmeum/bazel
that referenced
this pull request
Jun 23, 2026
* The cache was always written to, even if not enabled. (On 8.x this guard was already folded into the remote repo contents cache port, bazelbuild#29075, so only the second fix below is applied here.) * Google RBE doesn't accept `Command`s without the (deprecated) `Platform` field set. We set it both on `Command` and `Action`, just to be safe. Fixes bazelbuild#28294 (comment) Closes bazelbuild#28295. PiperOrigin-RevId: 856169835 Change-Id: I2479119a173e325a7d39643a36536569f5f831fc (cherry picked from commit a9946096847e22de98e0e11b1f5dfbb6ec6ecdbb)
fmeum
added a commit
to fmeum/bazel
that referenced
this pull request
Jun 24, 2026
* The cache was always written to, even if not enabled. (On 8.x this guard was already folded into the remote repo contents cache port, bazelbuild#29075, so only the second fix below is applied here.) * Google RBE doesn't accept `Command`s without the (deprecated) `Platform` field set. We set it both on `Command` and `Action`, just to be safe. Fixes bazelbuild#28294 (comment) Closes bazelbuild#28295. PiperOrigin-RevId: 856169835 Change-Id: I2479119a173e325a7d39643a36536569f5f831fc (cherry picked from commit a9946096847e22de98e0e11b1f5dfbb6ec6ecdbb)
fmeum
added a commit
to fmeum/bazel
that referenced
this pull request
Jun 25, 2026
* The cache was always written to, even if not enabled. (On 8.x this guard was already folded into the remote repo contents cache port, bazelbuild#29075, so only the second fix below is applied here.) * Google RBE doesn't accept `Command`s without the (deprecated) `Platform` field set. We set it both on `Command` and `Action`, just to be safe. Fixes bazelbuild#28294 (comment) Closes bazelbuild#28295. PiperOrigin-RevId: 856169835 Change-Id: I2479119a173e325a7d39643a36536569f5f831fc (cherry picked from commit a9946096847e22de98e0e11b1f5dfbb6ec6ecdbb)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Contains the following adaptions required for Bazel 8:
DetailedIOExceptionpropagation had to be added to ensure that rewinding works correctlynameset in WORKSPACE (gone in Bazel 9),AbstractActionInputPrefetcherneeded a combination of a lazySupplierfor the path as well as moving the temp directory to the output base, whose path is known at startup.Original commit info follows:
Repositories are cached in a regular remote cache as AC entries for a synthetic command with the predeclared input hash as the salt. The contents are represented as an output file for the marker file and an output directory for the contents.
Upon a cache hit, the metadata of the files comprising the repository is downloaded and injected into an in-memory file system that is overlaid on the
externaldirectory on the native file system. Downloads of file contents only occur when Bazel needs to read a file (e.g., a BUILD or .bzl file) or if a file is an input to an action executed locally. This can save time taken to execute repo rules and compute file digests and disk space required to store the contents of external repositories.The output of
du -h $(bazel info output_base)afterbazel build //src:bazel-devand a fully up-to-date remote cacheSome repos are still materialized eagerly, which may not be necessary. Patched
http_archivealso can't be cached yet, so these numbers are likely to improve with further work on this feature.Fixes #6359
RELNOTES[NEW]: The results of reproducible repository rules without dependencies added at runtime (e.g., via
repository_ctx.watchor.getenv) can now be cached in a regular HTTP or gRPC remote cache if the new--experimental_remote_repo_contents_cachestartup option is provided.Closes #26860.
PiperOrigin-RevId: 827569755
Change-Id: I3a4e20c8c717c8503b5eb43545a8db162daef585
(cherry-picked from b8589c3, also includes 4f46e82)