Skip to content

workflows: improve Bazel CI caching #134

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
copybara-service wants to merge 1 commit into
base: main
Choose a base branch
from
Open

workflows: improve Bazel CI caching #134

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 27 additions & 4 deletions .github/workflows/bazel_arm64_macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,23 +36,46 @@ jobs:
USE_BAZEL_VERSION: 9.0.0
steps:
- uses: actions/checkout@v6
# We use actions/cache for the disk cache rather than setup-bazel's
# built-in disk-cache feature because setup-bazel includes a hash of
# all BUILD files in the cache key. Since Bazel's disk cache is
# already content-addressed, this is redundant and causes the entire
# cache (~1.7 GB on Windows) to be re-uploaded whenever any BUILD
# file changes, quickly hitting GitHub's 10 GB limit.
#
# actions/cache entries are immutable (write-once), so we include the
# ISO week number to rotate the cache weekly. On a miss the
# restore-keys prefix match restores the previous week's cache,
# avoiding a fully cold build.
#
# See: https://github.com/bazel-contrib/setup-bazel/issues/18
- name: Get current week
id: week
run: echo "week=$(date +%Y-W%V)" >> "$GITHUB_OUTPUT"
- name: Cache Bazel disk cache
uses: actions/cache@v4
with:
path: ~/.cache/bazel-disk
key: bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-${{ steps.week.outputs.week }}
restore-keys: |
bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-
bazel-disk-cache-${{ runner.os }}-main-
bazel-disk-cache-${{ runner.os }}-
- name: Setup Bazel
uses: bazel-contrib/setup-bazel@c5acdfb288317d0b5c0bbd7a396a3dc868bb0f86 # 0.19.0
with:
# Avoid downloading Bazel every time.
bazelisk-cache: true
# Store build cache per workflow.
disk-cache: ${{github.workflow}}
# Share repository cache between workflows.
repository-cache: true
- name: Check Bazelisk
run: bazelisk --version
- name: Build
run: |
bazelisk build --config=dev -c fastbuild //...
bazelisk build --config=dev --disk_cache=~/.cache/bazel-disk -c fastbuild //...
- name: Test
run: >-
bazelisk test --config=dev -c fastbuild
bazelisk test --config=dev --disk_cache=~/.cache/bazel-disk -c fastbuild
--test_size_filters=-enormous
--test_output=errors
--test_filter="-PywrapFstTest.testRandGen"
Expand Down
31 changes: 27 additions & 4 deletions .github/workflows/bazel_x64_linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,22 +36,45 @@ jobs:
USE_BAZEL_VERSION: 9.0.0
steps:
- uses: actions/checkout@v6
# We use actions/cache for the disk cache rather than setup-bazel's
# built-in disk-cache feature because setup-bazel includes a hash of
# all BUILD files in the cache key. Since Bazel's disk cache is
# already content-addressed, this is redundant and causes the entire
# cache (~1.7 GB on Windows) to be re-uploaded whenever any BUILD
# file changes, quickly hitting GitHub's 10 GB limit.
#
# actions/cache entries are immutable (write-once), so we include the
# ISO week number to rotate the cache weekly. On a miss the
# restore-keys prefix match restores the previous week's cache,
# avoiding a fully cold build.
#
# See: https://github.com/bazel-contrib/setup-bazel/issues/18
- name: Get current week
id: week
run: echo "week=$(date +%Y-W%V)" >> "$GITHUB_OUTPUT"
- name: Cache Bazel disk cache
uses: actions/cache@v4
with:
path: ~/.cache/bazel-disk
key: bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-${{ steps.week.outputs.week }}
restore-keys: |
bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-
bazel-disk-cache-${{ runner.os }}-main-
bazel-disk-cache-${{ runner.os }}-
- name: Setup Bazel
uses: bazel-contrib/setup-bazel@c5acdfb288317d0b5c0bbd7a396a3dc868bb0f86 # 0.19.0
with:
# Avoid downloading Bazel every time.
bazelisk-cache: true
# Store build cache per workflow.
disk-cache: ${{github.workflow}}
# Share repository cache between workflows.
repository-cache: true
- name: Check Bazelisk
run: bazelisk --version
- name: Build
run: |
bazelisk build --config=dev -c fastbuild //...
bazelisk build --config=dev --disk_cache=~/.cache/bazel-disk -c fastbuild //...
- name: Test
run: >-
bazelisk test --config=dev -c fastbuild
bazelisk test --config=dev --disk_cache=~/.cache/bazel-disk -c fastbuild
--test_size_filters=-enormous --test_output=errors
//...
32 changes: 28 additions & 4 deletions .github/workflows/bazel_x64_windows.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,24 +39,48 @@ jobs:
USE_BAZEL_VERSION: 9.0.0
steps:
- uses: actions/checkout@v6
# We use actions/cache for the disk cache rather than setup-bazel's
# built-in disk-cache feature because setup-bazel includes a hash of
# all BUILD files in the cache key. Since Bazel's disk cache is
# already content-addressed, this is redundant and causes the entire
# cache (~1.7 GB on Windows) to be re-uploaded whenever any BUILD
# file changes, quickly hitting GitHub's 10 GB limit.
#
# actions/cache entries are immutable (write-once), so we include the
# ISO week number to rotate the cache weekly. On a miss the
# restore-keys prefix match restores the previous week's cache,
# avoiding a fully cold build.
#
# See: https://github.com/bazel-contrib/setup-bazel/issues/18
- name: Get current week
id: week
shell: bash
run: echo "week=$(date +%Y-W%V)" >> "$GITHUB_OUTPUT"
- name: Cache Bazel disk cache
uses: actions/cache@v4
with:
path: ~/.cache/bazel-disk
key: bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-${{ steps.week.outputs.week }}
restore-keys: |
bazel-disk-cache-${{ runner.os }}-${{ github.base_ref || github.ref }}-
bazel-disk-cache-${{ runner.os }}-main-
bazel-disk-cache-${{ runner.os }}-
- name: Setup Bazel
uses: bazel-contrib/setup-bazel@c5acdfb288317d0b5c0bbd7a396a3dc868bb0f86 # 0.19.0
with:
# Avoid downloading Bazel every time.
bazelisk-cache: true
# Store build cache per workflow.
disk-cache: ${{github.workflow}}
# Share repository cache between workflows.
repository-cache: true
- name: Check Bazelisk
run: bazelisk --version
- name: Build
run: |
bazelisk build -c fastbuild //...
bazelisk build -c fastbuild --disk_cache=~/.cache/bazel-disk //...
- name: Test
shell: bash # Avoid default PowerShell syntax headaches.
run: >-
bazelisk test -c fastbuild
bazelisk test -c fastbuild --disk_cache=~/.cache/bazel-disk
--test_size_filters=-enormous
--test_output=errors
--test_tag_filters=-no_windows_msvc
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/cmake_arm64_macos.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ jobs:
uses: hendrikmuhs/ccache-action@1bbbcda0748b3e340dee71a314fa68ffcbd6df79 # v1.2.21
with:
key: cmake-cache-${{env.cache-name}}
# Compressed cache is ~125MB; default 500MB max-size is sufficient.
- name: Configure
run: >-
cmake -S . -B build
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/cmake_x64_linux.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,6 +50,7 @@ jobs:
uses: hendrikmuhs/ccache-action@1bbbcda0748b3e340dee71a314fa68ffcbd6df79 # v1.2.21
with:
key: cmake-cache-${{env.cache-name}}
# Compressed cache is ~156MB; default 500MB max-size is sufficient.
- name: Configure
run: >-
cmake -S . -B build
Expand Down
Loading