Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
50 changes: 47 additions & 3 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,34 @@ jobs:
uuid-dev libudev-dev libblkid-dev libtirpc-dev libcurl4-openssl-dev \
libaio-dev libattr1-dev libffi-dev zlib1g-dev libpam0g-dev \
python3 python3-dev python3-all-dev python3-cffi python3-setuptools python3-packaging python3-sphinx \
python3-distlib parted
python3-distlib parted ccache

# Compiled objects carry over between builds through ccache, kept in
# GitHub's own Actions cache: a commit that leaves most of the kernel
# unchanged rebuilds only what changed. The cache holds only this
# workflow's own compiler output, saved per branch; a pull request
# cannot write to the cache that main restores. Hit rates are printed
# after each build.
- name: Restore compiler cache
uses: actions/cache/restore@v4
with:
path: ~/.ccache
key: ccache-${{ matrix.deb_arch }}-${{ needs.meta.outputs.kernel_version }}-${{ github.run_id }}-${{ github.run_attempt }}
restore-keys: |
ccache-${{ matrix.deb_arch }}-${{ needs.meta.outputs.kernel_version }}-
ccache-${{ matrix.deb_arch }}-

- name: Use the compiler cache
run: |
echo "/usr/lib/ccache" >> "$GITHUB_PATH"
cat >> "$GITHUB_ENV" <<EOF
CCACHE_DIR=$HOME/.ccache
CCACHE_BASEDIR=$GITHUB_WORKSPACE
CCACHE_NOHASHDIR=true
CCACHE_MAXSIZE=5G
EOF
/usr/lib/ccache/gcc --version | head -1
CCACHE_DIR=$HOME/.ccache ccache --zero-stats

- name: Check ROCm kernel config
run: make check-rocm-config
Expand All @@ -140,8 +167,10 @@ jobs:
SECUREBOOT_SIGNING_CERT_PEM: ${{ secrets.RAKUEN_SECUREBOOT_SIGNING_CERT }}
run: |
set -euo pipefail
d="$(mktemp -d /dev/shm/signing.XXXXXX)"
chmod 700 "$d"
# A fixed path: the key's path is part of the kernel config, which
# every file includes, so a random one would defeat ccache.
d=/dev/shm/smoothkernel-signing
mkdir -m 700 "$d"
echo "dir=$d" >> "$GITHUB_OUTPUT"

have_modsig=0
Expand Down Expand Up @@ -192,6 +221,21 @@ jobs:
- name: Build OpenZFS
run: make zfs DEB_ARCH=${{ matrix.deb_arch }} BUILD_THREADS="$(nproc)"

- name: Compiler cache statistics
if: always()
continue-on-error: true
run: ccache --show-stats

# Saved even when the build failed, so a re-run starts warm. Only an
# optimisation: a failure to save never fails the build.
- name: Save compiler cache
if: always()
continue-on-error: true
uses: actions/cache/save@v4
with:
path: ~/.ccache
key: ccache-${{ matrix.deb_arch }}-${{ needs.meta.outputs.kernel_version }}-${{ github.run_id }}-${{ github.run_attempt }}

- name: Validate packaged ZFS DKMS against the built kernel
run: sudo make ENV_FILE=examples/smooth.env check-zfs-dkms DEB_ARCH=${{ matrix.deb_arch }} BUILD_THREADS="$(nproc)"

Expand Down
37 changes: 28 additions & 9 deletions docs/CI_RELEASES.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,15 +60,34 @@ The workflow:

1. Checks out the repository.
2. Installs build dependencies.
3. Copies `examples/smooth.env` to `build.env`.
4. Runs `make kernel` for amd64 on `ubuntu-24.04` and arm64 on `ubuntu-24.04-arm`.
5. Runs `make zfs` for each architecture.
6. Uploads per-architecture package artifacts.
7. Merges amd64 and arm64 packages into `dist/`, skipping duplicate `_all.deb` assets.
8. Writes `dist/SHA256SUMS`.
9. Writes `dist/manifest.json` with artifact architectures and sha256 values.
10. Attests the build provenance of every `.deb` (see below).
11. Publishes a GitHub Release with `.deb`, checksum, and manifest assets.
3. Restores the compiler cache (see below).
4. Copies `examples/smooth.env` to `build.env`.
5. Runs `make kernel` for amd64 on `ubuntu-24.04` and arm64 on `ubuntu-24.04-arm`.
6. Runs `make zfs` for each architecture.
7. Saves the compiler cache and uploads per-architecture package artifacts.
8. Merges amd64 and arm64 packages into `dist/`, skipping duplicate `_all.deb` assets.
9. Writes `dist/SHA256SUMS`.
10. Writes `dist/manifest.json` with artifact architectures and sha256 values.
11. Attests the build provenance of every `.deb` (see below).
12. Publishes a GitHub Release with `.deb`, checksum, and manifest assets.

## Compiler Cache

Builds reuse compiled objects through ccache, stored in this repository's
GitHub Actions cache per architecture and kernel version. A commit that leaves
most of the kernel alone, such as a packaging change, recompiles only what it
changed; a kernel version bump starts mostly cold. Each build job prints its
hit rate in the "Compiler cache statistics" step.

The cache only holds what this workflow compiled in earlier runs, on GitHub's
runners. Caches are scoped by branch: `main` restores only caches saved by
`main`, so a pull request cannot put anything into a release build.

For the cache to hit, nothing that changes per build may reach the kernel
config, which every file includes. The build number is therefore not in
`CONFIG_LOCALVERSION`: it is a `localversion-00-build` file in the source
tree, which the kernel places ahead of `CONFIG_LOCALVERSION` in the release
string. Signing material is likewise written to a fixed path.

## Provenance

Expand Down
22 changes: 13 additions & 9 deletions recipes/build-kernel.sh
Original file line number Diff line number Diff line change
Expand Up @@ -116,15 +116,15 @@ PATCH_ROOT="$ROOT/patches"
VERSIONED_CONFIG_DIR="$ROOT/configs/$KERNEL_VERSION"
VERSIONED_CONFIG="$VERSIONED_CONFIG_DIR/smooth-$DEB_ARCH.config"

# The checked-in configs keep the plain LOCALVERSION; only a package build
# carries the build number.
if [[ "$MODE" == "update-config" ]]; then
CONFIG_LOCALVERSION="$LOCALVERSION"
else
# The config keeps the plain LOCALVERSION. A package build adds the build
# number through a localversion file in the source tree, which the kernel puts
# ahead of CONFIG_LOCALVERSION (scripts/setlocalversion). Kept out of the
# config, it stays out of autoconf.h, which every file includes, so a new
# build number does not change what is compiled and ccache can reuse it.
if [[ "$MODE" != "update-config" ]]; then
BUILD_NUMBER="$("$ROOT/recipes/build-number.sh")"
CONFIG_LOCALVERSION="-${BUILD_NUMBER}${LOCALVERSION}"
KERNEL_RELEASE="${KERNEL_VERSION}${CONFIG_LOCALVERSION}" # e.g. 7.2.10-68-smoothkernel
PKG_VERSION="${KERNEL_VERSION}-${BUILD_NUMBER}" # e.g. 7.2.10-68
KERNEL_RELEASE="${KERNEL_VERSION}-${BUILD_NUMBER}${LOCALVERSION}" # e.g. 7.2.10-68-smoothkernel
PKG_VERSION="${KERNEL_VERSION}-${BUILD_NUMBER}" # e.g. 7.2.10-68
fi

kernel_make_args=(ARCH="$KERNEL_ARCH" KBUILD_DEBARCH="$DEB_ARCH")
Expand Down Expand Up @@ -583,7 +583,7 @@ apply_smoothkernel_profile() {
scripts/config --module ACPI_CALL
fi

scripts/config --set-str LOCALVERSION "$CONFIG_LOCALVERSION"
scripts/config --set-str LOCALVERSION "$LOCALVERSION"
kernel_make olddefconfig </dev/null >/dev/null
}

Expand Down Expand Up @@ -646,6 +646,10 @@ if [[ "$MODE" == "update-config" ]]; then
exit 0
fi

# The build number, ahead of CONFIG_LOCALVERSION (see KERNEL_RELEASE above).
# The .deb names checked below confirm the resulting release.
printf '%s\n' "-${BUILD_NUMBER}" > localversion-00-build

echo "==> building (-j${BUILD_THREADS})"
date
# Recursive make must not inherit LOCALVERSION from the top-level wrapper,
Expand Down
Loading