Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions .github/actions/sccache-cpp/action.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: sccache for DuckDB's C++
description: >-
Route the bundled DuckDB's C and C++ compiles through sccache with the GitHub
Actions cache backend, so a cold Rust build (a new rustc, an evicted cache, a
tag) reuses the objects an earlier run compiled.

# **Only the C/C++ compiles, never rustc.** `libduckdb-sys` builds DuckDB through
# CMake and forwards `CMAKE_C_COMPILER_LAUNCHER` / `CMAKE_CXX_COMPILER_LAUNCHER`
# to it, so setting those two wraps exactly DuckDB's compiler calls. Rust stays
# with rust-cache: wrapping rustc too (`RUSTC_WRAPPER`) would store every crate a
# second time in the repository's 10 GB of cache, where rust-cache already keeps
# `target/`.
#
# **Windows needs Ninja and the MSVC environment.** CMake ignores a compiler
# launcher under its default Visual Studio generator. `libduckdb-sys` switches to
# Ninja by itself when `ninja` is on PATH, and the developer environment puts
# Visual Studio's bundled Ninja and `cl` there; the step after it fails the job
# if it did not, rather than letting the launcher be silently ignored.

runs:
using: composite
steps:
- uses: mozilla-actions/sccache-action@fc920bf0ec8de6ee65d409111f7ec508035751ba # v0.0.11
with:
version: v0.19.0
- name: Point CMake's compiler launchers at sccache
shell: bash
run: |
{
echo "SCCACHE_GHA_ENABLED=true"
echo "CMAKE_C_COMPILER_LAUNCHER=sccache"
echo "CMAKE_CXX_COMPILER_LAUNCHER=sccache"
} >> "$GITHUB_ENV"
- name: MSVC developer environment (Windows)
if: runner.os == 'Windows'
uses: ilammy/msvc-dev-cmd@0b201ec74fa43914dc39ae48a89fd1d8cb592756 # v1.13.0
- name: Require Ninja (Windows)
if: runner.os == 'Windows'
shell: bash
run: |
if ! command -v ninja >/dev/null; then
echo "::error::ninja is not on PATH, so CMake would use the Visual Studio generator and ignore the sccache launcher"
exit 1
fi
ninja --version
5 changes: 5 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,9 @@ jobs:
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2
with:
save-if: ${{ github.ref == 'refs/heads/main' }}
# DuckDB's C++ through sccache, here and in the two jobs below — see the
# action for why only the C/C++ compiles and not rustc.
- uses: ./.github/actions/sccache-cpp
- name: Formatting
run: cargo fmt --all -- --check
# `--features schemaic-db/live-tests` is what compiles the live engine
Expand Down Expand Up @@ -140,6 +143,7 @@ jobs:
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2
with:
save-if: ${{ github.ref == 'refs/heads/main' }}
- uses: ./.github/actions/sccache-cpp
# The lint job runs on Linux, where `#[cfg(windows)]` and
# `#[cfg(target_os = "macos")]` code is never compiled — so clippy had
# never seen it: Windows-only code was linted only when someone happened
Expand Down Expand Up @@ -265,6 +269,7 @@ jobs:
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2
with:
save-if: ${{ github.ref == 'refs/heads/main' }}
- uses: ./.github/actions/sccache-cpp
# **Contained database authentication on**, so SQL Server's contained-user
# round trip runs rather than reporting itself skipped. It is a server-wide
# setting, which is why the test only reads it and this step — on a
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,13 @@ jobs:
- uses: Swatinem/rust-cache@d1527c8f67ef9af357290475d4af4b0fab8f12e0 # v2
with:
save-if: false
# **DuckDB's C++ from `main`'s sccache.** A tag run can read caches saved
# from the default branch, and the dev profile builds dependencies at the
# release profile's `opt-level` and `debug`, so CMake should invoke the
# compiler identically and the release can reuse the objects CI compiled.
# Linux is the expected exception: zigbuild compiles with `zig cc`, not
# CI's gcc, so its objects are its own.
- uses: ./.github/actions/sccache-cpp

# Linux: build against an older glibc (2.31 = Ubuntu 20.04 / Debian 11) with
# cargo-zigbuild, so the binary runs on distros older than the CI runner
Expand Down
66 changes: 66 additions & 0 deletions .github/workflows/sccache-probe.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
name: sccache probe (temporary)

# **Delete before merging.** Measures the one thing docs/roadmap.md says to
# measure before relying on sccache: whether a cold `--release` build reuses the
# DuckDB objects a dev build compiled. No rust-cache here on purpose, so DuckDB
# is compiled every time and the sccache stats are the whole answer. Read the
# hit rate in each "stats" step, not the wall-clock time.
on:
pull_request:

concurrency:
group: sccache-probe-${{ github.ref }}
cancel-in-progress: true

env:
CARGO_TERM_COLOR: always

jobs:
probe:
name: Probe (${{ matrix.os }})
runs-on: ${{ matrix.os }}
timeout-minutes: 120
strategy:
fail-fast: false
matrix:
os: [ubuntu-latest, windows-latest, macos-latest]
steps:
- uses: actions/checkout@3d3c42e5aac5ba805825da76410c181273ba90b1 # v7.0.1
- uses: dtolnay/rust-toolchain@b725eda525362e7cf34af9a7243cac1e3f01a7ec # stable
with:
toolchain: stable
- uses: ./.github/actions/sccache-cpp
- name: Dev build (fills sccache)
shell: bash
run: |
sccache --zero-stats
cargo build -p schemaic-db
- name: Stats after the dev build
shell: bash
run: sccache --show-stats
- name: Release build (should hit)
shell: bash
run: |
sccache --zero-stats
cargo build --release -p schemaic-db
- name: Stats after the release build
shell: bash
run: sccache --show-stats
- name: Install Zig (Linux)
if: runner.os == 'Linux'
uses: mlugg/setup-zig@d1434d08867e3ee9daa34448df10607b98908d29 # v2
with:
version: 0.13.0
- name: Install cargo-zigbuild (Linux)
if: runner.os == 'Linux'
run: cargo install --locked cargo-zigbuild
- name: Zigbuild release (Linux, expected to miss)
if: runner.os == 'Linux'
shell: bash
run: |
sccache --zero-stats
cargo zigbuild --release -p schemaic-db --target x86_64-unknown-linux-gnu.2.31
- name: Stats after the zigbuild
if: runner.os == 'Linux'
shell: bash
run: sccache --show-stats
Loading