feat(ci): keep only what cache.nixos.org lacks in the Actions cache - #2
Merged
Merged
Conversation
The cache entry becomes a Nix binary cache on disk (narinfo files and zstd-compressed NARs) that the checks read as a trusted substituter, instead of an archive of the whole rooted store. After the checks, the run copies the closure of its outputs into it, asks cache.nixos.org which of those paths it serves (a HEAD request per narinfo file), records the rest as this build's root, and prunes the cache to the union of the surviving roots' lists before the save. Sizes for the 10 GB budget come from the narinfo FileSize fields, so they are the compressed sizes GitHub counts rather than the uncompressed closure sizes used before. Keys move from the nix- prefix to cache- so that no entry of the earlier form is restored; the earlier entries expire unused. The push to the clhodapp cache covers this run's outputs; earlier runs pushed theirs. This is the same change the other five pipeline repos carry, with their two follow-up fixes folded in. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The same change the other five pipeline repos carry (ch-ai-workbench #8, ch-emacs-config #15, merman #4, big-console #13, ch-zsh-config #4), with their two follow-up fixes folded in. This repository's copy of the pipeline landed after those branches were cut, so it gets its own.
The Actions cache entry this pipeline keeps between runs held the whole rooted Nix store: everything the checks' closures reach, including the paths cache.nixos.org serves, which every run could fetch from there instead. This changes what the entry holds so that those paths stay out of it.
How it works now
/home/runner/ci-cache: narinfo files and zstd-compressed NARs) plus the root tree underroots/. Nix reads it as a substituter, listed in nix.conf asfile:///home/runner/ci-cache?trusted=true; thetrustedstore parameter accepts that store's unsigned paths and no other's.nix flake check, the run copies the closure of the flake's checks and packages into the binary cache, asks cache.nixos.org which of those paths it serves (a HEAD request per path for its narinfo file, 32 in parallel; a failed request counts as not served, which only keeps more), and records the rest as this build's root (roots/<class>/<id>/paths). The whole closure has to go in first, because a binary cache refuses a path whose references it does not hold; the served paths are deleted again before the save, and compressing them with zstd costs seconds.FileSizefields, the compressed sizes GitHub counts, instead of uncompressed closure sizes.actions/cache/restoreandactions/cache/save(v4.3.0) directly.cache-nix-actionand its store garbage collection are gone, and the root step needs no sudo.Transition
Keys move from
nix-Linux-<sha>tocache-Linux-<sha>, so no run restores an entry of the earlier form (a store archive, which the new restore step would extract before Nix is installed). The first run on main after this lands is cold and saves the first new entry; the earlier entries expire unused within GitHub's seven-day window. This pull request changes.github/, socheck-prbuilds it under the request's own scope, which is the first exercise of the new pipeline here.cachix
The push to the clhodapp cache now covers the closure of this run's outputs; earlier runs pushed theirs when they were built, and
cachix pushstill skips what the cache holds. Before, each run re-pushed the closure of every kept root.Verified
actionlint and the repository formatter are clean, and the diff is line for line the one ch-zsh-config carries apart from this repository's two prose passages. In the other repos the check-pr runs on the final commits are green; the workbench entry went from 1.35 GB to 38 MB and ch-emacs-config's from 1.63 GB to 48 MB.
🤖 Generated with Claude Code
Result on this branch
The check-pr run saw 133 paths in the closure, 130 of them served by cache.nixos.org, and kept 3; the saved entry is 20 KB, against 277 MB for the whole-store entry main holds today.