Skip to content

Fail the recorded checksum check in CI instead of skipping it on a shallow checkout - #119

Open
only21mil wants to merge 1 commit into
omacom:mainfrom
only21mil:release-checksum-fail-under-ci
Open

only21mil wants to merge 1 commit into
omacom:mainfrom
only21mil:release-checksum-fail-under-ci

Conversation

@only21mil

@only21mil only21mil commented Nov 10, 2026 •

Copy link
Copy Markdown
Contributor

Problem

tools/release/test/omacom-lane checks that four omarchy-mac-pkgs commits hash to the checksums makepkg recorded for them in omacom/omarchy-pkgs. When the checkout lacks those commits, it prints ok - # skip: ... and passes. #113 made CI check out the full history, so the check runs there now. But nothing stops a later workflow edit from making that checkout shallow again. The check would then skip, CI would stay green, and a change that broke how releases compute a git source's checksum could merge. AGENTS.md: "a skipped test, or a job that failed before reaching the tests, is not a pass."

Change

  • tools/release/test/omacom-lane: when CI is true and the history is missing, the check fails and names the cause. A local shallow clone still skips, as before.
  • .github/workflows/tests.yml: the Tooling suites step passes -e CI to docker exec, so the test inside the Arch container sees the runner's CI=true.
  • Before the check, $tmp/out is cleared, so a failure here no longer prints the earlier hashing test's output under it.

Tests and CI only, so omarchy-mac/version stays as it is.

Verification

Setup. x86_64 Linux (Ubuntu 26.04), a depth-1 clone of each commit. An AI agent ran every check, each local run twice with the same output; no person tested this change.

Steps. CI=true bash tools/release/test/omacom-lane → read the exit status.

Before, main 28abebe: the check skips, exit 0:

ok - # skip: this checkout lacks the omarchy-mac-pkgs history the recorded checksums name
exit 0

After, 7d26b31: the check fails and names the missing history, exit 1:

not ok - CI's checkout lacks the omarchy-mac-pkgs history the recorded checksums name; check it out with fetch-depth: 0
exit 1

Without CI it still skips; a full clone with CI=true passes (details).

  • CI on ccd0c80, GitHub's merge of 7d26b31 into 28abebe (the same tree as 7d26b31; full checkout; Arch Linux container on ubuntu-24.04): ok - omarchy-mac-pkgs commits hash to the checksums makepkg recorded in omarchy-pkgs.

Links. Commits 28abebe (main) and 7d26b31; the regression test; #113.

Not shown

  • This PR's own CI run takes the passing branch, since its checkout is full. It shows the check still runs, not that CI reaches the container. That rests on how docker exec -e NAME works: the docker CLI fills in a bare name from its own environment.
  • tools/acceptance/test/mac-image-run-test.sh and asahi-fresh-run-test.sh were not run locally with CI set. Nothing in this repository reads it apart from omacom-lane.

Prior PRs

Follows #113 (above).

Unedited runs and detail

Terminal captures, as first posted:

Before: omacom-lane on main 28abebe in a depth-1 clone with CI=true skips the recorded checksum check and exits 0.

After: omacom-lane on 7d26b31 fails and names the missing history in a depth-1 clone with CI=true; without CI it skips; a full clone with CI=true runs the check and passes.

Full run on main:

$ # main at 28abebe, depth-1 clone
$ git rev-parse --short HEAD; git rev-parse --is-shallow-repository
28abebe
true
$ CI=true bash tools/release/test/omacom-lane
ok - with edge and rc serving one set there is nothing to promote
ok - a release pins the Mac recipes through one pull request and waits for its merge
ok - a pin never moves backwards, and promotion waits for CI to publish master's version to edge
ok - promotion needs VM evidence of the set's digest and hardware evidence of exactly its boot moves
ok - widening goes through a pull request, and one scoped aarch64 advance promotes exactly the moved packages
ok - a runtime-only move needs no hardware evidence, and an x86_64 change during the advance or an unreadable database stops
ok - a same-version conflict and a fast-ring package are refused on the real path before any pull request or advance
ok - a blocked set opens no pin, and a pin merged for part of the set does not stop the rest of the same commit's pin
ok - a pin takes pkgver from omarchy-mac/version and from the commit's UTC date, resetting pkgrel when it moves
ok - a version file that cannot be a pkgver blocks the pin
ok - a recipe that already pins the commit with a stale pkgver or checksum blocks the pin
ok - mac-aurora-pin pins omacom's linux-aurora forward only and promotes through mac-release
ok - a git source hashes to what makepkg -g records, whatever the operator's git settings
ok - # skip: this checkout lacks the omarchy-mac-pkgs history the recorded checksums name
ok - pinning a git source writes makepkg's checksum of the new commit, and refuses to leave one stale
ok - --lane fork runs the fork adapter, and an unknown lane is refused
exit 0

Full runs on this branch:

$ # this branch at 7d26b31, depth-1 clone
$ git rev-parse --short HEAD; git rev-parse --is-shallow-repository
7d26b31
true
$ CI=true bash tools/release/test/omacom-lane
ok - with edge and rc serving one set there is nothing to promote
ok - a release pins the Mac recipes through one pull request and waits for its merge
ok - a pin never moves backwards, and promotion waits for CI to publish master's version to edge
ok - promotion needs VM evidence of the set's digest and hardware evidence of exactly its boot moves
ok - widening goes through a pull request, and one scoped aarch64 advance promotes exactly the moved packages
ok - a runtime-only move needs no hardware evidence, and an x86_64 change during the advance or an unreadable database stops
ok - a same-version conflict and a fast-ring package are refused on the real path before any pull request or advance
ok - a blocked set opens no pin, and a pin merged for part of the set does not stop the rest of the same commit's pin
ok - a pin takes pkgver from omarchy-mac/version and from the commit's UTC date, resetting pkgrel when it moves
ok - a version file that cannot be a pkgver blocks the pin
ok - a recipe that already pins the commit with a stale pkgver or checksum blocks the pin
ok - mac-aurora-pin pins omacom's linux-aurora forward only and promotes through mac-release
ok - a git source hashes to what makepkg -g records, whatever the operator's git settings
not ok - CI's checkout lacks the omarchy-mac-pkgs history the recorded checksums name; check it out with fetch-depth: 0
exit 1

$ # the same depth-1 clone, run locally without CI
$ env -u CI bash tools/release/test/omacom-lane | grep -F 'recorded checksums'
ok - # skip: this checkout lacks the omarchy-mac-pkgs history the recorded checksums name
exit 0

$ # a full clone of this branch (fetch-depth: 0)
$ cd ../cand-full && git rev-parse --is-shallow-repository
false
$ CI=true bash tools/release/test/omacom-lane | grep -F 'recorded'
ok - omarchy-mac-pkgs commits hash to the checksums makepkg recorded in omarchy-pkgs
exit 0
  • The depth-1 clone of this branch with CI=false: exit 0, skip line.
  • Full clone of 7d26b31 with one recorded checksum changed, CI=true: exit 1, with no leftover output under it. CI=true bash tools/release/test/candidate-set and actionlint .github/workflows/tests.yml: exit 0.
  • The changed-checksum run printed not ok - the checksum of omarchy-mac-pkgs 5b2c815f007d is not the 00000000e396 makepkg recorded.
  • docker exec passes only the variables it is given, so without -e CI the test never sees the runner's CI=true. A bare -e CI takes the runner's value.
  • No other test reads CI, so passing it to the other three tooling suites changes nothing for them. Only the suites job runs omacom-lane, and its omarchy-mac-pkgs checkout already has fetch-depth: 0, so this change leaves CI green.
  • Scope: no package behaviour changes, so nothing here needs cold-boot evidence or a manual change.

Generated with Claude Opus 5.5 in T3 Code Nightly

…missing

tools/release/test/omacom-lane skips the check that omarchy-mac-pkgs commits
hash to the checksums makepkg recorded in omacom/omarchy-pkgs when the
checkout lacks those commits, and prints ok. omacom#113 made CI check out the full
history, but a later shallow checkout would hide the check again.

With CI=true the missing history is now a failure that names the cause; a
local shallow clone still skips. The tooling suites step passes CI into the
Arch container, since docker exec does not carry the runner's environment.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant