ci(release): add canary upgrade coverage#1981
Conversation
krishicks
requested review from
a team,
derekwaynecarr,
maxamillion and
mrunalp
as code owners
| - name: Clean install latest release and check status | ||
| run: | | ||
| set -euo pipefail | ||
| curl -LsSf "$INSTALL_SH_URL" | sh |
There was a problem hiding this comment.
Question: Should we test against a "stable" reference i.e. the latest release instead?
There was a problem hiding this comment.
So, pinning install.sh to main for the latest release, and using install.sh based on the branch otherwise (meaning when testing the dev release, or a branch's install.sh)? That makes sense.
There was a problem hiding this comment.
I would say that the questions that we want to answer as part of these tests include:
- Can we do a clean install of the current
HEAD(this could be a PR or the tip of a branch)? - Can we upgrade from a previous version? Here the version we're upgrading from should probably be some well-defined reference such as the latest
stablerelease. This allows us to define a multi-step upgrade path from any previous version to the version under test. Note that if we ever introduce major versions we may need to start with multiple references depending on what our support contract is. - Can dowgrade to a previous version? As with upgrades, the target needs to be a well-defined reference such as a release version so that this does not shift with every PR.
We don't need to solve / implement checks for all of these in this PR though.
Although not in scope for this PR, we also need to follow up to define what the expected behaviour of the various components is during an upgrade. For example, what do we expect to happen with running sandboxes or external driver connections?
krishicks
force-pushed
the
hicks/push-vprmssnwomyu
branch
from
e723955 to
9cc7300
Compare
|
Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually. Contributors can view more details about this message here. |
The release canary now exercises both install paths for dev artifacts: a clean install of the rolling dev release, and an upgrade from the latest stable release to dev. The upgrade jobs catch package-manager regressions that only appear when replacing an installed stable package, while the clean-install jobs keep coverage for first-time dev installs. Installer sourcing now follows the release under test. Stable installs use STABLE_INSTALL_SH_URL, which always points at main/install.sh, so the canary does not use a new dev installer to install an older stable release. Dev installs use DEV_INSTALL_SH_URL: automatic workflow_run canaries source install.sh from the moving dev tag, while manual dispatches source it from the dispatched branch ref so workflow and dev installer changes can be tested before they merge. Signed-off-by: Kris Hicks <khicks@nvidia.com>
krishicks
force-pushed
the
hicks/push-vprmssnwomyu
branch
2 times, most recently
from
4e944c2 to
79cd52c
Compare
2 participants
Summary
The release canary now exercises both install paths for dev artifacts: a clean install of the rolling dev release, and an upgrade from the latest stable release to dev. The upgrade jobs catch package-manager regressions that only appear when replacing an installed stable package, while the clean-install jobs keep coverage for first-time dev installs. This is mostly meant to test the packaging scripts do the right thing, not that the upgrade itself is otherwise successful (i.e. how to deal with running sandboxes, component upgrades, etc).
Related Issue
Changes
Testing
mise run pre-commitpassesChecklist