The macOS installer checks the file it downloaded - #99
Merged
Merged
Conversation
Installing Notary on macOS fails with a checksum mismatch right now. The
download is not the problem: the installer compares it against a different file.
installer expects: 61c71dc5... (notary-0.10.11-linux-aarch64.tar.gz)
the macOS zip is: 2ca2053b... (correct)
The digest came from `grep -o 'sha256:...' | head -1` over the whole release
response, which takes the first digest in the JSON whatever asset it belongs to.
Correct while a release carried one asset, and wrong from the moment the Linux
tarballs were added: GitHub lists assets in upload order, so the first digest
belongs to whichever packaging job finished first.
Plaza has the same bug and hit it at v0.19.0, where macOS happened to lose that
race. Notary's is not intermittent: its Linux job has been winning, so this has
been broken on every macOS install since the Linux tarballs landed.
That is the worst way for a checksum to fail. The bytes are right and the
comparison is against something else, so the tool tells people their download is
corrupt and refuses. A check that cries wolf teaches people to skip checks.
The digest now comes from the asset that names the file. The response is
pretty-printed, so it is flattened BEFORE being split on the `{` that starts
each object: without the flatten every field is already on its own line and the
name and the digest can never meet, which is a way of getting this wrong that
looks like it works when tested against the wrong sample.
The guard is the same one Plaza now carries, as a script rather than shell
buried in a workflow so it can be run by hand. A fixture with a Linux asset
listed first, because that is the case that breaks and it has to break
deterministically. The live release, because a fixture only encodes what I think
the response looks like and what can break this again is GitHub changing exactly
that. And that the shipped script is the one those checks describe.
This reaches people the moment it merges. The installer is fetched from main,
not from a release, so no version bump is involved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Installing Notary on macOS fails with a checksum mismatch right now. I checked against the live release:
The digest came from
grep -o 'sha256:...' | head -1over the whole release response, which takes the first digest in the JSON whatever asset it belongs to. Correct while a release carried one asset, and wrong from the moment the Linux tarballs were added: GitHub lists assets in upload order, so the first digest belongs to whichever packaging job finished first.Plaza has the same bug and hit it at v0.19.0, where macOS happened to lose that race. Notary's is not intermittent. Its Linux job has been winning, so this has been broken on every macOS install since the Linux tarballs landed.
This is the worst way for a checksum to fail. The bytes are right and the comparison is against something else, so the tool tells people their download is corrupt and refuses. A check that cries wolf teaches people to skip checks.
The fix
The digest now comes from the asset that names the file. The response is pretty-printed, so it is flattened before being split on the
{that starts each object. Without the flatten, every field is already on its own line and the name and the digest can never meet, which is a way of getting this wrong that looks like it works if you test it against the wrong sample.The guard
The same one Plaza now carries, as a script rather than shell buried in the workflow so it can be run by hand:
Reach
This fixes every macOS install the moment it merges. The installer is fetched from
main, not from a release, so no version bump is involved.