fix(deploy): report and gate on whether the app can sign anyone in - #22
Merged
Merged
Conversation
The Deploy check verifies that the VM is serving the commit that was built. It does not verify that the commit works, and for this particular rollout those are very different things. The image that switches sign-in to Auth.js can reach the VM before anyone sets the Auth.js environment variables. Nothing about that looks broken from outside: the app boots, serves every page, and /api/health returns "healthy" -- because its integration flags all predate Auth.js and check Clerk-era variables. The only symptom is that nobody can log in, and the deploy that caused it is green. So /api/health now reports an `auth` flag, and verify-rollout fails on it once the rollout itself has been confirmed. Publishing was not shipping and shipping is not working; this is the same lesson one step further along. The flag is a bare boolean. This endpoint is public -- the container healthcheck polls it with no session -- and publishing the names of the secrets a deployment is missing is a map for anyone probing it. The names go to the container log, where an operator can act on them. Five variables count as required: AUTH_SECRET, AUTH_ADAPTER_SECRET, and the three AUTH_JWT_* values. Deliberately not the OAuth client IDs -- a deployment with no Google app is degraded rather than broken, since credentials and magic-link sign-in still work, and firing this flag for something a user can work around would make it mean less when it fires for something they cannot. Two things that would have made the check useless: jq's // operator treats false as unset, so `.integrations.auth // empty` returns empty for auth:false -- precisely the case being checked -- and the gate would have passed silently. It tests for the key explicitly instead. An image built before this change has no `auth` field at all. That reads as empty and is deliberately not a failure: an old image has no opinion, and failing on it would block the very rollout that introduces the flag.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
verify-rolloutchecks that the VM is serving the commit that was built. It does not check that the commit works, and for this rollout those are very different things.The image that switches sign-in to Auth.js can reach the VM before anyone sets the Auth.js environment variables. Nothing looks broken from outside: the app boots, serves every page, and
/api/healthreturns"healthy"— because its integration flags all predate Auth.js and check Clerk-era variables. The only symptom is that nobody can log in, and the deploy that caused it is green.Publishing was not shipping; shipping is not working. Same lesson, one step further along.
What changes
/api/healthreportsintegrations.auth.verify-rolloutfails on it, but only after confirming the rollout itself — so it stays one clear failure at a time.Details worth checking
The flag is a bare boolean, on purpose. This endpoint is public — the container healthcheck polls it with no session — and publishing the names of the secrets a deployment is missing is a map for anyone probing it. The names go to the container log instead, where an operator can act on them.
Five variables count as required:
AUTH_SECRET,AUTH_ADAPTER_SECRET, and the threeAUTH_JWT_*values. Deliberately not the OAuth client IDs — a deployment with no Google app is degraded rather than broken, since credentials and magic-link sign-in still work. Firing this flag for something a user can work around would make it mean less when it fires for something they cannot.Two things that would have made the check useless, both fixed here:
//operator treatsfalseas unset, so.integrations.auth // emptyreturns empty forauth: false— precisely the case being checked — and the gate would have passed silently. It now tests for the key explicitly.authfield at all. That reads as empty and is deliberately not a failure: an old image has no opinion, and failing on it would block the very rollout that introduces the flag.Verification
tsc --noEmitclean, production build green.bash -n.authkey → empty (passes),auth:false→false(fails),auth:true→true(passes).Note on the current Deploy failure
This does not make the failing check on
mainpass. That failure is correct — production is still serving1d98fb9becausecommit-watchtowerdoes not exist, which was measured over a full poll interval. The fix for that remains the whole-stackdocker compose pull && docker compose up -don the VM.Right now that failing check is the only thing stopping
b8a4601from reaching users without theAUTH_*variables set. Set them before the redeploy.