fix(ci): make a green signal mean the tests actually ran - #40
Open
sym-bot wants to merge 1 commit into
Open
Conversation
Three defects, found while trying to decide whether a Node 18 hang belonged to PR #38 or to its base. None of them is exotic; together they mean this repo's green signal has not been evidence for some time. A PR TARGETING A STACKED BASE RAN NO CHECKS AT ALL. The pull_request trigger was filtered to `main`, so PR #39 — targeting the cutover branch feat/cmb-only-pin-core — ran zero jobs, while `gh pr view` reported mergeStateStatus CLEAN and `gh pr checks` said "no checks reported". At the surface a reviewer actually looks at, unverified and passing are the same colour. The filter is removed: every branch that opens a PR is checked. Stacked work is exactly the work most worth checking, because its base is not main. A HANG RECORDED ITSELF AS "cancelled", NOT "failed". The Node 18 matrix job has run past six hours twice while Node 22 finished in ~30s. With no job timeout it burns to the platform ceiling; with cancel-in-progress unconditional, any later run supersedes it first. So a persistent, reproducible break left a trail of cancelled runs that reads as CI noise. timeout-minutes: 15 makes a hang fail fast and red, and cancel-in-progress is now conditional: superseding a PR push is fine, superseding `main` destroys the only record of whether the mainline is green. fail-fast: false, so a Node 18 hang can no longer cancel the Node 22 job and take the one working signal down with it. workflow_dispatch, so `main` can be run on demand. Isolating "is this break mine or the base's?" previously required pushing to main — precisely when you least want to. NOT FIXED HERE, AND DELIBERATELY SO: whatever makes Node 18 hang. engines declares node >=18 and the matrix tests it, so dropping it would be quietly narrowing a supported range to make a board go green. This change makes the failure visible and diagnosable in fifteen minutes instead of invisible for six hours; the diagnosis is its own piece of work, and it needs a Node 18 environment to reproduce. The first run of this workflow is expected to go RED on Node 18. That is the point. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SpDHL4kTRWvkuw63KADT6R
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found while trying to decide whether a Node 18 hang belonged to PR #38 or to its base. Three defects; none exotic, but together they mean this repo's green signal has not been evidence for some time.
1. A PR targeting a stacked base ran no checks at all
pull_requestwas filtered tomain. PR #39 targets the cutover branchfeat/cmb-only-pin-core, so it ran zero jobs — whilegh pr viewreportedmergeStateStatus: CLEANandgh pr checkssaid "no checks reported".At the surface a reviewer actually looks at, unverified and passing are the same colour.
Filter removed. Stacked work is exactly the work most worth checking, because its base is not
main.2. A hang recorded itself as "cancelled", not "failed"
The Node 18 matrix job has run past six hours, twice, while Node 22 finished in ~30s. With no job timeout it burns to the platform ceiling; with
cancel-in-progressunconditional, any later run supersedes it first.So a persistent, reproducible break left a trail of cancelled runs that reads as CI noise. It is why
main's history shows almost nothing but cancellations and why the last genuine Node 18 result anyone has is from 26 Jul.timeout-minutes: 15— a hang fails fast and red.cancel-in-progressis now conditional. Superseding a PR push is fine; supersedingmaindestroys the only record of whether the mainline is green.fail-fast: false— a Node 18 hang can no longer cancel the Node 22 job and take the one working signal down with it.3.
maincould not be run on demandNo
workflow_dispatch, so isolating "is this break mine or the base's?" required pushing tomain— precisely when you least want to. Added.Not fixed here, deliberately
Whatever makes Node 18 hang.
enginesdeclaresnode >=18and the matrix tests it, so dropping it would be quietly narrowing a supported range to make a board go green.This change makes the failure visible and diagnosable in fifteen minutes instead of invisible for six hours. The diagnosis is its own piece of work and needs a Node 18 environment to reproduce — I have only Node 22 locally.
Expected result
The first run of this workflow should go RED on Node 18. That is the point. If it goes green, the hang was environmental and this PR still stands on defects 1 and 3.
🤖 Generated with Claude Code
https://claude.ai/code/session_01SpDHL4kTRWvkuw63KADT6R