Skip to content

Fix: Required CI gate fails when the lifecycle gate fails (#796) - #797

Merged
0xLeif merged 5 commits into
mainfrom
fix/required-gate-honors-lifecycle
Sep 26, 2026
Merged

0xLeif merged 5 commits into
mainfrom
fix/required-gate-honors-lifecycle

Conversation

@0xLeif

@0xLeif 0xLeif commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #796.

Summary

  • The bug. Required CI gate passes when SpecSync implementation ready (implementation-gate) passes. That job needed neither preflight nor lifecycle-gate, and it accepted success or skipped from every job it did need. When the lifecycle gate failed, test, audit, coverage and spec-check were skipped rather than failed, and both gates went green. Fix: lifecycle commits stage only what the change owns, never git add -A #795 showed it at bb1d80f2.
  • The fix (.github/workflows/ci.yml). implementation-gate now needs every job that can finish before it, preflight and lifecycle-gate included. Its step reads one GATES row per job, <job> <selected> <result>, where <selected> is that job's own if: (less a leading always() &&), evaluated again over the same classify outputs. skipped passes only where that says classify deselected the job. A selected job that was skipped had a dependency that did not succeed, and it fails the gate. The old "nothing in needs failed or was cancelled" check stays, and the gate also fails if its rows do not cover needs. ci-gate is unchanged.
  • The guard (.github/scripts/test-required-ci-gate.py, 29 tests). It parses ci.yml with Psych, like the other workflow validators, and fails if preflight or lifecycle-gate leaves implementation-gate.needs, if a job that gates on lifecycle-gate or otherwise runs before the gate is missing, or if a row's selection drifts from its job's if:. It then simulates the job graph (with GitHub's implicit, transitive success()) for every classify lane and runs the gate's own bash under bash -e, which is how the runner invokes a step that names no shell. The pre-CI: Required CI gate passes when the lifecycle gate fails (skipped jobs read as green) #796 gate, kept as a fixture, reproduces the bug in the same harness. It runs in validate-action and as the Fledge task ci-gate-test in the verify, ci and repo lanes.
  • Docs and spec. docs/HLD.md says what the implementation gate requires. The github delta adds the invariant and REQ-github-021; the github companions record the decision and the tests.
  • SpecSync change required-ci-gate-fails-when-the-lifecycle-gate-fails: a draft, waiting on Leif's definition approval.

This pull request proves the fix

The change is not approved yet, so Lifecycle gate fails on this head. Before this fix that left Required CI gate green, as on #795 at bb1d80f2. On this PR's first head 17e63f5d (run 36252382792), Lifecycle gate failed, test, audit, coverage and spec-check were skipped exactly as on #795, and this time SpecSync implementation ready and Required CI gate both failed, with one annotation per cause:

lifecycle-gate was selected and ended with: failure
test was selected but skipped, so a job it needs did not succeed
spec-check was selected but skipped, so a job it needs did not succeed
audit was selected but skipped, so a job it needs did not succeed
coverage was selected but skipped, so a job it needs did not succeed
a job this gate needs ended with: failure

They stay red until the change is approved and change check --commit lands.

Gate rule

Row selected Job result Verdict
true success pass
false skipped pass (classify deselected it)
true skipped fail: a job it needs did not succeed
true failure / cancelled fail
false anything but skipped fail: the row no longer matches the job's if:

Truth table

Generated by python3 .github/scripts/test-required-ci-gate.py --truth-table. classify, preflight and lifecycle-gate are selected on every path. "Required CI gate" verdicts:

Classify path Event Selected besides classify, preflight, lifecycle-gate Deselected, skipped passes All selected green lifecycle-gate fails preflight fails Any one selected job fails or is cancelled lifecycle-gate fails, before this fix
Full PR (src/, tests/, workflows, docs/, *.md) pull_request test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check corvid-pet green red red red in 26/26 green
Full PR, change awaiting scoped review pull_request test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check, corvid-pet none green red red red in 28/28 green
Site-only PR (site/**) pull_request site, validate-action, spec-check test, fmt, hi-check, audit, coverage, vscode-extension, action-consumer, corvid-pet green red red red in 12/12 green
VS Code-only PR (vscode-extension/**) pull_request vscode-extension, validate-action, spec-check test, fmt, hi-check, audit, coverage, site, action-consumer, corvid-pet green red red red in 12/12 green
Site and VS Code PR pull_request site, vscode-extension, validate-action, spec-check test, fmt, hi-check, audit, coverage, action-consumer, corvid-pet green red red red in 14/14 green
Specs/lifecycle-only PR (specs/, .specsync/changes/) pull_request validate-action, spec-check test, fmt, hi-check, audit, coverage, site, vscode-extension, action-consumer, corvid-pet green red red red in 10/10 green
Specs/lifecycle-only PR, awaiting scoped review pull_request validate-action, spec-check, corvid-pet test, fmt, hi-check, audit, coverage, site, vscode-extension, action-consumer green red red red in 12/12 green
Archive-only PR (workflow-v2 archive move) pull_request none test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check, corvid-pet green red red red in 6/6 green
Legacy archive-only PR (workflow-v1) pull_request test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check corvid-pet green red red red in 26/26 green
Review-only PR (review.json + review-attempts.json) pull_request none test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check, corvid-pet green red red red in 6/6 green
Push to main (full) push test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check corvid-pet green red red red in 26/26 green
Push to main, verifying change present push test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check corvid-pet green red red red in 26/26 green
Push to main (site-only) push site, validate-action, spec-check test, fmt, hi-check, audit, coverage, vscode-extension, action-consumer, corvid-pet green red red red in 12/12 green
Push to main (specs/lifecycle-only) push validate-action, spec-check test, fmt, hi-check, audit, coverage, site, vscode-extension, action-consumer, corvid-pet green red red red in 10/10 green
workflow_dispatch (forced full) workflow_dispatch test, fmt, hi-check, audit, coverage, site, vscode-extension, validate-action, action-consumer, spec-check corvid-pet green red red red in 26/26 green

On every one of the 192 combinations of the six classify flags the job conditions read, under each of pull_request, push and workflow_dispatch, the required gate is green when every selected job succeeds.

Found on the way, not changed here

  • attest ("Record attestation") has been skipped on each of the last 15 pushes to main, including runs where Required CI gate succeeded. Job-level success() looks at every transitive dependency, and corvid-pet, two levels up, is always skipped on pushes. I've left it for its own issue.
  • corvid-pet's status table has no rows for preflight or lifecycle-gate, and it labels every skipped as "Not selected". So on a pull request that needs a scoped review, a failed lifecycle gate still gets a passing review summary. The required gate is now red in that case, so merge is blocked, but the review comment is misleading.

Test Plan

  • python3 -S .github/scripts/test-required-ci-gate.py: 29 tests pass.
  • The same tests run against ci.yml from origin/main fail 65 cases in five tests (contract, needs, CI wiring, the CI: Required CI gate passes when the lifecycle gate fails (skipped jobs read as green) #796 reproduction, and 60 cases where preflight or lifecycle-gate fails or is cancelled across the 15 lanes).
  • validate-workflow-runtime-pins.py, validate-release-version.py and test-classify-ci-paths.sh pass on the changed workflow.
  • fledge lanes run verify: fmt, clippy, check, 2504 unit and 437 integration tests, release build, strict spec check (62/62, 100% coverage), release-candidate tests, ci-gate-test.
  • fledge lanes run pre-push and fledge trust verify.
  • CI on 17e63f5d: Lifecycle gate red on the unapproved draft, so SpecSync implementation ready and Required CI gate are red. validate-action ran the new tests (29, OK).
  • After approval and change check --commit: Required CI gate green.

🤖 Generated with Claude Code

https://claude.ai/code/session_01V3ZZAEiUP7xRJPozhZb6rL

`Required CI gate` passes when `implementation-gate` ("SpecSync
implementation ready") passes. That job needed neither `preflight` nor
`lifecycle-gate`, and it accepted `success` or `skipped` from every job
it did need. When the lifecycle gate failed, `test`, `audit`, `coverage`
and `spec-check` were skipped rather than failed, and both gates went
green. #795 showed it at bb1d80f.

`implementation-gate` now needs every job that can finish before it,
`preflight` and `lifecycle-gate` included. Its step reads one row per
job: the job's own `if:` (less a leading `always() &&`), evaluated again
over the same classify outputs, and the job's result. `skipped` passes
only where that says classify deselected the job. A selected job that
was skipped had a dependency that did not succeed, and it fails the
gate. The old check that nothing in `needs` failed or was cancelled
stays, and the gate fails if its rows do not cover `needs`.

`.github/scripts/test-required-ci-gate.py` guards it. It fails if
`preflight` or `lifecycle-gate` leaves `implementation-gate.needs`, if a
job that gates on `lifecycle-gate` or otherwise runs before the gate is
missing, or if a row drifts from its job's `if:`. It simulates the job
graph for every classify lane and flag combination under
pull_request, push and workflow_dispatch, runs the gate's own bash, and
requires the required gate to be green when every selected job succeeds
and red when any one fails or is cancelled. The pre-#796 gate,
kept as a fixture, reproduces the bug in the same simulation. It runs
in `validate-action` and as the Fledge task `ci-gate-test` in the
verify, ci and repo lanes.

SpecSync change: required-ci-gate-fails-when-the-lifecycle-gate-fails
(draft; definition approval is left to the owner).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3ZZAEiUP7xRJPozhZb6rL
@0xLeif
0xLeif requested a review from a team as a code owner September 26, 2026 15:33
@0xLeif
0xLeif requested review from 0xGaspar, Kyntrin and tofu-ux and removed request for a team September 26, 2026 15:33
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 26, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-26T17:06:08.145328Z 15d0e41 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

The CI log for the first push shows the gate's step running as
`/usr/bin/bash -e {0}`: a step that names no shell gets `bash -e`, not
the `bash --noprofile --norc -eo pipefail` that `shell: bash` gets. The
simulation now uses the same invocation for each case. Results are
unchanged.

Record the live check in the change's testing notes: on the unapproved
draft `Lifecycle gate` failed, `test`, `audit`, `coverage` and
`spec-check` were skipped, and `SpecSync implementation ready` and
`Required CI gate` both failed, with one annotation per cause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01V3ZZAEiUP7xRJPozhZb6rL

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Corvin says...

      _
    <(^\  .oO(Caw! ^v^)
     |/(\
      \(\\
      " "\\

"That's a nice looking export you've got there."

CI Summary

Check Status
Validate action.yml ✅ Passed
Packaged Action Consumer ✅ Passed
Dependency Audit ✅ Passed
Code Coverage ✅ Passed
Format Check ✅ Passed
Human intent check ✅ Passed
Docs Site ✅ Passed
Spec Validation ✅ Passed
Tests (build, test, clippy) ✅ Passed
VS Code Extension ✅ Passed
📋 Spec Validation Details

✅ SpecSync: Passed

Metric Value
Specs checked 62
Passed 62
Errors 0
Warnings 0
File coverage 100% (107/107)
LOC coverage 100% (149230/149230)

Generated by specsync · Run specsync check --format github to reproduce


Powered by corvid-pet

@0xLeif
0xLeif merged commit 6a47f2c into main Sep 26, 2026
22 checks passed
@0xLeif
0xLeif deleted the fix/required-gate-honors-lifecycle branch September 26, 2026 17:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

CI: Required CI gate passes when the lifecycle gate fails (skipped jobs read as green)

1 participant