Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1,035 changes: 1,035 additions & 0 deletions .github/scripts/test-required-ci-gate.py

Large diffs are not rendered by default.

68 changes: 61 additions & 7 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -430,6 +430,8 @@ jobs:
run: python3 .github/scripts/validate-workflow-runtime-pins.py
- name: Validate release version consistency
run: python3 .github/scripts/validate-release-version.py
- name: Test the required CI gate
run: python3 .github/scripts/test-required-ci-gate.py

action-consumer:
name: Packaged GitHub Action consumer
Expand Down Expand Up @@ -591,19 +593,70 @@ jobs:
runs-on: ubuntu-latest
timeout-minutes: 5
if: ${{ always() }}
needs: [classify, test, fmt, hi-check, validate-action, action-consumer, spec-check, audit, coverage, site, vscode-extension, corvid-pet]
# Every job that can finish before this one belongs in `needs`, preflight and
# the lifecycle gate included. A job missing from here cannot fail the gate
# (#796). .github/scripts/test-required-ci-gate.py fails when one is missing.
needs: [classify, preflight, lifecycle-gate, test, fmt, hi-check, validate-action, action-consumer, spec-check, audit, coverage, site, vscode-extension, corvid-pet]
steps:
- name: Require every selected gate
# One row per job in `needs`: `<job> <selected> <result>`. <selected> is
# that job's own `if:`, less a leading `always() &&`, evaluated again over
# the same classify outputs (a job with no `if:` is always selected). So
# `skipped` passes only where classify deselected the job. A selected job
# that was skipped had a dependency that did not succeed, and fails the
# gate: before #796 a failed lifecycle gate left test, audit, coverage and
# spec-check skipped and this gate green. test-required-ci-gate.py holds
# each row to its job's `if:` and runs this script on every classify path.
env:
RESULTS: >-
${{ join(needs.*.result, ' ') }}
NEEDS_RESULTS: ${{ join(needs.*.result, ' ') }}
GATES: |
classify true ${{ needs.classify.result }}
preflight true ${{ needs.preflight.result }}
lifecycle-gate true ${{ needs.lifecycle-gate.result }}
test ${{ needs.classify.outputs.full == 'true' }} ${{ needs.test.result }}
fmt ${{ needs.classify.outputs.full == 'true' }} ${{ needs.fmt.result }}
hi-check ${{ needs.classify.outputs.full == 'true' }} ${{ needs.hi-check.result }}
validate-action ${{ needs.classify.outputs.archive_only != 'true' && needs.classify.outputs.review_only != 'true' }} ${{ needs.validate-action.result }}
action-consumer ${{ needs.classify.outputs.full == 'true' }} ${{ needs.action-consumer.result }}
spec-check ${{ needs.classify.outputs.archive_only != 'true' && needs.classify.outputs.review_only != 'true' }} ${{ needs.spec-check.result }}
audit ${{ needs.classify.outputs.full == 'true' }} ${{ needs.audit.result }}
coverage ${{ needs.classify.outputs.full == 'true' }} ${{ needs.coverage.result }}
site ${{ needs.classify.outputs.full == 'true' || needs.classify.outputs.site == 'true' }} ${{ needs.site.result }}
vscode-extension ${{ needs.classify.outputs.full == 'true' || needs.classify.outputs.vscode == 'true' }} ${{ needs.vscode-extension.result }}
corvid-pet ${{ github.event_name == 'pull_request' && needs.classify.outputs.review_required == 'true' }} ${{ needs.corvid-pet.result }}
run: |
for result in $RESULTS; do
status=0
rows=0
while read -r job selected result; do
[[ -n "$job" ]] || continue
rows=$((rows + 1))
case "$selected:$result" in
true:success|false:skipped) continue ;;
true:skipped) why="was selected but skipped, so a job it needs did not succeed" ;;
true:*) why="was selected and ended with: ${result:-no result}" ;;
false:*) why="was not selected yet ended with: ${result:-no result}; its row no longer matches its if:" ;;
*) why="has no usable selection '${selected}'; its row is malformed" ;;
esac
echo "::error::${job} ${why}"
status=1
done <<<"$GATES"
# Belt and braces: the rows must cover `needs` exactly, and nothing in
# `needs` may have failed or been cancelled whatever its row says.
read -r -a results <<<"$NEEDS_RESULTS"
if [[ "$rows" -ne "${#results[@]}" ]]; then
echo "::error::this gate has ${rows} rows for ${#results[@]} jobs in needs"
status=1
fi
for result in "${results[@]}"; do
case "$result" in
success|skipped) ;;
*) echo "Selected CI gate ended with: $result" >&2; exit 1 ;;
*) echo "::error::a job this gate needs ended with: $result"; status=1 ;;
esac
done
if [[ "$status" -ne 0 ]]; then
exit 1
fi
echo "Every selected gate succeeded; every skipped job was deselected by classify."

ci-gate:
name: Required CI gate
Expand All @@ -613,8 +666,9 @@ jobs:
needs: [classify, implementation-gate]
steps:
# One aggregate context, suitable for a required status check. Lifecycle
# coherence is proven by `specsync change audit --strict` in spec-check,
# not by inspecting commit topology.
# coherence is proven by `specsync change audit --strict` in the lifecycle
# gate and in spec-check, not by inspecting commit topology, and the
# implementation gate requires both of them.
- name: Require the implementation gate
env:
IMPLEMENTATION_RESULT: ${{ needs.implementation-gate.result }}
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
{
"schema_version": 1,
"workflow_version": 2,
"workflow_origin_version": 2,
"id": "required-ci-gate-fails-when-the-lifecycle-gate-fails",
"slug": "required-ci-gate-fails-when-the-lifecycle-gate-fails",
"title": "Required CI gate fails when the lifecycle gate fails",
"description": "Required CI gate fails when the lifecycle gate fails",
"kind": "bug_fix",
"state": "accepted",
"canonical_applied": true,
"base_commit": "cddc39e478dcc1f111940a3cfb02134bba9804cc",
"created_at": 1790434419,
"updated_at": 1790441718,
"affected_specs": [
"github"
],
"affected_paths": [
".github/workflows/ci.yml",
".github/scripts/test-required-ci-gate.py",
"fledge.toml",
"docs/HLD.md"
],
"no_spec_change": false,
"no_spec_change_rationale": null,
"acceptance_criteria": [
"Required CI gate fails whenever Lifecycle preflight or Lifecycle gate fails or is cancelled. implementation-gate (SpecSync implementation ready) needs classify, preflight, lifecycle-gate and every other job that can finish before it, and accepts skipped for a job only when that job's own if: condition, evaluated again over the classify outputs, deselected it; a selected job that was skipped because something it needs did not succeed fails the gate. Full, site-only, VS Code-only, specs/lifecycle-only, archive-only, legacy archive-only and review-only pull requests, pushes to main and workflow_dispatch runs stay green when every selected job succeeds. .github/scripts/test-required-ci-gate.py runs in the validate-action CI job and in the Fledge verify lane; it fails if preflight or lifecycle-gate is missing from implementation-gate.needs, if a job that gates on lifecycle-gate or can otherwise finish before the gate is missing, or if a gate row no longer matches its job's if:, and it simulates every classify path to show the required gate red when any one selected job fails or is cancelled and green otherwise, and reproduces #796 against the pre-fix gate."
],
"selected_artifacts": [
"context",
"testing",
"tasks",
"research",
"design",
"plan"
],
"dependencies": [],
"answers": {
"architecture_risk": "yes",
"public_contract": "no"
}
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
{
"approvals": [
{
"gate": "definition",
"actor": "user:0xLeif",
"timestamp": 1790441012,
"digest": "baa6fc958f3c065adeafb9385c73380bedc2abdb580462737d18a48a55704d24",
"note": "Approved by Leif in the orc session, 2026-09-26: spec-sync #797",
"approved_scope": {
"schema_version": 1,
"change_id": "required-ci-gate-fails-when-the-lifecycle-gate-fails",
"title": "Required CI gate fails when the lifecycle gate fails",
"description": "Required CI gate fails when the lifecycle gate fails",
"kind": "bug_fix",
"affected_specs": [
"github"
],
"affected_paths": [
".github/scripts/test-required-ci-gate.py",
".github/workflows/ci.yml",
"docs/HLD.md",
"fledge.toml"
],
"no_spec_change": false,
"no_spec_change_rationale": null,
"acceptance_criteria": [
"Required CI gate fails whenever Lifecycle preflight or Lifecycle gate fails or is cancelled. implementation-gate (SpecSync implementation ready) needs classify, preflight, lifecycle-gate and every other job that can finish before it, and accepts skipped for a job only when that job's own if: condition, evaluated again over the classify outputs, deselected it; a selected job that was skipped because something it needs did not succeed fails the gate. Full, site-only, VS Code-only, specs/lifecycle-only, archive-only, legacy archive-only and review-only pull requests, pushes to main and workflow_dispatch runs stay green when every selected job succeeds. .github/scripts/test-required-ci-gate.py runs in the validate-action CI job and in the Fledge verify lane; it fails if preflight or lifecycle-gate is missing from implementation-gate.needs, if a job that gates on lifecycle-gate or can otherwise finish before the gate is missing, or if a gate row no longer matches its job's if:, and it simulates every classify path to show the required gate red when any one selected job fails or is cancelled and green otherwise, and reproduces #796 against the pre-fix gate."
],
"dependencies": [],
"supersedes": [],
"answers": {
"architecture_risk": "yes",
"public_contract": "no"
}
},
"approved_delta_digests": {
"github": "e1e69409edadb41bd7af8b1992f79007fa2138651348cb2d82b1c652b1490ec7"
}
},
{
"gate": "finalization",
"actor": "specsync:finalization",
"timestamp": 1790441717,
"digest": "846c770fee65169aee2f680122bf3db12bb73e23401ef8c3614dfeb633df9e7e",
"note": "Same-PR finalization closing digest"
}
],
"reopenings": []
}
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
id: required-ci-gate-fails-when-the-lifecycle-gate-fails
state: archived
type: bug_fix
base_commit: cddc39e478dcc1f111940a3cfb02134bba9804cc
---

# Required CI gate fails when the lifecycle gate fails

## Intent

Required CI gate fails when the lifecycle gate fails

## Affected Canonical Specs

- `github`

## Acceptance Criteria

- Required CI gate fails whenever Lifecycle preflight or Lifecycle gate fails or is cancelled. implementation-gate (SpecSync implementation ready) needs classify, preflight, lifecycle-gate and every other job that can finish before it, and accepts skipped for a job only when that job's own if: condition, evaluated again over the classify outputs, deselected it; a selected job that was skipped because something it needs did not succeed fails the gate. Full, site-only, VS Code-only, specs/lifecycle-only, archive-only, legacy archive-only and review-only pull requests, pushes to main and workflow_dispatch runs stay green when every selected job succeeds. .github/scripts/test-required-ci-gate.py runs in the validate-action CI job and in the Fledge verify lane; it fails if preflight or lifecycle-gate is missing from implementation-gate.needs, if a job that gates on lifecycle-gate or can otherwise finish before the gate is missing, or if a gate row no longer matches its job's if:, and it simulates every classify path to show the required gate red when any one selected job fails or is cancelled and green otherwise, and reproduces #796 against the pre-fix gate.

## No-spec Rationale

Not applicable
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
---
change: required-ci-gate-fails-when-the-lifecycle-gate-fails
artifact: context
---

# Context

`Required CI gate` is the only required check on `main`, and it passes when
`implementation-gate` ("SpecSync implementation ready") passes. On #795 at `bb1d80f2`,
`Lifecycle gate` and `trust` failed, `test`, `audit`, `coverage` and `spec-check` never ran,
and both gates were green (#796). orc found it while shipping #795.

The cause is in `.github/workflows/ci.yml`:

- `test`, `audit`, `coverage` and `spec-check` need `lifecycle-gate`. When it fails, GitHub
reports them as `skipped`, not failed.
- `implementation-gate` needed neither `preflight` nor `lifecycle-gate`, and it accepted
`success` or `skipped` from every job it did need.
- `ci-gate` only required `implementation-gate == success`.

So a failed lifecycle gate became four skipped jobs, and skipped read as green.

What a session picking this up needs to know:

- `skipped` alone cannot say why a job did not run. GitHub reports the same result for a job
classify deselected and for a job whose dependency failed. The gate can only tell them apart by
asking the question the job asked, so each gate row carries the job's own `if:` condition,
evaluated again over the same classify outputs.
- Classify outputs are fixed once `classify` finishes, and the gate and the job use the same
expression evaluator. The only way the two answers can differ is if the row and the job's `if:`
are different text. The test compares them as text.
- `preflight` and `lifecycle-gate` have no `if:`, so they are selected on every path, archive-only
and review-only included. Recent archive-only and review-only pull requests (#785, #790, #791)
and the #795 product and archive tips all show `Lifecycle gate: success`, so requiring success
blocks nothing that merges today.
- Job-level `success()` looks at every transitive dependency, not only the direct ones. That is
why `attest` has been skipped on every push to `main` (it needs `ci-gate`, and `corvid-pet`,
two levels up, is skipped on pushes). The simulation models this. Fixing `attest` is out of
scope here and is reported separately.
- `act` is not installed and CI cannot be run locally, so the proof is a simulation of the job
graph that executes the gate's own bash under GitHub's default invocation, plus the live run on
this pull request.
- Ruled out: requiring `success` from every job in `needs`. Archive-only and review-only pull
requests legitimately skip most jobs, and that would block them.
Original file line number Diff line number Diff line change
@@ -0,0 +1,56 @@
## MODIFIED

### SPEC SECTION Invariants

Every path that can be merged can reach the required CI gate; a path the CI
workflow cannot trigger can never report the gate and blocks its pull request.

The required CI gate fails whenever a job it depends on did not succeed, unless classify
deselected that job. A skipped job passes only when its own `if:` condition, evaluated over the
classify outputs, left it unselected. A selected job that was skipped was skipped because a job it
needs did not succeed, and it fails the gate: `skipped` is never read as green on its own. The
lifecycle preflight and the lifecycle gate are selected on every path, so either one failing turns
the required gate red.

Release qualification verifies exactly the tag protections this repository actually has, and names
every protection it does not verify on every run, green runs included. A gate that demands an
unprovisioned policy fails on every candidate and therefore verifies nothing — it is not a safe
default, because the protections that DO exist are never reached. Dropping a check from the gate is
permitted; dropping it silently is not. The tag protections that remain admit no bypass actor and
no broadening — where that can be observed. GitHub returns `bypass_actors` only to a caller with
admin access to repository settings, and the workflow token is not one, so the field is ABSENT
from every payload CI fetches. Absence means UNOBSERVED, never "no bypass actors": it is checked
when visible, refused when it grants anyone, and named in the unenforced disclosure when it cannot
be read. Requiring it made the gate impossible to satisfy from CI, which is how a lane stayed red
on every candidate while appearing to enforce something.

Release authority is stated wherever it is exercised. The final tag is created by the release
workflow's own token under a permission scoped to the single job that writes it, so the authority
to run the release lane is the authority to create a release tag; that equivalence is announced by
every run and recorded at the job itself, never left to be inferred from a green result. A named
deployment environment that does not exist is not a gate — GitHub materializes it unprotected on
first use — so the workflow names no environment rather than publish a gate that gates nothing.

## ADDED

### REQUIREMENT REQ-github-021

The required CI gate SHALL fail whenever the lifecycle preflight, the lifecycle gate, or any job
classify selected for the run fails, is cancelled, or is skipped because a job it needs did not
succeed.

Acceptance Criteria

- `implementation-gate` (SpecSync implementation ready) needs `classify`, `preflight`,
`lifecycle-gate` and every other job that can finish before it, and `ci-gate` (Required CI gate)
passes only when `implementation-gate` succeeds.
- A skipped job passes only when that job's own `if:` condition, evaluated again over the classify
outputs, deselected it.
- Full, site-only, VS Code-only, specs/lifecycle-only, archive-only, legacy archive-only and
review-only pull requests, pushes to `main` and `workflow_dispatch` runs stay green when every
selected job succeeds.
- `.github/scripts/test-required-ci-gate.py` fails when `preflight` or `lifecycle-gate` is missing
from the gate's `needs`, when a job that gates on `lifecycle-gate` or can otherwise finish before
the gate is missing, or when a gate row no longer matches its job's `if:`.
- The same test runs the gate's own script over every classify path and requires the required gate
to be red when any one selected job fails or is cancelled.
Loading
Loading