From 724469c180571fd9b6a043dd18c971f1b0b88133 Mon Sep 17 00:00:00 2001 From: Kyle Sexton <153232337+kyle-sexton@users.noreply.github.com> Date: Thu, 19 Nov 2026 02:17:03 -0500 Subject: [PATCH] chore(formal): move the carry-forward model to the report-ci-status sync commit PR #17283 (726f82db25) runs ci-status on ci-workflows' report-ci-status composite at v0.76.3 (9c5bedf). The carry-forward behavior is unchanged, only moved: - The composite fixes yield-to-full-run 'true' and carry-forward-wait-seconds '0' and calls aggregate-results through `$/` at the same commit, so RUN and ACTION are read at 9c5bedf. - Its contract and draft steps are continue-on-error, so the step the writer check reads concludes on the lanes alone, as the model's gate does. - The hold drops to 0 on a contract-only run whose job, contract or draft step failed: the Hold = FALSE case YieldNoHold.cfg already checks. - aggregate-results run.sh changed between v0.75.0 and v0.76.3 only by extracting in_flight_sibling_ids, run_attempt_of and finish_carry_forward; aggregate-results action.yml is unchanged. The model's comments re-cite WF, RUN and the new REPORT lines; the state machine is untouched. TLC at the new commit: Yield.cfg holds every rule (4,788,560 states), YieldNoHold.cfg too (880,975), the seeded break (Yield = FALSE) breaks NoStaleCarry after 5,056 and 7,972 distinct states, Yield.cfg without VIEW holds (21,746,774 states), and NoStaleCarryAtPass still breaks in the known 10 steps. Co-Authored-By: Claude Opus 5.5 Claude-Session: https://claude.ai/code/session_01Vcupoo79EqmECYFZ4MTv6C --- .../formal-ci-lanes-carry-forward/SKILL.md | 30 ++++--- .../model/CarryForward.tla | 80 ++++++++++--------- .../formal-ci-lanes-carry-forward/sync.yaml | 12 +-- 3 files changed, 65 insertions(+), 57 deletions(-) diff --git a/.claude/skills/formal-ci-lanes-carry-forward/SKILL.md b/.claude/skills/formal-ci-lanes-carry-forward/SKILL.md index c1b7f9a9a19..8f0251d6ec2 100644 --- a/.claude/skills/formal-ci-lanes-carry-forward/SKILL.md +++ b/.claude/skills/formal-ci-lanes-carry-forward/SKILL.md @@ -1,6 +1,6 @@ --- name: formal-ci-lanes-carry-forward -description: "TLA+ model of the `ci-lanes` commit status on one pull request head: full runs of PR Require Checks mark it pending and record the lanes verdict, and a contract-only run (a label flip or body edit) yields to a full run in flight or carries the recorded success forward. Use when a change touches .github/workflows/pr-require-checks.yml's mark-lanes job, the ci-status job's needs or its Aggregate lane results inputs, or repins the ci-workflows aggregate-results composite, or to ask whether a label flip can pass CI Status on an older verdict while a full run is in flight." +description: "TLA+ model of the `ci-lanes` commit status on one pull request head: full runs of PR Require Checks mark it pending and record the lanes verdict, and a contract-only run (a label flip or body edit) yields to a full run in flight or carries the recorded success forward. Use when a change touches .github/workflows/pr-require-checks.yml's mark-lanes job, the ci-status job's needs or its Aggregate lane results inputs, or repins the ci-workflows report-ci-status composite, or to ask whether a label flip can pass CI Status on an older verdict while a full run is in flight." disable-model-invocation: true --- @@ -9,10 +9,11 @@ disable-model-invocation: true ## Property Each rule is an INVARIANT in `model/CarryForward.tla`. `WF` is -`.github/workflows/pr-require-checks.yml`; `RUN` is +`.github/workflows/pr-require-checks.yml`; `REPORT` is +`.github/actions/pr-require-checks/report-ci-status/action.yml`, `RUN` is `.github/actions/pr-require-checks/aggregate-results/run.sh` and `ACTION` its `action.yml`, in `melodic-software/ci-workflows` at the commit `WF` pins on the ci-status job's Aggregate lane -results step. +results step. `REPORT` runs `aggregate-results` at that same commit. | Rule | Plain sentence | Source | |---|---|---| @@ -79,7 +80,7 @@ through an installed-plugin invocation of `/formal-verification:specify`. only compared, so states with the same order behave alike. Under it TLC's distinct-state count depends on exploration order; the chart command runs one worker, so its count is stable. The view does not decide the verdicts: without `VIEW`, `Yield.cfg` holds every rule at - c4fe439dfb47 (21,746,774 states, the chart command with one worker, 4 min 23 s). Rerun without + 726f82db25b4 (21,746,774 states, the chart command with one worker, 3 min 34 s). Rerun without it after any change to `View` or to what holds a tick. - The seeded break in `sync.yaml` was run against this model: with `yield-to-full-run: 'false'` (`Yield = FALSE`), `NoStaleCarry` breaks after 5,056 distinct states in `Yield.cfg` and 7,972 in @@ -111,10 +112,11 @@ One command per cfg; `Yield.cfg` takes about three minutes, `YieldNoHold.cfg` un /formal-verification:chart .claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla --config .claude/skills/formal-ci-lanes-carry-forward/model/YieldNoHold.cfg --view .claude/skills/formal-ci-lanes-carry-forward/model/view.mjs --rerun-states 6000000 ``` -`Yield.cfg` is the setting `WF` passes on an open pull request's label flip: `yield-to-full-run: -'true'` (`Yield`) and a `yield-wait-seconds` of 600 (`Hold`). `YieldNoHold.cfg` is the draft's -setting, `yield-wait-seconds` `0`, with the marker allowed to fail. `--rerun-states` must cover the -whole space (4,788,560 and 880,975 states): after a rule breaks, the command rechecks the others +`Yield.cfg` is the setting on an open pull request's label flip: `yield-to-full-run: 'true'`, which +`REPORT` fixes (`Yield`), and the `yield-wait-seconds` of 600 `WF` passes (`Hold`). +`YieldNoHold.cfg` is a `yield-wait-seconds` of `0` (a draft, or a contract-only run whose job, +contract or draft step already failed), with the marker allowed to fail. `--rerun-states` must +cover the whole space (4,788,560 and 880,975 states): after a rule breaks, the command rechecks the others under that bound, and a smaller bound leaves them not checked. Pass a new `--out` folder each run. @@ -158,15 +160,17 @@ Known results at the sync commit (a run that shows only these has found nothing When a covered file in `sync.yaml` changes, read the change against the model: a moved marker, a changed `needs`, a changed Aggregate lane results input, or a repin of the ci-workflows composite -means the model may change. Read the pinned `run.sh` and `action.yml` at the new commit +means the model may change. Read the pinned `report-ci-status/action.yml`, `run.sh` and +`aggregate-results/action.yml` at the new commit (`gh api -H 'Accept: application/vnd.github.raw' 'repos/melodic-software/ci-workflows/contents/.github/actions/pr-require-checks/aggregate-results/run.sh?ref='`) and re-cite the changed lines in the model's comments. The names the model reads, for a search of new code that touches the same state: in `WF`, the `ci-lanes` context, the `mark-lanes` job, the ci-status job's `needs` and `if:`, the contract-only predicate (`labeled`, `unlabeled`, `edited`), -the `concurrency` group, and the Aggregate lane results step's `aggregate-results` pin and its -`yield-to-full-run`, `yield-wait-seconds`, `carry-forward-wait-seconds` and -`rerun-contract-only-siblings` inputs; in `RUN`, `write_status`, `list_in_flight_full_runs`, -`read_newest_entry`, `read_carried_state`, `verify_carried_writer`, `verify_writer_step`, +the `concurrency` group, and the Aggregate lane results step's `report-ci-status` pin and its +`yield-wait-seconds`, `rerun-contract-only-siblings` and `step-name` inputs; in `REPORT`, the +`continue-on-error` contract and draft steps and the `yield-to-full-run`, +`carry-forward-wait-seconds` and `yield-wait-seconds` it passes to `aggregate-results`; in `RUN`, +`write_status`, `list_in_flight_full_runs`, `read_newest_entry`, `read_carried_state`, `verify_carried_writer`, `verify_writer_step`, `gate_step_outcome`, `verify_no_newer_verdict`, `writer_attempt_wrote`, `carried_postdates`, `held_success_carries`, `yield_hold` and `fail_superseded`. A promise changed on purpose means a rule changes, and only with the owner's yes. Never weaken a rule to make it hold. Move `commit` forward diff --git a/.claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla b/.claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla index c9db8e18381..86092db3287 100644 --- a/.claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla +++ b/.claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla @@ -4,9 +4,13 @@ \* body edit) whose `CI Status` job carries the recorded verdict forward. \* \* Line numbers: WF is .github/workflows/pr-require-checks.yml in this -\* repository; RUN is .github/actions/pr-require-checks/aggregate-results/run.sh -\* and ACTION its action.yml, in melodic-software/ci-workflows at -\* 24067add42f5fa04743c587c6a22aa7d4059e7b8, the commit WF:7327 pins. +\* repository; REPORT is .github/actions/pr-require-checks/report-ci-status/action.yml, +\* RUN is .github/actions/pr-require-checks/aggregate-results/run.sh and ACTION +\* its action.yml, in melodic-software/ci-workflows at +\* 9c5bedfbfdbdfe733b9ce4cd7ca80c1574bd4cb8, the commit WF:7018 pins. REPORT +\* calls aggregate-results through `$/`, so at that same commit (REPORT:119-123). +\* Its contract and draft steps are continue-on-error (REPORT:94-105), so the +\* step WF:7015-7049 concludes on the lanes alone, as the model's gate does. EXTENDS Naturals, FiniteSets @@ -14,9 +18,9 @@ CONSTANTS FullRuns, \* the full runs that may start on the SHA MaxAttempts, \* attempts per full run (a re-run is attempt 2) Rerunnable, \* the full runs a person may re-run - Yield, \* WF:7348 yield-to-full-run - Hold, \* WF:7349 yield-wait-seconds above 0 - MarkMayFail \* mark-lanes may fail after its three tries (WF:948-959) + Yield, \* REPORT:128 yield-to-full-run + Hold, \* WF:7045 yield-wait-seconds above 0 (REPORT:134-140) + MarkMayFail \* mark-lanes may fail after its three tries (WF:761-772) NoRun == "none" States == {"pending", "success", "failure"} @@ -57,7 +61,7 @@ cvars == <> rvars == <> -\* Readers take the newest entry by id (RUN:381-400), so the model keeps only +\* Readers take the newest entry by id (RUN:367-386), so the model keeps only \* that one. Write(s, r) == /\ newest' = [state |-> s, run |-> r, att |-> att[r], t |-> clock + 1] /\ clock' = clock + 1 @@ -104,25 +108,25 @@ Start(r) == /\ clock' = clock + 1 /\ UNCHANGED <> /\ UNCHANGED cvars -\* mark-lanes POSTs ci-lanes=pending on the head (WF:948-954). It runs beside -\* the lanes; ci-status needs it (WF:4239-4240), so it lands before the verdict. +\* mark-lanes POSTs ci-lanes=pending on the head (WF:761-767). It runs beside +\* the lanes; ci-status needs it (WF:4109-4110), so it lands before the verdict. Mark(r) == /\ Moving /\ live[r] /\ mark[r] = "todo" /\ Write("pending", r) /\ mark' = [mark EXCEPT ![r] = "done"] /\ UNCHANGED <> /\ UNCHANGED cvars -\* Three failed POSTs fail the job (WF:956-959); ci-status still runs on -\* !cancelled() (WF:4262) and aggregates that failure. +\* Three failed POSTs fail the job (WF:769-772); ci-status still runs on +\* !cancelled() (WF:4132) and aggregates that failure. MarkFails(r) == /\ Moving /\ MarkMayFail /\ live[r] /\ mark[r] = "todo" /\ mark' = [mark EXCEPT ![r] = "failed"] /\ UNCHANGED <> /\ UNCHANGED cvars -\* ci-status's Aggregate lane results step (WF:7322-7355) records the lanes -\* verdict (RUN:1561-1566, write_status at RUN:250-271). A failure exits 1 -\* (RUN:1568-1570); a success keeps the step running while it re-runs failed -\* contract-only siblings (RUN:1710-1713). +\* ci-status's Aggregate lane results step (WF:7015-7049) records the lanes +\* verdict (RUN:1554-1559, write_status at RUN:236-257). A failure exits 1 +\* (RUN:1561-1563); a success keeps the step running while it re-runs failed +\* contract-only siblings (RUN:1702-1705). Aggregate(r) == /\ Moving /\ live[r] /\ mark[r] # "todo" /\ gate[r] = "none" /\ \E v \in (IF mark[r] = "failed" THEN {"failure"} ELSE {"success", "failure"}) : @@ -130,7 +134,7 @@ Aggregate(r) == /\ gate' = [gate EXCEPT ![r] = IF v = "success" THEN "running" ELSE "failure"] /\ UNCHANGED <> /\ UNCHANGED cvars -\* The aggregate step ends; the sibling re-run cannot fail it (RUN:1607-1608). +\* The aggregate step ends; the sibling re-run cannot fail it (RUN:1600-1601). StepEnds(r) == /\ Moving /\ live[r] /\ gate[r] = "running" /\ gate' = [gate EXCEPT ![r] = "success"] @@ -144,7 +148,7 @@ Complete(r) == \* A newer event on the pull request cancels it (WF:106-115), or a person \* does. A step that was running ends cancelled; ci-status never starts in a -\* cancelled run (WF:4262). +\* cancelled run (WF:4132). Cancel(r) == /\ Moving /\ live[r] /\ live' = [live EXCEPT ![r] = FALSE] @@ -167,7 +171,7 @@ Rerun(r) == \* ------------------------------------------------------------------------ \* The contract-only run's CI Status job, Aggregate lane results step. It may -\* start at any point; a re-run of it (RUN:1609-1713) is a later start. +\* start at any point; a re-run of it (RUN:1602-1705) is a later start. CStart == /\ cpc = "idle" @@ -176,7 +180,7 @@ CStart == holdEntry, timedOut, liveAtPass, listClock>> /\ UNCHANGED rvars \* Yield, step 1: list this workflow's runs on the SHA (list_in_flight_full_runs, -\* RUN:1209-1239, called at RUN:1408). Every full run reads full by its jobs: +\* RUN:1216-1245, called at RUN:1413). Every full run reads full by its jobs: \* mark-lanes runs in every full run (WF:234). CList == /\ cpc = "list" @@ -187,7 +191,7 @@ CList == holdEntry, timedOut, liveAtPass>> /\ UNCHANGED rvars \* Yield, step 2: read the newest bot-written ci-lanes status -\* (read_newest_entry RUN:367-401, from read_carried_state at RUN:1413). +\* (read_newest_entry RUN:353-387, from read_carried_state at RUN:1418). CRead == /\ cpc = "read" /\ entry' = newest @@ -197,10 +201,10 @@ CRead == /\ UNCHANGED <> /\ UNCHANGED rvars -\* The writer check of a success (verify_carried_writer RUN:415-486, -\* verify_writer_step RUN:709-747, gate_step_outcome RUN:763-772, -\* verify_no_newer_verdict RUN:839-880). The writer's row comes from the -\* listing, else from a live read (RUN:460-471); its gate step and each rival's +\* The writer check of a success (verify_carried_writer RUN:401-472, +\* verify_writer_step RUN:695-733, gate_step_outcome RUN:749-758, +\* verify_no_newer_verdict RUN:825-866). The writer's row comes from the +\* listing, else from a live read (RUN:446-457); its gate step and each rival's \* are read live, as one step. WriterStart == IF snap[entry.run].created THEN snap[entry.run].start ELSE start[entry.run] @@ -224,19 +228,19 @@ CVerify == /\ UNCHANGED <> /\ UNCHANGED rvars -\* writer_attempt_wrote (RUN:1374-1379): the run is the writer and its attempt +\* writer_attempt_wrote (RUN:1381-1386): the run is the writer and its attempt \* in the listing started at or before the status was created. Wrote(r) == r = entry.run /\ snap[r].created /\ snap[r].start <= entry.t -\* RUN:1418-1426: a run whose attempt wrote the carried success (or, under a +\* RUN:1423-1431: a run whose attempt wrote the carried success (or, under a \* hold, failure) does not supersede. Skips(r) == (carried = "success" \/ (Hold /\ carried = "failure")) /\ Wrote(r) Superseding == {r \in InFlightSnap : ~Skips(r)} -\* carried_postdates (RUN:1328-1334) over the held runs, held_success_carries -\* (RUN:1339-1353): each held run in the listing started at or before the success. +\* carried_postdates (RUN:1335-1341) over the held runs, held_success_carries +\* (RUN:1346-1360): each held run in the listing started at or before the success. Postdates == \A r \in held : snap[r].created /\ snap[r].start <= entry.t -\* Yield, steps 3-6 (RUN:1417-1467). The single read passes a success once its -\* writer is verified (RUN:1484-1495). +\* Yield, steps 3-6 (RUN:1422-1468). The single read passes a success once its +\* writer is verified (RUN:1481-1488). CDecide == /\ cpc = "decide" /\ IF ~Yield @@ -248,18 +252,18 @@ CDecide == /\ UNCHANGED <> [] wRunning /\ Superseding = {entry.run} /\ Wrote(entry.run) /\ ~timedOut -> - \* The writer's step is finishing (RUN:1431-1449): list again. + \* The writer's step is finishing (RUN:1436-1454): list again. /\ cpc' = "list" /\ UNCHANGED <> [] Hold /\ ~timedOut -> - \* yield_hold (RUN:1452-1454, RUN:1287-1324). + \* yield_hold (RUN:1457-1459, RUN:1294-1331). /\ cpc' = "hold" /\ waiting' = Superseding /\ held' = held \cup Superseding /\ heldStarts' = heldStarts \cup {snap[r].start : r \in Superseding} /\ holdEntry' = entry [] OTHER -> - \* fail_superseded (RUN:1357-1369). + \* fail_superseded (RUN:1364-1376). /\ cpc' = "fail" /\ UNCHANGED <> /\ UNCHANGED <> @@ -269,7 +273,7 @@ CDecide == /\ UNCHANGED rvars \* The hold ends when the runs it waits on complete, the newest status changes, -\* or the ceiling passes (RUN:1294-1323); the loop then lists again (RUN:1454). +\* or the ceiling passes (RUN:1301-1330); the loop then lists again (RUN:1459). CHoldEnds == /\ cpc = "hold" /\ \/ \A r \in waiting : ~live[r] @@ -280,16 +284,16 @@ CHoldEnds == /\ UNCHANGED <> /\ UNCHANGED rvars -\* yield-wait-seconds (600 at WF:7349) or the writer wait's 30 s passes. +\* yield-wait-seconds (600 at WF:7045) or the writer wait's 30 s passes. CTimeout == /\ Yield /\ cpc \in {"list", "read", "verify", "decide", "hold"} /\ ~timedOut /\ timedOut' = TRUE /\ UNCHANGED <> /\ UNCHANGED rvars -\* Single read (carry-forward-wait-seconds '0', WF:7351, without yield): read -\* the status (RUN:1484-1490), then verify_carried_writer lists the runs -\* (RUN:438-443) and reads the status again (RUN:450-454). +\* Single read (carry-forward-wait-seconds '0', REPORT:129, without yield): read +\* the status (RUN:1481-1487), then verify_carried_writer lists the runs +\* (RUN:424-429) and reads the status again (RUN:436-440). SRead == /\ cpc = "sRead" /\ entry' = newest diff --git a/.claude/skills/formal-ci-lanes-carry-forward/sync.yaml b/.claude/skills/formal-ci-lanes-carry-forward/sync.yaml index 73907e3ad43..051b2af85fd 100644 --- a/.claude/skills/formal-ci-lanes-carry-forward/sync.yaml +++ b/.claude/skills/formal-ci-lanes-carry-forward/sync.yaml @@ -2,19 +2,19 @@ checks: - id: carry-forward-yield covers: - .github/workflows/pr-require-checks.yml - commit: c4fe439dfb47955c34b07dc40af22b9a6b879ccb + commit: 726f82db25b4e98cc90dd1454f9bd82bcf6efbdd command: /formal-verification:chart .claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla --config .claude/skills/formal-ci-lanes-carry-forward/model/Yield.cfg --view .claude/skills/formal-ci-lanes-carry-forward/model/view.mjs --rerun-states 6000000 seeded_break: - change: ci-status's Aggregate lane results step passes yield-to-full-run 'false', so a contract-only run reads ci-lanes once and never yields to a full run in flight (Yield = FALSE in the cfg); NoStaleCarry breaks after 5,056 distinct states + change: the report-ci-status composite passes yield-to-full-run 'false' to aggregate-results, so a contract-only run reads ci-lanes once and never yields to a full run in flight (Yield = FALSE in the cfg); NoStaleCarry breaks after 5,056 distinct states caught: true - commit: c4fe439dfb47955c34b07dc40af22b9a6b879ccb + commit: 726f82db25b4e98cc90dd1454f9bd82bcf6efbdd - id: carry-forward-yield-no-hold covers: - .github/workflows/pr-require-checks.yml - commit: c4fe439dfb47955c34b07dc40af22b9a6b879ccb + commit: 726f82db25b4e98cc90dd1454f9bd82bcf6efbdd command: /formal-verification:chart .claude/skills/formal-ci-lanes-carry-forward/model/CarryForward.tla --config .claude/skills/formal-ci-lanes-carry-forward/model/YieldNoHold.cfg --view .claude/skills/formal-ci-lanes-carry-forward/model/view.mjs --rerun-states 6000000 seeded_break: - change: ci-status's Aggregate lane results step passes yield-to-full-run 'false', so a contract-only run reads ci-lanes once and never yields to a full run in flight (Yield = FALSE in the cfg); NoStaleCarry breaks after 7,972 distinct states + change: the report-ci-status composite passes yield-to-full-run 'false' to aggregate-results, so a contract-only run reads ci-lanes once and never yields to a full run in flight (Yield = FALSE in the cfg); NoStaleCarry breaks after 7,972 distinct states caught: true - commit: c4fe439dfb47955c34b07dc40af22b9a6b879ccb + commit: 726f82db25b4e98cc90dd1454f9bd82bcf6efbdd outside_model: []