Skip to content

test(ci): fail when ci-workflows raises a wait report-ci-status keeps - #17410

Merged
kyle-sexton merged 2 commits into
mainfrom
test/ci-status-wait-default-budget
Nov 19, 2026
Merged

kyle-sexton merged 2 commits into
mainfrom
test/ci-status-wait-default-budget

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 19, 2026 •

Copy link
Copy Markdown
Member

No related issue: a guard test for the ci-workflows wait defaults ci-status keeps since #17283.

Summary

Since #17283, ci-status calls ci-workflows' report-ci-status composite with no wait input, so check-contract's and aggregate-results' rate-limit-wait-seconds stay at ci-workflows' defaults. Nothing here failed if a pin bump brought a raised default that outgrows the 435 s GH_READ_MARGIN_SECONDS holding the composite's waits.

Fix

  • New test in .github/actions/resolve-config/callers.test.mjs reads, at the ci-workflows SHA ci-status pins (fetchAtPin), report-ci-status/action.yml and each same-repository action it runs.
  • It asserts check-contract and aggregate-results both carry rate-limit-wait-seconds, each value (the step's with, else the input default) is at most WAIT_BUDGET_SECONDS.step (60, rate-guard.mjs), and those waits plus aggregate-results' rerun-wait-seconds, its own-gate retries (OWN_GATE_RETRY_DELAYS in aggregate-results/run.sh at the pin) and the margin comment's 7 s of status writes (STATUS_WRITE_RETRY_SECONDS) fit GH_READ_MARGIN_SECONDS.
  • At 9c5bedfb (v0.76.3): 60 + 60 + 90 + 120 + 7 = 337 s against 435 s.

Verification

  • node --test --test-name-pattern="report-ci-status's waits" .github/actions/resolve-config/callers.test.mjs: pass.
  • Feeding check-contract's default as '600' through the same fetch-and-parse path fails: check-contract's rate-limit-wait-seconds at 9c5bedfb… is 600, over WAIT_BUDGET_SECONDS.step 60. Feeding aggregate-results' rerun-wait-seconds as '250' fails the sum: total 497 s, over GH_READ_MARGIN_SECONDS 435. Both restored.
  • scripts/affected-tests.sh .github/workflows/pr-require-checks.yml selects callers.test.mjs (it references the workflow), so a pin bump runs the test.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01Vcupoo79EqmECYFZ4MTv6C

ci-status calls report-ci-status without a wait input, so check-contract's
and aggregate-results' rate-limit-wait-seconds stay at ci-workflows'
defaults. A raise there would land with the next pin bump and silently
outgrow the 435 s GH_READ_MARGIN_SECONDS that holds the composite's waits.

The test reads, at the SHA ci-status pins, report-ci-status's action.yml
and every same-repository action it runs, then asserts each
rate-limit-wait-seconds is at most WAIT_BUDGET_SECONDS.step (the value
rate-guard.mjs documents for these defaults) and that those waits plus
aggregate-results' rerun-wait-seconds fit GH_READ_MARGIN_SECONDS. At
9c5bedfb (v0.76.3): 60 + 60 + 90 = 210 <= 435.

It lives in callers.test.mjs, which already reads ci-workflows at pin and
has yaml; affected-tests.sh selects it when pr-require-checks.yml (and so
the pin) changes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vcupoo79EqmECYFZ4MTv6C
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 19, 2026 07:14
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-19T07:16:23.300751Z 724fb6d Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 70bba39e76ecae7af8a76e06396eb465da5fbb1a: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #17410 (incremental, 724fb6d → 70bba39)

I found no security issues. This was an incremental review of the one file changed since the last reviewed head, .github/actions/resolve-config/callers.test.mjs. I read every hunk of it. The lane has already reviewed an earlier head, so this review reported only blocking findings (CRITICAL and IMPORTANT).

What changed: the test now also counts two more waits against GH_READ_MARGIN_SECONDS:

  • the own-gate retries, which are the OWN_GATE_RETRY_DELAYS values in aggregate-results/run.sh, fetched at the pinned ci-workflows SHA;
  • a fixed 7 s of status-write retries, held in the new constant STATUS_WRITE_RETRY_SECONDS.

Checks I made:

  • Trust controls. The file is one of resolve-config's tests, which the skill excludes from the token-broker control. The change doesn't touch resolve.mjs, its sibling modules, effect-grants.json, lane-rules.json, activity-rules.json or pr-pipeline.yaml. No token scope, trusted actor, kill switch or merge hold changes.
  • What the test fetches. The new code reads one more file through fetchAtPin, at the same pinned SHA as before. It pulls the delays out with an anchored regex that accepts only digits and spaces (^OWN_GATE_RETRY_DELAYS='([0-9 ]+)'$) and adds them up as numbers. Nothing fetched is executed and no credentials are added. The only extra effect is one more network read when the test runs.
  • Instruction surface and manifests. The change removes or softens no standing instruction and touches no plugin manifest.
  • PR context. The description matches the change and contains no embedded directives.

Prior findings and settled threads. The only prior entry is the lane's last summary, which reported no findings, so nothing carries over.

critical-findings: 0

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on 724fb6ddc01fa2c5f291ba2aea13105113b5aed7; the summary is in this lane's pull request comment.

claude-lane: claude-review

@github-actions

github-actions Bot commented Nov 19, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 70bba39e76ecae7af8a76e06396eb465da5fbb1a: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Incremental review, 724fb6d → 70bba39: no blocking findings

Scope: one file, .github/actions/resolve-config/callers.test.mjs. I read every hunk of its incremental diff and checked the test at head (L2812-L2843). This lane already reviewed an earlier head, so this review covers blocking findings only.

What changed: the sum check now adds two figures before comparing against GH_READ_MARGIN_SECONDS:

  • the own-gate retries, read at the pin from aggregate-results/run.sh (OWN_GATE_RETRY_DELAYS);
  • the 7 s of status writes, as STATUS_WRITE_RETRY_SECONDS.

This answers this lane's earlier finding (discussion_r4625580805). That thread is resolved with an owner reply and stays settled. Using the PR's figures, raising rerun-wait-seconds to 250 now gives 497 s against 435 s and fails the check.

Fail-closed behavior I checked in the new code:

  • If run.sh moves, or OWN_GATE_RETRY_DELAYS is renamed, double-quoted or holds anything other than digits and spaces, the regex finds nothing and assert.ok(delays, …) fails. The check cannot quietly count 0 s.
  • Extra spaces in the delay list turn into Number("") = 0, which leaves the sum correct.

Other checks:

  • Closing issue: none (the PR says it has no related issue).
  • Path-scoped rules: none match the changed file.
  • Loosened checks: the change is test-only and tightens a check. The prompt had no loosened_check_verdict:, loosened_check_patterns: or decision-record line, so I used fail with no consumer entries and did not check the citation requirement.
  • Trusted path: the file matches .github/actions/resolve-config/**, so I read its whole change. It is a test and gates nothing at runtime.
  • Verification: I did not run the test. The PR's pass result and its 600 s / 250 s failure cases are the author's claims and unverified.
  • Spend: one extra raw-file fetch per run of this network test in CI.

Loosened-check verdict: fail (default; no verdict line in the prompt); patterns: 0

Verifier dropped 0 of 0 candidates.

The sum compared only the input waits (210 s) with the whole 435 s
GH_READ_MARGIN_SECONDS, though the margin's comment in
pr-require-checks.yml also budgets 120 s of own-gate retries and 7 s of
status writes, so a rerun-wait default of 250 passed and overflowed.

The own-gate retries are now read at the pin from aggregate-results/run.sh
(OWN_GATE_RETRY_DELAYS, summing to 120). The 7 s of status writes exists
in no ci-workflows file as a value, so STATUS_WRITE_RETRY_SECONDS carries
the margin comment's figure. At 9c5bedfb: 60 + 60 + 90 + 120 + 7 = 337
<= 435; with a 250 rerun wait, 497 > 435 fails.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vcupoo79EqmECYFZ4MTv6C
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 19, 2026
Merged via the queue into main with commit 7339564 Nov 19, 2026
129 of 134 checks passed
@kyle-sexton
kyle-sexton deleted the test/ci-status-wait-default-budget branch November 19, 2026 09:11
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 19, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant