Skip to content

harness: reset lists open PRs with --limit 200 and stops if any survive (closes #321) - #322

Merged
mmcky merged 2 commits into
mainfrom
harness-reset-limit
Sep 23, 2026
Merged

mmcky merged 2 commits into
mainfrom
harness-reset-limit

Conversation

@mmcky

@mmcky mmcky commented Sep 23, 2026

Copy link
Copy Markdown
Contributor

Closes #321.

The harness reset's three gh pr list calls had no --limit, so they returned gh's default of 30. A normal gate leaves exactly 28 open PRs per repo, which is why this never showed — but any diagnosis work between two gates pushes a lane past 30, and on 2026-09-21 the .ml lane had 31: the oldest (test-translation-sync.ml#192, from the v0.29.1 gate) was never listed, survived the v0.29.2 reset, and made the lane read 29 PRs / 28 stamped until it was found and closed by hand (tally on #318).

Change

  • --limit 200 on all three list calls (source repo, dry-run and live; each target repo).
  • A post-reset check: after the close loops, every repo is listed again and the script stops with the leftover PR numbers if any is still open. This catches the class, not just this cause — a listing that fails now counts as a failure rather than reading as "nothing to close" (the target loop's 2>/dev/null || echo "" had made it silent), and a close that did not take is caught too. It runs only outside --dry-run.
  • README: one line on the check under "The script will", and its message under Troubleshooting.
  • CHANGELOG under [Unreleased].

Checks

bash -n clean; shellcheck warning count unchanged (2, both pre-existing); the leftover-formatting line exercised on a fake two-PR list and on the empty case; workflow-templates.test.ts (the one Jest file that reads the script) green. Not run end-to-end: the harness is currently at @v0 after the v0.29.2 smoke with one open PR per repo, so a run would not exercise the >30 path anyway; the next gate is the first real exercise of the check.

🤖 Generated with Claude Code

gh pr list defaults to 30; a lane past that kept its oldest PR through the
reset and it read as a missing verdict in the v0.29.2 tally. Post-reset check
makes any survivor — page limit, failed listing, failed close — a hard stop
before fixture PRs are created.

Closes #321.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Copilot AI lite review requested due to automatic review settings September 23, 2026 03:11

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟢 Approval recommended

The changes directly address the documented gh pr list pagination issue and add a sensible safety check without altering core harness behavior beyond improved robustness.

Review effort: Lite
Findings: 1 Low severity

Open (1)
What changed in this PR

This PR hardens the end-to-end test harness reset logic so it reliably closes all open PRs (not just the first 30 returned by gh pr list) and fails fast if any PRs remain open after reset, preventing stale PRs from contaminating gate tallies.

Changes:

  • Add --limit 200 to all gh pr list calls used during reset to avoid missing older PRs when a repo has >30 open PRs.
  • Add a post-reset verification step (non---dry-run) that re-lists each repo and exits with a clear summary if any open PRs remain.
  • Document the new behavior and record the fix in the CHANGELOG under [Unreleased].
File Description
tool-test-action-on-github/​test-action-on-github.sh Adds --limit 200 to PR listings and introduces a post-reset “no open PRs remain” assertion to prevent survivors from skewing gate results.
tool-test-action-on-github/​README.md Notes the new “stop if any PR survives reset” behavior and adds troubleshooting guidance for the new failure mode.
CHANGELOG.md Records the bugfix and explains the historical failure scenario and the new guardrail.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread tool-test-action-on-github/test-action-on-github.sh Outdated
mmcky added a commit that referenced this pull request Sep 23, 2026
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
mmcky added a commit that referenced this pull request Sep 23, 2026
…4d74; round-4 arm (#319)

* dev: v0.29.2 released — §4a gate completed, @v0 = 5f74d74; round-4 numpy arm at v0.29.2

Gate 84/84 sync, 28/28 delivery + 28/28 engineVersion 0.29.2 verdicts per
lane; scenario 17 on .ml delivered. Round 4 regenerated at @v0: three draws
archived, draw 1 sent with one disclosed ml_repair.py comma.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* dev: STATE.md — 2026-09-23 close-out (#322 for #321, #323 for #320)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

* arm: note the garbage token in round-4 draw 2 (left as generated; a lint gap, #301)

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…the error

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@mmcky
mmcky merged commit 079b3c7 into main Sep 23, 2026
1 check passed
@mmcky
mmcky deleted the harness-reset-limit branch September 23, 2026 03:55
@mmcky mmcky mentioned this pull request Sep 23, 2026
mmcky added a commit that referenced this pull request Sep 23, 2026
* chore: release v0.29.3

Patch: fr glossary v1.2 (#328) from the French editor's second review
round, which reaches French syncs only once @v0 moves; plus the harness
reset fix (#322) and the release rate check (#323, step 4b), both of
which first run on this gate. No source change in this commit;
dist-action/ rebuilds byte-identical.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

* dev: STATE.md — wrap the v0.29.3 entry to the file's style

Raised by Copilot review on #329.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Harness reset misses open PRs beyond 30: gh pr list has no --limit (one survivor read as a missing verdict)

2 participants