Skip to content

fix(scripts): run the comment-lint gate in check-affected.sh - #18081

Merged
kyle-sexton merged 4 commits into
mainfrom
fix/check-affected-comment-lint
Nov 22, 2026
Merged

kyle-sexton merged 4 commits into
mainfrom
fix/check-affected-comment-lint

Conversation

@kyle-sexton

@kyle-sexton kyle-sexton commented Nov 22, 2026 •

Copy link
Copy Markdown
Member

Closes #18087

Summary

Issue-lane PRs keep failing CI's Lint Repo comment-lint gate, for example run 44962011745 at plugins/multi-agent/skills/setup/scripts/setup.sh:21-41. The lane's only self-check is scripts/check-affected.sh --uncommitted (docs/conventions/pr-pipeline.yaml:385), and scripts/check-affected-gates.txt listed comment-lint as CI only on the grounds that the commit-time git hook runs it. The lane commits through the API, and its check wrappers set core.hooksPath=/dev/null, so the hook never ran before the push.

Fix

  • scripts/check-affected.sh runs plugins/code-tidying/git-hooks/comment-lint.sh over <merge-base>..HEAD. The hook only judges its staged diff (checks_init pre-commit), and --worktree would see nothing inside the --uncommitted snapshot. So the check builds a scratch repository with HEAD at the merge-base and the index at HEAD's tree, the same thing scripts/check-comment-lint.sh gets from reset --soft. It checks out nothing except AGENTS.md, CLAUDE.md and the code-tidying*.yaml settings, all read from the merge-base as CI reads them, and it writes nothing under the checked repository's .git. A python3 that cannot import pygments is a SKIP, not a failure.
  • scripts/check-affected-gates.txt: comment-lint is now a run gate.
  • .github/actions/collect-base-activity/action.yml: the lane's base copy of the checks also carries plugins/code-tidying/{git-hooks,lib,scripts}, so the hook runs from the base. The lane installs .github/requirements-ci.txt, which pins pygments, through scripts/setup-env.sh for skill activities, and that venv is on PATH.

Verification

  • New scripts/check-affected-comment-lint.test.sh runs the real hook against a fixture repository. It covers: the hook is absent (SKIP); pygments is missing (SKIP); a clean change passes; a planted comment fails at its line (the planted line comes from the hook's own shape2 fixture), both committed and under --uncommitted; and the merge-base's comment_lint level wins over one the branch sets. With the hook switched to --worktree, the three failing cases go red. PASS=6.
  • A seventh case, added after review, gives the merge-base an AGENTS.md that points at a non-default convention home (team-conventions). The check must read comment_lint: block there, because check-affected.sh now resolves the pointer with --ref at the merge-base, as scripts/check-comment-lint.sh does. This case has not run locally yet; CI runs it.
  • scripts/check-affected.test.sh PASS=155, scripts/check-affected-gates.test.sh PASS=6, and the new collect-base-activity.test.mjs staging case passes.
  • scripts/check-affected.sh --uncommitted --base origin/main on this branch: 16 passed, 5 skipped. Before I removed my own comments, the new gate flagged them.

Related

🤖 Generated with Claude Code

https://claude.ai/code/session_01RAoAtV6H7kcRVG5xeX1rtT

Issue-lane PRs failed CI's comment-lint gate because the lane's only
self-check, check-affected.sh --uncommitted, listed it as CI only, and
the lane commits through the API with hooks off, so the commit-time
hook never ran.

check-affected.sh now runs the code-tidying comment-lint hook over
<merge-base>..HEAD: a scratch repository holds HEAD at the merge-base
and the index at HEAD's tree, so the hook's staged diff is the branch's,
with the merge-base's settings files as scripts/check-comment-lint.sh
uses in CI. A python3 without pygments is a SKIP. The lane's base copy
of the checks now carries code-tidying's git-hooks, lib and scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAoAtV6H7kcRVG5xeX1rtT
@kyle-sexton
kyle-sexton marked this pull request as ready for review November 22, 2026 00:05
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-22T00:08:55.621355Z ebc23fe Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

claude-security-review reviewed 0d05631a0193213a2d6704d37b4bb885a9e970d7: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Security review: PR #18081, head 0d05631 (incremental since 1cfa4ac)

No security issues found.

This review was blocking only. An earlier head was already reviewed, so this pass reported only CRITICAL and IMPORTANT findings.

What changed. The push changes one file, scripts/check-affected-comment-lint.test.sh, and adds one test case to it. The case builds a fixture repository where the merge-base's AGENTS.md points the convention home at team-conventions/. It moves code-tidying.yaml there, plants a disallowed comment in tool.sh, and asserts that check-affected.sh --base home-base fails comment-lint at tool.sh:8.

Why it is safe.

  • The change only touches a test fixture. It runs git inside a scratch repository the test controls, and the only input is a fixed printf string.
  • No untrusted text reaches it. It adds no runtime code and changes no credential handling.
  • The file doesn't match any TRUSTED PATHS glob.
  • It doesn't touch a trust control, merge hold, kill switch or token scope, and it changes no standing instruction.

Spend. No new spend beyond a few seconds of CI time for the extra test case. That is not a hold.

critical-findings: 0

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebc23fe955

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread scripts/check-affected.sh Resolved

@github-actions github-actions Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

claude-review: 1 inline finding(s) on ebc23fe955a8af55e811d9553a6f931a0be6b964; the summary is in this lane's pull request comment.

claude-lane: claude-review

Comment thread scripts/check-affected.sh Resolved
@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

claude-review reviewed 0d05631a0193213a2d6704d37b4bb885a9e970d7: read 1 of 1 changed file(s); 0 finding(s), 0 as inline comments.

Code review: incremental pass from 1cfa4ac8 to 0d05631a (blocking only)

This lane already reviewed earlier heads of this pull request, so this pass reports blocking findings only. One file changed since the last review: scripts/check-affected-comment-lint.test.sh. I read all of that change.

The change. It adds a test where the merge-base's AGENTS.md sends the convention files to a folder other than docs/conventions. This is the test the earlier finding on scripts/check-affected.sh:542 asked for. The test:

  • resets the fixture to lint-base;
  • uses git mv to move docs/conventions/code-tidying.yaml, which has comment_lint: block, to team-conventions/;
  • writes an AGENTS.md whose convention-home section points at `team-conventions`;
  • commits that as home-base, then commits the planted comment;
  • expects FAIL comment-lint at tool.sh:8:.

The test would catch the old bug. With the old lookup, which had no --ref, home came back empty and the check fell back to docs/conventions. home-base has no docs/conventions/code-tidying.yaml, so nothing was copied. The hook's comment-settings.sh then found no team-conventions/ folder in the scratch tree and treated every setting as unset, so the lint passed. This test would have gone red there.

With the fix at scripts/check-affected.sh:542, the lookup gets --ref "$merge_base". $merge_base is a 40-hex commit id, which resolve-convention-home.sh accepts. The lookup returns team-conventions and the settings are copied there, so the test passes. The AGENTS.md text matches the parser's format: a BEGIN and END marker and one line holding a backticked path (plugins/code-tidying/lib/resolve-convention-home.sh:19-31).

Prior findings. The inline finding on scripts/check-affected.sh:542 was fixed at an earlier head. This push adds the test case that finding suggested.

Closing issue: none. The PR says "No related issue", so there are no acceptance criteria to check. The PR context contains no instructions aimed at the lane.

Other checks:

  • Loosened checks would be marked: fail, with 0 patterns. That is the bundled default, because the prompt had no loosened_check_verdict: or loosened_check_patterns: line. There was no loosened_check_decision_record: line, so I did not check the citation requirement. Adding a test loosens no check.
  • No path-scoped rule matches the changed file, and this push touches no plugin manifest.

Verifier dropped 0 of 0 candidates.

@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/address-feedback

Run 44966256250, attempt 1

Field Value
Model rung 0
Leg handoff
Run job success
Activity step success
Commits pushed none (read lane)
Run 44966256250

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM66d9g0
Lane-Thread: PRRT_kwDOTCGFQM66d92Q
@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/apply-feedback-patch

Run 44966256250, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed 0d05631
Run 44966256250

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Test Selection traces a changed suite, and this one reads the base's
resolve-config package.json, apply-patch files and the skill-quality
plugin, which no test-scope line declared (scripts/affected-tests.sh R8).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01RAoAtV6H7kcRVG5xeX1rtT
@github-actions

github-actions Bot commented Nov 22, 2026 •

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/publish-feedback

Run 44966256250, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44966256250

Failed GitHub API calls

No failed call recorded by the run job's scripts.

Lane-Activity: pr-unblock/address-feedback
Lane-Thread: PRRT_kwDOTCGFQM66d92Q
@melodic-automation-lanes

Copy link
Copy Markdown
Contributor

pr-unblock / address-feedback finished round 1 of max 4 on 0d05631.

@github-actions

Copy link
Copy Markdown
Contributor

Lane run report

Field Value
Activity pr-unblock/resolve-feedback-threads

Run 44966256250, attempt 1

Field Value
Model rung 0
Leg base
Run job success
Activity step success
Commits pushed none recorded
Run 44966256250

Failed GitHub API calls

No failed call recorded by the run job's scripts.

@github-actions

Copy link
Copy Markdown
Contributor

The pr-verify verifier failed head 0d05631a0193213a2d6704d37b4bb885a9e970d7. What it wrote, as data:

Summary: PR #18081 (run the comment-lint gate in scripts/check-affected.sh) links no issue and has
  no Refs: line, and spec-text.json is an empty array: there is no trusted spec to judge the diff
  against, so the verdict must be fail. The code itself shows no blocking defect. The new gate
  builds a scratch repository at the merge-base with HEAD's tree as its index and reads AGENTS.md,
  CLAUDE.md and the code-tidying settings from the merge-base. The gates list moves comment-lint
  from the CI-only list to the run list, and collect-base-activity stages the base's
  plugins/code-tidying hook, lib and scripts directories. New tests cover all three changes. Lint
  Repo, Test Bash (0)/(1) and Test Node (0)/(1) all concluded success on the judged head (run
  44966831424). There are no binary files, no change to the takeover set, and no text trying to
  steer the review.
- PR body: links no issue and has no `Refs: #<n>` line, and no trusted spec entry exists
  (spec-text.json is empty). File an issue that states the outcome and done-when criteria (for
  example, check-affected.sh runs comment-lint so lane PRs catch it before CI) and link it with
  `Closes #<n>`, or have a trusted account author the PR body so it counts as the spec.

This pull request leaves the merge queue, and ci-status holds it out while it carries pr-verify-failed. A pr-verify pass removes the label, at a new commit or at this same head: once a trusted person edits the body, and only trusted people have edited it since the fail, this head is judged again, so a fail on the body or the issues it links needs only the body fixed, no new commit.

@github-actions github-actions Bot added the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 22, 2026
@github-actions github-actions Bot removed the pr-verify-failed pr-verify failed this head; a pass at a new commit, or after a trusted body edit, removes it. label Nov 22, 2026
@kyle-sexton
kyle-sexton added this pull request to the merge queue Nov 22, 2026
Merged via the queue into main with commit a5747a9 Nov 22, 2026
89 checks passed
@kyle-sexton
kyle-sexton deleted the fix/check-affected-comment-lint branch November 22, 2026 01:26
@github-actions github-actions Bot added the merge-path: owner-agent Post-merge accounting record (README.md, Merge paths) label Nov 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

merge-path: owner-agent Post-merge accounting record (README.md, Merge paths)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

check-affected.sh should run the comment-lint gate so lane PRs catch it before CI

1 participant