Repository navigation
feat(hook): gate PR-claims lacking same-turn mutation - #880
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
🚧 Files skipped from review as they are similar to previous changes (2)
📝 WalkthroughWalkthroughAdds a Stop hook that detects PR completion claims and checks for same-turn PR mutation evidence, with advisory, strict-block, bypass, fail-open, documentation, registration, and scenario-test coverage. ChangesPR claim mutation verification
Estimated code review effort: 3 (Moderate) | ~20 minutes Sequence Diagram(s)sequenceDiagram
participant StopEvent
participant PRClaimMutationGate
participant Transcript
participant HookLedger
StopEvent->>PRClaimMutationGate: invoke Stop hook
PRClaimMutationGate->>Transcript: read final message and current-turn tool calls
Transcript-->>PRClaimMutationGate: claim text and mutation evidence
PRClaimMutationGate->>HookLedger: record advisory or block decision
PRClaimMutationGate-->>StopEvent: emit JSON decision or silent pass
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Confidence: high Not-tested: not registered in hooks/manifest.json (orchestrator integration step) Session-Id: e43a3532-3c9d-47e1-a23d-4f160433db90
The hook shipped unregistered, which is not a deferrable step: CI's test_baseline_check_clean runs the plugin-manifest check, so an unregistered hook fails the `test` job and the PR can never go green. Registration has to live in the PR that adds the hook. Adds the manifest entry (Stop, after pr-report-destination-gate), the matching Stop-ordering entry in the check script, the INDEX.md and ARCHITECTURE.md rows, the strict/opt-out rows in bypass-vars.md, and the regenerated per-platform hooks.json / wrapper / hook doc. Pre-commit verified: python3 scripts/check-plugin-manifests.py -> plugin- manifest check OK (rc=0); python3 -m pytest tests/ -q -> 503 passed; shellcheck --severity=warning on the generated wrapper -> clean. Confidence: high Scope-risk: narrow Refs #868 Session-Id: e43a3532-3c9d-47e1-a23d-4f160433db90
a22e7b0 to
287e3f5
Compare
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@hooks/completion-verify/pr-claim-mutation-gate/impl.py`:
- Around line 19-26: Update _GH_API_WRITE_RE so it matches only explicit
write-method gh api calls whose endpoint contains a comments, reviews, or
threads path segment. Preserve the existing method detection and
case-insensitive behavior, while ensuring unrelated write endpoints such as
labels or milestones do not satisfy the PR-surface mutation gate.
- Line 152: Update _MCP_GH_MUTATION_RE and its mutation-detection usage to
recognize only explicitly write-oriented GitHub MCP tool names or predicates,
rather than matching comment, review, merge, or thread substrings anywhere in a
raw name. Ensure consolidated read-only tools such as pull_request_read do not
suppress the PR-mutation claim.
In `@hooks/completion-verify/pr-claim-mutation-gate/spec.md`:
- Around line 96-101: Update the documented PR claim gate behavior so normal
installations enforce blocking by default rather than emitting only an advisory.
Ensure the manifest registration or equivalent deployment configuration enables
PRAXIS_PR_CLAIM_STRICT=1 by default, while preserving PRAXIS_PR_CLAIM_BYPASS=1
as the explicit opt-out.
- Around line 78-87: Update the completion gate logic described in the spec so
it clears only for successful tool calls with verified PR-scoped mutations.
Correlate each matching tool_use with its successful tool result, reject
dry-run/help/echoed or otherwise non-mutating Bash commands, and replace the
broad GitHub MCP name match with an explicit allowlist whose arguments validate
the target PR. Preserve the existing mutation categories while excluding
read-only and issue-scoped operations.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: d014cbe8-e535-48b2-ab45-ebf37ea208d2
📒 Files selected for processing (15)
.claude-plugin/hooks/hooks.json.cursor-plugin/hooks/hooks.json.opencode/hooks/hooks.jsonARCHITECTURE.mddocs/bypass-vars.mddocs/hook-operating-matrix.mddocs/hook/INDEX.mddocs/hook/pr-claim-mutation-gate.mdhooks/completion-verify/pr-claim-mutation-gate/impl.pyhooks/completion-verify/pr-claim-mutation-gate/spec.mdhooks/manifest.jsonhooks/pr-claim-mutation-gate.shplugins/praxis/.codex-plugin/hooks/hooks.jsonscripts/check-plugin-manifests.pytests/hooks/completion-verify/test_pr_claim_mutation_gate.sh
Four CodeRabbit findings, all applied. Each one let something that is not a PR-surface mutation clear the gate -- the precise failure the hook exists to prevent. 1. `_GH_API_WRITE_RE` matched any write-method `gh api`, though the docstring promised a comments/reviews/threads endpoint restriction. Probe: the old regex accepted `gh api repos/o/r/issues/1/labels --method POST`. The endpoint segment is now required. 2. `_MCP_GH_MUTATION_RE` matched `comment|review|merge|thread` anywhere in a raw tool name, so consolidated READ tools cleared the claim. Probe: the old regex accepted `mcp__github__get_review_comments`. Matching now runs on write verbs, and a get/list/search/read prefix always loses. 3. Only the tool_use text was checked, so a failed, dry-run, help or echoed command counted as a mutation. tool_use `id` is now correlated with its tool_result `is_error` (same mechanism as pr-report-destination-gate), and rehearsal/echo forms are excluded. A tool_use with no result yet counts as success -- the gate is the accuser, so it biases to silence. 4. The hook shipped advisory-by-default while issue #868 asks for the opposite verbatim: "부재 시 차단 ... advisory 티어의 실측 효과가 0 이므로 advisory 로는 같은 실패가 반복된다", and its AC reads "mutation 없음 -> block". Default flipped to block; `PRAXIS_PR_CLAIM_STRICT` is replaced by `PRAXIS_PR_CLAIM_ADVISORY` as a demote switch, mirroring negative-existence-verdict-gate, which already blocks-by-default with an advisory escape hatch. Six fixtures added, one per hole: labels-endpoint write, consolidated MCP reader, dry-run push, echoed push, failed push, succeeded push. The last two are a matched pair -- identical commands, opposite is_error -- so the correlation is what the test distinguishes, not the command text. Also fixes the two reviewdog markdownlint errors in spec.md (h1->h3 jump, unlabelled fence). Pre-commit verified: bash tests/hooks/completion-verify/test_pr_claim_ mutation_gate.sh -> PASS=33 FAIL=0; python3 -m pytest tests/ -q -> 503 passed; scripts/check-plugin-manifests.py -> OK (rc=0); ruff check hooks/ -> clean; shellcheck --severity=warning on the changed test -> clean; markdownlint-cli2 on the changed docs -> 0 issues. Raised by CodeRabbit on PR #880 (1 Critical, 3 Major). Confidence: high Not-tested: the block tier's live re-prompt behaviour is exercised only through the fixture's stdout JSON, not against a running Stop lane. Scope-risk: moderate Directive: the endpoint segment in _GH_API_WRITE_RE and the read-prefix loss in _is_mcp_mutation_tool are the gate, not decoration -- widening either one restores the defect this commit closes. Refs #868 Session-Id: e43a3532-3c9d-47e1-a23d-4f160433db90
There was a problem hiding this comment.
Actionable comments posted: 1
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
hooks/completion-verify/pr-claim-mutation-gate/impl.py (1)
165-215: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick winTreat consolidated GitHub MCP PR-review write tools as mutations.
_is_mcp_mutation_toolrejects"mcp__github__pull_request_review_write"because its suffix has no^verb_prefix or_verb_infix from the current list. Since that tool’s write methods includecreate,submit_pending,resolve_thread, andunresolve_thread, a claim backed by that tool should clear the gate instead of being blocked. Match consolidated write tool names and inspectinput.methodfor write-only method values.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@hooks/completion-verify/pr-claim-mutation-gate/impl.py` around lines 165 - 215, Update _is_mcp_mutation_tool to recognize the consolidated pull_request_review_write tool as mutation-capable, including its fully qualified MCP name. For that tool, inspect input.method and return true only for write methods such as create, submit_pending, resolve_thread, and unresolve_thread; preserve read-tool precedence and existing verb matching for other tools.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@tests/hooks/completion-verify/test_pr_claim_mutation_gate.sh`:
- Around line 180-186: Update the inline comments for the
“write-is-not-pr-mutation” and “prev-turn-mutation-not-evidence” cases to state
that the hook blocks by default, matching their existing run_case block
assertions. Do not change the test commands or expectations.
---
Outside diff comments:
In `@hooks/completion-verify/pr-claim-mutation-gate/impl.py`:
- Around line 165-215: Update _is_mcp_mutation_tool to recognize the
consolidated pull_request_review_write tool as mutation-capable, including its
fully qualified MCP name. For that tool, inspect input.method and return true
only for write methods such as create, submit_pending, resolve_thread, and
unresolve_thread; preserve read-tool precedence and existing verb matching for
other tools.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: 973deea9-3595-4722-b6ac-2c7b41bff140
📒 Files selected for processing (8)
ARCHITECTURE.mddocs/bypass-vars.mddocs/hook-operating-matrix.mddocs/hook/INDEX.mdhooks/completion-verify/pr-claim-mutation-gate/impl.pyhooks/completion-verify/pr-claim-mutation-gate/spec.mdhooks/manifest.jsontests/hooks/completion-verify/test_pr_claim_mutation_gate.sh
🚧 Files skipped from review as they are similar to previous changes (1)
- docs/hook-operating-matrix.md
The block-by-default flip left three case headers and the spec's Tests paragraph still describing advisory outcomes, so the prose contradicted the assertions directly beneath it -- the shape that invites a future reader to "fix" a correct assertion. Also updates the count (27 -> 33) and describes what the six new cases pin. Pre-commit verified: bash tests/hooks/completion-verify/test_pr_claim_mutation_gate.sh -> PASS=33 FAIL=0; markdownlint-cli2 on the spec -> 0 issues; shellcheck --severity=warning -> clean. Confidence: high Scope-risk: narrow Refs #868 Session-Id: e43a3532-3c9d-47e1-a23d-4f160433db90
배경
2026-07-27 세션 retrospect finding #1 (HIGH, repeat 3x). CodeRabbit 리뷰 3건을 local worktree 에만 적용한 상태에서 사용자에게 처리 완료로 보고했다. 그 시점에 push /
gh pr comment/ 리뷰 코멘트 API / thread resolve 는 전부 0건이었다.기존
completion-signal-gate(#392) 는 이 케이스를 잡지 못한다: (a) advisory 티어가 이 세션 fire ledger 기준 42건 발화 대비 행동 변화 0건, (b) 증거 검사가 "턴 내 임의의 Bash/Read 호출"이라 PR 을 전혀 건드리지 않은git status도 통과시킨다.변경 사항
신규 sibling 훅
hooks/completion-verify/pr-claim-mutation-gate/impl.py(Stop, completion-verify role) 추가:PR,리뷰 코멘트,피드백,coderabbit등) + 처리-완료 CLAIM 동사(처리했,반영했,resolved,applied,handled,fixed, ...)가 같은 줄에 공존>) — 각각 test case 로 인코딩git push,gh pr comment,gh pr review, write-methodgh api, GraphQLresolveReviewThread, GitHub MCP comment/review 툴) — read-onlygh api목록 조회는 증거로 인정하지 않음입력 표면 enumeration (praxis:surface-enumeration)
것 같습니다,might have addressed(2) — 결정: silent 처리>인용줄 — 결정: silent (claim 을 보고하는 것과 claim 자체를 구분)27 fixture case 전부
tests/hooks/completion-verify/test_pr_claim_mutation_gate.sh에 인코딩.검증
검증하지 못한 것
hooks/manifest.json/ 생성된hooks.json등록은 이번 PR 범위 밖 (Phase 2 lane 병렬 작업 — 오케스트레이터가 통합 시 일괄 등록).scripts/check-plugin-manifests.py실패는 예상된 것.Pre-commit verified:
bash tests/hooks/completion-verify/test_pr_claim_mutation_gate.sh→ PASS=27 FAIL=0;ruff check hooks/completion-verify/pr-claim-mutation-gate/→ All checks passedCaller chain verified: N/A -- new hook, no caller expected (Stop hook invoked by harness via hooks.json registration, deferred to Phase 2 통합)
Closes #868
Summary by CodeRabbit
New Features
PRAXIS_PR_CLAIM_BYPASSto silence the gate, andPRAXIS_PR_CLAIM_ADVISORYto downgrade it to advisory behavior.Documentation
Tests
Chores