review-code metrics file intermittently not written despite Write permission (recurs after #331 fix)

[nsheaps]

Summary

The review-code skill's step 11 (emit ${REVIEW_METRICS_PATH}) is intermittently skipped by the review agent even when the Write(<runner-temp>/**) permission fix from #331 is in effect. This causes the receiver to fail closed with "No metrics file ... — review agent did not complete", even though the review itself completed successfully and was posted.

Evidence

Run: https://github.com/nsheaps/.ai-agent-henry/actions/runs/31554465238 (job review / review, job ID 93983907122), triggered by PR #334's own request-review label (self-review).

• --allowedTools included Write(/home/runner/work/_temp/**) (the fix(review-utils): allow review agent to write metrics file #331 fix, confirmed present in the SDK options dump in the job log).
• SKILL.md step 11 instructed the agent to use the Write tool for /home/runner/work/_temp/review-metrics.yaml (confirmed present in the prompt dump in the job log).
• The agent's Claude Code SDK result: "is_error": false, "duration_ms": 229920, "num_turns": 27, "total_cost_usd": 1.17778225, "permission_denials_count": 4.
• The agent did successfully post a full review via MCP: fix(review-utils): COMMENT verdict must block required checks #334 (review) (APPROVE, with a proper findings table, etc. — clearly not a truncated/crashed run).
• The metrics file was never created: ##[error]No metrics file at /home/runner/work/_temp/review-metrics.yaml or /home/runner/work/_temp/review-metrics.json — review agent did not complete.
• permission_denials_count: 4 does not appear to correlate with the Write call — the only deny rules in effect are mcp__github_ci__get_ci_status, mcp__github_ci__get_workflow_run_details, mcp__github_ci__download_job_log, Bash(gh pr checks:*), Bash(git push:*) (all CI/push related, consistent with "Do not base review on CI output" in the prompt — plausibly the agent tried one of these and was correctly denied, unrelated to metrics).

Hypothesis

This is not a permissions bug (the #331 fix is confirmed present and correctly scoped). The more likely cause: the agent treats "submit the review" (step 10) as task completion and doesn't reliably continue on to steps 11–13 (emit metrics, verify). This is an LLM step-adherence issue, not a tooling gap — a second prompt-wording fix is unlikely to reliably solve it, since step 11 is already marked "REQUIRED" with an explanation of why.

Suggested direction (not yet implemented — needs discussion, not a drive-by fix)

Make the receiver's conclusion computation less dependent on the agent faithfully writing a side-channel file. When ${METRICS_PATH} is missing, fall back to querying the actual PR review the agent just posted (gh api repos/$REPO/pulls/$PR_NUMBER/reviews, filtered to the bot's review on HEAD_SHA) and derive verdict/conclusion from the review's real state (APPROVED/COMMENTED/CHANGES_REQUESTED) instead of failing closed outright. This is still a real, workflow-computed outcome (not a hand-patched check) — it just uses a more reliable data source (the GitHub review object itself, which the review actually needs to exist for) instead of an LLM-authored file that can be silently skipped.

Related: #331 (first fix attempt for this general failure mode — Write tool permission), #334 (COMMENT-verdict strict-blocking, where this recurrence was discovered).

Verification steps once a fix lands

• Trigger a review that reaches each of APPROVE / COMMENT / REQUEST_CHANGES and confirm the check conclusion is correct in all three cases even if the metrics file write is deliberately skipped/blocked.