Skip to content

ci: add AI PR review workflow (Codex spike) - #11

Merged
IsraelAraujo70 merged 2 commits into
mainfrom
feat/ai-pr-review
Apr 30, 2026
Merged

IsraelAraujo70 merged 2 commits into
mainfrom
feat/ai-pr-review

Conversation

@IsraelAraujo70

Copy link
Copy Markdown
Owner

Summary

  • Adds .github/workflows/ai-pr-review.yml — runs OpenAI Codex CLI against every PR using a ChatGPT subscription (no API key spend).
  • First step in a roadmap toward a YAML-driven, per-area PR review that gates merges via REQUEST_CHANGES.
  • Setup doc at .github/AI_PR_REVIEW_SETUP.md.

How auth works

~/.codex/auth.json is seeded from the CODEX_AUTH_JSON repo secret on cache miss, then persisted between runs via actions/cache@v4 so Codex's auto-refreshed token survives.

Roadmap (next PRs)

  1. Spike (this PR): verify auth + Codex execution work in GHA, post a single smoke-test comment.
  2. Per-area pr-review.yml config (different rubrics for src-tauri/, src/, docs/).
  3. Structured JSON output via --output-schema, severity-based findings.
  4. Switch from comment to GitHub review with REQUEST_CHANGES to block merges; add branch protection.
  5. Extract action to its own repo (e.g. IsraelAraujo70/ai-pr-review@v1) for reuse across other projects.

Test plan

  • Workflow triggers on this PR
  • CODEX_AUTH_JSON secret successfully seeds ~/.codex/auth.json
  • codex --version and codex auth status succeed
  • codex exec runs and returns a final message
  • Workflow posts a single comment with the AI summary (or a clear error if it fails)
  • On a second run, cache restores the refreshed token instead of re-seeding

🤖 Generated with Claude Code

Smoke-test workflow that runs OpenAI Codex CLI against every PR using
a ChatGPT subscription auth (no API key). First step toward a YAML-driven,
per-area PR review that gates merges via REQUEST_CHANGES.

Auth flow: ~/.codex/auth.json is seeded from the CODEX_AUTH_JSON repo
secret on cache miss, then persisted between runs via actions/cache so
Codex's auto-refreshed token survives.

Setup instructions in .github/AI_PR_REVIEW_SETUP.md.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@github-actions

github-actions Bot commented Apr 30, 2026

Copy link
Copy Markdown

🔍 PR checks

Type: ci
Areas: area/ci

⏭️ Merging will not trigger a release

ci: is non-functional, no release.

Auto-generated by pr-checks.yml. Edit the PR title to change the verdict.

@github-actions

github-actions Bot commented Apr 30, 2026

Copy link
Copy Markdown

🤖 AI PR review

Area Verdict Files
Backend (Rust / Tauri) ⏭️ skipped
Frontend (React / TypeScript) ⏭️ skipped
CI / GitHub Actions ⚠️ CONCERNS 3
Documentation ⏭️ skipped

CI / GitHub Actions — ⚠️ CONCERNS

VERDICT: CONCERNS

  1. .github/workflows/ai-pr-review.yml:38 — The workflow seeds CODEX_AUTH_JSON into ~/.codex/auth.json, then later executes PR-controlled code from .github/scripts/pr_review.py, which can log or exfiltrate that secret. Suggestion: run trusted workflow/script content from the base branch before exposing secrets, or avoid using repo secrets in PR-triggered runs.

Reviewed with Codex CLI · rubrics in .github/pr-review.yml

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ee11c2d316

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ai-pr-review.yml Outdated
Replaces the single smoke-test prompt with a YAML-driven, per-area review:

- pr-review.yml defines areas (rust, frontend, ci, docs) with a path glob
  and a rubric derived from CLAUDE.md.
- pr_review.py parses the YAML, computes which areas the PR touches, and
  runs one `codex exec` invocation per area with a scoped diff and the
  area's rubric. Each area returns a VERDICT (PASS/CONCERNS) plus findings.
- Workflow drops the auth.json cache (Codex itself flagged the leak risk
  on PR #11) and re-seeds from the secret on every run; the refresh_token
  in CODEX_AUTH_JSON is stable so token refresh still works.
- Comment now shows a verdict table + per-area sections instead of a
  single freeform paragraph.
- Reasoning effort bumped to high; depth matters more than speed for
  reviewer role.

REQUEST_CHANGES gating + branch protection still on the roadmap.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@IsraelAraujo70
IsraelAraujo70 merged commit 0b08b3d into main Apr 30, 2026
2 checks passed
@IsraelAraujo70
IsraelAraujo70 deleted the feat/ai-pr-review branch April 30, 2026 17:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant