Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/workflows/opencode-review.yml
Original file line number Diff line number Diff line change
Expand Up @@ -425,6 +425,7 @@ jobs:
prompt_evidence_bytes="${OPENCODE_PROMPT_EVIDENCE_BYTES:-3200}"
cat >"$prompt_file" <<EOF
Review PR #${PR_NUMBER} in ${GITHUB_WORKSPACE}. Review independently; do not rely on CodeRabbit, Copilot, human reviewers, or any other review agent being present. Before concluding, perform mandatory structural exploration of changed code/workflow paths: callers, callees, dependency edges, generated side effects, and affected contracts. Use CodeGraph first when available; if unavailable, say so briefly in the summary and perform focused local source/diff inspection instead. Actively use available review tools when relevant, including DeepWiki, Context7, web_search, and local file inspection. Focus on real bugs, security regressions, broken workflow contracts, and missing tests.
Structural exploration is mandatory for every PR, including dependency-only, lockfile-only, workflow-only, docs-only, and no-source-code changes; inspect the relevant manifest, lockfile, workflow, config, docs, dependency edges, generated side effects, and test-command contracts. Never state that structural exploration, structural analysis, or structural review is not required or unnecessary.
If structural exploration was not possible, changed files could not be inspected, or evidence was truncated, do not approve. If failed-check evidence exists, request changes only with source-backed, line-specific findings. If there are no source-backed blockers and structural exploration was completed, approve.
Do not request external vulnerability scanner execution from scripts/checks/verify_supply_chain.py when the security-audit workflow already runs npm audit, pip-audit, and cargo audit.
Follow CodeRabbit/Copilot review style without depending on either tool: concise overview, findings first, source-backed path:line references, severity, problem, root cause, fix direction, regression-test direction, and a source-backed suggested diff. Avoid mechanical log dumps.
Expand Down Expand Up @@ -530,6 +531,7 @@ jobs:
prompt_evidence_bytes="${OPENCODE_PROMPT_EVIDENCE_BYTES:-3200}"
cat >"$prompt_file" <<EOF
GPT-5 failed. Review PR #${PR_NUMBER}. Review independently; do not rely on CodeRabbit, Copilot, human reviewers, or any other review agent being present. Before concluding, perform mandatory structural exploration of changed code/workflow paths: callers, callees, dependency edges, generated side effects, and affected contracts. Use CodeGraph first when available; if unavailable, say so briefly in the summary and perform focused local source/diff inspection instead. Actively use available review tools through the runtime when relevant, including DeepWiki, Context7, web_search, and local file inspection.
Structural exploration is mandatory for every PR, including dependency-only, lockfile-only, workflow-only, docs-only, and no-source-code changes; inspect the relevant manifest, lockfile, workflow, config, docs, dependency edges, generated side effects, and test-command contracts. Never state that structural exploration, structural analysis, or structural review is not required or unnecessary.
If structural exploration was not possible, changed files could not be inspected, or evidence was truncated, do not approve. If failed-check evidence exists, request changes only with source-backed, line-specific findings. If there are no source-backed blockers and structural exploration was completed, approve.
Follow CodeRabbit/Copilot review style without depending on either tool: concise overview, findings first, source-backed path:line references, severity, problem, root cause, fix direction, regression-test direction, and a source-backed suggested diff. Avoid mechanical log dumps.
Return only the review body. Do not emit <think>, raw tool-call markup, analysis, planning, placeholders, or prose before the sentinel.
Expand Down Expand Up @@ -634,6 +636,7 @@ jobs:
prompt_evidence_bytes="${OPENCODE_PROMPT_EVIDENCE_BYTES:-3200}"
cat >"$prompt_file" <<EOF
GPT-5 and DeepSeek R1 failed. Review PR #${PR_NUMBER}. Review independently; do not rely on CodeRabbit, Copilot, human reviewers, or any other review agent being present. Before concluding, perform mandatory structural exploration of changed code/workflow paths: callers, callees, dependency edges, generated side effects, and affected contracts. Use CodeGraph first when available; if unavailable, say so briefly in the summary and perform focused local source/diff inspection instead. Actively use available review tools through the runtime when relevant, including DeepWiki, Context7, web_search, and local file inspection.
Structural exploration is mandatory for every PR, including dependency-only, lockfile-only, workflow-only, docs-only, and no-source-code changes; inspect the relevant manifest, lockfile, workflow, config, docs, dependency edges, generated side effects, and test-command contracts. Never state that structural exploration, structural analysis, or structural review is not required or unnecessary.
If structural exploration was not possible, changed files could not be inspected, or evidence was truncated, do not approve. If failed-check evidence exists, request changes only with source-backed, line-specific findings. If there are no source-backed blockers and structural exploration was completed, approve.
Follow CodeRabbit/Copilot review style without depending on either tool: concise overview, findings first, source-backed path:line references, severity, problem, root cause, fix direction, regression-test direction, and a source-backed suggested diff. Avoid mechanical log dumps.
Return only the review body. Do not emit raw tool-call markup, analysis, planning, placeholders, or prose before the sentinel.
Expand Down
12 changes: 12 additions & 0 deletions scripts/ci/opencode_review_approve_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -136,6 +136,18 @@ if ! jq -e '
| (
($text | contains("structural exploration was not possible"))
or ($text | contains("structural exploration not possible"))
or ($text | contains("structural exploration is not required"))
or ($text | contains("structural exploration not required"))
or ($text | contains("structural analysis is not required"))
or ($text | contains("structural analysis not required"))
or ($text | contains("structural review is not required"))
or ($text | contains("structural review not required"))
or ($text | contains("no structural exploration required"))
or ($text | contains("no structural analysis required"))
or ($text | contains("no structural review required"))
or ($text | contains("structural exploration is unnecessary"))
or ($text | contains("structural analysis is unnecessary"))
Comment thread
Copilot marked this conversation as resolved.
or ($text | contains("structural review is unnecessary"))
or ($text | contains("could not be reviewed"))
or ($text | contains("could not inspect"))
or ($text | contains("could not be inspected"))
Expand Down
12 changes: 12 additions & 0 deletions scripts/ci/opencode_review_normalize_output.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,18 @@
STRUCTURAL_FAILURE_PHRASES = (
"structural exploration was not possible",
"structural exploration not possible",
"structural exploration is not required",
"structural exploration not required",
"structural analysis is not required",
"structural analysis not required",
"structural review is not required",
Comment thread
seonghobae marked this conversation as resolved.
"structural review not required",
"no structural exploration required",
"no structural analysis required",
"no structural review required",
"structural exploration is unnecessary",
"structural analysis is unnecessary",
Comment thread
Copilot marked this conversation as resolved.
"structural review is unnecessary",
"could not be reviewed",
"could not inspect",
"could not be inspected",
Expand Down
108 changes: 108 additions & 0 deletions services/analysis-engine/tests/test_supply_chain_policy.py
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,21 @@
import pytest
from conftest import load_module

OPTIONAL_STRUCTURAL_REVIEW_PHRASES = (
"structural exploration is not required",
"structural exploration not required",
"structural analysis is not required",
"structural analysis not required",
"structural review is not required",
"structural review not required",
"no structural exploration required",
"no structural analysis required",
"no structural review required",
"structural exploration is unnecessary",
"structural analysis is unnecessary",
"structural review is unnecessary",
)


def test_supply_chain_check_requires_multi_arch_runner_labels(
monkeypatch: pytest.MonkeyPatch, tmp_path: Path
Expand Down Expand Up @@ -5166,6 +5181,48 @@ def test_opencode_normalizer_rejects_approve_without_structural_review(
assert output_file.read_text(encoding="utf-8") == original_output


def test_opencode_normalizer_rejects_optional_structural_review_variants(
tmp_path: Path,
) -> None:
"""Ensure optional structural-review phrasing cannot be normalized."""
normalizer = load_module(
"scripts/ci/opencode_review_normalize_output.py",
"opencode_review_normalize_optional_structure",
)

assert set(OPTIONAL_STRUCTURAL_REVIEW_PHRASES).issubset(normalizer.STRUCTURAL_FAILURE_PHRASES)

for field in ("reason", "summary"):
for phrase in OPTIONAL_STRUCTURAL_REVIEW_PHRASES:
output_file = tmp_path / f"{field}-{phrase.replace(' ', '-')}.md"
reason = phrase if field == "reason" else "no blockers found"
summary = phrase if field == "summary" else "structural exploration completed"
original_output = "\n".join(
[
"review text",
'{"head_sha":"abc123","run_id":"456","run_attempt":"1",'
'"result":"APPROVE",'
f'"reason":"{reason}",'
f'"summary":"{summary}",'
'"findings":[]}',
]
)
output_file.write_text(original_output, encoding="utf-8")

result = normalizer.main(
[
"opencode_review_normalize_output.py",
"abc123",
"456",
"1",
str(output_file),
]
)

assert result == 4
assert output_file.read_text(encoding="utf-8") == original_output


def test_opencode_review_gate_rejects_approve_without_structural_review(
tmp_path: Path,
) -> None:
Expand Down Expand Up @@ -5211,6 +5268,57 @@ def test_opencode_review_gate_rejects_approve_without_structural_review(
assert not normalized_file.exists()


def test_opencode_review_gate_rejects_optional_structural_review_variants(
tmp_path: Path,
) -> None:
"""Ensure approval gate rejects optional structural-review phrasing."""
repo_root = Path(__file__).resolve().parents[3]

for field in ("reason", "summary"):
for phrase in OPTIONAL_STRUCTURAL_REVIEW_PHRASES:
comment_file = tmp_path / f"{field}-{phrase.replace(' ', '-')}.md"
normalized_file = tmp_path / f"{field}-{phrase.replace(' ', '-')}.json"
reason = phrase if field == "reason" else "no blockers found"
summary = phrase if field == "summary" else "structural exploration completed"
comment_file.write_text(
"\n".join(
[
"<!-- opencode-review-gate head_sha=abc123 run_id=456 run_attempt=1 -->",
"",
"<!-- opencode-review-control-v1",
'{"head_sha":"abc123","run_id":"456","run_attempt":"1",'
'"result":"APPROVE",'
f'"reason":"{reason}",'
f'"summary":"{summary}",'
'"findings":[]}',
"-->",
"",
]
),
encoding="utf-8",
)

result = subprocess.run(
[
"bash",
str(repo_root / "scripts" / "ci" / "opencode_review_approve_gate.sh"),
"abc123",
"456",
"1",
str(comment_file),
str(normalized_file),
],
cwd=repo_root,
capture_output=True,
text=True,
check=False,
)

assert result.returncode == 4
assert result.stdout.strip() == "NO_CONCLUSION"
assert not normalized_file.exists()


def test_opencode_normalizer_accepts_completed_local_structural_fallback(
tmp_path: Path,
) -> None:
Expand Down