Skip to content

ci: validate self-dogfood evidence before upload - #41

Merged
yui-stingray merged 2 commits into
masterfrom
codex/document-unreleased-hardening
Aug 11, 2026
Merged

ci: validate self-dogfood evidence before upload#41
yui-stingray merged 2 commits into
masterfrom
codex/document-unreleased-hardening

Conversation

@yui-stingray

@yui-stingray yui-stingray commented Aug 11, 2026

Copy link
Copy Markdown
Owner

Summary

  • use canonical allowlisted filenames for every self-dogfood report consumer
  • synchronize the self-dogfood workflow policy, digest pin, generated public sample, and command-count contract
  • run the packaged public evidence consumer before artifact upload so unexpected or invalid bundle entries fail closed
  • record the reviewed post-v0.3.4 maintenance changes under Unreleased without changing release state

Verification

  • TMPDIR=/tmp TEMP=/tmp TMP=/tmp .venv/bin/python -m pytest -s -q
  • targeted workflow, changelog, sample-schema, sample-size, and evidence-consumer contract tests
  • actionlint .github/workflows/ci.yml .github/workflows/release.yml .github/workflows/github-release.yml
  • .venv/bin/python scripts/check_changelog.py
  • self-dogfood workflow and digest checks
  • git diff --check

Boundary check

  • Self-infrastructure and release-note maintenance only; scanner behavior, evidence schemas, CLI exit codes, package version, and release state are unchanged.
  • No release, tag, credential, private path, or private repository data is included.

@coderabbitai

coderabbitai Bot commented Aug 11, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

The CI self-dogfood workflow now uses agent-guard-report.json, validates it with an isolated consumer, and updates all related rendering and packaging commands. Contract tests and changelog entries document the canonical paths and release-validation requirements.

Changes

Canonical evidence flow

Layer / File(s) Summary
Canonical report generation and validation
.github/workflows/ci.yml, CHANGELOG.md
The workflow uses agent-guard-report.json for rendering, conformance checks, evidence packaging, and annotations. It validates the report and evidence directory before artifact upload. The changelog records related release and provenance requirements.
Workflow contract assertions
tests/test_workflow_release_contract.py
The contract test checks the canonical report paths, rejects obsolete filenames, and verifies consumer validation before artifact upload.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Possibly related PRs

Poem

A rabbit checks the report with care,
Canonical paths now match everywhere.
The consumer hops before upload,
Old filenames vanish from the route.
CI leaves a tidy evidence trail.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly summarizes the main change: validating self-dogfood evidence before upload.
Description check ✅ Passed The description explains the changes, provides validation commands, and states the scope and boundary constraints.
✨ Finishing Touches 💡 1
🛠️ Fix failing CI checks 💡
  • Create stacked PR
  • Commit on current branch
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch codex/document-unreleased-hardening

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 159987893b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@tests/test_workflow_release_contract.py`:
- Around line 589-619: Extend the assertions in the self-dogfood workflow check
around normalized_self_dogfood to require the canonical renamed report-consumer
commands: conformance check using --evidence and evidence-pack manifest using
both --report and --artifact. Assert each command’s exact expected path,
alongside the existing report and consumer_command checks.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository YAML (base), Organization UI (inherited)

Review profile: ASSERTIVE

Plan: Pro Plus

Run ID: 5ed92dc3-906b-4c2c-856d-f8e6878f777d

📥 Commits

Reviewing files that changed from the base of the PR and between c9a88fa and 1599878.

📒 Files selected for processing (3)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • tests/test_workflow_release_contract.py

Comment thread tests/test_workflow_release_contract.py
@yui-stingray
yui-stingray merged commit 0b4f985 into master Aug 11, 2026
13 checks passed
@yui-stingray
yui-stingray deleted the codex/document-unreleased-hardening branch August 11, 2026 08:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant