Repository navigation
feat: enforce signed V3 release evidence gates - #312
Draft
EvanProgramming wants to merge 51 commits into
Draft
EvanProgramming wants to merge 51 commits into
EvanProgramming wants to merge 51 commits into
Conversation
|
Important Draft PR not reviewedDraft PRs are not automatically reviewed by default.
To automatically review draft PRs, update your CodeRabbit configuration: reviews:
auto_review:
drafts: true
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
EvanProgramming
force-pushed
the
Evan/v3-release-evidence
branch
15 times, most recently
from
October 22, 2026 19:17
7276872 to
992b027
Compare
EvanProgramming
force-pushed
the
Evan/v3-release-evidence
branch
3 times, most recently
from
October 22, 2026 22:03
22543f7 to
068b8f0
Compare
EvanProgramming
changed the base branch from
Evan/v3-matched-evaluation
to
main
October 28, 2026 12:49
EvanProgramming
changed the base branch from
main
to
Evan/v3-matched-evaluation
October 28, 2026 18:30
EvanProgramming
changed the base branch from
Evan/v3-matched-evaluation
to
main
October 29, 2026 02:38
EvanProgramming
changed the base branch from
main
to
Evan/v3-matched-evaluation
October 29, 2026 10:34
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracks #268; stacked on PR #311. Closes #268 only after the required release evidence validates.
Change
The release validator preserves passed, failed, inconclusive, and waived states. A limited-evidence preview requires all non-waived gates to pass, and each eligible inconclusive gate requires a GPG-signed waiver bound to the exact candidate and preview scope. Failed controls, missing safety evidence, stale or tampered evidence, and an ordinary full release with inconclusive gates remain blocked. The validator checks the exact safety-log bytes against the signed platform record before interpreting them.
The eligible evaluation gates are R259 (zero native protocol recovery), R262 (dynamic replanning quality), R263 (matched task success), and R264 (matched request and token efficiency). A positive R259 recovery count is a failure and cannot be waived. A waiver leaves the original evidence inconclusive; it is not a pass. The rationale is to allow an explicitly scoped preview after independently verified safety controls pass, without weakening full-release requirements. The alternative of treating missing results as passes would conceal the actual evidence gap.
The preview version is
3.0.0.dev1. The local publisher revalidates the signed manifest and scoped evidence, builds from the signed candidate, signs an artifact attestation, verifies a signed tag and uploaded draft assets, and publishes only a GitHub prerelease. Automatic V3 tag publication is excluded from the ordinary release workflow. Preview updates fail closed if the source version cannot be verified or would downgrade the installation. These changes add no Python dependency.Candidate and evidence
Candidate
c942491405fa4130a75891dd45f95f789cfa523fhas a locally and GitHub-verified commit signature from the pinned release-owner key. Six-platform CI and 30-minute soak run 41082489932 passed all six platform jobs, their 30-minute soaks, and the R266 collector. The R266 artifact was inspected for candidate binding, artifact hashes and required safety tests and signed with the release-owner key. The preceding candidate run failed in all six platform jobs and skipped the R266 collector. An earlier successful six-platform run is bound to668a3e43f1b17365ab24d4f0fee3bf8af6186e4cand cannot establish results for this candidate.No complete candidate-bound 17-gate evidence set, signed waivers, or accepted manifest exists. The original R259 and R262–R264 source reports were not found; further searching was stopped at the user’s request. Their results cannot be inferred from summaries. Missing safety evidence blocks the preview. Proposed scope: Linux and macOS preview with
KYROZEN_SANDBOX_REQUIRED=1; Windows distribution, production use, general availability, and claims that full V3 gates passed are excluded.The matched benchmark remains incomplete: the 3.5 run stopped at 133/180 rows with incomplete usage; the correctly configured 3.1 run stopped after one request with unknown usage. A report labeled 3.1 that actually used 3.5 is excluded as 3.1 evidence. Live benchmark work and budget-ledger reconciliation are paused as requested; unknown usage remains reserved. No new model calls were made.
Validation and risks
make check,make lint,make docs-check, Go tests and Go build passed locally on the candidate source.test_cancel_during_provider_call_prevents_late_writeafter a worker timed out; this does not establish that a late write occurred, but the safety check did not pass. The cancellation test now emits worker stacks on timeout without weakening its assertions. The new exact-SHA CI and all six soaks passed, but the previous Windows timeout did not recur and its underlying cause remains unconfirmed. The R266 record is signed; the other required candidate-bound gate records and waivers remain incomplete. Publication is blocked.Stacked native planning follow-up
PR #317 is stacked on this PR. It addresses a reproduced R252 gap in which a complex native turn could execute tools without first establishing a Working Plan. Its signed candidate and fresh CI run supersede this PR head for proposed preview evidence. The preview remains blocked until that run, gate records, and scoped waivers validate.
PR #318 is stacked on #317 and replaces a fragile deadline test assertion; the preceding run also showed an unresolved Windows subagent acceptance failure. The proposed preview candidate is PR #318 head, not this PR head.