feat(suite): supply_chain end-to-end proof + flip to proven (H-next-5a)#131
Merged
Conversation
…xt-5a, pre-flip)
Add the hermetic recipe that proves the supply_chain row end-to-end against a
released Assay binary, plus the committed input descriptor it runs on. The row
flip (declared -> proven) follows from a hosted recipe run in this PR.
- harness/fixtures/suite-compatibility/supply-chain/input.example.json: the
committed assay.supply_chain_conformance.input.v0 descriptor (input intent).
- .github/workflows/harness-ci.yml: dispatch-only assay-supply-chain-recipe job.
Downloads + checksum-verifies the released linux-gnu asset, emits the carrier
via `assay registry supply-chain-conformance --offline`, and consumes it via
`carrier supply-chain`.
Honest end-to-end semantics: the v1 emitter (none/unsupported provenance) yields
policy_result=incomplete, so the gate exits 6 (valid, not-clean). For this proof
exit {0,6} with a valid report is a successful consumption; exit {2,3,7} is a
contract/config failure. The proof establishes producer->consumer compatibility
and incomplete-never-clean, NOT a clean carrier. The recipe-provenance result
records recipe-execution success (0/"success") per the Evidence Pack verifier;
the not-clean verdict lives in the carrier (policy_result) and projection.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Fold the hermetic end-to-end proof from the released v3.28.0 supply-chain recipe
(hosted run 27724742076) into the supply_chain carrier row.
- end_to_end: declared -> proven; drop end_to_end_gap (no_released_binary_emitter).
- proof binds the released v3.28.0 emitter: hosted_run, assay_version=v3.28.0,
assay_binary_digest (extracted executable), fixture_digest (committed descriptor),
artifact_digest (the EMITTED CARRIER), runner_os, proof_scope{hosted, ambient_scan:false}.
- emits.min_version: v3.27.0 -> v3.28.0 (the release with the one-shot CLI emitter).
- manifest.digest recomputed via computeMatrixDigest.
Honest semantics: the v1 emitter (none/unsupported provenance) yields a valid carrier
with policy_result=incomplete, so the consumer exits 6 (valid, not-clean). The proof
establishes producer->consumer compatibility and incomplete-never-clean, NOT a clean
carrier; the row note records this.
Tests: e2e_proven_count 2 -> 3, e2e_declared_count 4 -> 3; the declared-carrier example
moves to render_safety; a new test locks the supply_chain proof (proven + bound to
v3.28.0 + fixture-scoped + honestly not-clean).
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (1)
📝 WalkthroughWalkthroughA new dispatch-only ChangesSupply-chain conformance E2E proof
Sequence DiagramsequenceDiagram
participant Runner as GitHub Actions Runner
participant GHRelease as GitHub Releases
participant AssayBin as Assay Binary (v3.28.0)
participant HarnessCLI as Harness CLI
Runner->>GHRelease: resolve & download tarball (default v3.28.0)
GHRelease-->>Runner: tarball + checksum
Runner->>Runner: verify checksum, export ASSAY_BIN
Runner->>AssayBin: emit carrier offline (input.example.json)
AssayBin-->>Runner: supply-chain carrier JSON
Runner->>Runner: assert schema + policy_result=incomplete, compute digests
Runner->>HarnessCLI: consume carrier
HarnessCLI-->>Runner: consumer report JSON (exit 0 or 6)
Runner->>Runner: validate consumer report JSON
Runner->>Runner: emit + validate suite.recipe_provenance.v0
Runner->>Runner: upload carrier, report, provenance artifacts
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
harness/test/suite_compatibility.test.mjs (1)
122-131: ⚡ Quick winAdd explicit row-existence assertions before property access.
These tests currently rely on implicit TypeErrors when
.find(...)misses. Explicit existence checks make CI failures much faster to diagnose.♻️ Suggested refactor
const inv = m.carrier_rows.find((r) => r.carrier === "assay.mcp_server_inventory.v0"); + assert.ok(inv, "missing assay.mcp_server_inventory.v0 row"); assert.equal(inv.proof.end_to_end, "proven"); @@ const rs = m.carrier_rows.find((r) => r.carrier === "assay.render_safety_conformance.v0"); + assert.ok(rs, "missing assay.render_safety_conformance.v0 row"); assert.equal(rs.proof.end_to_end, "declared"); @@ const sc = m.carrier_rows.find((r) => r.carrier === "assay.supply_chain_conformance.v0"); + assert.ok(sc, "missing assay.supply_chain_conformance.v0 row"); assert.equal(sc.proof.end_to_end, "proven");Also applies to: 136-151
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@harness/test/suite_compatibility.test.mjs` around lines 122 - 131, The test code uses `.find()` to locate carrier rows and directly accesses their properties without verifying the find operation returned a result, which can lead to implicit TypeErrors that are hard to diagnose. Add explicit `assert.ok()` checks immediately after each `.find()` call: one after the find for carrier "assay.mcp_server_inventory.v0" (assigned to inv) and another after the find for carrier "assay.render_safety_conformance.v0" (assigned to rs), ensuring each row exists before accessing its proof and end_to_end_gap properties on the following lines.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/harness-ci.yml:
- Around line 749-756: The test currently accepts both exit codes 0 and 6 as
valid consumption outcomes regardless of the policy_result value, which can mask
regressions where incomplete policies are incorrectly treated as clean. Modify
the case statement that checks consume_rc to only accept exit code 0 when the
policy_result is not "incomplete", or restrict acceptable exit codes based on
the policy_result value. Ensure that when policy_result equals "incomplete",
only exit code 6 (not 0) is treated as valid to properly enforce the stated
proof claim.
- Around line 675-683: The hardcoded fallback version in the assay_version
initialization block creates a mismatch: the script falls back to v3.28.0, but
the shared workflow_dispatch input defaults to v3.27.0, which violates the
min_version requirement of v3.28.0. Update the fallback assignment in the line
assay_version="${ASSAY_VERSION_INPUT:-}" by changing the hardcoded default
version from v3.28.0 to align with both the actual workflow_dispatch input
default and the minimum version requirement to ensure the job cannot silently
run with an incorrect version.
---
Nitpick comments:
In `@harness/test/suite_compatibility.test.mjs`:
- Around line 122-131: The test code uses `.find()` to locate carrier rows and
directly accesses their properties without verifying the find operation returned
a result, which can lead to implicit TypeErrors that are hard to diagnose. Add
explicit `assert.ok()` checks immediately after each `.find()` call: one after
the find for carrier "assay.mcp_server_inventory.v0" (assigned to inv) and
another after the find for carrier "assay.render_safety_conformance.v0"
(assigned to rs), ensuring each row exists before accessing its proof and
end_to_end_gap properties on the following lines.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro Plus
Run ID: cc02ce62-e64d-4a96-a263-82cb0ce7a10d
📒 Files selected for processing (4)
.github/workflows/harness-ci.ymlharness/fixtures/suite-compatibility/supply-chain/input.example.jsonharness/suite-compatibility.jsonharness/test/suite_compatibility.test.mjs
…it-6 strictness) CodeRabbit (2 Major + 1 nitpick), all folded: - Version guard: the shared workflow_dispatch input defaults to v3.27.0 (no supply-chain emitter), so default to v3.28.0 and FAIL CLOSED on any resolved version < v3.28.0 — the proof binds min_version v3.28.0 and must not run against an emitter-less release. - Exit-6 strictness: the recipe asserts the emitted carrier is policy_result=incomplete, so the gate MUST exit exactly 6 (incomplete => not-clean). Accepting exit 0 would mask an incomplete-read-as-clean regression; require exit 6 + a report with policy_result==incomplete. - Tests: explicit row-existence assertions before property access. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Re-ran the hardened supply-chain recipe (run 27725477694) after folding the review fixes; it reproduced byte-identical binary/asset/fixture/carrier digests (determinism), so only hosted_run changes. Rebind the proof to the run that executed the merged recipe code; manifest.digest recomputed via computeMatrixDigest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Owner
Author
|
@coderabbitai review |
✅ Action performedReview finished.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
H-next-5a — supply_chain end-to-end, proven against released Assay v3.28.0
Flips the
supply_chaincarrier row fromdeclared(gapno_released_binary_emitter) toend_to_end: proven, using the one-shot supply-chain emitter released in Assay v3.28.0. The consumer (carrier supply-chain), the matrix row, the recipe-provenance schema, and the proven-needs-evidence gate already existed; this PR adds the committed fixture, the hermetic recipe, and the proof.What it proves (and what it does not)
The released v3.28.0 binary emits a valid
assay.supply_chain_conformance.v0carrier from a committed descriptor, and Harness validly consumes/classifies it viacarrier supply-chain. The v1 emitter (none/unsupported provenance) yieldspolicy_result: incomplete, so the gate exits 6 (valid, not-clean) — positive evidence that Harness recognizes the carrier, reads the producer statuses, and refuses to readincompleteas clean.The proof establishes producer→consumer compatibility + incomplete-never-clean, NOT a clean carrier. Non-claims (frozen): emitted/consumed ≠ safe · verified checksum ≠ policy approval · carrier emitted ≠ clean · row proven ≠ category complete. No Sigstore/Rekor verification, no policy approval, no review.
How
harness/fixtures/suite-compatibility/supply-chain/input.example.json— the committed input descriptor (input intent).assay-supply-chain-recipe(dispatch-only, harness-ci.yml) — downloads + checksum-verifies the releasedx86_64-unknown-linux-gnuasset, emits the carrier--offline, consumes it, and writes asuite.recipe_provenance.v0sidecar. Accepts consumer exit{0,6}with a valid report; fails on{2,3,7}.artifact_digest), and fixture (descriptor).resultrecords recipe-execution success (0/success) per the Evidence Pack verifier; the not-clean verdict lives in the carrier (policy_result) and the row note.Tests
Full suite green (453 pass, 1 skip).
e2e_proven_count2→3; a new test locks the supply_chain proof: proven, bound to v3.28.0, fixture-scoped (ambient_scan: false), and honestly not-clean.🤖 Generated with Claude Code
Summary by CodeRabbit
Release Notes
CI / New Features
Bug Fixes / Compatibility
Tests
Chores