ci: wire full Sigstore/Rekor verification (cosign verify-blob) — currently structural presence only

[AlbinoGeek]

From TODO.md ("Publication pipeline") and related to closed #8 (signers.yaml trust bootstrap).

Today validate.py checks only the structural presence of signature.sigstore.json; it performs no cryptographic verification, and the publish job's --require-signature enforces presence, not validity. AGENTS.md/HUMANS.md both flag full Sigstore/Rekor verification as deferred.

Ask: wire cosign verify-blob (or equivalent Sigstore tooling) into CI to verify bundle attestations against the trusted signers in signers.yaml, including Rekor transparency-log consistency. Enforce in the publish job once the submitting workflow in Rethunk-AI/bakeoff stabilises.

Acceptance: an unsigned or invalidly-signed bundle is rejected by publish on cryptographic grounds, not merely on missing-file grounds.

[AlbinoGeek]

Reopening — the merge of #25 auto-closed this, but #25 only shipped the advisory (non-blocking) cosign verify-blob step (commit f3ffe05). This issue's scope is full Sigstore/Rekor verification as a hard publish gate, which remains open: it is blocked on upstream Rethunk-AI/bakeoff emitting real Sigstore bundles (current submissions carry stub signatures that real verification would reject). Promotion path = remove continue-on-error + wire signers.yaml identity constraints once upstream stabilises.

[AlbinoGeek]

Bastion // J-4 tracking update // 290522ZMAY26

Re-open acknowledged. State recorded:

BLOCKED — upstream Rethunk-AI/bakeoff not yet emitting real Sigstore bundles; stub signatures in current submissions would fail hard verification.

Promotion path captured:

1. Upstream gate: Rethunk-AI/bakeoff must emit real Sigstore bundles
2. Remove continue-on-error from cosign verify step in publish pipeline
3. Wire signers.yaml identity constraints

Ticket remains open as hard-gate tracking. Will resurface when upstream stabilizes.

— Bastion // AGI Overwatch

[AlbinoGeek]

050139ZJUN26 — Sigstore hard-gate dependency analysis

Advisory cosign verify-blob step is already in CI and correctly structured with continue-on-error: true and a TODO pointing here. No change needed in bakeoff-results CI today. Blocking issue is upstream.

Why the hard gate can't land yet

signers.yaml trusts bundles signed by:

repo:Rethunk-AI/bakeoff:ref:refs/heads/main
  via .github/workflows/package-results.yml

That workflow does not exist in Rethunk-AI/bakeoff. Until it does, every submitted bundle will fail OIDC subject verification — so promoting continue-on-error: false would reject all submissions including legitimate ones.

Current signing gap

bench/publish.py already wires cosign sign-blob when --sign is passed (calls _sign_result), but this is a local invocation using whatever ambient credentials the operator has. The resulting Sigstore bundle carries the local OIDC identity, not repo:Rethunk-AI/bakeoff:ref:refs/heads/main. Signers.yaml won't accept it.

Additionally, all existing bundles under submissions/ carry stub signature.sigstore.json files — not real Sigstore bundles produced by a GHA workload-identity flow. These will need to be either re-signed (by the new workflow retroactively) or grandfathered under a migration flag when enforcement lands.

Two-phase path to hard gate

Phase 1 — implement package-results.yml in Rethunk-AI/bakeoff (new issue filed, see below)

• GHA workflow that triggers when a run result is ready to publish
• Signs result.json via cosign sign-blob + GitHub Actions OIDC (workload identity) — no long-lived keys
• Submits signed bundle to bakeoff-results via PR or direct push
• Output bundle will satisfy signers.yaml exactly as written

Phase 2 — promote advisory → hard gate here (this issue)

• Once Phase 1 is shipping bundles, drop continue-on-error: true from the cosign verify-blob step
• Add a migration note or --allow-unsigned exemption for pre-enforcement submissions if needed
• Close this issue

New bakeoff issue tracking Phase 1 filed as Rethunk-AI/bakeoff — "CI: package-results.yml — automated Sigstore bundle signing and submission workflow".

— Bastion