Skip to content

Phase B: rescan with VEX, observe-only images, stable IDs, contract hygiene - #76

Merged
northcutted merged 15 commits into
mainfrom
feat/phase-b-rescan
Oct 26, 2026
Merged

northcutted merged 15 commits into
mainfrom
feat/phase-b-rescan

Conversation

@northcutted

@northcutted northcutted commented Oct 26, 2026 •

Copy link
Copy Markdown
Owner

Wave 2 for clearcutt-verify: Phase B (structural gaps) plus contract hygiene, per the assessment's §2 gap A, §3 (verify), and §4 Phase B/C/D, and the items the orchestrator appended (VEX scoping, the estate signer role, the no-check exit code, the live cosign trusted-root bug).

Item What changed Test that proves it
1. --rescan (gap A) grype over each image's verified SBOMs, per platform; images[].rescan beside vulnerabilities (scanner, database.built, findings); policy.scanSource (attestation/rescan/worst, worst by default with --rescan); maxScanAgeDays by database build time; no verified SBOM or unreadable platforms ⇒ unknown, never clean; grype missing ⇒ exit 1 before the run; grype's database updated once before the run (grype db update, failure ⇒ exit 1 before reading anything) and scans run with GRYPE_DB_AUTO_UPDATE=false (with it already false, only grype db status checks a database is there); grype scanning nothing ⇒ exit 1 after writing TestRescanScansOnlyVerifiedSBOMs, TestRescanGrypeFailureIsUnknown, TestScanSourceGating, TestMaxScanAgeUsesTheRescansDatabase, TestGrypeDatabaseIsPreparedOnce (stand-in grype binary: env and order), TestEstateVerifyRescan (recorded grype output; database update failing), e2e --rescan case (registry:3 leg)
2. VEX for rescans vex: list (relative to the policy), scoped per image before grype sees it (image products by digest / repository_url / plain reference rewritten to subcomponents or every package; package products estate-wide; rule in docs/verify-estate.md); every suppressed finding listed with suppressedBy; unused statements warned TestVEXScoping, TestReadVEXRefusesWhatIsntOpenVEX, the rescan tests above
3. Observe-only tier observe: patterns ⇒ images[].observeOnly, summary.verdicts.observeOnly, ignored by --fail-on, marked in history rows TestObserveOnly, TestEstateVerifyObserveOnly
4. Contract hygiene IDs from the full reference, order-independent (hash suffix on collisions); uniqueImages (observations still sharing an ID numbered -2, -3 by canonical reference, then digest, not input order); report and history validated against the embedded schemas before writing (exit 1); deterministic checkedAt; legacy history IDs renamed when unambiguous, computed from each image's observed reference as the old IDs were (a tagless reference had the repository's ID, a digest reference the digest's) TestImageID, TestAssignIDsIsUniqueAndOrderIndependent, TestUniqueImages (incl. one reference at two digests, both orders; SourceRefs), TestValidate, TestValidateUsesThePublishedSchemas, TestEstateVerifyWritesNothingInvalid, TestHistoryKeepsImagesAcrossTheIDChange (incl. bare, digest, and tag+digest references; unknown reference left alone)
5. Staleness from claims name/history edges have no drift (no version identified) and link to no image (base.imageId empty; an unreadable image, which has no digest either, is never matched); maxDaysBehind fails only on proven drift, claimed drift ⇒ unverified; --reproduce doesn't skip an unknown recipe TestBuildGraphNameAndHistoryEdgesHaveNoDrift, TestNamedBaseIsNotLinkedToAnUnreadableImage, TestStalenessFromClaimsNeverFails, TestReproduceDoesNotSkipAnUnknownRecipe
6. sourceRepositoryOwner value validated (https://github.com/OWNER exactly, a trailing slash allowed); every trust-policy field described in the schema; exact rule documented TestSourceRepositoryOwnerIsValidated (a repository URL, no scheme, http, another forge, no owner: refused at policy read naming https://github.com/OWNER, for trusted signers, stack signers, and trust-policy signers; the owner with and without / accepted), TestAccepts (owner with a trailing slash matches its repositories, not acme-evil); schema regenerated
7. Provenance honesty images[].provenance (builderId, buildType, builderChecked); detail says what was/wasn't checked; provenanceBuilders: makes the builder a check (another builder, or none, ⇒ failed, and neither source nor images[].provenance is read from failed provenance; an empty entry is refused at policy read); golden test recomputes every verdict in the contract bundles; contract README records the v0.16.0 meaning change TestProvenanceBuilders (listed ⇒ verified + builderChecked; another builder, none, and an unvalidated empty entry ⇒ failed, no source, no panic; unchecked ⇒ read), TestReadProvenanceDetails (SLSA v1, v0.2), TestReadPolicy (provenanceBuilders: [''] refused), TestVerdictsInContractBundlesArePinned (pinned the bundles; the minimal example's hand-written verdicts were corrected)
8. Contract gate portal-contract job: clearcutt-portal (pinned to main b183f2e, the documented invocation isn't released yet; move the pin at its next release) builds the fresh e2e bundle and the v0.16.0 fixture; in the PR gate summary contract workflow + TestPRGateSummaryContractMatchesItsJobs
estate role trust-policy role estate; estate pull/history --trust-policy/--policy; estate history now verifies the index signature (breaking: needs a signer or --allow-unverified) TestEstatePullAndHistoryUseTheEstateRole, e2e history round trip
Nothing checked ⇒ exit 1 an all-unknown report is written and the run exits 1 TestEstateVerifyNothingCheckedIsNotAPass
Live cosign bug root cause: a concurrent cosign process losing Sigstore's trusted root (each run refreshes TUF into the shared cache; v3.0.6 only warns, then fails the bundle; verify kept the last stderr line and read it as failed). Fix: cosign initialize once (for keyless signers and key signers that check the transparency log), then --trusted-root on every verification that needs it (keyless, and key with tlog on), using the root of the mirror initialize used (TUF_MIRROR, else remote.json, else the default), else the newest trusted_root.json in the cache; none found ⇒ a loud warning (stderr, ::warning, report warnings) and the old per-run fetch; TUF errors (the legacy client's initializing tuf, tuf: failed to decode included) ⇒ unknown with TUF's message; --trusted-root FILE; action pins cosign v3.1.4; keyless e2e job against the public Sigstore TestLostTrustedRootIsUnknownNotFailed, TestLegacyTUFErrorsAreUnknown, TestPrepareTrustedRoot (two mirrors, another layout, none found, key with and without tlog), TestEstateVerifyFetchesTheTrustedRootOnce (incl. the warning in the report), cosign-keyless-e2e

Not shown deterministically:

  • The live "trusted root is required when using new bundle format" failure was not reproduced on demand. The root-cause analysis comes from the stderr quoted in TestLostTrustedRootIsUnknownNotFailed (cosign v3.0.6 in the showcase's CI) and cosign's source; the keyless e2e job exercises the fixed path (one cosign initialize, --trusted-root on every verification) against the public Sigstore, but it doesn't force the race.

Not done:

  • Refusing, when the policy is read, a signer whose identity is a reusable workflow and that names neither sourceRepository nor sourceRepositoryOwner (nor sourceMatchesImage). That case stays decided at evidence time: such evidence is present, never verified (callerUnchecked).

Deferred: the northcutted-images fixture wasn't regenerated with --rescan (it is a real estate; the minimal example carries the new fields); the keyless e2e runs only for same-repository PRs (forks get no OIDC token).

🤖 Generated with Claude Code

northcutted and others added 15 commits October 18, 2026 20:29
…ting

Image IDs were the repository's last path segment and tag, with -2, -3
added to repeats in input order, so IDs collided across registries and
changed when refs.txt was reordered, though the contract promises they
are unique and stable. They now come from the whole reference (registry,
repository path, tag or short digest); references that spell the same ID
each get a hash of the reference appended, independent of order. Build
gives any observation with a missing, repeated, or non-path-segment ID the
ID its reference gives, and history rows under the old short IDs are
renamed when that is unambiguous.

estate verify now validates the report and the history against the
contract schemas (compiled into the binary from the same types) before
writing either, and writes nothing (exit 1) when one doesn't match.
reproducibility.checkedAt for rebuilt images is the report's time, so
--generated-at makes the report reproducible.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
maxDaysBehind failed images on drift from bases they only claimed, and
oci-base-name and history edges matched the oldest observed version of
the repository, so a label could make an image stale. Name and history
edges now identify no version (drift unknown, no digest, not stale), and
maxDaysBehind fails only on drift from a base proven by layer digests;
claimed drift leaves the image unverified. --reproduce no longer skips an
image whose recipe or rebase record is unknown: it is not-checked and
unverified.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…y images, provenance detail

estate verify --rescan scans each image's verified SBOMs with grype and
reports the result (images[].rescan: scanner, database build time,
findings) beside the publisher's attested scan. Only SBOMs a trusted
signer verified are scanned; an image with none on a platform, not all of
whose platform images could be read, or that grype couldn't scan is
rescan.status unknown, never clean. The policy's scanSource (attestation,
rescan, worst; worst by default with --rescan) says which scan failOn
gates on; maxScanAgeDays measures a rescan by its database's build time.
grype missing exits 1 before the run; grype scanning nothing exits 1
after writing the report.

A policy's vex: OpenVEX documents are applied to the rescans, scoped per
image before grype sees them (image products by digest or repository_url
rewritten to their subcomponents or every package; package products
estate-wide); every suppressed finding is listed with the statement that
suppressed it.

A policy's observe: patterns mark images reported with a verdict but not
gating (observeOnly, summary.verdicts.observeOnly, ignored by --fail-on).

Verified provenance records the builder and build type as read and says
in its detail what was checked (signature, subject, source) and what
wasn't; provenanceBuilders makes the builder a check. A golden test pins
every verdict in the contract bundles, so a change of meaning can't pass
unnoticed; the bundles' verdicts are pinned from this code (the minimal
example's hand-written verdicts were corrected).

A run in which every piece of evidence is unknown writes its report and
exits 1: a report in which nothing could be checked is not a pass.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…, not failed

On the showcase estate, one verified attestation a day came out failed
with "not signed by a trusted signer: … trusted root is required when
using new bundle format". Each cosign run refreshes TUF metadata over the
network into the shared ~/.sigstore/root cache, so with verifications
running concurrently one could find the TUF repository unreachable, or a
sibling's half-written file, and lose the trusted root; cosign v3.0.6
(what the action installed) only warns what TUF said and then fails the
bundle for the lack of a root, and verify kept only cosign's last stderr
line and didn't classify that text as "couldn't check".

estate verify and estate pull now fetch the trusted root once (cosign
initialize) when the policy trusts a keyless signer and hand the cached
trusted_root.json to every verification (--trusted-root), so none fetches
it itself; a fetch that fails is an operational error. --trusted-root
FILE supplies one; SIGSTORE_ROOT_FILE and the other individual-target
variables are left to cosign. TUF and trusted-root errors are classified
as couldn't-check (unknown), with TUF's own message kept in the detail.
The action installs cosign v3.1.4.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…policy

A trust policy's signers may have role estate: who may sign the estate's
own artifacts in the registry (the report bundle estate push stores and
the history index it appends to). estate pull and estate history take
--trust-policy FILE (or --policy FILE, through its trustPolicy) and verify
against those signers, beside the cosign flags Phase A added.

estate history read the index on nobody's word; it now verifies the
index's signature like a bundle and refuses an unsigned or wrongly signed
index unless --allow-unverified (breaking: callers name a signer, and a
job that appends to a history signs the index digest the push prints).

sourceRepositoryOwner must be exactly https://github.com/OWNER (a wrong
value silently matched nothing), the trust-policy schema describes every
signer field, and the docs state the exact owner rule.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…te in the real-cosign e2e

The real-cosign e2e now stores evidence both ways cosign v3 can (bundles,
and cosign v2's .sig/.att tags for a legacy image) and checks every kind
verifies the same way, pushes to and reads back a signed history index,
and, on the registry:3 leg, rescans the verified SBOMs with a pinned,
checksum-verified grype and keeps its bundle. A separate job signs an
image keylessly with the job's own OIDC identity against the public
Sigstore (skipped for forks' pull requests), exercising the trusted-root
fetch that failed one attestation a day on the showcase. A portal
contract gate builds the fresh bundle and the previous release's fixture
with clearcutt-portal, pinned to its main by commit (the documented
invocation landed after v0.1.0; move the pin when the next release
ships), so a breaking contract change fails here first. All three are in
the PR gate summary's contract.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The keyless signer block was appended after failOn, so the policy file
didn't parse in the keyless job.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…eadable one

An image that couldn't be read has no digest, and the ID map keyed it by
the empty string; edges from base.name or history now carry no base
digest, so they resolved to that image. Empty keys are skipped and an
edge without a digest links to no image.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A reference without a tag (ghcr.io/acme/app) had the old ID app, since
the tag was latest, but the rename rebuilt it from the report's tags and
digest and computed the digest's ID, so its rows were never renamed; a
tagged digest reference had the digest's old ID, not the tag's. The
rename now computes the old ID from each image's observed reference,
the way the old IDs were made, and leaves images whose reference isn't
known alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ng from failed provenance

A policy listing an empty builder matched provenance that named no
builder, and then dereferenced the details it hadn't read (a nil pointer
panic). Empty entries are refused when the policy is validated, and
provenance naming no builder fails whatever the list holds. Provenance
whose builder check failed still set the image's source (basis
provenance) and images[].provenance; nothing is read from it now.

Tests cover a listed builder (verified, builderChecked), another builder
and none (failed, no source, no provenance), the empty entry reaching
Build unvalidated (no panic), readProvenanceDetails for SLSA v1 and v0.2,
and the policy refusal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…com/OWNER

The PR table named tests for the validation that didn't exist. A
repository URL, a value without the scheme, http, another forge, or no
owner is refused when the policy is read, with a message naming the
exact form, whether the signer is a trusted signer, a stack signer, or
comes from a trust policy; the owner with or without a trailing slash
is accepted and matches at evidence time.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… so when it can't

After cosign initialize, the cached trusted_root.json was used only when
exactly one <cache>/*/targets/trusted_root.json existed; with two
mirrors cached or another layout, every verification silently went
back to fetching TUF concurrently, the race the trusted root fixes.
The root is now the one from the mirror initialize used (TUF_MIRROR,
else the mirror it recorded in remote.json, else the default), else the
newest trusted_root.json in the cache (initialize clears it first). When
none is found the run warns on stderr, as a GitHub Actions annotation,
and in the report's warnings, and verifies as before.

Key signers that check the transparency log need the trusted root too
(Rekor's keys), so they now trigger the one fetch and get --trusted-root;
keys with tlog: false don't. The legacy TUF client's cache errors
(initializing tuf, tuf: failed to decode, local cache may be corrupt)
are unknown, not failed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
--rescan checked only that grype ran; the first scans then ran four at
a time and each checked for, and on a fresh runner downloaded, the
vulnerability database into the same cache at once. The run now runs
grype db update once before reading anything (a failed update exits 1,
like a missing grype), and every scan uses that database as it is
(GRYPE_DB_AUTO_UPDATE=false). With GRYPE_DB_AUTO_UPDATE=false already
set, nothing is downloaded: grype db status checks a database is there.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…, not input order

uniqueImages gave -2, -3 to observations that AssignIDs still left with
one ID (the same reference observed at two digests, for one) in the
order they came in, so the IDs swapped when the input order did. They
are now numbered by canonical reference, then digest, then the rest of
what tells them apart.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ransparency log

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@northcutted
northcutted merged commit 8e1aec5 into main Oct 26, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant