Skip to content

github_image_signing 0.1.0: cosign sign and verify steps in workflow … - #385

Open
soya-beep wants to merge 3 commits into
mainfrom
feat/github-image-signing
Open

soya-beep wants to merge 3 commits into
mainfrom
feat/github-image-signing

Conversation

@soya-beep

Copy link
Copy Markdown
Contributor

…files, and how the signing workflows ran

Reads the workflow files on each default branch and records every cosign sign step (kind of key, whether the image is named by digest, transparency log, whether the step can be skipped or fail quietly) and every verify step. For each workflow that signs, reads the runs in the window by outcome. Summary lists repositories that sign without verifying. KSI-SVC-VRI.

Needs Contents for the files and Actions for the runs; include_runs: false reads the files only and leaves every run figure null.

The workflow file reader and the newest-first run walk move from github_actions_security and github_workflow_runs into the shared client so the three fetchers use one copy. Their evidence is unchanged.

…files, and how the signing workflows ran

Reads the workflow files on each default branch and records every cosign
sign step (kind of key, whether the image is named by digest, transparency
log, whether the step can be skipped or fail quietly) and every verify step.
For each workflow that signs, reads the runs in the window by outcome.
Summary lists repositories that sign without verifying. KSI-SVC-VRI.

Needs Contents for the files and Actions for the runs; include_runs: false
reads the files only and leaves every run figure null.

The workflow file reader and the newest-first run walk move from
github_actions_security and github_workflow_runs into the shared client so
the three fetchers use one copy. Their evidence is unchanged.

@bruceweygandt bruceweygandt left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The cosign parser is careful and well tested. It handles quoting, ${{ }} expressions, || true, the key kinds, and the docker-publish starter workflow. The move of workflow_file_texts and newest_since into the shared client does not change the output of github_actions_security or github_workflow_runs.

There is one must-fix and two should-fixes.

1. Must-fix: fetcher.yaml text does not follow the format, so CI fails

fetchers/github/image_signing/fetcher.yaml:3-6 and :83-90

Problem. tests/test_contracts.py::test_fetcher_text_follows_the_format[fetchers/github/image_signing/fetcher.yaml] fails. This makes the test (py 3.11) and test (py 3.14) jobs fail. The test reports three errors:

  • The description has 31 words. The limit is 30.
  • instructions does not start with Script: github/image_signing/fetcher.py on a line of its own.
  • After Script, the test expects Access, then Commands Executed: on its own line, then Output, with a blank line between each. These sections are missing.

Why this is a problem. The PR cannot merge while CI fails. Also, the token permissions that the fetcher needs (Contents read and Actions read) are only in the PR description. The person who sets up the token reads instructions, not the PR.

Fix. Shorten the description to 30 words or fewer. Rewrite instructions in the same form as other fetchers, for example fetchers/github/workflow_runs/fetcher.yaml: Script, Access (Contents read; Actions read when include_runs is true), Commands Executed (the GraphQL workflow-file read and GET repos/{org}/{repo}/actions/workflows/{file}/runs), Output, and the summary fields.

2. Should-fix: run history counts pull request runs that do not sign

fetchers/github/image_signing/fetcher.py:218-234

Problem. run_history counts every run of a signing workflow, from every event and branch. Many signing workflows also run on pull_request. GitHub's docker-publish starter workflow is an example. On a pull request run, the sign step is skipped by if: github.event_name != 'pull_request', but the run still ends with success.

Why this is a problem. signing_workflow_runs_successful and latest_successful_signing_run then include runs that did not sign an image. The evidence can show recent successful signing when no image was signed in the window.

Fix. Do not count runs whose event is pull_request or pull_request_target. Or report the run counts per event, so a validator can choose. Add a test with a pull request run of a workflow whose sign step has that if:.

3. Should-fix: the summary does not show reusable workflow calls

fetchers/github/image_signing/fetcher.py:205-207 and :280-313

Problem. A job that calls a reusable workflow (jobs.<id>.uses) is recorded only in reusable_workflows_called on its workflow row. No summary field counts or lists these workflows. workflows_mentioning_cosign_without_a_cosign_command finds them only when the called path contains "cosign".

Why this is a problem. Many organizations sign images in one shared reusable workflow. A repository that signs this way reads as not signing, and the summary gives no sign that the evidence stops at that call.

Fix. Add a summary field that lists the workflows that call a reusable workflow, for example workflows_calling_reusable_workflows, in the same <repository>:<path> form as the other lists.

@bruceweygandt bruceweygandt added the enhancement New feature or request label Nov 17, 2026
claude and others added 2 commits November 17, 2026 14:11
…ary format

The contract test now checks every new fetcher's text format. Shorten the
description to 30 words, write the instructions in the required blocks with
every summary field, check them against the summary, and keep the 80-word
cap only for old-style github instructions.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MBcaFn4pMcbUNLVCX6bo1j
… list reusable workflow calls

- Pull request runs (pull_request, pull_request_target) are left out of the
  signing run figures and counted apart in signing_workflow_pull_request_runs:
  a sign step is usually skipped on them while the run still succeeds, so they
  could show signing that never happened.
- New summary field workflows_calling_reusable_workflows: signing done in a
  called workflow is not read, so the summary now says where the evidence stops.
- Tests for both, README and CHANGELOG to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MBcaFptEwsTWWrKsxJfcyE
@soya-beep

Copy link
Copy Markdown
Contributor Author
  1. fetcher.yaml text format (41b3a5d)

The description is under 30 words.
instructions now follows the Script, Access, Commands Executed, Output, Summary fields format.
Access names the token permissions: Metadata and Contents read, plus Actions read when include_runs is true.
test_image_signing_verdicts now calls assert_summary_documented, so the summary fields in the yaml are checked against the real summary.
Note: workflow_runs/fetcher.yaml is still on TEXT_FORMAT_PENDING, so it is not a format example yet.
Also: the GitHub 80-word cap in test_every_fetcher_follows_the_shape cannot hold for text in the new format. It now applies only to fetchers whose instructions are still in the old one-paragraph form.

  1. Pull request runs counted as signing runs (c925b77)

run_history leaves out pull_request and pull_request_target runs from total, successful, by_outcome and the latest_* fields.
Each workflow row has pull_request_runs, and the summary has signing_workflow_pull_request_runs, so the excluded runs are still visible.
New test: a docker-publish style workflow with the sign step under if: github.event_name != 'pull_request'. A successful pull request run no longer counts as a successful signing run.

  1. Reusable workflow calls not in the summary (c925b77)

New summary field workflows_calling_reusable_workflows, in the same : form as the other lists.
New test: a repository that signs only through a called workflow is listed there and is not counted as signing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants