Skip to content

ci(mutants): decide in a selector job whether the gate runs - #1420

Draft
auxesis wants to merge 2 commits into
ci/test-eql-selector-jobfrom
ci/mutants-selector-job
Draft

auxesis wants to merge 2 commits into
ci/test-eql-selector-jobfrom
ci/mutants-selector-job

Conversation

@auxesis

@auxesis auxesis commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Mutants is the mutation-testing gate for stack-auth and stack-encrypt. It used a workflow-level path filter, so it did not run at all on most pull requests. A workflow skipped that way leaves its required checks pending, and GitHub's documentation on required status checks says that blocks a merge. So Mutants could not become a required check.

After this change, Mutants starts on every pull request. A selector job decides whether the gate runs, and a summary job reports the result. The gate runs for the same files as before.

This pull request is stacked on #1417. Merge that one first.

Changes

  • Selector job: mutants.yml starts with a selector job on ubuntu-latest. It uses dorny/paths-filter with the old filter's paths. It fails when the filter gives no answer on a pull request.
  • Excluding documentation: the filter uses dorny's every quantifier. Under the default, a !**/*.md rule matches every file that is not Markdown. So the trigger paths are one brace pattern, followed by the two ! rules.
  • Gate: the mutants job needs the selector. Its if: is github.event_name != 'pull_request' || needs.selector.outputs.mutants == 'true'. A manual dispatch still sweeps both crates.
  • Summary job: Mutants summary needs both jobs. It passes when each passed or was skipped.
  • No path filter: on.pull_request is now {}.
  • Shared test code: scripts/__tests__/lib/selector.mjs reads a selector job, runs its script, and works out which jobs a pull request runs.
  • New tests: workflow-selectors.test.mjs checks the same rules for every workflow with a selector job, including Test EQL. mutants-selector.test.mjs pins what the Mutants selector decides.
  • Changed tests: the parity test compares a selector workflow's push paths with its selector filters, by what each selects. That replaces Test EQL's written exemption. In the dispatch test, every selector output is set to true.

Verification

  • pnpm run test:scripts: 83 files, 1,463 tests passed, 3 skipped.
  • actionlint 1.7.12 with shellcheck 0.11.0 on mutants.yml: no findings.
  • I ran the filter's rules through picomatch 2.3.2, the matcher dorny uses, with dot: true. It matched the crates' code and the configuration files. It did not match Markdown, .example files, stack-kms, or stack-encrypt-derive.
  • I read the source of the pinned dorny/paths-filter (v3.0.4). On a pull request it lists changed files through the GitHub API, so the selector needs no checkout.
  • Not yet verified: a run on GitHub. This pull request changes mutants.yml, so the gate runs here.

Related

Review notes

  • Look first at the selector job and the multi-line brace pattern. YAML drops a line break that ends in \, so dorny reads one pattern.
  • The other pull request workflows get selector jobs in separate pull requests, also stacked on ci(test-eql): run the Postgres suite only for changes to EQL #1417. Each one has the same first commit, test(ci): check every selector job the same way. Git merges identical changes cleanly. Their edits to the lists in workflow-paths-filter-parity.test.mjs sit next to each other, so the second to merge may need a rebase.
  • The new jobs have timeout-minutes: 10. That matches the floor in the change that adds timeouts.
  • Merge queue runs select every job. Comparing a queue commit with the entry ahead of it is a separate change.

Test EQL has a selector job, and the other pull request workflows are
getting one each. Each workflow's selector must follow the same rules:
no path filter on pull_request, one dorny/paths-filter step, an output
named after each filter, every job selected on other events, a failure
when a filter gives no answer, `if:` conditions that exclude
pull_request, and a summary job that needs every other job.

workflow-selectors.test.mjs checks those rules for every workflow that
has a selector job, found by scanning. It runs the selector's own
decision script and the summary job's own check.

lib/selector.mjs holds the shared code. It reads a selector job, runs
its script, and works out which jobs a pull request runs from the files
it changes. It reads dorny's `every` quantifier and brace patterns,
which a filter needs to exclude documentation.

The parity test now compares a selector workflow's push paths with the
union of its selector filters, by what each selects from the tracked
files. That replaces Test EQL's written exemption. The dispatch test
holds every selector output open, whatever its name.
Mutants used a workflow-level path filter on pull_request. A workflow
that a path filter skips leaves its required checks pending, and
GitHub's documentation on required status checks says that blocks a
merge. So Mutants could not become a required check.

The workflow now runs on every pull request. A `selector` job on a
GitHub-hosted runner compares the changed files with the old filter's
paths and decides whether the gate runs. When the filter gives no
answer, the selector fails. A `Mutants summary` job ends the workflow.
It passes when every needed job passed or was skipped. It is the job
that branch rules will require.

The trigger paths are the old filter's paths, so the gate runs for the
same changes as before. The filter uses dorny's `every` quantifier, so
its `!` rules still exclude Markdown and `.example` files. A manual
dispatch still sweeps both crates.

mutants-selector.test.mjs pins what the selector decides for code,
configuration, documentation and other crates.
@changeset-bot

changeset-bot Bot commented Oct 28, 2026

Copy link
Copy Markdown

⚠️ No Changeset found

Latest commit: f299db3

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@coderabbitai

coderabbitai Bot commented Oct 28, 2026

Copy link
Copy Markdown
Contributor

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

  • Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

@github-actions

Copy link
Copy Markdown
Contributor

Mutation testing (cargo-mutants, --in-diff, stack-auth + stack-encrypt)

No mutants were generated for the changed lines.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant