Skip to content

Run cargo-mutants audit on security-critical modules (v1.0 prep) #35

@Metbcy

Description

@Metbcy

Context

bomdrift has 432 tests and clippy-strict CI, but no proof its tests meaningfully exercise the security-critical modules. v0.9.8 reviewer flagged this as a v1.0 audit candidate: run cargo-mutants against modules where false negatives = security holes (diff/, enrich/typosquat, enrich/license, enrich/maintainer) to find tests that don't actually pin behavior.

Scope

  • Install cargo-mutants locally (cargo install cargo-mutants --locked).

  • Run on the security-critical modules:

    cargo mutants --in-place --no-shuffle --file 'src/diff/**'              --timeout 60
    cargo mutants --in-place --no-shuffle --file 'src/enrich/typosquat*'    --timeout 60
    cargo mutants --in-place --no-shuffle --file 'src/enrich/license.rs'    --timeout 60
    cargo mutants --in-place --no-shuffle --file 'src/enrich/maintainer.rs' --timeout 60
    cargo mutants --in-place --no-shuffle --file 'src/baseline.rs'          --timeout 60
    cargo mutants --in-place --no-shuffle --file 'src/vex.rs'               --timeout 60
  • For each module, document the surviving-mutants count (i.e. mutants the test suite didn't catch).

  • For surviving mutants that genuinely indicate weak coverage (not false-positive mutants like Vec::is_empty()false), add tests to kill them.

  • Goal: <10% surviving-mutant rate per module after the audit pass.

Output

Open one PR per module so each is reviewable on its own:

  • test(diff): kill surviving mutants from cargo-mutants audit
  • test(enrich/typosquat): kill surviving mutants from cargo-mutants audit
  • ... (one per module)

Each PR's body should include:

  • Total mutants generated.
  • Mutants caught (test failed).
  • Mutants surviving (test passed despite mutation).
  • Mutants you decided NOT to address (with brief rationale per).

Notes

  • Don't add mutation testing as a CI gate yet. It's slow (hours per full run) and noisy. v1.0 audit only; revisit gating in v2.x if it stabilizes.
  • Use --no-shuffle so the run order is reproducible.
  • Some mutants are uninteresting (e.g. mutating a log::debug! arg) — cargo-mutants config can exclude them.

A note on commit signing

main requires verified signatures (the repo ships cosign-signed releases — we hold our own commits to the same bar).

You usually don't need to set up signing as a contributor — when a maintainer merges via "Merge" or "Squash", GitHub auto-signs the resulting commit and your unsigned PR-branch commits are fine. The friendlier path for everyone.

If you'd like your individual commits to land verbatim on main (so your name shows up in git blame), set up local signing once and your PR can be rebase-merged:

git config --global gpg.format ssh
git config --global user.signingkey ~/.ssh/id_ed25519.pub
git config --global commit.gpgsign true

Then add the same SSH public key under GitHub → Settings → SSH and GPG keys → Signing keys.

See CONTRIBUTING.md → Commit signing on main for the full picture. Either way, please don't sweat it — if your PR is otherwise great, the maintainer will pick a merge mode that works.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions