Skip to content

Parallel workers inflate the mutation score: overlay directories collide on non-unique mutant IDs #1

Description

@mcgoughprogress

Summary

Running gomu run with more than one worker reports a higher mutation score than a serial run of the same code, and the inflated figure is not stable between runs. The extra "kills" are phantoms: mutants that were never actually executed under test.

Serial runs are correct, so the defect is invisible unless you compare worker counts.

Reproduction

Measured against a single package (internal/ha of a private module, 108 mutants), --incremental=false, .gomu_history.json deleted between runs:

workers killed score
1 (ground truth) 79/108 91.9%
4 80/108 93.0%
8 80/108 93.0%

That package generates 108 mutants but only 93 distinct mutant IDs.

Root cause

Two things combine:

  1. internal/mutation/engine.go does not produce unique mutant IDs — see GenerateMutants produces non-unique mutant IDs #2 for the mechanism. Duplicates are pervasive: 87 of this repository's own 123 source files generate at least one repeated ID.

  2. internal/execution/overlay.go:53 derives each mutant's temp directory from that ID:

    mutantDir := filepath.Join(om.baseDir, fmt.Sprintf("mutant_%s", mutant.ID))

Two mutants with the same ID therefore share a directory. They overwrite each other's mutated source and overlay.json, and whichever finishes first calls CleanupMutation and deletes the directory out from under the other. The victim's go test -overlay ... then fails to load the package and exits non-zero, which internal/execution/engine.go:175 classifies as StatusKilled.

Net effect: phantom kills that inflate the score, non-deterministically, whenever --workers > 1.

Suggested fix

Fixing (1) alone would stop the collisions, but (2) is worth fixing on its own terms: mutant.ID carries no uniqueness contract that execution can rely on, it is owned by a different package, and it embeds the source file path — so the "directory" was really a nested path, mutant_internal/ha/ha.go_12.

Allocate the mutant directory from a counter private to the OverlayMutator instead. With that change applied, 4- and 8-worker runs return 79/108 (91.9%), matching the serial ground truth exactly.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions