Gremlins mutation testing tracker

[cplieger]

Gremlins mutation testing tracker

Auto-updated weekly by cplieger/ci/.github/workflows/weekly-gremlins.yaml.
Last update: 2026-07-07 23:28

This week: 99.0% efficacy (±0.0% across 3 runs), 98.7% mutant coverage. 3 confirmed live mutants.
Trend: ↗ +2.1% from 12-week mean (96.9%).

Rolling 12-week history

Run (UTC)	Mean efficacy	Stddev	Mutant coverage	Live mutants	Δ efficacy
2026-07-07 23:28	99.0%	±0.0%	98.7%	3	+0.0%
2026-07-06 01:31	99.0%	±0.0%	98.7%	3	+2.6%
2026-06-29 19:29	96.4%	±0.0%	98.7%	11	-1.8%
2026-06-29 00:22	98.2%	±0.0%	96.6%	5	-1.1%
2026-06-22	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-22	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-18	99.3%	±0.0%	93.6%	2	+0.0%
2026-06-18	99.3%	±0.0%	93.6%	2	+1.8%
2026-06-17	97.5%	±0.0%	93.6%	7	+0.0%
2026-06-17	97.5%	±0.0%	93.6%	7	+7.7%
2026-06-15	89.8%	±0.0%	92.8%	26	+0.0%
2026-06-15	89.8%	±0.0%	92.8%	26	—

Current live mutants (3, this week)

Solid gap — LIVED in all 3 runs (known coverage hole) — 3

internal/cache/processed.go

• L24 — CONDITIONALS_BOUNDARY
• L57 — CONDITIONALS_BOUNDARY

internal/streams/score.go

• L34 — CONDITIONALS_BOUNDARY

How to read

• Mean efficacy: % of runnable mutants killed (or timed-out, treated as caught), averaged across the N runs
• Stddev: variance across runs — high stddev (>3%) signals flaky tests
• Mutant coverage: % of mutants reached by the test suite (test depth)

The "Current live mutants" section is bucketed by how many of the {N} runs the
mutant LIVED in:

• Rare flake (1/N): a mutant your tests usually KILL but occasionally let
  through. Most actionable — almost always means a flaky test that's not
  reliable. Open the section to see the file:line; expect to fix the test, not
  the production code.
• Weakly flaky (k/N for 1<k<N): tests catch this mutant inconsistently.
  Less ideal than rare-flake but still worth investigating.
• Solid gap (N/N): every run lets this mutant through. Either there's no
  test exercising the path, or the test asserts the wrong thing. Action: add a
  test or strengthen an existing assertion.

The mutation-regression label is added when this week's mean efficacy drops

5% below the rolling 12-week mean.

Free-form notes

Add anything below — won't be touched by the auto-updater.