You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
test-parity: known_failures.json is a pure suppression list — a stale entry hid a data-loss bug for six days #7582
test-parity/known_failures.json is a pure suppression list. parity_known_failures.py computes failures - allowed, so an entry whose test
has started passing is simply inert. It never fails, never reports, and never
asks to be removed.
That is a fifth way a gate can be unable to fail, alongside the four already
named in CLAUDE.md — and it is the most insidious, because unlike those four the
job is genuinely green and genuinely running. The entry silently converts a
future regression of that exact test into a non-event, forever.
It just cost us a real bug
test_gap_diagchannel_3082_3084_3085_3086 was added to known_failures.json on
2026-07-04 for the original #3082/#3084/#3085/#3086 feature cluster. Those
features were implemented; the entry stayed. When #7105 (2026-08-01) broke
the test again for a completely unrelated reason — PreallocateBoxes shadowing a
module-level global — the suppression absorbed it.
The impact was not one gap test. The defect silently emptied every let/const in a top-level bare block of an ES module that a sibling function declaration read. Nothing threw; the binding just read back as
whatever it was defined with. It sat for six days and was found by accident.
Fixed in #7580.
Compounding it, parity is tag-gated (CLAUDE.md), so even an unsuppressed
failure there would not have blocked a merge.
The fix that already exists in this repo
scripts/gc_root_dominance_allowlist.json solves exactly this, and its own
header states the rule:
an entry that matches nothing FAILS the build. When the bug is fixed, delete
the entry in the same PR. That is the ratchet.
known_failures.json should adopt the same property: an entry whose test now
passes fails the run, with a message naming the entry to delete. That converts
the list from a place bugs go to die into a ratchet that empties itself.
Audit the existing entries for others that have started passing. Every one is
a live suppression of a test nobody is watching, and this exercise just found
one that was hiding a data-loss bug.
Care needed on the flip
Turning this on will go red immediately for every already-passing entry, which
is the point but also the reason it cannot be flipped blind — CLAUDE.md's
corollary applies: a new gate has never been green, so promoting it to required
immediately blocks every open PR. Run it once, empty the entries it names in
the same PR, then promote. Note also that a test can pass on one platform and
fail on another, so the "now passes" verdict has to be per the platform the gate
runs on, not asserted globally.
Activity
test(parity): retire the 28 stale known_failures entries the ratchet …
docs(changelog): fragment for the known_failures ratchet (#7582)
docs(changelog): record the audit result and the macOS #6847 reproduc…
test(parity): make known_failures.json a ratchet, not a suppression l…
test(parity): retire the 28 stale known_failures entries the ratchet …
docs(changelog): fragment for the known_failures ratchet (#7582)
docs(changelog): record the audit result and the macOS #6847 reproduc…
test(parity): make known_failures.json a ratchet, not a suppression l…
test(parity): drop the stale test_stress_promises known-failure entry
test(parity): drop the stale test_stress_promises known-failure entry
test(parity): drop the stale test_stress_promises known-failure entry (…