You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
US-32 — Tell me which of my policy rules never fire #324
Story and acceptance criteria come from Discussion #296 unchanged, so anyone who reviewed the discussion has already reviewed this issue.
Part of Phase 3 - market readiness. The distribution of this milestone is deliberately uneven and is explained in docs/design/phase-3-backlog.md and on #249.
Started on this, and stopped deliberately before writing code, because the acceptance criteria leave the one thing that matters undefined: what "no flow can match" means for a policy rule. I measured it against live Batfish before deciding, and the obvious readings do not work.
rule's header space permit deny
real LAN -> real server 1 1
addresses NOT in the snapshot at all 0 1
LAN, all ports 1 1
icmp with a dstPort (unsatisfiable) error error
Two consequences:
A prohibition about addresses nowhere in the snapshot reports permit=0 — exactly what a prohibition that genuinely passes reports. Today both render as a clean tick. That is the gap this story exists for, and it is real.
But "no flow matches" cannot be the test, because every satisfiable space returns deny ≥ 1 — the ACL's implicit deny decides everything. Under that reading no rule is ever dead.
Proposed definition — please agree or correct before I build
The honest analogue of filterLineReachability is:
A policy rule never fires when every flow in its header space is decided by the filter's implicit default, and none by a line anyone wrote.
filterLineReachability finds config lines no packet reaches. This finds policy rules that reach no config line — the rule is asserting about traffic the ACL's author never wrote anything for. searchFilters already returns the deciding line per flow, so it is measurable without new machinery.
It would be reported as hygiene (its own status wording, never found), and kept distinct from "matched and passed" — the two cases measured as identical above.
Raising it here rather than picking one silently, because on #310 I learned the hard way what it costs to settle an open semantic question by just deciding it. If nobody objects I will build to this definition.
Correcting the definition I proposed above, before anyone builds to it. I said I would build to it if nobody objected. Nobody did. I went to build it today, tested it against live Batfish first, and it is wrong in two ways.
1. It does not catch its own motivating case
The definition was: a rule never fires when every flow in its space is decided by the filter's implicit default, and none by a line anyone wrote.
rtr-us5-secure's ACL ends in an explicitdeny ip any any. For a rule about addresses nowhere in the snapshot:
addresses nowhere in the snapshot deny -> decided by: deny ip any any
real LAN -> real server, tcp/443 permit -> decided by: permit tcp 10.10.10.0 0.0.0.255 host 10.20.0.5 eq 443
The catch-all is a line someone wrote, so under my definition that rule fires — and the check would be silent on the exact case it exists for, on our own fixture. Most real ACLs end in an explicit catch-all.
Corrected: a rule never fires when the only lines that touch its traffic are catch-alls (match all traffic) or the implicit default. Batfish gives us what this needs — namedStructures(structureTypes="IP_Access_List") returns every line's header space, and the catch-all's is 0.0.0.0/0 both ways with no protocol or port. Intersection can reuse analysis/policy_conflicts.overlap() from #387, including its "undecidable" answer for anything outside the parsed subset.
It also narrows what "never fires" can mean, and I think correctly. A violation decided by a catch-all is real — a requirement whose traffic only deny ip any any touches really is blocked. So only a pass decided by a catch-all is vacuous. That gives the story's "distinguished from a rule that matched and passed" a precise meaning: passed because of a line written for it, versus passed only because everything is denied.
2. Reporting it as found would wipe out true all-clears
access_control reports a dead ACL line as status="found", so symmetry says the same here. But #364 made the all-clear per device — coverage.devices_still_clean() removes any device with a found finding. One vacuous rule on rtr-us5 would then take away the all-clear for every other rule on that device that genuinely holds. A hygiene note would make the user's report less accurate.
The options:
A — found, severity low, matching access_control and accepting the lost all-clear. Symmetric, and wrong about the other rules.
B — none, on its own card, with summary "Policy rule N never takes effect". Keeps the all-clear honest, but breaks the symmetry the story asks for.
C — found, and teach devices_still_clean() to ignore hygiene findings. Keeps both, but it means a new idea in the coverage logic, and that file is Arsh's.
I lean towards B, and towards saying in the code that it is deliberately not symmetric on status, only on wording and meaning. But the choice is between consistency with access_control and the correctness of #364, and that is not mine alone.
Not building until one of you picks. On #310 I learned what settling an open question by just deciding it costs. This time the question is partly about my own earlier answer, which makes it more important to say so.
Activity
shubhamkataria2005 commentedon Oct 12, 2026
Started on this, and stopped deliberately before writing code, because the acceptance criteria leave the one thing that matters undefined: what "no flow can match" means for a policy rule. I measured it against live Batfish before deciding, and the obvious readings do not work.
Measurement —
rtr-us5-secure,acl_in,searchFiltersTwo consequences:
permit=0— exactly what a prohibition that genuinely passes reports. Today both render as a clean tick. That is the gap this story exists for, and it is real.deny ≥ 1— the ACL's implicit deny decides everything. Under that reading no rule is ever dead.Proposed definition — please agree or correct before I build
The honest analogue of
filterLineReachabilityis:filterLineReachabilityfinds config lines no packet reaches. This finds policy rules that reach no config line — the rule is asserting about traffic the ACL's author never wrote anything for.searchFiltersalready returns the deciding line per flow, so it is measurable without new machinery.It would be reported as hygiene (its own status wording, never
found), and kept distinct from "matched and passed" — the two cases measured as identical above.Raising it here rather than picking one silently, because on #310 I learned the hard way what it costs to settle an open semantic question by just deciding it. If nobody objects I will build to this definition.
#322 is up as #387.
shubhamkataria2005 commentedon Oct 13, 2026
Correcting the definition I proposed above, before anyone builds to it. I said I would build to it if nobody objected. Nobody did. I went to build it today, tested it against live Batfish first, and it is wrong in two ways.
1. It does not catch its own motivating case
The definition was: a rule never fires when every flow in its space is decided by the filter's implicit default, and none by a line anyone wrote.
rtr-us5-secure's ACL ends in an explicitdeny ip any any. For a rule about addresses nowhere in the snapshot:The catch-all is a line someone wrote, so under my definition that rule fires — and the check would be silent on the exact case it exists for, on our own fixture. Most real ACLs end in an explicit catch-all.
Corrected: a rule never fires when the only lines that touch its traffic are catch-alls (match all traffic) or the implicit default. Batfish gives us what this needs —
namedStructures(structureTypes="IP_Access_List")returns every line's header space, and the catch-all's is0.0.0.0/0both ways with no protocol or port. Intersection can reuseanalysis/policy_conflicts.overlap()from #387, including its "undecidable" answer for anything outside the parsed subset.It also narrows what "never fires" can mean, and I think correctly. A violation decided by a catch-all is real — a requirement whose traffic only
deny ip any anytouches really is blocked. So only a pass decided by a catch-all is vacuous. That gives the story's "distinguished from a rule that matched and passed" a precise meaning: passed because of a line written for it, versus passed only because everything is denied.2. Reporting it as
foundwould wipe out true all-clearsaccess_controlreports a dead ACL line asstatus="found", so symmetry says the same here. But #364 made the all-clear per device —coverage.devices_still_clean()removes any device with afoundfinding. One vacuous rule onrtr-us5would then take away the all-clear for every other rule on that device that genuinely holds. A hygiene note would make the user's report less accurate.The options:
found, severitylow, matchingaccess_controland accepting the lost all-clear. Symmetric, and wrong about the other rules.none, on its own card, with summary "Policy rule N never takes effect". Keeps the all-clear honest, but breaks the symmetry the story asks for.found, and teachdevices_still_clean()to ignore hygiene findings. Keeps both, but it means a new idea in the coverage logic, and that file is Arsh's.I lean towards B, and towards saying in the code that it is deliberately not symmetric on status, only on wording and meaning. But the choice is between consistency with
access_controland the correctness of #364, and that is not mine alone.Not building until one of you picks. On #310 I learned what settling an open question by just deciding it costs. This time the question is partly about my own earlier answer, which makes it more important to say so.