Repository navigation
Compare the traffic before warning about a filter's spelling (#440 N1, N2) - #464
Conversation
…, N2) N1: a pair whose filter names differ only in spelling was reported as 'could not be compared' before its traffic was compared, so two rules about different hosts warned although they cannot collide whichever ACL Batfish resolves the names to. The warning now needs traffic that overlaps or cannot be decided. between() shares the loop, so #426's check is fixed too. N2: pin each protocol number to its own protocol, and acl_in,acl_out as a specifier. Changing '17' to tcp, '1' to udp, or dropping ',' from the specifier pattern used to pass every test; all three now fail one.
SamikaPerera
left a comment
There was a problem hiding this comment.
Approving 6033b38, with one question that does not block.
The new check at analysis/policy_conflicts.py:349-352 warns only if some pair of queries is not provably disjoint. If either rule has no queries, pairs is empty, any([]) is False, and the "could not be compared" warning that main gives is gone.
A rule with "queries": [] (or null) loads on main today. I measured it with python -m analysis.policy_conflicts a.json b.json, where a.json is one prohibition on filter ACL_IN with no queries and b.json is one requirement on filter acl_in (UDP 53 to 10.0.0.53), both on rtr-us5. On main (2833a1e) the note "could not be compared" is printed. On 6033b38 it is not. On #468 (55c8f85) the loader refuses the rule: "'queries' must be a non-empty list of traffic descriptions, got []".
Question: is #468 meant to cover this? If #468 lands, the zero-query case cannot reach this branch. If #464 lands first, could the warning be kept when either side has no queries, with a test that uses one empty-queries rule? The scan itself still reports such a rule as "could not check" (analysis/checks/policy_compliance.py:699, once this is merged with main), so only the load-time note is affected.
Tests at 6033b38 (clean checkout, Batfish up, 11 Nov): test_policy_contradictions.py 53 passed; full suite 2266 passed, 13 skipped.
Not checked: #464 merged together with #468 (each was tested on its own); a queries value that is not a list.
With either rule's queries empty there are no pairs, any([]) is False, and the 'could not be compared' warning main gives went away. Nothing proves such a pair disjoint, so it warns again. #468 refuses an empty list at load, but policy_conflicts can still be handed one, and this does not depend on the order the two PRs land in.
|
@SamikaPerera, thank you, and yes, that was a real regression on this branch. Fixed in |
patelankeet2
left a comment
There was a problem hiding this comment.
Re-reviewed with real testing (Samika's approval went stale after a follow-up commit addressing her own review).
Live-reproduced the empty-queries edge case directly through contradictions(): a prohibition and a requirement with no queries on either side still warns about the spelling difference (the any([]) gap the follow-up fixes).
Mutation-tested the fix: removed the not pairs or guard -- both parametrized regression tests failed exactly as expected (the warning silently disappeared), reverted after confirming.
Full suite with Batfish running: 2494 passed, 13 skipped, 0 failed.
Approving.
Part of #440 (N1, N2). N5 is in #463, because it touches
change_impact.py, which #406 rewrites. Thanks @ARSH871-bot for the list.N1. A pair whose filters differ only in spelling (
ACL_INvsacl_in,/acl_in/,acl_in,acl_out) was reported as "could not be compared" before its traffic was compared. So two rules about different hosts warned, although they can't collide whichever ACL Batfish resolves the names to. Now the traffic is compared first, and the spelling warning only fires if some pair of queries overlaps or can't be decided.between()(#426) uses the same loop, so it gets the fix too.N2. The tests you asked for:
6/17/1each match their own protocol and not another;acl_in,acl_outadded to the specifier cases.Mutation, 5 of 5 killed: the three that used to survive (
"17": "tcp","1": "udp",,dropped from_FILTER_SPECIFIER), plus warning before comparing the traffic, and never warning at all.docs/policy-rules.mdgets one sentence on the new order. 2266 passed, 13 skipped on this branch, with Batfish running; ruff clean on CI's selection.