Repository navigation
Description
What is wrong
R-1 in analysis/checks/risk.py is meant for dead ACL lines. In the code, it matches any finding whose evidence contains the word "unreachable" anywhere. So findings that have nothing to do with a dead ACL line are rated low.
One example. A routing assertion that says the guest LAN must NOT reach the finance LAN, and it does:
check routing
evidence Expected UNREACHABLE for a flow from 10.10.10.5 to 10.20.20.5, but
traceroute unexpectedly succeeded: ...
severity high (set by the check) -> low (after risk)
The rules say a routing finding is medium (R-3). It gets low because the word "UNREACHABLE" is in the evidence, and R-1 runs first. Low sorts to the bottom of the list, below every medium.
Measured
I had Claude Code measure this. It built one finding per case with findings.make_finding() and the evidence wording each check writes, then ran the real risk.severity_for() on both main (db59b00) and #443's head (2563f1e). The results were the same on both, except the dead deny line in the last row.
| finding type | where the wording comes from | check sets | today | per the rules |
|---|---|---|---|---|
| routing, expected UNREACHABLE, path succeeded | routing.py:484-486, :291 |
high | low | medium (R-3) |
routing, expected REACHABLE, path ended NEIGHBOR_UNREACHABLE |
routing.py:484-486, :281 |
high | low | medium (R-3) |
routing, expected REACHABLE, path ended NO_ROUTE (control) |
same | high | medium | medium (R-3) |
access_control testFilters, decided by permit icmp any any unreachable |
access_control.py:458-460 |
high | low | high (R-4) |
access_control searchFilters, allowed by an ICMP host-unreachable line |
access_control.py:543-545 |
high | low | high (R-4) |
policy_compliance, decided by an ICMP port-unreachable line |
policy_compliance.py:389-391 |
high | low | high (R-4) |
| access_control undefined reference to a name containing "UNREACHABLE" | access_control.py:667-670 |
high | low | high (R-4) |
| access_control unused filter whose name contains "UNREACHABLE" | access_control.py:740-745 |
medium | low | medium (R-4) |
| dead permit line (what R-1 is for) | access_control.py:606-610 |
medium | low | low (R-1) |
| dead deny line (what R-1 is for) | access_control.py:606-610 |
high | low on main, high on #443 |
high (R-1 as #443 writes it) |
The first row was also run end to end against real Batfish. That run used tests/fixtures/routing-secure plus a user policy with one UNREACHABLE route assertion, through pipeline.analyse(). It came out as RT-003 found low.
unreachable, host-unreachable and port-unreachable are ordinary Cisco IOS ICMP keywords, so the ICMP rows come from text a real config can contain. The name rows come from names the config author chooses.
Doc versus code
docs/severity-rules.md:51-52 on main:
Matches:
status="found"and the evidence describes an unreachable / shadowed ACL line.
analysis/checks/risk.py:67 on main:
if "unreachable" in _evidence(finding):
return "low"docs/severity-rules.md:9-10 says: "if the two disagree, one of them is a bug." The doc describes a dead ACL line. The code matches a word.
The only evidence that describes a dead ACL line is the one access_control writes at access_control.py:608-610, which starts Unreachable line: <line> (action PERMIT|DENY). Blocked by: ....
Relation to #443
Found while building #443. #443 does not change this. It adds an exception for a dead deny line, but it keeps the same substring test (risk.py:92 on #443's head), so every row above marked bold is still low with #443 merged.
Not checked
- The
NEIGHBOR_UNREACHABLErow was built with the finding constructor. No fixture was run that ends a trace that way. - The ICMP and name rows were built with the finding constructor, not run through Batfish. I did not check whether Batfish's
Line_Contentkeeps the ICMP keyword word for word. - Whether any fixture or the client's export has a line or a name like these. Not searched.
- The routing hygiene findings numbered from RT-100 (duplicate IPs, loops, BGP, OSPF). Their fixed wording does not contain the word, but they quote Batfish status values. I did not list every value Batfish can return.
change_impactfindings.analysis/change_impact.pydoes not callrefine(); I did not check whether anything else applies R-1 to them.- How business-context escalation interacts with a finding R-1 has already lowered.
Activity
Pin the redundant-case agreement sentence, not just the absent phrases