Skip to content

R-1 lowers any finding whose evidence contains the word "unreachable", not only dead ACL lines #444

Open

Description

@SamikaPerera

What is wrong

R-1 in analysis/checks/risk.py is meant for dead ACL lines. In the code, it matches any finding whose evidence contains the word "unreachable" anywhere. So findings that have nothing to do with a dead ACL line are rated low.

One example. A routing assertion that says the guest LAN must NOT reach the finance LAN, and it does:

check     routing
evidence  Expected UNREACHABLE for a flow from 10.10.10.5 to 10.20.20.5, but
          traceroute unexpectedly succeeded: ...
severity  high (set by the check)  ->  low (after risk)

The rules say a routing finding is medium (R-3). It gets low because the word "UNREACHABLE" is in the evidence, and R-1 runs first. Low sorts to the bottom of the list, below every medium.

Measured

I had Claude Code measure this. It built one finding per case with findings.make_finding() and the evidence wording each check writes, then ran the real risk.severity_for() on both main (db59b00) and #443's head (2563f1e). The results were the same on both, except the dead deny line in the last row.

finding type where the wording comes from check sets today per the rules
routing, expected UNREACHABLE, path succeeded routing.py:484-486, :291 high low medium (R-3)
routing, expected REACHABLE, path ended NEIGHBOR_UNREACHABLE routing.py:484-486, :281 high low medium (R-3)
routing, expected REACHABLE, path ended NO_ROUTE (control) same high medium medium (R-3)
access_control testFilters, decided by permit icmp any any unreachable access_control.py:458-460 high low high (R-4)
access_control searchFilters, allowed by an ICMP host-unreachable line access_control.py:543-545 high low high (R-4)
policy_compliance, decided by an ICMP port-unreachable line policy_compliance.py:389-391 high low high (R-4)
access_control undefined reference to a name containing "UNREACHABLE" access_control.py:667-670 high low high (R-4)
access_control unused filter whose name contains "UNREACHABLE" access_control.py:740-745 medium low medium (R-4)
dead permit line (what R-1 is for) access_control.py:606-610 medium low low (R-1)
dead deny line (what R-1 is for) access_control.py:606-610 high low on main, high on #443 high (R-1 as #443 writes it)

The first row was also run end to end against real Batfish. That run used tests/fixtures/routing-secure plus a user policy with one UNREACHABLE route assertion, through pipeline.analyse(). It came out as RT-003 found low.

unreachable, host-unreachable and port-unreachable are ordinary Cisco IOS ICMP keywords, so the ICMP rows come from text a real config can contain. The name rows come from names the config author chooses.

Doc versus code

docs/severity-rules.md:51-52 on main:

Matches: status="found" and the evidence describes an unreachable / shadowed ACL line.

analysis/checks/risk.py:67 on main:

if "unreachable" in _evidence(finding):
    return "low"

docs/severity-rules.md:9-10 says: "if the two disagree, one of them is a bug." The doc describes a dead ACL line. The code matches a word.

The only evidence that describes a dead ACL line is the one access_control writes at access_control.py:608-610, which starts Unreachable line: <line> (action PERMIT|DENY). Blocked by: ....

Relation to #443

Found while building #443. #443 does not change this. It adds an exception for a dead deny line, but it keeps the same substring test (risk.py:92 on #443's head), so every row above marked bold is still low with #443 merged.

Not checked

  • The NEIGHBOR_UNREACHABLE row was built with the finding constructor. No fixture was run that ends a trace that way.
  • The ICMP and name rows were built with the finding constructor, not run through Batfish. I did not check whether Batfish's Line_Content keeps the ICMP keyword word for word.
  • Whether any fixture or the client's export has a line or a name like these. Not searched.
  • The routing hygiene findings numbered from RT-100 (duplicate IPs, loops, BGP, OSPF). Their fixed wording does not contain the word, but they quote Batfish status values. I did not list every value Batfish can return.
  • change_impact findings. analysis/change_impact.py does not call refine(); I did not check whether anything else applies R-1 to them.
  • How business-context escalation interacts with a finding R-1 has already lowered.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinglayer: analysisLayer 1 — Batfish orchestration, checks, the pipeline

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions