Skip to content

Propose-a-change raises no warning when a change breaks a policy rule rated below high #451

Open

Description

@ARSH871-bot

What's wrong

warning is true only when a finding is rated high (ai/propose.py:492). A change that breaks a rule rated medium or low raises no warning, and the summary says the change narrows access.

Measured (rtr-us5-secure, live Batfish)

block 10.10.10.0/24 to 10.20.0.5 on tcp/443 on rtr-us5

warning: False
answer:  "... Simulating this change shows it narrows access; it does not appear to
          newly open anything -- see the impact list."
impact:  CH-002 medium  This change BREAKS policy rule 5 (HTTPS to the internal server must work)

The same on main db59b00 and on #405's head f4668d0. Rule 5 is violation_severity: medium (analysis/checks/policy_compliance.py:269). The rule's own severity carries through (analysis/change_impact.py:441), so a rule rated high would warn. An allow that opens traffic warns correctly on both.

Why it matters

Nothing here is false: the impact list names the broken rule. But the client asked for the system to push back, and someone who reads only the summary is told nothing is wrong after blocking the one service their policy says must work.

Decision needed

Should any broken policy rule raise the warning, whatever its rating? I think so, with wording that says a rule was broken rather than "newly opens access". This overlaps #402's second item (key warning on direction, not severity), and #405 currently keeps the trigger on severity, so it is probably best done on top of #405. Samika for how it reads on screen.

Activity

  1. added
    bugSomething isn't working
    layer: aiLayer 2 — local LLM explanation and Q&A
    safetyTouches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care
    on Nov 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinglayer: aiLayer 2 — local LLM explanation and Q&AsafetyTouches F-4, grounding, or the refuse-rather-than-guess boundary. Review with care

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions