Repository navigation
Refuse an all-clear explanation of a found or error finding - #456
Conversation
The module docstring says what the gap is and why it matters (F-4): the check that stops a claim about the network runs only for status="error", and it is a short word list, so "Everything looks clean." passes for a check that could not run, and "No issues found." passes for a confirmed problem. Two helpers build a real finding of a given status and report whether a stubbed model answer would reach the user as model output. No tests yet, so pytest reports that none ran.
…tatus One accurate sentence per status (found, none, error), each repeating only what its finding says. All three pass today. This stops a future fix from making the gap tests pass by rejecting everything.
A status="error" finding means the check could not run. Four sentences that tell the user the opposite are accepted by the output check today, because none of their words is on the error-only result-claim list: "There are no problems with this rule.", "This is fine and nothing needs fixing.", "Everything looks clean." and "This check passed." That is F-4's failure. The test asserts each is rejected, so it fails on main for that reason, and is left failing on purpose rather than marked xfail.
A status="found" finding is a confirmed problem, and it is the status the dashboard sends to the model. The output check runs no result-claim test for it, so all six clean-sounding sentences tried are accepted today, including "No issues found." and "...the rule is safe.", which are refused for an error finding. The test asserts each is rejected, so it fails on main for that reason, and is left failing on purpose rather than marked xfail.
The output check only rejected claims about the network for status "error", and only by single words. A model answer such as "No issues found." for a confirmed problem, or "Everything looks clean." for a check that could not run, reached the user labelled as model output. Add a short list of all-clear phrases and check it for "found" and "error" findings. A match is treated like any other rejected answer: one retry, then the existing deterministic fallback. "none" findings are not checked, because there an all-clear is true. The existing error-only word list is unchanged; reusing it for "found" would have rejected 3 of the 4 rated explanations in the holdout set, which say "reach".
patelankeet2
left a comment
There was a problem hiding this comment.
Re-verified 8c22f2e in an isolated worktree, with Batfish running.
tests/test_explain_output_check_gap.py: 13 passed, matching the PR body. Mutation-tested by reverting ai/explain.py to the parent commit: 10 failed, 3 passed, matching the PR body exactly (both red-on-purpose counts and the control test still passing). Restored and re-confirmed 13 passed. Full suite on the branch: 2255 passed, 13 skipped, 0 failed, matching your number exactly.
Checked every example in the PR body's own false-positive section live: all four sentences you say would be wrongly rejected do get rejected, and all three you say should not match, do not match. Nothing in that section overstates what the pattern does.
One real gap, since you asked specifically about the phrase list's shape. This is the exact wording from the research question Samika and I looked at together: 'There are no security problems with this configuration.' does NOT match _ALL_CLEAR_PATTERN, because the inserted word 'security' breaks the literal phrase 'no problems'. I checked this live:
'There are no problems with this rule.' MATCH
'There are no security problems with this ...' NO MATCH
'The scan found no real issues of concern.' NO MATCH
So the specific case that started this whole line of research is still not caught by the fix meant to close it. This isn't a surprise regression, your own 'Not done' section already says other all-clear wording will get through, and I'm not asking you to solve open-ended paraphrase detection in this PR. But this particular phrase is not just 'other wording', it's the one that was actually measured getting through in the first place, so I think it's worth naming explicitly rather than folding into the general disclaimer. I'll file a follow-up issue for a version of the pattern that tolerates one inserted word (something like no (\w+\s+)?problems) rather than block this PR on it.
Approving. The fix does exactly what it claims, is proven by a real mutation test, and the false-positive measurement holds up under my own independent check.
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Approving 8c22f2e, with two small tests I'd like added before merge. Red tests first, then the fix, plus false positives measured on 23 real findings: this is how a safety-net change should arrive.
Checked:
- Merged onto current
main(2833a1e): 2324 passed, 13 skipped, ruff clean on CI's selection. - Removing the
foundcheck fails 6. The gap is pinned. - One more false-positive sample, a kind your seven fixtures don't stress: 100 real
netwise-wardenexplanations of dead-rule findings, from this week's #401 and #461 measurements. Dead rules are where I'd expect "this line is safe to remove". 0 of 100 match any of the new phrases, and none say "safe to". That supports your measurement from a different angle.
Two mutants survive, both cheap to kill:
- Checking
nonefindings too (status in ("found", "none")) passes every test. The description makes a point of not checkingnone, because there an all-clear is the truth, but nothing pins it. One test would: anonefinding explained as "No issues were found by policy compliance." must be accepted. - Dropping
is safefrom the pattern passes every test. Each of the seven phrases deserves one rejectedfoundsentence of its own, e.g. as a parametrised test, so the list can't quietly shrink.
Neither blocks the idea, which is right. They're just the two places where the code currently has no tests protecting it.
…each-phrase Add the two tests asked for on #456 (none findings accepted, each phrase refused)
This reports a gap in the output check in
ai/explain.pyand closes it. It does not claim the AI is unsafe: no real model has been shown producing an all-clear for a found finding.The gap
_is_unacceptable()only rejects a claim about the network forstatus="error"(ai/explain.py:860onmain), and only by single words (_RESULT_CLAIM_WORDS). Nothing checks astatus="found"explanation for an all-clear. The dashboard only sendsfoundfindings to the model (web/main.py:1036). So if a model answered "No issues found." about a confirmed problem, the user would see it labelled as model output.The same list also misses ordinary all-clear wording for
errorfindings, such as "Everything looks clean." or "This check passed." for a check that could not run.What changed
ai/explain.py, one commit, 15 lines added:_ALL_CLEAR_PATTERN, a short list of whole phrases:no problems,no issues,nothing needs fixing,looks clean,check passed,is fine,is safe._is_unacceptable()checks that list forfoundanderrorfindings. A match is handled like any other rejected answer: one retry, then the deterministic fallback that already exists (_fallback_plain_restatement()forfound,_fallback_error_explanation()forerror).nonefindings are not checked, because there an all-clear is true. The existing error-only word list is unchanged. Reusing it forfoundwould reject honest answers, because they need words like "reach" and "blocked".Nothing else changed: no prompt change, no input sanitising, nothing in
web/.Commits
The first four commits add
tests/test_explain_output_check_gap.pyand are red on purpose. They show the gap onmainbefore anything is fixed. The fifth is the fix. Please keep them in that order.53eaacethe test file and helperse7f2ec3the control: one faithful explanation per status (found,none,error) must still be accepted, so a fix that rejects everything cannot passdbe3edc4 all-clear sentences for anerrorfinding (red)2890ab76 all-clear sentences for afoundfinding (red)8c22f2ethe fixFalse positives, measured
The risk is rejecting an honest explanation of a real problem.
Rated holdout set (
tests/fixtures/explanation_quality/holdout.json, llama3.2:3b, rated by Ankeet): 4foundexplanations, 0 rejected by the new phrases. The old error-only word list would have rejected 3 of the 4, for "reach".Fresh model output:
netwise-warden(local Ollama) explained everyfoundfinding in seven fixtures (rtr-us5-insecure, rtr-us5-messy, vendor-asa, vendor-arista, vendor-juniper, routing-faults, sample-network). That was 23 findings, one explanation each, 0 matched the new phrases. The old word list would have matched 15. 3 of the 23 are already rejected onmainfor other reasons (speculation or an invented address), which this PR does not change.Test strings: 37 string literals in
tests/contain one of the phrases. They are docstrings,nonesummaries ("No issues found by access control") and samples meant to be rejected. The full suite passes.Sentences I wrote to probe the phrases. None appeared in any real output above. These four are honest and would be rejected:
If a model wrote one, the user would get the deterministic restatement instead of the model's text. These do not match: "There are no other issues on this filter.", "The rule is not safe.", "This is safely ignored.".
Verification
tests/test_explain_output_check_gap.py: 13 passed. Withai/explain.pyreverted to the parent commit: 10 failed, 3 passed.main(d424e05), on a plain copy: 2250 passed, 13 skipped.tests/test_no_configs_in_git.py(24 tests) was left out because the copy was not a git checkout.ruff check --select E4,E7,E9,Fon the changed files: all checks passed.Not done
foundfinding. This closes the gap; it does not fix an observed failure.ai/is Ankeet's (@patelankeet2), so he should review this, including whether the phrase list is the right shape.