Repository navigation
Catch a one-word-inserted all-clear phrase (#457, stacked on #456) - #459
patelankeet2 wants to merge 1 commit into
Conversation
_ALL_CLEAR_PATTERN (#456) refuses literal "no problems"/"no issues" for a found or error finding, but an inserted adjective still escapes it. Measured live on #456's own commit: 'There are no problems with this rule.' MATCH 'There are no security problems with this configuration.' NO MATCH 'The scan found no real issues of concern.' NO MATCH 'no security problems' is the exact sentence from the prompt-injection research that motivated this whole check in the first place. Widens exactly those two alternatives to allow one inserted word: no (\w+\s+)?problems, no (\w+\s+)?issues. Scope is deliberately narrow, per the issue: one adjective, not open-ended paraphrase detection, which is #410's separate problem. Tests: the two measured real sentences added to both existing gap-test lists, a new parametrized test pinning both statuses, and a regression test that checks the widened pattern against every captured_explanation in tests/fixtures/explanation_quality/holdout.json so a future widening that starts rejecting real answers is caught immediately. Mutation-tested: reverting to the literal pattern breaks 3 of the 4 new phrase tests for the right reason; the 4th (error status, 'no real issues') is independently caught by the pre-existing result-claim word list, which already lists 'issues' for error-only -- not a gap. Full suite: 2231 passed, 46 skipped, 0 failed. Live-tested against the real model: 15 generations each for a found and an error finding. All 15 found generations accepted as genuine model output; across all 30 runs, 0 final outputs match _ALL_CLEAR_PATTERN, so the widening does not start rejecting real explanations.
SamikaPerera
left a comment
There was a problem hiding this comment.
Approving d31d8a4, with three small asks. None blocks this. I checked it on a scratch merge with main (2833a1e), with Batfish and Ollama up.
What holds. The pattern change is as described (ai/explain.py:663-667). tests/test_explain_output_check_gap.py: 22 passed. With the pattern put back to #456's, 6 tests fail. Full suite merged with main: 2333 passed, 13 skipped. ruff is clean. No backtracking on 100,000 characters (17 ms and 8 ms). On real output: holdout.json 4 of 4 accepted, and fresh netwise-warden explanations for the 7 fixtures, one generation per found finding, 29 of 29 accepted by both patterns.
-
The pattern allows any one word, not an adjective.
\w+also matches "no other issues" and "no known problems". Two honest sentences that #456 accepted are now rejected: "There are no other issues on this filter." and "There are no known problems with this rule, but it is never used." A rejected answer is retried once and then replaced by the existing fallback, so this fails safe, and I measured 0 of 29 real explanations hit by it. But the body says "one adjective" and that the widening does not start rejecting real explanations. Please say what it does, and list these two next to #456's four known false rejections. -
Nothing records how many inserted words are allowed. With the pattern widened to allow two words, no test fails. If one word is the intended limit, a test should say so. It would document a known limit, not wanted behaviour: "no real security problems" is still an all-clear sentence that gets through.
-
Stacking. The body says GitHub retargets this PR when #456 merges. The repo deletes merged branches, and Arsh wrote on #445: "the repo deletes merged branches… merging #443 closes this PR". Please retarget this PR to main before #456 merges. Merged against main it is clean, with no conflicts. Please tell me when it is done.
Not checked: what GitHub does to this PR when #456's branch is deleted, error-status generations, and whether other one-word all-clears appear in real output.
shubhamkataria2005
left a comment
There was a problem hiding this comment.
Approving d31d8a4. It catches the sentence that matters, and I agree with the trade-off. But the trade-off should be written down, because it reverses one of #456's own promises.
Checked: #456 + #459 merged onto current main (2833a1e): 2333 passed, 13 skipped, ruff clean on CI's selection. "There are no security problems with this configuration." is now rejected.
The cost, measured. One inserted word also catches scope words:
#456 #459
✓ There are no security problems with this configuration. <- the target
✓ There are no other issues on this filter. <- #456 lists this as must-NOT-match
✓ This line has no other problems; the dead permit is the only one.
✓ No further issues were checked, because this check could not run.
✓ It causes no routing problems, but it does let telnet through.
All four of the extra matches are honest sentences about a real problem. In practice it rarely bites: 0 of 100 real dead-rule explanations from this week match the widened pattern, and your own 30 runs agree. And a false match costs the deterministic fallback, which is the safe direction. Missing an all-clear about a confirmed problem is worse. So: approve.
Two small asks:
- #456's description promises "There are no other issues on this filter." does not match. After this PR it does. Please note the change in this PR's description, so the record doesn't contradict itself.
- Consider excluding the scope words (
other,further,known) from the inserted slot, e.g.no (?!other\b|further\b|known\b)(\w+\s+)?issues. They all signal "something else is a problem", which is the opposite of an all-clear. Measure it against the holdout set before adopting it, as you did here. Fine as a follow-up.
ARSH871-bot
left a comment
There was a problem hiding this comment.
Reviewed d31d8a4. The widening is right and narrow, but the branch conflicts with main in tests/test_explain_output_check_gap.py (#473 added tests there), so please merge main and re-request.
- On its own head: CI-style 2223 passed, 54 skipped, ruff clean.
no (\w+\s+)?problems/no (\w+\s+)?issuesis the narrow change #457 asked for, and checking it against every holdout capture is the right false-positive guard. A false positive here sends the card to the deterministic fallback, which is the safe direction, so one adjective is a reasonable bound.- One case worth pinning while you merge: a correct explanation of a found finding that says "there are no other problems on this device" would now be rejected. That is safe, but if it shows up in the holdout re-run it is a quality cost worth knowing about.
Closes #457.
Stacked on #456 rather than main: this is a one-line widening of the exact pattern #456 adds (_ALL_CLEAR_PATTERN does not exist on main yet), so basing on #456's branch is the literal edit to it rather than a second, duplicate copy that would conflict with it at merge time. Once #456 merges, GitHub retargets this PR onto main automatically.
What's wrong: #456's literal phrases (no problems, no issues, ...) break the moment the model inserts one adjective. Measured live on #456's own commit 8c22f2e:
'no security problems' is the exact sentence from the prompt-injection research that motivated this whole check in the first place (the model said this about a confirmed finding). It still gets through #456's fix.
What changed: widened exactly the two alternatives named in the issue to allow one inserted word,
no (\w+\s+)?problemsandno (\w+\s+)?issues. Scope is deliberately narrow per the issue body: one adjective, not open-ended paraphrase detection, which is #410's separate, unstarted, bigger problem and stays untouched by this.Testing:
pytest tests/test_explain_output_check_gap.py -v- 22 passed.pytest tests/ -q- 2231 passed, 46 skipped, 0 failed.