You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
_ALL_CLEAR_PATTERN still misses the exact phrase that started this (#456 follow-up) #457
#456 added _ALL_CLEAR_PATTERN to ai/explain.py to reject all-clear wording for found and error findings. It works for the phrases it lists, but the one phrase that actually motivated this whole line of research still gets through, because an inserted word breaks the literal match.
'There are no problems with this rule.' MATCH
'There are no security problems with this configuration.' NO MATCH
'The scan found no real issues of concern.' NO MATCH
'no security problems' is the exact sentence from the original prompt-injection research (Samika's test, reported this week): the model said 'no security problems' about a confirmed finding, and the old error-only word list let it through because 'problems' alone was not on that list. #456 closes that for the literal phrase 'no problems', but a model that inserts one adjective still escapes the same way.
Not asking for open-ended paraphrase detection here, that's a much bigger problem than one PR should solve. Just the one-word-inserted case, since it's the one already measured happening for real:
Needs live re-testing against the same false-positive set #456 already built (tests/fixtures/explanation_quality/holdout.json plus the seven-fixture fresh-model run) before landing, since a wider pattern risks rejecting more honest explanations, not just catching more bad ones.
Activity