You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
The model can copy Modelfile Example 1 word for word, and the check passes it #455
The explanation for a finding can come back as ai/Modelfile's Example 1, unrelated to the finding, and _is_unacceptable() passes it, so it is shown labelled as AI.
Measured
finding: a dead DNS permit (AC-002), and a dead SSH deny
text: "The internal server can be reached over unencrypted web traffic that should have
been blocked. The device's rule set currently allows all traffic through with no
restriction, which is what let this through."
source: model
Across AC-002 and a redundant DENY, 30 live runs each: 2/60 on #401 and 0/60 on main. Shubham's run on #401 had 0/60 and 2/60, so about 2 in 120 on either. The 9/60 rate under #401's earlier wording is gone, but this remains.
Suggested fix
Reject any text containing a sentence from the Modelfile's examples, so it falls back to the fixed text. It needs no model. #410 is a broader, statistical check on drift, and this is a much smaller exact one.
Activity