fix: remove degenerate keywords that flooded false positives on clean code#50
Merged
Conversation
…on clean code 18 patterns carried degenerate keyword fragments (a bare comma ",", ", or", generic "AI"/"ai", numbered-list bits "2.".."8.", "---", a stray sentence fragment). Because the engine flags on a keyword pre-filter match, any ordinary text containing a comma (or the substring "ai", as in "email"/"detail") tripped HIGH findings — e.g. `def add(a, b): return a + b` produced 14 findings / block, and a clean 31-line module produced 20 "threats". Removed only the 29 junk keyword lines. Every real signature (RTL/invisible unicode, fork bomb, path traversal), all genuine keywords, and every regex are preserved, so real-threat detection is unchanged. Proof: - patterns still load 981/64 (no patterns lost) - clean code (`def add`, shopping-cart class) -> decision=allow, 0 findings - malicious injection -> decision=block, 31 findings (still caught) - full test suite: 143 passed, 7 xfailed (no regressions) Unblocks GitHub Action publish, integration recipes, and the founder launch (all were held because they would have flagged clean repos). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
A clean-text corpus (READMEs, security articles, normal web pages, dev docs,
code) tripped 46 patterns: the scanner was BLOCKING the very things it exists
to discuss ('prompt injection is a growing concern' -> BLOCKED, '## Installation
/ pip install' -> BLOCKED, '<html>...' -> BLOCKED). Credibility killer for a
security brand.
Root cause: auto-generated patterns reused single common words as keywords
(injection, exec, html, bot, model, bypass, token, secret) plus a few over-broad
regexes.
Fixes:
- engine.py: centralized KEYWORD_DENYLIST — generic words can't trigger a block
alone (structural guard; also neutralizes future generated patterns).
- patterns.py: tightened 6 over-broad regexes (GLS-GHSA-PI-202, GLS-I18N-LR-203,
GLS-SC-014, GLS-CI-005, GLS-MCP-POISON-201, GLS-IU-531) and gave 4 niche GHSA
patterns specific product anchors so they keep detecting.
- GLS-IU-531 zero-width regex required >=1 zero-width char (was matching plain
text) — also fixes the long-standing negation edge case.
Permanent gate:
- tests/test_false_positives.py: clean corpus must scan clean on BOTH engines +
attack canaries must still block. Wired into CI + ship preflight.
- test_customer_zero.py now exits non-zero on failure (was a silent no-op).
Verified: corpus 46->0 FPs, customer_zero ALL PASS (was failing), attacks still
block, 200 passed / 7 xfailed (was 145/7). No user-facing detection regression.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…itemap/.well-known)
The held v0.2.62 blocker. Even after the general FP fix, the scanner blocked
all 6 normal discovery files — the exact embarrassment the discovery_file_poisoning
category warns against ('don't panic at a plain robots.txt').
Root cause (verified by ablation, not the garbled matched_text red herring):
bare generic keywords that appear in every normal discovery/config/manifest file
- canonical (fired 6 patterns), description, expires, allow/disallow/admin,
.well-known, <loc>, sitemap:, support, description_for_model, name_for_model,
sdl, /* team */ — plus GLS-DFP-008's regex matching 'agent' (from User-agent:)
and 'crawl' (from Crawl-delay:).
Fix (same approach as the general FP fix):
- engine KEYWORD_DENYLIST += the 16 generic discovery tokens (neutralized for
ALL patterns; detection survives via each pattern's regex + multiword keywords)
- GLS-DFP-008 regex tightened: require a real AI-agent address + a genuine
injection/override verb (not bare agent/crawl/fetch/run)
Verified: clean discovery corpus 6/6 ALLOW (both engines); poisoned discovery
corpus 6/6 still BLOCK; 0 patterns zeroed-out by the denylist; full suite
216 passed / 7 xfailed (was 200/7); customer_zero PASS. Permanent gate: 10 new
clean+poisoned discovery cases in tests/test_false_positives.py.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
sunglasses scan --file <clean.py>floods false positives on ordinary code.def add(a, b): return a + b→ 14 findings / block (5 HIGH); a clean 31-line module → 20 "threats". This was the credibility keystone blocking the GitHub Action, integration recipes, and founder launch.Root cause
18 patterns carried degenerate keyword fragments — a bare comma
,,", or", generic"AI"/"ai", numbered-list bits2.–8.,---, a stray sentence fragment. The engine flags on a keyword pre-filter match, so any text with a comma (or the substringai, as in "email"/"detail") tripped HIGH findings.Fix
Removed only the 29 junk keyword lines. Every real signature (RTL/invisible unicode, fork bomb, path traversal), all genuine keywords, and every regex are preserved.
Proof
decision=allow, 0 findingsdecision=block, 31 findings (still caught)🤖 Generated with Claude Code