feat: add a control anomaly kind, closing seven CVE rows nothing reported (#612)

CHANGELOG.md

@@ -548,6 +548,45 @@ compatibility (see [RELEASING.md](RELEASING.md)).
   the invariant that actually defines the mode: `Preserve` output is never shorter than
   `Ignore` output, which needs no generator exclusions at all.
 
+- **A new `control` anomaly kind — `has_anomalies` goes from 11 CVE rows to 18 (#612).**
+  A non-whitespace control (`NUL`, `ESC`, `BEL`, `DEL`, the C1 block) is never
+  legitimate in text, and nothing reported one. `strip_control_chars` has removed them
+  since #433, so the transform existed and the detector did not.
+
+  The reason they were invisible is worth recording: the introducers are plain ASCII, so
+  the ASCII fast path in the token classifier — which exists because the invisible, bidi,
+  zalgo and mixed-script branches can only fire above `U+007F` — skipped them entirely.
+  The new branch runs before that gate.
+
+  **Presence, not position.** #612 framed this as an "edge" question because it started
+  from whitespace trimming, but a control hides things wherever it sits: the last
+  character of `"malicious\u001b\\"` is a backslash, so an edge-only rule would call
+  that token clean while the escape introducer sits one place in.
+
+  The whitespace-class controls are excluded, reusing `is_fold_whitespace` rather than
+  restating the set. TAB, LF, VT, FF, CR, `U+001C`–`U+001F` and NEL are real separators
+  that `collapse_whitespace` folds to a space, and flagging them would fire on every
+  multi-line string.
+
+  This closes seven rows that `docs/security/cve-validation.md` listed as reported by
+  nothing: CVE-2023-24329 (leading NUL) and the whole terminal-control class
+  (CVE-2008-2383, CVE-2019-9535, CVE-2025-55754, CVE-2024-52005, CVE-2023-43620,
+  CVE-2023-37275). The three that remain undetected are a different shape — a fold
+  collision, a length budget, a table lookup — so no further character class will close
+  them, and the page now says so.
+
+  Deliberately *not* added: leading/trailing whitespace detection, which #612 also asked
+  for. `inspect_anomalies` documents itself as flagging characters "disguising a real
+  word", and padding disguises nothing; a kind for it would fire on ordinary text.
+
+- **The Node `AnomalyKind` union shipped without `bidi_mixed`.** It was added to the Rust
+  enum in #412 and never mirrored, so a TypeScript caller matching on it got a type error
+  for a kind the library really returns. Nothing caught it, because the value crosses
+  napi as a bare `String` and `index.ts` casts. Node is the only binding that restates
+  the set — every other surface passes it through as a string — so a drift gate now reads
+  the `as_str` arms out of `src/anomalies.rs` and compares them to the union, plus a
+  second test asserting every kind is reachable from some input.
+
 - **`PRESETS["ml_normalize"]` was missing two of the nine steps it claims to describe
   (#600).** `PRESETS` is a hand-maintained Python mirror of the `const STEPS` arrays in
   `src/presets.rs`; nothing executes it, and it had drifted. The mirror listed seven

bindings/node/index.ts

@@ -49,7 +49,7 @@ export { Lexicon }
 export { Pipeline }
 
 /** The anomaly branch that fired for a finding. */
-export type AnomalyKind = 'invisible' | 'bidi' | 'zalgo' | 'mixed_script' | 'leet' | 'segmentation'
+export type AnomalyKind = 'invisible' | 'bidi' | 'bidi_mixed' | 'zalgo' | 'mixed_script' | 'leet' | 'segmentation' | 'control'
 
 /**
  * One reason a token is anomalous. Re-typed over the generated {@link NativeFinding}

docs/security/cve-validation.md

@@ -58,10 +58,10 @@ ranking.
 | [CVE-2017-7832](https://nvd.nist.gov/vuln/detail/CVE-2017-7832) | Firefox — dotless-i address-bar spoof evading punycode display | 5.3 (v3.0) | Neutralized + detected | `canonicalize`, `normalize_confusables` | `is_confusable`, `is_suspicious_hostname` |
 | [CVE-2017-5383](https://nvd.nist.gov/vuln/detail/CVE-2017-5383) | Firefox — alternative hyphens and quotes evading punycode display | 5.3 (v3.0) | Neutralized + detected | `canonicalize`, `normalize_confusables`, `catalog_key` | `is_confusable` |
 | [CVE-2017-7833](https://nvd.nist.gov/vuln/detail/CVE-2017-7833) | Firefox — combining vowel mark eclipsing a Latin letter in a domain | 5.3 (v3.0) | Neutralized + detected | `strip_obfuscation`, `catalog_key`, `strip_zalgo` | `has_anomalies`, `has_bidi_conflict`, `is_mixed_script`, `is_suspicious_hostname` |
-| [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329) | Python urllib.parse — blocklist bypass via leading blank characters | 7.5 (v3.1) | Neutralized | `canonicalize`, `strip_obfuscation` | — |
+| [CVE-2023-24329](https://nvd.nist.gov/vuln/detail/CVE-2023-24329) | Python urllib.parse — blocklist bypass via leading blank characters | 7.5 (v3.1) | Neutralized + detected | `canonicalize`, `strip_obfuscation` | `has_anomalies` |
 | [CVE-2019-9636](https://nvd.nist.gov/vuln/detail/CVE-2019-9636) | Python urlsplit — netloc misparse under NFKC normalization | 9.8 (v3.1) | Out of scope | — | — |
-| [CVE-2008-2383](https://nvd.nist.gov/vuln/detail/CVE-2008-2383) | xterm — command execution via DECRQSS escape sequence | 9.3 (v2.0) | Neutralized | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | — |
-| [CVE-2019-9535](https://nvd.nist.gov/vuln/detail/CVE-2019-9535) | iTerm2 — command execution via tmux control-mode output | 9.8 (v3.1) | Neutralized | `strip_log_injection`, `canonicalize` | — |
+| [CVE-2008-2383](https://nvd.nist.gov/vuln/detail/CVE-2008-2383) | xterm — command execution via DECRQSS escape sequence | 9.3 (v2.0) | Neutralized + detected | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | `has_anomalies` |
+| [CVE-2019-9535](https://nvd.nist.gov/vuln/detail/CVE-2019-9535) | iTerm2 — command execution via tmux control-mode output | 9.8 (v3.1) | Neutralized + detected | `strip_log_injection`, `canonicalize` | `has_anomalies` |
 | [CVE-2025-32711](https://nvd.nist.gov/vuln/detail/CVE-2025-32711) | Microsoft 365 Copilot (EchoLeak) — AI command injection | 9.3 (v3.1) | Neutralized | `strip_tags`, `llm_guardrail`, `canonicalize`, `strip_obfuscation` | — |
 | [CVE-2024-5184](https://nvd.nist.gov/vuln/detail/CVE-2024-5184) | EmailGPT — prompt injection via untrusted message text | 9.1 (v3.1) | Out of scope | — | — |
 | [CVE-2024-5565](https://nvd.nist.gov/vuln/detail/CVE-2024-5565) | Vanna.AI — prompt injection to arbitrary Python execution | 8.1 (v3.1) | Out of scope | — | — |
@@ -73,10 +73,10 @@ ranking.
 | [CVE-2024-43093](https://nvd.nist.gov/vuln/detail/CVE-2024-43093) | Android — path filter bypass via improper Unicode normalization (CISA KEV) | 7.3 (v3.1) | Out of scope | — | — |
 | [CVE-2023-41889](https://nvd.nist.gov/vuln/detail/CVE-2023-41889) | SHIRASAGI — validation performed before Unicode normalization | 5.3 (v3.1) | Out of scope | — | — |
 | [CVE-2023-52081](https://nvd.nist.gov/vuln/detail/CVE-2023-52081) | ffcss — regex filter re-populated by NFKC-equivalent characters | 5.3 (v3.1) | Out of scope | — | — |
-| [CVE-2025-55754](https://nvd.nist.gov/vuln/detail/CVE-2025-55754) | Apache Tomcat — ANSI escape injection into Windows console logs | 9.6 (v3.1) | Neutralized | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | — |
-| [CVE-2024-52005](https://nvd.nist.gov/vuln/detail/CVE-2024-52005) | Git — ANSI escape sequences in sideband channel messages | 8.8 (v3.1) | Neutralized | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | — |
-| [CVE-2023-43620](https://nvd.nist.gov/vuln/detail/CVE-2023-43620) | Croc — ANSI escape sequences placed in a filename | 7.8 (v3.1) | Neutralized | `sanitize_filename`, `strip_log_injection`, `canonicalize` | — |
-| [CVE-2023-37275](https://nvd.nist.gov/vuln/detail/CVE-2023-37275) | Auto-GPT — console spoofing via ANSI relayed through an LLM | 4.3 (v3.1) | Neutralized | `strip_log_injection`, `canonicalize` | — |
+| [CVE-2025-55754](https://nvd.nist.gov/vuln/detail/CVE-2025-55754) | Apache Tomcat — ANSI escape injection into Windows console logs | 9.6 (v3.1) | Neutralized + detected | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | `has_anomalies` |
+| [CVE-2024-52005](https://nvd.nist.gov/vuln/detail/CVE-2024-52005) | Git — ANSI escape sequences in sideband channel messages | 8.8 (v3.1) | Neutralized + detected | `strip_log_injection`, `canonicalize`, `strip_obfuscation` | `has_anomalies` |
+| [CVE-2023-43620](https://nvd.nist.gov/vuln/detail/CVE-2023-43620) | Croc — ANSI escape sequences placed in a filename | 7.8 (v3.1) | Neutralized + detected | `sanitize_filename`, `strip_log_injection`, `canonicalize` | `has_anomalies` |
+| [CVE-2023-37275](https://nvd.nist.gov/vuln/detail/CVE-2023-37275) | Auto-GPT — console spoofing via ANSI relayed through an LLM | 4.3 (v3.1) | Neutralized + detected | `strip_log_injection`, `canonicalize` | `has_anomalies` |
 | [CVE-2019-11721](https://nvd.nist.gov/vuln/detail/CVE-2019-11721) | Firefox — Latin kra spoofing 'k' in the address bar | 6.5 (v3.1) | Neutralized + detected | `normalize_confusables`, `canonicalize`, `strip_obfuscation` | `is_confusable`, `is_suspicious_hostname` |
 | [CVE-2023-4399](https://nvd.nist.gov/vuln/detail/CVE-2023-4399) | Grafana — request deny list bypassed by punycode encoding | 7.2 (v3.1) | Detected only | — | `is_suspicious_hostname` |
 | [CVE-2026-23950](https://nvd.nist.gov/vuln/detail/CVE-2026-23950) | node-tar — symlink poisoning via a Unicode path collision | 5.9 (v3.1) | Neutralized | `fold_case`, `search_key`, `catalog_key` | — |
@@ -85,7 +85,7 @@ ranking.
 | [CVE-2017-20190](https://nvd.nist.gov/vuln/detail/CVE-2017-20190) | Windows — performance degradation from piled combining marks (Zalgo) | none (SSVC only) | Neutralized + detected | `strip_zalgo`, `canonicalize`, `strip_obfuscation` | `is_zalgo`, `has_anomalies` |
 | [CVE-2024-46954](https://nvd.nist.gov/vuln/detail/CVE-2024-46954) | Ghostscript — overlong UTF-8 decoded to a real ../ traversal | 7.8 (v3.1) | Not affected | `decode_to_utf8` | — |
 | [CVE-2026-44288](https://nvd.nist.gov/vuln/detail/CVE-2026-44288) | protobufjs — overlong UTF-8 decoded to canonical characters | 5.3 (v3.1) | Not affected | `decode_to_utf8` | — |
-| [CVE-2009-4142](https://nvd.nist.gov/vuln/detail/CVE-2009-4142) | PHP htmlspecialchars — overlong UTF-8 and invalid Shift_JIS/EUC-JP | 4.3 (v2.0) | Not affected | `decode_to_utf8` | — |
+| [CVE-2009-4142](https://nvd.nist.gov/vuln/detail/CVE-2009-4142) | PHP htmlspecialchars — overlong UTF-8 and invalid Shift_JIS/EUC-JP | 4.3 (v2.0) | Not affected + detected | `decode_to_utf8` | `has_anomalies` |
 | [CVE-2022-31116](https://nvd.nist.gov/vuln/detail/CVE-2022-31116) | UltraJSON — lone surrogates causing dictionary key confusion | 7.5 (v3.1) | Neutralized | `canonicalize`, `canonicalize_strict`, `strip_obfuscation` | — |
 | [CVE-2025-64439](https://nvd.nist.gov/vuln/detail/CVE-2025-64439) | LangGraph — illegal surrogates falling back to insecure deserialization | 7.4 (v4.0) | Neutralized | `canonicalize`, `canonicalize_strict`, `strip_obfuscation` | — |
 | [CVE-2008-4066](https://nvd.nist.gov/vuln/detail/CVE-2008-4066) | Firefox — HTML-escaped low surrogate ignored by the parser | 4.3 (v2.0) | Out of scope | — | — |
@@ -160,22 +160,33 @@ Treat this as "no vector here needs a third call", not as coverage.
 ### Detection has no equivalent
 
 The symmetry breaks here, and it decides how a pipeline should be built. No
-single detector covers the matrix, and neither does all of them together: five
-vectors are silent to every detector disarm exposes.
+single detector covers the matrix, and neither does all of them together.
+
+The gap used to be wider. Eight rows went unreported because their introducers
+are plain ASCII controls, and the ASCII fast path in the token classifier skipped
+them entirely: a leading NUL, the whole terminal-control class, and one byte-level
+row whose probe is itself a NUL injection. The `control` anomaly kind (#612)
+closed all eight in one branch, and `has_anomalies` went from 11 rows to 19.
+
+What remains is a different shape, which is the useful part:
 
 | Vector | Why nothing flags it |
 |---|---|
-| CVE-2023-24329 | A leading NUL is not an anomaly kind |
-| CVE-2008-2383, CVE-2019-9535 | Nor is a terminal escape sequence |
-| CVE-2025-32711 | Nor is the Unicode Tags block |
-| CVE-2019-9636 | Nor is compatibility-fold unmasking |
-
-All five are still *neutralized* by `canonicalize`. So the rule follows from the
+| CVE-2025-32711 | The Unicode Tags block is not an anomaly kind |
+| CVE-2026-23950 | Nor is a case-folding path collision |
+| CVE-2023-46695 | Nor is a long run of already-normalized characters |
+| CVE-2022-31116, CVE-2025-64439, CVE-2007-2688 | Nor is a malformed encoding — by the time text reaches a detector it has already been decoded |
+| CVE-2024-46954, CVE-2026-44288 | Nor is a decode failure: `decode_to_utf8` returns `had_errors`, which is a return value rather than a panel predicate |
+
+Not one of them is a character you can look for. Each needs a *comparison* — a
+fold collision against another string, a length budget, a decode result — so no
+number of additional character classes will close any of them. That is the line
+this section is really drawing, and it is why the rule below follows from the
 measurement rather than from taste:
 
 **Clean unconditionally. Use the detectors to decide whether to alert, never
-whether to clean.** A pipeline that screens first and cleans only what it
-flagged forwards those five untouched.
+whether to clean.** Every one is still *neutralized*, so a pipeline that screens
+first and cleans only what it flagged forwards them untouched.
 
 ## How other tools handle the same vectors
 

docs/user-guide/anomaly-detection.md

@@ -15,8 +15,17 @@ the caller — it never claims intent.
 
 ## Detected classes
 
-Six branches fire, in order; the first four need no lexicon and are
-script-agnostic, so they port across writing systems.
+Eight branches fire. Six need no lexicon — only `leet` and `segmentation` do.
+
+The table below is grouped by kind, not by evaluation order. `control` is checked
+**first**, ahead of the ASCII fast-path, because `NUL`, `ESC`, `BEL` and `DEL` are
+themselves ASCII: a check placed after that fast-path would never see the vectors it
+exists for. The remaining branches split on `!tok.is_ascii()`, so `invisible`, `bidi`,
+`zalgo`, `bidi_mixed` and `mixed_script` only run on non-ASCII tokens, and `leet` and
+`segmentation` run last on everything.
+
+Most branches are script-agnostic and port across writing systems. `mixed_script` is the
+exception — it is anchored on Latin, and fires on Latin combined with Cyrillic or Greek.
 
 | Kind | Fires on | Spared (false-positive guards) |
 |---|---|---|
@@ -27,6 +36,7 @@ script-agnostic, so they port across writing systems.
 | `bidi_mixed` | one token mixes strong left-to-right and strong right-to-left **letters** (`varonisו`), which can visually reorder ("BiDi Swap") — no `U+202x` override (that is `bidi`) | single-direction text (all-LTR or all-RTL); digits are neutral |
 | `leet` | every out-of-place char substitutes a letter and the result is a common word (`fr33` → `free`) | a literal number that maps to no letter (`win32`, `Power5`, `21st`, `3pm`) |
 | `segmentation` | dense separators splitting single letters into a real word (`v.i.a.g.r.a`) | multi-letter parts (`6-foot-6`); a lone separator (`e-mail`) |
+| `control` | a non-whitespace control anywhere in the token — `NUL`, `ESC`, `BEL`, `DEL`, the C1 block. Never legitimate in text, and the introducer for terminal-escape injection and leading-blank blocklist bypass | the whitespace-class controls (TAB, LF, VT, FF, CR, `U+001C`–`U+001F`, NEL), which are real separators `collapse_whitespace` folds to a space |
 
 The **leet** and **segmentation** branches take a caller-supplied **lexicon** — a
 set of common words for the language being protected. The defining rule: a real

src/anomalies.rs

@@ -70,6 +70,15 @@ pub enum AnomalyKind {
     Leet,
     /// Dense separators splitting single letters into a real word (`v.i.a.g.r.a`).
     Segmentation,
+    /// A non-whitespace control character (`NUL`, `ESC`, `BEL`, `DEL`, the C1 block).
+    ///
+    /// Never legitimate in text, and the introducer for terminal-escape injection
+    /// (CVE-2008-2383, CVE-2019-9535) and leading-blank blocklist bypass
+    /// (CVE-2023-24329). The whitespace-class controls — TAB, LF, VT, FF, CR, the
+    /// information separators `U+001C`–`U+001F`, NEL — are excluded: they are real
+    /// separators that [`crate::whitespace::collapse_whitespace`] folds to a space,
+    /// so flagging them would fire on ordinary multi-line text (#612).
+    Control,
 }
 
 impl AnomalyKind {
@@ -84,6 +93,7 @@ impl AnomalyKind {
             AnomalyKind::BidiMixed => "bidi_mixed",
             AnomalyKind::Leet => "leet",
             AnomalyKind::Segmentation => "segmentation",
+            AnomalyKind::Control => "control",
         }
     }
 }
@@ -135,6 +145,9 @@ impl Finding {
             AnomalyKind::Segmentation => {
                 format!("{:?} splits the word {:?}", self.token, self.detail)
             }
+            AnomalyKind::Control => {
+                format!("{:?} contains the control character {}", self.token, self.detail)
+            }
         }
     }
 }
@@ -338,6 +351,27 @@ fn classify(tok: &str, start: usize, lexicon: &HashSet<String>) -> Option<Findin
     // branch and the leet/segmentation branches; compute it once.
     let core = tok.trim_matches(|c: char| WRAP.contains(&c));
 
+    // Non-whitespace controls (#612). Checked BEFORE the ASCII fast-path below, because
+    // NUL, ESC, BEL and DEL are all ASCII — a pure-ASCII token skips that whole block, so
+    // a check placed inside it would never see the vectors this exists for.
+    //
+    // Presence, not position. #612 framed this as an "edge" question because it started
+    // from whitespace trimming, but a control hides things wherever it sits: the last
+    // character of `"malicious\u{1b}\\"` is a backslash, so an edge-only rule would call
+    // that token clean while the escape introducer sits one place in.
+    //
+    // The whitespace-class controls are excluded via `is_fold_whitespace` — TAB, LF, VT,
+    // FF, CR, `U+001C`-`U+001F` and NEL are real separators that `collapse_whitespace`
+    // folds to a space, and flagging them would fire on ordinary multi-line text. That is
+    // the same split `strip_control_chars` has drawn since #433, reused rather than
+    // restated so the two cannot drift.
+    if let Some(c) = tok
+        .chars()
+        .find(|&c| c.is_control() && !crate::whitespace::is_fold_whitespace(c))
+    {
+        return Some(mk(AnomalyKind::Control, codepoint(c)));
+    }
+
     // ASCII fast-path: the invisible / bidi / zalgo / mixed-script branches can only fire
     // above U+007F, so a pure-ASCII token skips every script and zalgo call.
     if !tok.is_ascii() {