Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 36 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,42 @@ compatibility (see [RELEASING.md](RELEASING.md)).

### Added

- **`is_case_fold_stable` — ask whether a value is a stable identity key before you key a
table on it (#619).** Answers `fold_case(x) == x.lower()`. A `False` says some *other*
string folds to the same value, which is the precondition node-tar's `PathReservations`
guard missed in CVE-2026-23950: `groß.txt` and `gross.txt` are one path on a
case-insensitive filesystem. Available in Rust (`api::is_case_fold_stable`, and on
`DisarmStr`), Python (`is_case_fold_stable`, `Text.is_case_fold_stable`), Node
(`isCaseFoldStable`), Ruby (`Disarm.case_fold_stable?`), the C ABI and Java.

**It states a fact about the string, not suspicion.** `groß` is an ordinary German word
and `file` an ordinary ligature, so the predicate reads `True` for ordinary text and is
deliberately kept out of `has_anomalies` and out of the CVE detector panel — folding it
in would flag ordinary German and every Greek word ending in sigma. What to do about a
`False` is the caller's decision: reserve both forms, reject the name, or key the table
on `fold_case` rather than `str.lower()`.

**`str.lower()` is the comparison basis and `str.casefold()` is not.** Casefolding
performs the very transform under test, so a predicate written against it answers
`True` for every string in Unicode — the substitution #617 already made once, now
pinned by a test.

**Not a per-character table, because a per-character table is wrong for Greek.**
`ΟΔΟΣ` ("street") lowercases to `οδος` and folds to `οδοσ`, yet `Σ` agrees with itself
in isolation; `U+03A3` is the only code point in Unicode whose lowercase mapping depends
on its neighbours, which a Tier-3 test asserts by enumeration rather than by assertion.
The implementation is allocation-free for anything that contains no capital sigma —
ASCII short-circuits, everything else scans the folding table in place — and falls back
to the exact string comparison for the rest, so it cannot drift from what `fold_case`
actually does.

CVE-2026-23950's *Detected by* column stops reading `—`. The collision itself is still a
property of a pair of names and no single-string predicate can report it (#620 tracks
that); the precondition is what moved. **Measured limit:** the issue paired this with
CVE-2019-19844, and that half does not hold — that row's probe turns on `U+0131` DOTLESS
I, which folds *and* lowercases to itself and collides through `.upper()` instead, so
the predicate is silent on it.

- **Ten CVEs on encoding rather than code points, and the survey method behind them
(36 → 46 rows).** Found by sweeping NVD across the operations disarm performs, then
verified one ID at a time.
Expand Down
8 changes: 8 additions & 0 deletions bindings/cabi/disarm.h
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,14 @@ char *
disarm_inspect_auto_lang (
char const * text);

/** \brief
* Whether case folding and simple lowercasing agree, so `text` is a stable
* identity key ("groß.txt" is not; "gross.txt" is).
*/
bool
disarm_is_case_fold_stable (
char const * text);

/** \brief
* Whether `text` mixes characters from more than one script.
*/
Expand Down
7 changes: 7 additions & 0 deletions bindings/cabi/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -175,6 +175,13 @@ fn disarm_fold_case(text: char_p::Ref<'_>) -> char_p::Box {
to_c(api::fold_case(text.to_str()).into_owned())
}

/// Whether case folding and simple lowercasing agree, so `text` is a stable
/// identity key ("groß.txt" is not; "gross.txt" is).
#[ffi_export]
fn disarm_is_case_fold_stable(text: char_p::Ref<'_>) -> bool {
api::is_case_fold_stable(text.to_str())
}

/// Replace emoji with their plain names; `strip_modifiers` drops skin-tone marks.
#[ffi_export]
fn disarm_demojize(text: char_p::Ref<'_>, strip_modifiers: bool) -> char_p::Box {
Expand Down
12 changes: 12 additions & 0 deletions bindings/java/disarm-java/src/main/java/dev/disarm/Disarm.java
Original file line number Diff line number Diff line change
Expand Up @@ -90,6 +90,18 @@ public static String foldCase(String text) {
return Native.foldCase(req(text));
}

/**
* Whether {@code text} is a stable identity key under case folding — whether
* {@link #foldCase} and {@code String.toLowerCase()} agree on it.
*
* <p>{@code false} means some other string folds to the same value, so a table keyed on
* this one can collide: {@code "groß.txt"} and {@code "gross.txt"} are the pair node-tar
* collided on (CVE-2026-23950). It states a fact about the string, not suspicion.
*/
public static boolean isCaseFoldStable(String text) {
return Native.isCaseFoldStable(req(text));
}

/** Replace emoji with their plain names (skin-tone modifiers preserved). */
public static String demojize(String text) {
return demojize(text, false);
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,8 @@ public static native String normalizeConfusables(

public static native String foldCase(String text);

public static native boolean isCaseFoldStable(String text);

public static native String demojize(String text, boolean stripModifiers);

// ── Normalization ──────────────────────────────────────────────────────────
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,13 @@ void stripPua() {
assertEquals("ab", Disarm.stripPua("ab")); // Private Use Area
}

@Test
void isCaseFoldStable() {
assertTrue(Disarm.isCaseFoldStable("gross.txt"));
assertFalse(Disarm.isCaseFoldStable("groß.txt")); // collides with gross.txt
assertThrows(NullPointerException.class, () -> Disarm.isCaseFoldStable(null));
}

// ── Overloads with explicit optional arguments ──────────────────────────────

@Test
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,14 @@ fun String.stripAccents(): String = JDisarm.stripAccents(this)

fun String.foldCase(): String = JDisarm.foldCase(this)

/**
* Whether this value is a stable identity key under case folding — whether [foldCase]
* and `lowercase()` agree on it. `false` means some other string folds to the same
* value, the collision node-tar hit in CVE-2026-23950. A fact about the string, not
* suspicion: `groß` is an ordinary German word.
*/
fun String.isCaseFoldStable(): Boolean = JDisarm.isCaseFoldStable(this)

@JvmOverloads
fun String.demojize(stripModifiers: Boolean = false): String = JDisarm.demojize(this, stripModifiers)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,8 @@ class DisarmKtTest {
fun canonicalizationPrimitives() {
assertEquals("cafe", "café".stripAccents())
assertEquals("hello", "Hello".foldCase())
assertTrue("gross.txt".isCaseFoldStable())
assertFalse("groß.txt".isCaseFoldStable())
assertFalse("😀".demojize().isBlank())
assertFalse("👍🏽".demojize(stripModifiers = true).isBlank())
}
Expand Down
15 changes: 15 additions & 0 deletions bindings/java/rust/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -529,6 +529,21 @@ pub fn foldCase<'l>(env: EnvUnowned<'l>, _class: JClass<'l>, input: JString<'l>)
map_str(env, input, |t| api::fold_case(t).into_owned())
}

/// Whether case folding and simple lowercasing agree, so `text` is a stable
/// identity key ("groß.txt" is not; "gross.txt" is).
#[jni_mangle("dev.disarm.internal.Native")]
pub fn isCaseFoldStable<'l>(
mut env: EnvUnowned<'l>,
_class: JClass<'l>,
input: JString<'l>,
) -> jboolean {
env.with_env(|env| -> JniResult<jboolean> {
let text = input.mutf8_chars(env)?.to_string();
Ok(api::is_case_fold_stable(&text))
})
.resolve::<Policy>()
}

/// Replace emoji with their plain names; `stripModifiers` drops skin-tone marks.
#[jni_mangle("dev.disarm.internal.Native")]
pub fn demojize<'l>(
Expand Down
8 changes: 8 additions & 0 deletions bindings/node/__test__/disarm.test.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,14 @@ describe('slugify', () => {
describe('canonicalization', () => {
test('stripAccents', () => expect(disarm.stripAccents('café')).toBe('cafe'))
test('foldCase', () => expect(disarm.foldCase('HELLO')).toBe('hello'))
test('isCaseFoldStable', () => {
expect(disarm.isCaseFoldStable('gross.txt')).toBe(true)
// The pair node-tar collided on: both sides reduce to gross.txt.
expect(disarm.isCaseFoldStable('groß.txt')).toBe(false)
// Greek final sigma — the whole-string answer, not a per-character one.
expect(disarm.isCaseFoldStable('ΟΔΟΣ')).toBe(false)
expect(disarm.isCaseFoldStable('ΣΑΒΒΑΤΟ')).toBe(true)
})
test('demojize', () => expect(disarm.demojize('hi 👍')).toBe('hi thumbs up'))
})

Expand Down
14 changes: 14 additions & 0 deletions bindings/node/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -254,6 +254,20 @@ export function foldCase(text: string): string {
return native.foldCase(text)
}

/**
* Whether `text` is a stable identity key under case folding — that is, whether
* {@link foldCase} and `String.toLowerCase()` agree on it (#619).
*
* `false` means some *other* string folds to the same value, so a table keyed on
* this one can collide: `'groß.txt'` and `'gross.txt'` are the pair node-tar
* collided on (CVE-2026-23950). It is a fact about the string and not an
* accusation — `groß` is an ordinary German word — so it is deliberately not
* folded into {@link hasAnomalies}.
*/
export function isCaseFoldStable(text: string): boolean {
return native.isCaseFoldStable(text)
}

/** Replace emoji with their plain names. `stripModifiers` drops skin-tone/variation marks. */
export function demojize(text: string, options: { stripModifiers?: boolean } = {}): string {
return native.demojize(text, options.stripModifiers ?? false)
Expand Down
7 changes: 7 additions & 0 deletions bindings/node/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,13 @@ pub fn fold_case(text: String) -> String {
api::fold_case(&text).into_owned()
}

/// Whether case folding and simple lowercasing agree, so the value is a stable
/// identity key (#619).
#[napi]
pub fn is_case_fold_stable(text: String) -> bool {
api::is_case_fold_stable(&text)
}

/// Replace emoji with their plain names; `strip_modifiers` drops skin-tone marks.
#[napi]
pub fn demojize(text: String, strip_modifiers: bool) -> String {
Expand Down
10 changes: 10 additions & 0 deletions bindings/ruby/ext/disarm/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -255,6 +255,12 @@ fn fold_case(text: Wtf8Text) -> String {
api::fold_case(&text).into_owned()
}

/// `Disarm._is_case_fold_stable?(text)` — whether case folding and simple
/// lowercasing agree, so the value is a stable identity key (#619).
fn is_case_fold_stable(text: Wtf8Text) -> bool {
api::is_case_fold_stable(&text)
}

/// `Disarm._slugify(text, …)` — the full slug option surface, positional. The
/// Ruby layer maps its keyword arguments (with the core's documented defaults)
/// onto this order. `regex_pattern` and `replacements` are intentionally not
Expand Down Expand Up @@ -779,6 +785,10 @@ fn init(ruby: &Ruby) -> Result<(), Error> {
// keeping `rescue Disarm::Error` exhaustive across the whole public surface.
module.define_singleton_method("_strip_accents", function!(strip_accents, 1))?;
module.define_singleton_method("_fold_case", function!(fold_case, 1))?;
module.define_singleton_method(
"_is_case_fold_stable?",
function!(is_case_fold_stable, 1),
)?;
module.define_singleton_method("_suspicious_hostname?", function!(suspicious_hostname, 1))?;
module.define_singleton_method("_analyze_hostname", function!(analyze_hostname, 2))?;

Expand Down
9 changes: 9 additions & 0 deletions bindings/ruby/lib/disarm.rb
Original file line number Diff line number Diff line change
Expand Up @@ -161,6 +161,15 @@ def fold_case(text)
translate_errors { _fold_case(text) }
end

# Whether `text` is a stable identity key under case folding — whether
# `fold_case` and `String#downcase` agree on it (#619). A false result means
# some other string folds to the same value, so a table keyed on this one can
# collide: "groß.txt" and "gross.txt" are the pair node-tar collided on
# (CVE-2026-23950). It is a fact about the string, not an accusation.
def case_fold_stable?(text)
translate_errors { _is_case_fold_stable?(text) }
end

# Whether the hostname looks like a mixed-script / confusable / bidi-reorder
# IDN spoof. Flags a mixed-script label, a Latin confusable, or a
# bidi-direction conflict (see #bidi_conflict?, the "BiDi Swap" precondition),
Expand Down
9 changes: 9 additions & 0 deletions bindings/ruby/spec/disarm_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -120,6 +120,15 @@
it "case-folds" do
expect(Disarm.fold_case("HELLO")).to eq("hello")
end

it "reports whether a value is a stable key under case folding" do
expect(Disarm.case_fold_stable?("gross.txt")).to be(true)
# The pair node-tar collided on (CVE-2026-23950).
expect(Disarm.case_fold_stable?("groß.txt")).to be(false)
# Greek final sigma — the whole-string answer, not a per-character one.
expect(Disarm.case_fold_stable?("ΟΔΟΣ")).to be(false)
expect(Disarm.case_fold_stable?("ΣΑΒΒΑΤΟ")).to be(true)
end
end

describe "security" do
Expand Down
1 change: 1 addition & 0 deletions bindings/ruby/spec/surrogate_spec.rb
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
"transliterate" => ->(s) { Disarm.transliterate(s) },
"strip_accents" => ->(s) { Disarm.strip_accents(s) },
"fold_case" => ->(s) { Disarm.fold_case(s) },
"case_fold_stable?" => ->(s) { Disarm.case_fold_stable?(s) },
"search_key" => ->(s) { Disarm.search_key(s) },
"sort_key" => ->(s) { Disarm.sort_key(s) },
"catalog_key" => ->(s) { Disarm.catalog_key(s) }
Expand Down
24 changes: 24 additions & 0 deletions docs/api/predicates.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,30 @@ See [Language Detection](../user-guide/language-detection.md#inspecting-detectio

---

## is_case_fold_stable

::: disarm.is_case_fold_stable

```python
from disarm import is_case_fold_stable

is_case_fold_stable("gross.txt") # True
is_case_fold_stable("groß.txt") # False — folds to gross.txt, so the two collide
is_case_fold_stable("file") # False — folds to file
is_case_fold_stable("ΟΔΟΣ") # False — lowercases to οδος, folds to οδοσ
```

Use it before a name becomes a key: a reservation table, a username registry, an
extraction path. `False` says the value shares its folded form with some other
string, which is the precondition node-tar's `PathReservations` guard missed in
CVE-2026-23950. It says nothing about intent, since `groß` is an ordinary German
word, so the predicate is kept out of
[anomaly detection](../user-guide/anomaly-detection.md) and the response is the
caller's to choose: reserve both forms, reject the name, or key the table on
[`fold_case`](transforms.md#fold_case) instead of `str.lower()`.

---

## is_normalized

::: disarm.is_normalized
Expand Down
14 changes: 14 additions & 0 deletions docs/node/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,6 +112,20 @@ foldCase('Straße') // => 'strasse'
demojize('Café ☕') // => 'Café hot beverage'
```

### `isCaseFoldStable(text)`

Whether `foldCase` and `String.toLowerCase()` agree on `text`, so it is a stable
identity key. `false` means some other string folds to the same value, which is
the collision node-tar's `PathReservations` guard missed in CVE-2026-23950. Ask
it before a name becomes a key; a `false` is a fact about the string rather than
a report of an attack, since `groß` is an ordinary German word.

```ts
isCaseFoldStable('gross.txt') // => true
isCaseFoldStable('groß.txt') // => false, folds to gross.txt
isCaseFoldStable('ΟΔΟΣ') // => false, lowercases to οδος and folds to οδοσ
```

## Normalization

### `normalize(text, options?)` · `isNormalized(text, options?)`
Expand Down
14 changes: 14 additions & 0 deletions docs/ruby/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,20 @@ Disarm.fold_case("HELLO") # => "hello"
Disarm.fold_case("Straße") # => "strasse"
```

### `Disarm.case_fold_stable?(text)`

Whether `fold_case` and `String#downcase` agree on `text`, so it is a stable
identity key. `false` means some other string folds to the same value, which is
the collision node-tar's `PathReservations` guard missed in CVE-2026-23950. Ask
it before a name becomes a key; a `false` is a fact about the string rather than
a report of an attack, since `groß` is an ordinary German word.

```ruby
Disarm.case_fold_stable?("gross.txt") # => true
Disarm.case_fold_stable?("groß.txt") # => false, folds to gross.txt
Disarm.case_fold_stable?("ΟΔΟΣ") # => false, downcases to οδος
```

### `Disarm.demojize(text, strip_modifiers: false)`

Replace emoji with their plain names. `strip_modifiers:` drops skin-tone /
Expand Down
Loading
Loading