Skip to content

feat: regexp filters on re2js instead of the native re2 addon - #85

Draft
rolljee wants to merge 1 commit into
betafrom
feat/re2js
Draft

rolljee wants to merge 1 commit into
betafrom
feat/re2js

Conversation

@rolljee

@rolljee rolljee commented Sep 27, 2026

Copy link
Copy Markdown
Member

Draft — parked, decision pending. The port is done and proven equivalent; what is left is a product call on the matching speed (see Trade-off).

Follow-up of kuzzleio/kuzzle#2839: after the three Kuzzle-maintained addons moved to prebuilt N-API binaries, re2 is the last native module that touches the network or a compiler at install time.

Why

re2 (upstream uhop/node-re2, not ours) is a NAN addon: its install script downloads a binary per (os, arch, Node ABI) from GitHub releases, and runs node-gyp rebuild when that fails — the same failure class as kuzzleio/kuzzle#2790. It is used for one thing: the regexp keyword, where it protects Kuzzle from ReDoS (a client can subscribe with any regex).

re2js is a pure JavaScript port of RE2 (v2.8.6, 0 dependencies): same linear-time guarantee, same syntax, nothing to download or compile.

What

  • lib/util/RE2.js: the subset of re2's RE2 class Koncorde uses (constructor, test(), toString()), on re2js. It reproduces what re2 1.22 did on top of the RE2 engine:
    • its JS → RE2 syntax translation, ported from re2's lib/new.cc (\uXXXX, \u{…}, \cX, \p{Long_Name}, (?<name>, the m flag as (?m), /) — re2js' own translateRegExp accepts more than re2 did (e.g. \8), so it is not used;
    • its flags: i/m/s honoured, g/y making test() stateful through lastIndex, any other character ignored;
    • RE2's syntax error messages (Koncorde forwards them in its parse error): Go-style codes mapped to RE2's, flag prefixes stripped, lookbehinds reported as Perl operators.
  • regexpCondition.js, standardize.js use it; re2 dropped from dependencies, re2js added.
  • The regExpEngine: "re2" option keeps its name and meaning (RE2 semantics); "js" is unchanged.

Proof of equivalence

scripts/re2-parity.js — seeded random patterns (literals incl. non-ASCII/astral, classes, Unicode properties, JS escapes, groups, quantifiers, and what RE2 rejects) × random flags × 12 test() calls on the same object, against re2 1.22.3:

PARITY OK: 100000 patterns (44216 compiled, 55784 rejected), 530592 test() calls,
0 behaviour mismatch(es) (+552 known \B byte-position ones);
syntax error wording identical for 55231/55784

The PR workflow gets a re2-parity job (Node 20/22/24). Two deliberate differences, both because RE2 (C++) walks UTF-8 bytes where re2js walks characters:

  • \C (any byte) reads as any character — identical on ASCII;
  • \B no longer matches between the bytes of a multi-byte character (/\B/ on "Aé1": true with re2, false with re2js — re2js is the one that is right).

Koncorde's own suite: 423 passing, lint clean.

Trade-off: matching is slower

scripts/re2-bench.js (darwin-arm64, Node 24; the CI job prints it too):

Filter re2 (native) re2js ratio
^user_ 82 ns 147 ns 1.8×
@example\.com$ /i 81 ns 405 ns 5×
e-mail ^[a-z0-9._%+-]+@…$ /i 112 ns 1.1 µs 10×
(error|warn|fatal) 178 ns 570 ns 3.2×
\p{L}+ville$ /u 77 ns 2.6 µs 33×
literal in 1 KB of text 95 ns 48 ns 0.5×
(a+)+b (ReDoS-shaped) 104 ns 44 ns 0.4×

The cost is per written document × regexp subscriptions on the field: 1 000 regexp subscriptions on one field ≈ 0.1 ms → 0.5–2.5 ms per document. No regexp subscription, no cost.

Options (to decide before undrafting)

  1. Ship as is: no install-time download/compile left, same behavior everywhere; regex matching 2–10× slower (33× on Unicode classes).
  2. re2 as an optionalDependency, re2js as fallback: native speed where the binary installs, npm ci no longer fails when it does not — but the GitHub download stays, and behavior (the two differences above, error wording) would depend on the environment.
  3. Keep re2 and close this: with-retry.sh covers its download failures in Kuzzle's CI.

Deciding factor: how many regexp subscriptions real projects carry.

🤖 Generated with Claude Code

re2 is a NAN addon that downloads a per-Node-ABI binary from GitHub at
install time and compiles when that fails: the last install-time network
dependency and compiler use left in Koncorde. re2js is a pure JavaScript
port of RE2, with the same linear-time guarantee against ReDoS.

lib/util/RE2.js reproduces on re2js what `re2` 1.22 did on top of RE2:
its JS-to-RE2 syntax translation (ported from its new.cc), its flags
(i/m/s honoured, g/y making test() stateful through lastIndex, anything
else ignored) and RE2's syntax error messages, which Koncorde forwards.

scripts/re2-parity.js compares it with re2 1.22.3 on seeded random
patterns: 100 000 patterns, 530 592 test() calls, no behaviour mismatch;
error wording identical for 99 %. Two deliberate differences, both from
RE2 walking UTF-8 bytes where re2js walks characters: `\C` reads one
character, and `\B` no longer matches between the bytes of a multi-byte
character. The PR workflow runs the parity on Node 20, 22 and 24.

Matching is slower: 2–10x on typical filters, up to 33x with Unicode
classes, faster on literal search (scripts/re2-bench.js).

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant