Skip to content

Pattern "\uDC21"` matches inside a surrogate pair; equivalent classes don't #207

Description

@jemmix

A pattern whose literal prefix is a lone low surrogate matches inside a well-formed surrogate pair; the equivalent alternation/char-class patterns do not:

Pattern.compile("\uDC21").matcher("a\uD801\uDC21b").find()          // true — U+10421's low unit
Pattern.compile("\uDC21|\uDC22").matcher("a\uD801\uDC21b").find()   // false
Pattern.compile("[\uD800-\uDFFF]").matcher("a\uD801\uDC21b").find() // false

Whether a match may start between the units of a pair depends on whether the pattern compiles to a single literal prefix.

Repro (SurrogatePairTest):

assertFalse(Pattern.compile("\uDC21").matcher("a\uD801\uDC21b").find());
// search must resume past the skipped interior hit:
Matcher m = Pattern.compile("\uDC21").matcher("\uD801\uDC21\uDC21");
assertTrue(m.find());
assertEquals(2, m.start());

Cause: MachineInput.StringInput.index() jumps to raw String.indexOf hits. Such a hit can be the low unit of a pair — not a codepoint boundary — and step() then decodes the lone low unit as a rune. Paths without the fast path decode pairs and never enter the interior.

Fix: skip indexOf hits that are a low surrogate preceded by a high surrogate; keep searching.

Sidenote: RE2 proper operates on UTF-8 runes, where pair-interior positions cannot exist; this is a UTF-16 port artifact.

Disclosure: this was discovered when fuzzing another library (https://github.com/jemmix/tdfa-jvm) against re2j. Patch and fix composed agentically using GLM 5.3.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions