You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Tests/test_backend.py::test_tokenize_ascii_apostrophe_contraction verifies that stats.py::_WORD_RE tokenizes "don't can't it's" as whole tokens. The test was added to fix issue #37.
backend/src/edcmbone/parser/turns_rounds.py::_WORD_RE is a visually similar regex but has no equivalent regression test. It is possible that the same apostrophe fix was not applied to the parser regex, or that it was applied but is untested and could regress.
Action required:
Byte-compare stats.py::_WORD_RE and parser/turns_rounds.py::_WORD_RE to confirm whether the character class ['‘’] (or equivalent) is present in both.
Add a test to Tests/test_backend.py (or a new file) that feeds "don't can't it's" through the parser tokenization path (not just stats.tokenize) and asserts whole-token output.
Without step 3, even if the regex is currently correct, the fix is unguarded and will not be caught if it regresses.
Tests/test_backend.py::test_tokenize_ascii_apostrophe_contractionverifies thatstats.py::_WORD_REtokenizes "don't can't it's" as whole tokens. The test was added to fix issue #37.backend/src/edcmbone/parser/turns_rounds.py::_WORD_REis a visually similar regex but has no equivalent regression test. It is possible that the same apostrophe fix was not applied to the parser regex, or that it was applied but is untested and could regress.Action required:
stats.py::_WORD_REandparser/turns_rounds.py::_WORD_REto confirm whether the character class['‘’](or equivalent) is present in both.stats.pyfor issue stats.py _WORD_RE misses ASCII apostrophe — contractions split on raw input #37.Tests/test_backend.py(or a new file) that feeds "don't can't it's" through the parser tokenization path (not juststats.tokenize) and asserts whole-token output.Without step 3, even if the regex is currently correct, the fix is unguarded and will not be caught if it regresses.