Unicode range matched mistake.

[iDalink]

Hi. Regex [\u4e00-\u9fa5]+ seems match out of range. Input txt is Check (✓) the ways 你yo㸀u study English., Charactor ✓ `s unicode value is 0x2713, but it matched by the regex. Something get failed?

THX.

Code here:

#include <stdio.h>
#include <locale.h>
#include <tre/tre.h>

int main() {
setlocale(LC_ALL, "en_US.UTF-8");

regex_t preg;
regmatch_t pmatch[1];
int ret;

// 正则表达式：匹配连续的中文字符（Unicode 范围）
// const char *pattern = "([\xE4\xB8\x80-\xE9\xBE\xBF]+)"; // UTF-8 编码的汉字范围
// const char *pattern = "[^\u4e00-\u9fa5]+"; // 匹配中文字符的 Unicode 范围

const char *pattern = "[\u4e00-\u9fa5]+"; // 匹配中文字符的 Unicode 范围
const char *text = "Check (✓) the ways 你yo㸀u study English.";
// 编译正则表达式（无需 REG_UTF8 标志）
if (tre_regcomp(&preg, pattern, REG_EXTENDED) != 0) {
    char errbuf[256];
    tre_regerror(ret, &preg, errbuf, sizeof(errbuf));
    fprintf(stderr, "Compile err: %s\n", errbuf);
    return 1;
}

// 循环匹配所有中文字符
int offset = 0;
while (1) {
    ret = tre_regexec(&preg, text + offset, 1, pmatch, 0);
    if (ret != 0) break;

    // 输出匹配结果
    printf("Mached: ");
    for (int i = pmatch[0].rm_so; i < pmatch[0].rm_eo; i++) {
        putchar(text[offset + i]);
    }
    printf(" (Pos: %d-%d)\n", offset + pmatch[0].rm_so, offset + pmatch[0].rm_eo);

    offset += pmatch[0].rm_eo; // 更新偏移量
}

tre_regfree(&preg);
return 0;

}

Output here:
Mached: ✓ (Pos: 7-10)
Mached: 你 (Pos: 21-24)
Mached: 㸀 (Pos: 26-29)

[le0pard]

possible problem, that library is quietly falling back to byte-by-byte matching because the C runtime environment failed to set the UTF-8 locale correctly. So no fix for TRE, we can fix this code:

#include <stdio.h>
#include <locale.h>
#include <stdlib.h>
#include <tre/tre.h>

int main() {
    // Try cross-platform locale strings and assert success.
    if (!setlocale(LC_ALL, "en_US.UTF-8") && !setlocale(LC_ALL, ".UTF8")) {
        fprintf(stderr, "Compile err: Failed to set UTF-8 locale!\n");
        return 1;
    }

    // Ensure the C runtime acknowledges we are in multibyte mode.
    if (MB_CUR_MAX == 1) {
        fprintf(stderr, "Compile err: Environment does not support multibyte UTF-8.\n");
        return 1;
    }

    regex_t preg;
    regmatch_t pmatch[1];
    int ret;

    const char *pattern = "[\u4e00-\u9fa5]+"; 
    const char *text = "Check (✓) the ways 你yo㸀u study English.";
    
    if (tre_regcomp(&preg, pattern, REG_EXTENDED) != 0) {
        char errbuf[256];
        tre_regerror(ret, &preg, errbuf, sizeof(errbuf));
        fprintf(stderr, "Compile err: %s\n", errbuf);
        return 1;
    }

    int offset = 0;
    while (1) {
        ret = tre_regexec(&preg, text + offset, 1, pmatch, 0);
        if (ret != 0) break;

        printf("Matched: ");
        for (int i = pmatch[0].rm_so; i < pmatch[0].rm_eo; i++) {
            putchar(text[offset + i]);
        }
        printf(" (Pos: %d-%d)\n", offset + pmatch[0].rm_so, offset + pmatch[0].rm_eo);

        offset += pmatch[0].rm_eo; 
    }

    tre_regfree(&preg);
    return 0;
}

I hope this may help