Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion src/languages/ru/profanity/huy.json
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
"id": "ru.obscene.huy.base",
"category": "OBSCENE_MAT",
"severity": "high",
"source": "ху[йяиёею]",
"source": "(?:хуй|хуя|хуи|хуё|хуе|хую)",

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restore reviewed хуей/ху-ей loose matches

With this source split into fixed alternatives, stretch: true can only repeat the selected final letter, so inputs like хуей or ху-ей now only produce a хуе prefix match; the loose boundary check then rejects it because the same token still has a trailing й. These are existing reviewed loose corpus cases in tests/loose-corpus.spec.ts that should be fully masked, so this change creates a bypass while fixing the following-conjunction case.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Stop absorbing same-letter following words

Because each explicit alternative is still stretched, a following standalone word that starts with the same final letter is consumed as another repeat of that final atom. For example, привет хуи и мир still gets a loose range over хуи и rather than just хуи (and хуя я ... has the same shape), so the range leak this patch is meant to fix remains for base variants whose last letter matches the next word.

Useful? React with 👍 / 👎.

"match": {
"strict": {},
"loose": {
Expand Down
39 changes: 39 additions & 0 deletions tests/loose-corpus.spec.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import { describe, expect, it } from "vitest";
import { mergeRanges } from "@textfilters/core";

import { createProfanityFilter, filter } from "../src";

Expand Down Expand Up @@ -188,4 +189,42 @@ describe("loose corpus", () => {
"********",
);
});

it("keeps Russian obscene root ranges from absorbing a following conjunction", () => {
const input = "привет хуй и мир";
const ranges = filter.analyze(input).map(([start, end]) => [start, end]);

expect(ranges).toContainEqual([7, 10]);
expect(ranges).not.toContainEqual([7, 12]);
expect(ranges.every(([, end]) => end <= 10)).toBe(true);
expect(filter.censor(input)).toBe("привет *** и мир");
});

it("keeps Telegram HTML spoiler output limited to the obscene token", () => {
const input = "💬 привет хуй и мир <script>";

expect(renderTelegramSpoilers(input)).toContain(
"💬 привет <tg-spoiler>хуй</tg-spoiler> и мир &lt;script&gt;",
);
});
});

const renderTelegramSpoilers = (input: string): string => {
const ranges = mergeRanges(filter.analyze(input));
let output = "";
let offset = 0;

for (const [start, end] of ranges) {
output += escapeTelegramHtml(input.slice(offset, start));
output += `<tg-spoiler>${escapeTelegramHtml(input.slice(start, end))}</tg-spoiler>`;
offset = end;
}

return output + escapeTelegramHtml(input.slice(offset));
};

const escapeTelegramHtml = (input: string): string =>
input
.replaceAll("&", "&amp;")
.replaceAll("<", "&lt;")
.replaceAll(">", "&gt;");