Skip to content

Enhance slop detection and update sloplist.json #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Louis27940 wants to merge 5 commits into
base: main
Choose a base branch
from
Open

Enhance slop detection and update sloplist.json #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5a7ed0a
Add new phrases to sloplist.json
Louis27940 Apr 4, 2026
f5da852
Add ending slop detection and update sloplist.json
Louis27940 Apr 4, 2026
e25be8e
more ending slop
Louis27940 Apr 4, 2026
8d76fea
Add 'ending_slop' category to ResponseExplorer component
Louis27940 Apr 5, 2026
c5475ba
Added different apostrophe
Louis27940 Apr 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 19 additions & 2 deletions data/sloplist.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,9 @@
"Happy to help!",
"I'd be happy to",
"Sure thing!",
"What a great question"
"What a great question",
"Let’s break it down",
"Let's break it down"
]
},

Expand All @@ -33,7 +35,8 @@
"What a fascinating question",
"I completely understand where you're coming from",
"That's a very thoughtful question",
"Great observation"
"Great observation",
"Great catch"
]
},

Expand Down Expand Up @@ -169,6 +172,20 @@
"that being said",
"with that in mind"
]
},

"ending_slop": {
"description": "Overused assistant-style closing phrases that often appear at the end of responses.",
"type": "behavioral",
"detection": "string_match_response_end",
"items": [
"I hope this helps",
"Does that make sense",
"Let me know if you need any further clarification",
"Feel free to ask if you have any more questions",
"If you want, I can help you",
"Feel free to ask"
Comment on lines +186 to +187

Copilot AI Apr 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ending_slop.items contains both "Feel free to ask if you have any more questions" and the shorter substring "Feel free to ask". With the current substring matching, the longer phrase will trigger both entries and double-count hits. Consider removing/rewriting the shorter entry for this category, or updating the detection logic to prefer the longest match / deduplicate overlaps.

Suggested change
"If you want, I can help you",
"Feel free to ask"
"If you want, I can help you"

Copilot uses AI. Check for mistakes.
]
}
}
}
13 changes: 13 additions & 0 deletions scripts/scoring.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,17 @@ export function detectPhraseLevel(text: string, items: string[]): SlopHit[] {
return hits;
}

export function detectEnding(text: string, items: string[]): SlopHit[] {
const tail = text.slice(-220).toLowerCase();
const hits: SlopHit[] = [];
Comment on lines +67 to +68

Copilot AI Apr 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The 220 tail length in text.slice(-220) is an unexplained magic number. Please pull this into a named constant (and ideally comment/justify it) so it’s clear how much of the response is considered “the end” and easy to tune later.

Copilot uses AI. Check for mistakes.
for (const item of items) {
if (tail.includes(item.toLowerCase())) {
Comment on lines +67 to +70

Copilot AI Apr 5, 2026

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

detectEnding is labeled/used as an end-of-response detector, but it currently uses tail.includes(...), which will flag phrases anywhere in the last 220 chars (not necessarily as a closing). This can create false positives and makes the implementation inconsistent with string_match_response_end semantics; consider trimming trailing whitespace/punctuation and using endsWith, or requiring the match index to be within a small window of the actual end.

Suggested change
const tail = text.slice(-220).toLowerCase();
const hits: SlopHit[] = [];
for (const item of items) {
if (tail.includes(item.toLowerCase())) {
const normalizeEnding = (value: string): string =>
value.toLowerCase().replace(/[\s.!?,;:'")\]]+$/g, "");
const tail = normalizeEnding(text.slice(-220));
const hits: SlopHit[] = [];
for (const item of items) {
const normalizedItem = normalizeEnding(item);
if (normalizedItem && tail.endsWith(normalizedItem)) {

Copilot uses AI. Check for mistakes.
hits.push({ type: "ending_slop", match: item });
}
}
return hits;
}

export function detectReframe(text: string): SlopHit[] {
const re = /not (?:just )?[\w\s]+, but (?:also )?[\w\s]+/gi;
return re.test(text) ? [{ type: "reframe_slop", match: "reframe pattern" }] : [];
Expand All @@ -84,6 +95,7 @@ type SlopList = {
validation_slop: { items: string[] };
word_level_slop: { items: string[] };
phrase_level_slop: { items: string[] };
ending_slop: { items: string[] };
[key: string]: unknown;
};
};
Expand All @@ -95,6 +107,7 @@ export function scoreResponse(text: string, slopList: SlopList): ScoredResponse
...detectValidation(text, categories.validation_slop.items),
...detectWordLevel(text, categories.word_level_slop.items),
...detectPhraseLevel(text, categories.phrase_level_slop.items),
...detectEnding(text, categories.ending_slop.items),
...detectReframe(text),
...detectEmDash(text),
...detectStructural(text),
Expand Down
2 changes: 1 addition & 1 deletion src/components/ResponseExplorer.jsx
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ export default function ResponseExplorer({ runId }) {
}, {})
const categories = [
'structural_slop', 'word_level_slop', 'em_dash_slop',
'phrase_level_slop', 'opener_slop', 'validation_slop', 'reframe_slop'
'phrase_level_slop', 'opener_slop', 'validation_slop', 'reframe_slop', 'ending_slop'
].filter(c => breakdown[c])
Comment thread
Louis27940 marked this conversation as resolved.

return (
Expand Down