Skip to content

fix(ttsr): interrupt paragraph loops by near-duplicate frequency - #1763

Merged
code-yeongyu merged 5 commits into
mainfrom
fix/1330-near-duplicate-paragraph-frequency
Sep 16, 2026
Merged

code-yeongyu merged 5 commits into
mainfrom
fix/1330-near-duplicate-paragraph-frequency

Conversation

@code-yeongyu

@code-yeongyu code-yeongyu commented Sep 16, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the near-repeat gap #1330 describes, with a frequency rule instead of a looser match.

paragraph-repeat (#1326) compares paragraphs byte for byte over a 64-entry ring, so it misses two shapes. A model that restates the same step in different words every time never produces three identical hashes. And a cycle longer than the ring stays invisible even where it repeats verbatim.

A captured incident hit both at once: claude-fable-5-1 streamed narration for 12 minutes with zero tool calls, paraphrasing one action ("I'm assembling the final delivery ... — task complete." / "Both batches sent with photos and reaction cleared — done."), then settled into an exact 114-paragraph cycle. Replaying that text through the shipped collapseDetector returns no match on any mechanism. Nothing ended the generation; the user killed the session.

What this adds

detectors/collapse-near-duplicates.ts (near-duplicate-paragraphs), wired last in the collapse chain for text and thinking streams:

  • a paragraph is an echo when its normalized word set reaches 0.5 Jaccard against any of the last 32 eligible paragraphs (eligibility is the existing 64 chars / 24 word chars);
  • the detector fires only when at least 8 of the last 12 eligible paragraphs are echoes.

The density gate is what keeps ordinary prose out. #1330 warns that a looser match alone would interrupt a legitimate "processing item N" traversal; a rate gate answers that directly, because one similar pair, a callback to an earlier point, or a closing summary never reaches 8-of-12. Cycle length does not matter, so the 114-paragraph case is covered too. Paragraphs inside fenced code blocks are skipped and never enter the history.

Remediation reuses the existing collapse path: abort, truncate from the first echoed paragraph in the firing window (its anchor is kept), then the collapse nudge.

Calibration evidence

Replaying the shipped detector over the local session store - 12,499 real assistant text and thinking parts, 28.9 MB - fires twice, and both firings are known runaway generations from one 2026-09-03 session (38,193 chars, and 363,042 chars ending in stopReason: length). No other match.

input result
incident tail (sanitized fixture) fires after 1,935 chars
12,499 real assistant parts 2 matches, both known loops
distinct multi-sentence prose, 40 paragraphs silent
10 echoes spread across 40 paragraphs silent
64 KB healthy prefix silent
24 near-identical fenced code blocks silent

Tests

test/ttsr/detector-collapse-near-duplicates.test.ts replays the sanitized incident tail, pins chunk-boundary independence, and pins every negative above plus tool-stream exclusion and the hand-off to paragraph-repeat on byte-identical cycles.

Mutation proof: removing the one wiring line in collapse.ts fails exactly the three positive tests (incident replay, synthetic paraphrases, thinking stream); restoring it returns 9/9.

Fixture changes, and why they are not loosened assertions

Two synthetic generators varied only by a counter, so after digit normalization every paragraph carried the same tokens - healthy input for a byte-exact rule, a narration loop for a normalized one:

  • collapse-test-inputs.ts buildHealthyPrefix now composes varied vocabulary;
  • detector-collapse-paragraphs.test.ts narration() now emits seven lexically distinct steps.

Their assertions are untouched, so the exact-repeat contract still tests exact repetition. The digit-only-drift shape those fixtures used to have is the shape #1330 asks to catch, and it is now covered by the new test.

Real-CLI QA

A new senpi-qa mock-loop scenario, ttsr-near-duplicate-loop, streams fifteen paraphrases of one action (none byte-identical) from the local fake model server through the real CLI:

[PASS] CLI exits zero - code=0
[PASS] exactly two model turns (abort + one bounded recovery) - requests=2
[PASS] no paragraph repeats byte-exactly, so only the frequency rule can fire - distinct=15/15
[PASS] persisted aborted message is truncated before the streamed tail - stopReason=aborted chars=527/2446
[PASS] recovery request never replays the streamed tail - tailPresent=false
[PASS] collapse-repetition system-interrupt injected into the recovery request
[PASS] recovery answer returned
7/7 passed

The existing scenarios still pass on the same build: ttsr-paragraph-loop 6/6, ttsr-collapse 4/4.

Verification

Run on a separate macOS machine (bun 1.4.2) against origin/main 9e1f97f634:

  • bunx vitest run test/ttsr - 18 files, 321 tests passed
  • bun run check - exit 0
  • bun scripts/check-pr-changelog.mjs - PASS

Evidence for the incident itself is held locally; the in-tree fixture is the sanitized tail.

Fixes #1330

A captured incident streamed a 12-minute narration loop whose paragraphs were
paraphrases of one another, so the byte-exact paragraph-repeat mechanism never
counted three occurrences. The fixture is the sanitized incident tail; the
negatives pin that ordinary prose, a minority of echoes, fenced code and tool
argument streams stay untouched.
buildHealthyPrefix emitted one sentence template whose only varying token was a
counter, and the paragraph-repeat cycle did the same, so every paragraph carried
the same words. That is healthy input for a byte-exact rule and a narration loop
for a normalized one. Both generators now vary their wording; the assertions are
unchanged.
A paragraph counts as an echo when its normalized word set reaches 0.5 Jaccard
against any of the last 32 eligible paragraphs, and the detector fires only when
at least 8 of the last 12 eligible paragraphs are echoes. The density gate is
what keeps ordinary prose out: a repeated sentence, a callback or a closing
summary never reaches it. Fenced code blocks are skipped, and truncation keeps
the anchor paragraph and drops the echoed run.
Fifteen paraphrases of one action, none byte-identical, streamed from the local
fake model server. The scenario asserts the abort, the truncated persisted
message, the collapse-repetition interrupt in the recovery request, and the
recovered answer.
@code-yeongyu
code-yeongyu merged commit 76dfe17 into main Sep 16, 2026
26 checks passed
@code-yeongyu
code-yeongyu deleted the fix/1330-near-duplicate-paragraph-frequency branch September 16, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

TTSR paragraph-repeat detector misses near-repeat drift (exact hash match); needs normalized match gated by progress

1 participant