fix: Strip system references footer from conversation history (closes #146)

[vlad-ko]

Summary

Fixes the doubled references-block + 👍/👎 hint observed in production multi-turn threads (#146). The system appends its footer exactly once — but buildConversationHistory was feeding prior bot replies back to the model with their footers, so after a few turns the model learned the footer as part of the house answer format and emitted its own copy mid-answer, ahead of the genuine one.

• REFERENCES_FOOTER_MARKER exported from references.ts and used by both formatReferences and the new stripReferencesFooter — a round-trip test (strip(answer + format(refs)) === answer) makes the pair drift-proof
• buildConversationHistory strips from the marker onward on assistant turns only; user messages quoting footer-lookalike text pass through untouched; a footer-only bot message vanishes instead of becoming an empty turn
• The compaction transcript is cleaned transitively (built from the same history)
• Deliberately no prompt-text change ("never write a References section") — that would invalidate the behavior-eval cassettes for cosmetic benefit; the history strip removes the imitation source structurally

Testing

TDD: 8 new tests RED-first (round-trip drift guard, marker-in-output pairing, assistant-strip, user-passthrough, footer-only-skip). Full suite 960 passing, typecheck clean, and keyless behavior-eval replay verified 5/5 — no cassette drift.

Closes #146

🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes
  • Improved conversation history handling so assistant messages no longer include system-added references footers.
  • Cleaned-up footer text is now removed before message processing, preventing stray footer content from appearing in chat history.
  • Empty assistant messages that contain only a footer are now omitted, allowing surrounding user messages to flow together correctly.
  • Added coverage for footer stripping and related message-history edge cases.

[coderabbitai]

Warning

Review limit reached

@vlad-ko, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 52 minutes

Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available.
You're only billed for reviews past your plan's rate limits ($0.25/file).

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: a0be67d7-69c0-4d85-8554-f18ac7929bbe

📥 Commits

Reviewing files that changed from the base of the PR and between b137cfc and b4dcdab.

📒 Files selected for processing (2)

• src/lib/thread-filter.test.ts
• src/lib/thread-filter.ts

📝 Walkthrough

Walkthrough

Adds an exported REFERENCES_FOOTER_MARKER constant and a stripReferencesFooter function in references.ts, refactors formatReferences to use the shared marker, and updates buildConversationHistory in thread-filter.ts to strip footers from assistant-role message text before cleaning/truncating, with accompanying unit tests.

Changes

Footer Marker and Stripping

Layer / File(s)	Summary
Footer marker constant and stripReferencesFooter utility src/lib/references.ts, src/lib/references.test.ts	Exports REFERENCES_FOOTER_MARKER, refactors formatReferences to use it, adds stripReferencesFooter(text) that removes footer content from the marker onward, and adds tests for round-trip, marker presence, no-op, truncation, and footer-only cases.
Wire footer stripping into buildConversationHistory src/lib/thread-filter.ts, src/lib/thread-filter.test.ts	Imports stripReferencesFooter and reorders buildConversationHistory to determine role first, strip footers from assistant text before cleaning/truncating, and skip empty results; adds tests verifying assistant-only stripping and merging when footer-only turns are omitted.

Estimated code review effort: 2 (Simple) | ~12 minutes

Possibly related issues

• Model echoes references footer mid-answer in long threads #146: Implements the footer format/strip round-trip logic and assistant-only stripping behavior described in this issue.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly matches the main change: stripping the system references footer from conversation history.
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/footer-echo-history

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[qodo-code-review]

PR Summary by Qodo

Fix footer echo by stripping references block from assistant history

🐞 Bug fix 🧪 Tests 🕐 20-40 Minutes

AI Description

• Strip system references footer from prior assistant replies before rebuilding model context.
• Share an exported footer marker between formatter and stripper to prevent drift.
• Add regression tests covering assistant stripping, user passthrough, and footer-only turns.

Diagram

graph TD
  A{{"Slack thread messages"}} --> B["buildConversationHistory"] --> C["stripReferencesFooter"] --> D["cleanText + truncate"] --> E["LLM conversation context"]
  F["formatReferences"] --> G["REFERENCES_FOOTER_MARKER"]
  C --> G
  F --> H["Assistant reply (footer appended)"]
  subgraph Legend
    direction LR
    _ext{{"External input"}} ~~~ _fn["Function" ] ~~~ _const["Shared constant"]
  end

Loading

High-Level Assessment

The following are alternative approaches to this PR:

1. Prompt-only mitigation (instruct model not to emit references)

• ➕ No code changes to message history processing
• ➕ May reduce accidental footer imitation in some cases
• ➖ Does not remove the imitation source from context; model can still learn the pattern
• ➖ High risk of eval/cassette drift from prompt text changes
• ➖ Less reliable than structural sanitization

2. Heuristic regex stripping (no shared marker)

• ➕ Quick to implement without changing formatter output
• ➖ Brittle across formatting tweaks (easy to drift)
• ➖ Higher false-positive/false-negative risk vs a single authoritative marker

3. Store references out-of-band (structured message fields)

• ➕ Clean separation of answer content vs system-added footer
• ➕ Avoids any parsing/stripping
• ➖ Larger refactor across message persistence and rendering paths
• ➖ Not justified for this narrow regression fix

Recommendation: Keep the PR’s approach: strip the footer structurally when rebuilding assistant history, and share a single exported marker between formatting and stripping with a round-trip test. This is the smallest change that eliminates the imitation source reliably while remaining robust to future footer formatting edits.

Files changed (4) +106 / -5

Bug fix (2) +25 / -4

references.tsExport footer marker and add stripReferencesFooter helper +16/-1

Export footer marker and add stripReferencesFooter helper

• Introduces a shared REFERENCES_FOOTER_MARKER and updates formatReferences to use it. Adds stripReferencesFooter to remove the system-appended footer (marker through end) while preserving the original answer text.

src/lib/references.ts

thread-filter.tsStrip references footer from assistant turns before building history +9/-3

Strip references footer from assistant turns before building history

• Updates buildConversationHistory to run stripReferencesFooter on assistant messages prior to cleanText/truncation. Ensures previously appended system footer does not re-enter the model context and cause learned footer imitation in multi-turn threads.

src/lib/thread-filter.ts

Tests (2) +81 / -1

references.test.tsAdd drift-guard tests for references footer stripping +35/-1

Add drift-guard tests for references footer stripping

• Imports the new footer marker and stripping helper and adds a focused test suite. Validates format/strip round-tripping, marker presence, no-op behavior when absent, and full removal from marker to end (including the reaction hint).

src/lib/references.test.ts

thread-filter.test.tsTest conversation history footer stripping behavior in buildConversationHistory +46/-0

Test conversation history footer stripping behavior in buildConversationHistory

• Adds regression tests ensuring assistant turns have the references footer removed, user turns are not modified even if they contain footer-like text, and footer-only assistant messages are skipped rather than producing empty turns.

src/lib/thread-filter.test.ts

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@src/lib/thread-filter.ts`:
- Around line 88-96: `extractTranscriptTail` is still passing assistant replies
with the references footer into the classifier transcript, so update the
transcript-building logic to strip that footer for assistant turns the same way
`m.text` is handled in the main message path. Reuse `stripReferencesFooter`
inside `extractTranscriptTail` for assistant messages, alongside the existing
`@mention` cleanup, so the classifier only sees cleaned assistant text before
truncation and filtering.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ff4b1ce0-cc8e-41cc-b361-a8eab3753541

📥 Commits

Reviewing files that changed from the base of the PR and between d85e10f and b137cfc.

📒 Files selected for processing (4)

• src/lib/references.test.ts
• src/lib/references.ts
• src/lib/thread-filter.test.ts
• src/lib/thread-filter.ts

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
battle-mage	Ready	Preview, Comment	Jul 9, 2026 8:57pm