Skip to content

fix: keep AI transcript capture honest across write failures and pauses - #441

Merged
guan4tou2 merged 4 commits into
mainfrom
fix/tailer-per-message-durability
Oct 24, 2026
Merged

fix: keep AI transcript capture honest across write failures and pauses#441
guan4tou2 merged 4 commits into
mainfrom
fix/tailer-per-message-durability

Conversation

@guan4tou2

@guan4tou2 guan4tou2 commented Oct 24, 2026

Copy link
Copy Markdown
Owner

問題

AI transcript tailer（tailer-host.ts）有三條路徑，會讓已經觀測到的內容沒進到紀錄，或把暫停期間的內容留了下來：

  1. Per-message（OpenCode 這種一則訊息一個檔案的格式）：檔名在處理前就寫進索引。寫入失敗，或讀到還沒寫完的檔案時，這個檔案仍被標成已處理，之後再也不會被讀。
  2. JSONL 寫入失敗：emitTurn() 只把錯誤回報給 capture health，sidecar 照樣前進，失敗的 turn 不會再被讀。
  3. 暫停期間：不寫事件，但原始的 transcript 位元組仍持續寫進 sidecar，私人對話還是留在專案裡。

三種情況下，來源看起來都很正常。

變更

  • 寫入成功後才推進進度：per-message 的檔案在所有 turn 都寫入成功後才進索引；JSONL 有任何 turn 失敗，就把讀取狀態倒回、不推進 sidecar，下一輪重讀，重讀時已寫入的 turn 靠 uuid 去重。
  • 失敗 turn 的佇列：寫入失敗的 turn 依 uuid 保存，每一輪（不在暫停中時）優先重送。這涵蓋了沒有位元組可以重讀的情況，例如等待 parent 的 child、暫停前就失敗的 turn。
  • 有上限的重試：1s、2s、4s… 最多 5 次。只有計時器觸發的輪次才計次；計時器在暫停中觸發時順延、不計次。重試用完時兩條路徑處理方式一致：推進位置，並寫入 capture_gap（reason: write_failed），記下失敗的 turn 數和無法解析的檔案數。
  • 暫停時改寫等長的換行字元：不保存任何內容，sidecar 大小仍然等於讀取位置，所以恢復後不會重讀暫停期間的資料。跨越暫停的那一行也一併丟掉；這個狀態寫成 .pause-tail 標記檔，重啟後仍有效。sidecar 被 retention 清掉時不採信這個標記檔；證據包也不打包它。
  • logged tier（thinking turn）寫入失敗改成拋例外，和 chain tier 一致；原本回傳 null，看起來像「刻意不記錄」。
  • OpenCode 遇到不完整的訊息檔時拋例外，不再回傳空結果，所以不會被標成已處理。
  • 順帶修正：per-message 原本每一輪都會重讀、重新解析所有訊息檔。

審查

這個 PR 經過一次獨立的對抗式審查。前三個 commit 被找出 6 個缺陷，還有一個不成立的宣稱（OpenCode 寫到一半的檔案）。第 4 個 commit 116c1813 修正了全部問題，reviewer 的重現測試已改寫成正式的回歸測試。

驗證

  • 新增 3 個測試檔（per-message 9 項、暫停 7 項、JSONL 11 項），另外在 opencode-adapter.test.ts 加了 1 項。最初的新測試在修正前的程式上大多會失敗。
  • 突變檢查：9 個突變裡有 8 個會被測試抓到。剩下的那個是縱深防禦：重試計時器的清除和 stale session 檢查同時拿掉也無害，因為 catchUpSession() 找不到 session 時本來就會直接返回。
  • tailer、bundle、logged tier 相關測試 17 個檔案、161 項通過；四道檢查都通過。
  • 完整 npm test：16 個 shell/PTY 測試檔在本機失敗，失敗清單和修改前的基準完全相同，而且這些測試都沒有用到 tailer-host。以 CI 結果為準。

已知限制（修改前就存在）

  • 失敗 turn 的佇列只存在記憶體裡，重啟前還沒寫入的 child 會遺失。
  • 暫停是在「讀到」的時候判定，不是「寫入」的時候：暫停開始時還沒讀完的內容，會被當成暫停期間的內容。

合併注意

#437 也改過 catchUpJsonl 和 catchUpPerMessageDir，把 main 合進 #437 時預期會有衝突。

🤖 Generated with Claude Code

guan4tou2 and others added 4 commits October 24, 2026 16:36
@guan4tou2 @claude
fix: index a per-message file only after its events are written
9bba82d 
The per-message tailer (OpenCode's layout) appended each file's name to
its sidecar index before processing it. A write that failed, or a file
read before its writer finished and so failing to parse, was then marked
handled and never read again: a message missing from the record while
the source looked healthy.

A turn whose write throws is now counted on the session, processUnit
reports whether the unit fully landed, and the name is indexed only when
it did. A failed unit gets a bounded retry, since nothing may touch the
directory again. A paused file is still indexed unread, so resume does
not record the pause.

Handled names also join the in-memory set now. Before, every pass
re-read and re-parsed every message file the session had, with only the
turn uuid dedup keeping the events single.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@guan4tou2 @claude
fix: keep none of an AI transcript read while recording is paused
e1f72cb 
During a pause the JSONL tailer wrote no events but went on appending
the transcript's raw bytes to its sidecar, so a private conversation was
still kept in the project. Simply not appending would move the next read
back to the pause, since the sidecar's size is the read position.

Paused bytes are now replaced by newlines of the same length: nothing of
the content is kept, the size still marks how far the source was read,
and a restart's line alignment ends on a boundary. A line that began
during the pause and finishes after it is dropped and filled the same
way. The capture_gap description no longer says the raw data was kept.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@guan4tou2 @claude
fix: hold the JSONL sidecar back when a transcript turn is not written
1a630dd 
The JSONL tailer reads from where its sidecar ends and appends what it
read after processing. emitTurn reported a failed write only to capture
health, so the sidecar advanced past the turn and it was never read
again, with the source still looking healthy.

When any turn in a pass fails, the read state is put back and the same
bytes are read again on a bounded retry; turns that did land are skipped
by uuid then, and the counters a re-read would double are restored.
After the last retry the bytes are consumed and a capture_gap with
reason write_failed records how many turns are missing, so one turn the
store keeps refusing cannot hold back the rest of the session.

The pause gap is now recorded after the pass succeeds, so a rewind
cannot record it twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@guan4tou2 @claude
fix: close the gaps review found in the tailer durability fixes
116c181 
An independent review reproduced six defects in the three fixes before
this one, and one claim that did not hold:

- A child turn queued for its parent was lost when its flush failed:
  nothing was left to re-read. Turns whose write throws now wait in a
  per-session queue, retried first on every pass that is not paused.
- A compaction after a paused partial line kept pauseTail set and
  dropped the first line of the new content. The reset clears it.
- File events spent the retry budget in milliseconds. Only the retry
  timer's own passes count now, a pending retry keeps its schedule, a
  timer that fires while paused waits without spending an attempt, and
  re-arming clears the timer it replaces.
- A turn that failed before a pause was swallowed by the pause. The
  queue keeps it until recording resumes.
- A restart while a paused line was unfinished kept the rest of that
  line in the sidecar. pauseTail is now a marker beside the sidecar,
  ignored and removed if retention pruned the sidecar, and left out of
  evidence bundles.
- A failed logged-tier insert (thinking turns) returned null, which read
  as deliberately not recorded. It throws now, as a chained insert does.
- OpenCode returned an empty result for a message file still being
  written, so the host marked it handled. parseUnit throws instead.

Retry decisions now live in catchUpSession for both paths, and when the
retries are spent both give up the same way: consume, and write a
capture_gap with reason write_failed naming the failed turns and
unreadable units. A re-read no longer queues the same child twice.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 24, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 1c06deae-24d1-473f-8822-389e9c091020
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@guan4tou2
guan4tou2 merged commit 5bb1382 into main Oct 24, 2026
6 checks passed
@guan4tou2 guan4tou2 mentioned this pull request Oct 24, 2026
Merged
@guan4tou2
guan4tou2 deleted the fix/tailer-per-message-durability branch October 24, 2026 13:53
This was referenced Oct 24, 2026
Open
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@guan4tou2