Skip to content

fix(exec): decode captureTail streams with StringDecoder for split UTF-8 - #1313

Merged
SaulMoro merged 1 commit into
Tencent:mainfrom
hobostay:fix/capture-tail-utf8
Oct 23, 2026
Merged

SaulMoro merged 1 commit into
Tencent:mainfrom
hobostay:fix/capture-tail-utf8

Conversation

@hobostay

Copy link
Copy Markdown
Contributor

Summary

captureTail in src/utils/exec.ts decoded child process output chunks using chunk.toString('utf8'). When multi-byte UTF-8 sequences (such as Chinese characters, emojis, or accented characters in script output or error logs) straddled chunk boundaries across pipe reads, toString('utf8') immediately turned partial bytes into \uFFFD, corrupting the error tail. In addition, stdout and stderr shared the same unbuffered callback, allowing separate streams to interleave partial byte sequences.

captureTail now uses an independent StringDecoder('utf8') for stdout and stderr and flushes any buffered remainder when the tail reader is called, matching the approach in execCommand and clone.ts (#1281).

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature causing existing behavior to change)
  • Documentation only
  • Refactor / internal cleanup

Test Plan

  • npm run build — tsup built cleanly
  • npx tsc --noEmit — 0 type errors
  • npm run lint — oxlint passed with 0 errors and 0 warnings on 1102 files
  • npx vitest run src/__tests__/exec.test.ts — 11 passed (including 6 new unit tests for captureTail covering split UTF-8 across chunks, separate stream decoders, whitespace collapsing, length bounds, and idempotent reader calls)
  • npx vitest run src/__tests__/post-pull.test.ts src/__tests__/team-scripts.test.ts — 20 passed
  • git diff --check — clean whitespace

Real CLI Verification

Tested with the built CLI (dist/index.js) in an isolated temporary workspace configured with a postPull lifecycle script in teamai.yaml. The child script emitted a multi-byte Chinese character (标) split across two pipe writes followed by exit 1.

Before the fix (unbuffered chunk.toString('utf8')):

[DEBUG] postPull: exited 1 in 74ms (...) — 错误:会话\ufffd\ufffd\ufffd题 café 😀 失败

After the fix (using StringDecoder('utf8') in captureTail):

[DEBUG] postPull: exited 1 in 75ms (/private/tmp/teamai-self-buwQ2F/.teamai/fail-script.mjs) — 错误:会话标题 café 😀 失败

The split multi-byte characters and emojis are decoded completely intact with no replacement characters in the captured tail.

Related Issues

Follow-up to #1281 (addressing the captureTail follow-up in src/utils/exec.ts).

@github-actions

Copy link
Copy Markdown

No findings in the specified diff.

The PR description includes sufficient testing: a representative built-CLI run with split UTF-8 output and before/after results, satisfying the real-CLI verification requirement.

Review was read-only; I did not run, build, or install anything from the PR.

@SaulMoro SaulMoro left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was generated by AI during triage.

@hobostay Thanks, approving.

We ran teamai pull --silent with the built CLI on this head and on main 2401d047. The team's postPull script writes a Chinese message split in the middle of 标 across two writes:

  • main: the postPull line in debug.log has U+FFFD replacement characters.
  • this PR: the message is intact (错误:会话标题 café 😀 失败).

The four new split-UTF-8 cases fail on main and pass here. tsc, lint, and the exec and post-pull tests pass. Only pull --silent and the headless prePull/postDeploy paths reach captureTail with piped output. Plain pull inherits stdio.

Optional:

  • [P3 nit] The reader calls end() on the decoders. A read in the middle of a character would emit U+FFFD for the bytes held so far. Every current caller reads after close, so nothing hits it today.
  • [P3 nit] The body names a second cause, interleaving through a shared callback. The old code kept no state between chunks, so per-chunk decoding is the only defect. Wording only.

@SaulMoro
SaulMoro merged commit a6b1a9f into Tencent:main Oct 23, 2026
18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants