perf(ios/chat): coalesce assistant stream tokens#3460
Conversation
Streaming applied every assistant chunk immediately: mutate the message then onChange() on EVERY token. Because messages is an observed array on an @observable ChatThread, each mutate reassigns messages[idx] and invalidates the whole message list — so a fast stream (tokens arriving faster than a frame) drove a per-token render and scroll storm. Introduce a small @mainactor StreamCoalescer that buffers chunk text and flushes on a ~50ms cadence (~20 UI updates/sec) instead of per token. All terminal paths (.done/.error, end of the send loop, and the resume loop) flush unconditionally, so the final text is always complete; a single coalescer instance is shared across the send + resume streams so a resumed turn keeps the same buffer. Verified: xcodebuild build (scheme CovenCave, iOS Simulator) succeeds.
Ensure a paused stream drains its final buffered assistant chunk within the coalescing interval rather than waiting for a later frame or terminal event. Co-authored-by: Val Alexander <68980965+BunsDev@users.noreply.github.com>
|
Follow-up pushed in Validated locally with |
|
Validation complete on the final PR head: all required CI checks passed (Frontend build, E2E, Rust, cross-environment, both sidecar-runtime jobs, and CodeQL); review-thread audit found no unresolved threads or review findings. This replacement is mergeable and ready for maintainer review. It preserves @BunsDev's original performance commit and includes the buffered-text recovery fix from #3455's review. |
2 participants
Supersedes #3455, whose OpenCoven-owned head branch cannot be updated by maintainers (
maintainer_can_modify: false; direct push returns HTTP 403). This branch retains the original performance commit by @BunsDev and adds the review correction.Behavior
Validation
pnpm typecheckpnpm check:tests-wiredpnpm test:mobile(81 files)The normal PR CI is enabled because this replacement targets
main.