feat(storage): integrate replay buffer and reconnect into appendable upload worker - #6607
vsharonlynn wants to merge 7 commits into
Conversation
product-auto-label
Bot
added
the
api: storage
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request implements automatic reconnection and unacknowledged chunk replay for bidirectional streaming writes. It introduces a new ReplayBuffer module to retain unacknowledged data chunks and trim them upon server confirmation. The background Worker and Connector are updated to handle stream failures, perform reconnection, and replay any unpersisted chunks and pending flush/finalize requests. Comprehensive unit and integration tests are also added to verify the reconnection and replay behavior. I have no feedback to provide as there are no review comments.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #6607 +/- ##
==========================================
- Coverage 97.17% 97.17% -0.01%
==========================================
Files 328 329 +1
Lines 109802 110452 +650
==========================================
+ Hits 106702 107333 +631
- Misses 3100 3119 +19 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
5ead722 to
a12c61d
Compare
- Drop the unreachable expect() by reading the front chunk length before popping it, removing the redundant panic branch. - Relabel the // SAFETY: comment as // Invariant:, since Bytes::slice is safe Rust and the comment documents an algorithmic invariant, not an unsafe contract.
… bidi write worker
The replay buffer gates the worker's intent branch on `is_full()`, but the buffer is only drained by `ack()`, which requires a server response. Nothing in the append path requested one, so ~32 MiB of appends without an explicit flush stalled the worker permanently and blocked the caller inside `append()`. Inject `flush` + `state_lookup` on the append that crosses a high watermark, and re-establish the invariant after reconnect.
a12c61d to
a361574
Compare
1 participant
Issue #5716 .
This PR follows PR #6606 .