fix(stream): node v26 read() chunk-at-a-time, no spurious EOF readable, pipe dedup#5096
Conversation
…e, pipe dedup
Closes the largest real-gap cluster in test-parity/node-suite/stream by
matching node v26's internal/streams/readable byte-for-byte. Three distinct
root causes (6 previously-failing node-suite tests):
1. read() with no size returns the HEAD chunk, not the whole buffer.
node v26 howMuchToRead(NaN) takes a fast path: WITHOUT a string decoder it
always returns state.buffer[bufferIndex].length (one buffer at a time), even
when paused. Only a setEncoding (decoded) paused stream concatenates the
whole buffer into a single string. Perry drained the entire buffer whenever
paused, so multi-chunk paused read() over-concatenated.
Fixes: readable-mode-only, read-in-readable-listener, unshift-after-read,
iter-yields-buffer-no-encoding.
2. objectMode read() of the last buffered item no longer emits a final
'readable'. node never emits 'readable' just to hand the consumer a null at
EOF — once read() returns the last item from an ended stream it transitions
straight to 'end' (endReadable). The extra emit made a fixed-count consumer
observe a spurious read()===null pair.
Fixes: object-mode-read-returns-object.
3. pipe() destinations now consume-on-emit. A flowing destination (piped-into
PassThrough/Duplex) must drop each chunk from its own readable buffer when it
emits 'data' live; otherwise the chunk lingers and the destination's drain
microtask re-emits it, duplicating every piped chunk. pipeline() already
marks both ends via mark_live_pipe_consume_on_emit; pipe() now marks the
destination too.
Fixes: pipe/repipe-after-unpipe.
Also updates two perry-runtime unit tests in node_stream_tests_extra.rs that
asserted the pre-v26 drain-whole behavior (read() after unshift returning the
concatenated buffer). Verified against node v26: read() returns the unshifted
head chunk and leaves the remainder, so the tests now assert chunk-at-a-time.
Verification (node v26.3.0, macOS): the 6 target tests pass; a focused
232-test regression over the read/pipe/objectMode/flowing/transform/duplex/
consumers surface is clean (226 pass, 6 node-fail, 0 diffs, 0 new regressions);
pipe fan-out and chained-pipe cross-checks match node. cargo test -p
perry-runtime --lib green (the Date.prototype global-state test passes in
isolation, a pre-existing flake unrelated to this change).
Not addressed (separate codegen gap): subclass/extends-{readable,writable,
duplex,transform} still throw "Class extends value is not a constructor" —
user-defined `class X extends Readable` requires native-stream subclassing
support in codegen (synthesized super() initialising native stream state on
`this`, routing push()/_read to the user object), which is a much larger,
independent change.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
📝 WalkthroughWalkthroughThis PR refines Perry's readable stream implementation to align with Node v26 semantics. Buffered chunk draining in paused mode now requires both non-flowing state and active encoding. End-of-stream handling removes duplicate readable events. Pipe destination streams are explicitly marked for live consumption. Tests validate chunk-at-a-time read behavior. ChangesStream Read and Pipe Behavior
Estimated Code Review Effort🎯 3 (Moderate) | ⏱️ ~22 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
Summary
Closes the largest real-gap cluster in
test-parity/node-suite/streamby matching node v26'sinternal/streams/readablebyte-for-byte. Baseline (pristineorigin/main, node v26.3.0): 758 pass / 10 real fail (after excluding NODE-FAIL tests where node itself throws). This fixes 6 of the 10 via three distinct root causes; the remaining 4 are a single separate codegen gap (see below).Root causes & fixes
1.
read()with no size returns the head chunk, not the whole buffer.node v26
howMuchToRead(NaN)has a fast path: without a string decoder it always returnsstate.buffer[bufferIndex].length(one buffer at a time), even when paused. Only asetEncoding(decoded) paused stream concatenates the whole buffer into one string. Perry drained the entire buffer whenever paused.→ fixes
readable/readable-mode-only,readable/read-in-readable-listener,readable/unshift-after-read,readable/iter-yields-buffer-no-encoding2. objectMode
read()of the last item no longer emits a finalreadable.node never emits
readablejust to deliver anullat EOF — after the last item of an ended stream it goes straight toend(endReadable). The extra emit made a fixed-count consumer observe a spuriousread()===nullpair.→ fixes
readable/object-mode-read-returns-object3.
pipe()destinations now consume-on-emit.A flowing destination (piped-into PassThrough/Duplex) must drop each chunk from its own readable buffer when it emits
datalive; otherwise the chunk lingers and the destination's drain microtask re-emits it, duplicating every piped chunk.pipeline()already marks both ends viamark_live_pipe_consume_on_emit;pipe()now marks the destination too.→ fixes
pipe/repipe-after-unpipeAlso updates two
perry-runtimeunit tests (node_stream_tests_extra.rs) that asserted the pre-v26 drain-whole behavior; verified against node v26,read()afterunshiftreturns the unshifted head chunk and leaves the remainder.Verification (node v26.3.0, macOS)
cargo test -p perry-runtime --lib: green (theDate.prototypeglobal-state test passes in isolation — a pre-existing flake unrelated to this change).Not addressed — separate codegen gap
subclass/extends-{readable,writable,duplex,transform}still throwTypeError: Class extends value is not a constructor. User-definedclass X extends Readablerequires native-stream subclassing support in codegen (synthesizedsuper()initialising native stream state onthis, routingpush()/_readto the user object) — a much larger, independent change, not a stream-runtime bug.Summary by CodeRabbit
Bug Fixes
Tests