Skip to content

feat(runtime): node:stream + node:stream/web (Readable/Writable/Transform + WHATWG) (#1545)#5017

Merged
proggeramlug merged 1 commit into
mainfrom
node-streams-parity
Jun 11, 2026
Merged

feat(runtime): node:stream + node:stream/web (Readable/Writable/Transform + WHATWG) (#1545)#5017
proggeramlug merged 1 commit into
mainfrom
node-streams-parity

Conversation

@proggeramlug

Copy link
Copy Markdown
Contributor

node:stream + node:stream/web parity (#1545)

node:stream and node:stream/web were floored/skip-listed in the #812 matrix. This makes the in-repo test-parity/node-suite/stream suite (801 differential tests vs node v26.3.0) go 739 → 761 PASS (+22) with zero regressions.

What now works (root causes)

1. No more spurious Uncaught (in promise) on WHATWG stream abort/error/cancel (+13).
WritableStream.abort(), ReadableStreamDefaultController.error(), reader/writer cancel, pipeTo to a failing sink, and TransformStream cancel-propagation all reject the stream's internal closed/ready control promises. Those had no user-attached reaction, so Perry reported them as unhandled — Node marks them markPromiseAsHandled at creation. Added js_promise_mark_internally_handled (a persistent, hot-path-free set consulted only when non-empty) and applied it to every reader/writer/stream closed + ready promise at creation.

2. Async stream operations that observe a rejection by polling no longer leak it (+6).
Readable iterator-helpers (map/filter/every/some/…), pipeline(), and compose() drive an async callback by polling the returned promise's state and reading reason on rejection — without attaching on_rejected. They now mark_rejection_handled the promise they consumed, matching Node (the user still sees the error through the normal channel; only the duplicate internal leak is silenced).

3. Readable.read() (no size) returns the head chunk, not the whole buffer (+4).
Matches Node's howMuchToRead(NaN): a readable-event drain loop (while ((c = r.read()) !== null)), for await, unshift()-then-read(), and Readable.toWeb() now preserve chunk boundaries instead of getting one concatenated blob. Sized read(n) still spans chunks.

4. Async data-listener rejections are swallowed (+1).
Node's Readable neither captures an async data listener's rejection to error nor reports it as unhandled; Perry now marks it handled to match.

node-suite stream: 739 → 761 PASS / 0 CFAIL

The remaining 40 "diffs" are not Perry failures: 33 are tests where node itself exits non-zero (intentional TransformStream HWM-0 backpressure deadlocks → unsettled top-level await exit 13, or Readable.from(promise)-style ERR_INVALID_ARG_TYPE throws), which gen_feature_matrix.py scores as NODE-FAIL and excludes. The genuine remainder (subclassing class X extends Readable, an extra readable event in one objectMode case, a repipe-after-unpipe double-delivery) are left as follow-ups — each is a distinct, higher-risk area.

Differential repros (node v26 vs Perry — byte-identical)

r1_web_cancel.ts      first: chunk-1 / cancelled cleanly        (no Uncaught on cancel)
r2_pipeline_error.ts  pipeline err: boom                        (async generator throw → cb)
r3_chunk_boundaries.ts chunks: 3 => ["header\n","body-line-1\n","body-line-2\n"]

Regression guard

  • test-parity/node-suite/stream: 0 regressions (every previously-passing test still passes; pass-set diff is empty).
  • test262 built-ins language shard 0/16 (scripts/test262_subset.py, --jobs 40): pass=1873, diff=14, runtime-fail=92, compile-fail=5, parity 94.4%byte-identical before and after (the only core change, the unhandled-rejection set, is empty for any program that never touches streams, so non-stream behavior is provably unchanged).

Files

perry-runtime: promise/then.rs (+js_promise_mark_internally_handled), promise/mod.rs, lib.rs, node_stream_iter_helpers.rs, node_stream_pipeline.rs, node_stream_readable_read.rs, node_stream_event_emitter.rs. perry-stdlib: streams.rs (+internal_promise() helper), streams/{byob,transform,writable}.rs. (+134 / −48)

…us unhandled rejections, fix read() chunk boundaries (#1545)

node-suite stream: 739 -> 761 PASS (+22), zero regressions.

- WHATWG stream abort/error/cancel no longer emit a spurious
  'Uncaught (in promise)': reader/writer closed+ready promises are
  marked internally-handled at creation (Node's markPromiseAsHandled).
- Readable async iterator-helpers (map/filter/every/...), pipeline(),
  and compose() mark a callback rejection handled when they consume it
  by polling promise state instead of attaching a reaction.
- Readable.read() with no size returns the head chunk (one at a time)
  matching Node's howMuchToRead(NaN) instead of concatenating the whole
  buffer; preserves chunk boundaries for 'readable' drain loops,
  'for await', unshift(), and Readable.toWeb().
- Readable swallows an async 'data' listener rejection (captureRejections
  path) instead of surfacing it as unhandled.
@proggeramlug proggeramlug merged commit 7e9e206 into main Jun 11, 2026
11 of 13 checks passed
@proggeramlug proggeramlug deleted the node-streams-parity branch June 11, 2026 19:40
proggeramlug pushed a commit that referenced this pull request Jun 11, 2026
…d chunk

#5017 changed read() with no size to always return the head chunk. Node's
howMuchToRead(NaN) is flowing-conditional: it yields one chunk at a time only
while flowing; while paused it returns the entire buffer (state.length)
concatenated. The blanket change dropped everything past the first chunk for
paused readers — e.g. push("world"); unshift("hello "); read() returned
"hello " instead of "hello world" — regressing the #2484/#4386 unshift unit
tests (cargo-test gate red on main).

Restore the flowing branch: flowing -> head chunk (keeps #1545 boundary
behavior for for-await / 'readable' drain loops); paused -> drain-all concat.
Both node_stream unshift tests green again; full perry-runtime suite passes
(the unrelated date::test_full_year_setters_revive_invalid_date_only failure is
a pre-existing macOS-only timezone flake, green on CI Linux).
proggeramlug pushed a commit that referenced this pull request Jun 12, 2026
…1545 follow-up)

PR #5017 made read() with no size return only the head chunk of the
internal buffer unconditionally. Node's howMuchToRead(NaN) only does
that when the stream is FLOWING (state.flowing && state.length); a
paused stream drains the entire buffer and returns it as one value.

This broke two cargo-test unit tests on main (unshift prepend tests
expect 'hello world' from a paused read()) and with it every PR's
required cargo-test check.

Fix: keep the head-chunk path for flowing streams, restore the
full-drain concat path (drain_whole_buffer) for paused streams.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant