Skip to content

node:stream/iter: call transform() of a transform object with the object as this - #44131

Open
robobun wants to merge 2 commits into
mainfrom
robobun/6b0bb0ae/stream-iter-transform-receiver
Open

robobun wants to merge 2 commits into
mainfrom
robobun/6b0bb0ae/stream-iter-transform-receiver

Conversation

@robobun

@robobun robobun commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

  • pull(), pullSync(), pipeTo() and pipeToSync() of node:stream/iter call the transform function of a transform object with this undefined. Node.js v26.10.0 passes the object.
  • Cause: src/js/internal/streams/iter/pull.ts. The call sites (lines 448, 642, 644) pass only transform.transform. The helpers (lines 418, 539, 571) call it with no receiver.

Fix

  • applyStatefulSyncTransform, applyStatefulAsyncTransform and applyValidatedStatefulAsyncTransform take the transform object and call the function with .$call(receiver, ...).
  • Port of nodejs/node@649a4ab. Every entry point reaches these three helpers.
  • Verified on the debug build: the new case in test/js/node/stream/node-stream.test.js and test/js/node/test/parallel/test-stream-iter-transform-roundtrip.js. Both fail on bun 1.4.3. All 42 test-stream-iter-*.js files pass. Other cases of node-stream.test.js timed out (see Notes).

Background

  • A transform is a function (stateless) or an object with a transform() method (stateful).
  • Transform objects of node:zlib/iter carry an internal marker. pull() runs them through the third helper, which skips output validation.
  • .$call is the built-in Function.prototype.call. User code cannot replace it.
  • Considered .bind(transform): it allocates per transform object. Considered object.transform(...) in the helper: it reads the property later than Node.js. No design review ran.

Downsides

Notes

Repro, run with --experimental-stream-iter:

const { bytes, from, pull } = require("node:stream/iter");
const { compressGzip } = require("node:zlib/iter");
const descriptor = compressGzip();
const transform = descriptor.transform;
descriptor.transform = function (source, options) {
  console.log("this is the descriptor:", this === descriptor);
  return Reflect.apply(transform, this, [source, options]);
};
bytes(pull(from("receiver"), descriptor)).then(r => console.log("bytes:", r.byteLength > 0));
Runtime Output
Node.js v26.10.0 this is the descriptor: true
Node.js v26.3.0 this is the descriptor: false
bun 1.4.3 canary 367d939 this is the descriptor: false
this PR (debug build) this is the descriptor: true

The script of the new test prints the same JSON on Node.js v26.10.0 and on this PR. On Node.js v26.3.0 and bun 1.4.3 all four receivers are false.

What each test covers. The case in node-stream.test.js covers the plain async helper and the sync helper, through all four entry points. It does not cover the node:zlib/iter path. The upstream function testValidatedTransformReceiver in the vendored file covers that path. On bun 1.4.3 the vendored file exits 1 at test-stream-iter-transform-roundtrip.js:49:12. With this PR it exits 0.

The vendored file is not the v26.10.0 copy. It is the upstream file at commit 649a4aba93, byte for byte (sha256 7cafd1427d25621acad8f5d6b8e4dcfd0255d0c30a970c63a96da3b5811bb884). The v26.10.0 copy adds one more function, testZstdConcatenatedFrames, from nodejs/node@678ff3561a. That function fails on main: pull(from([first, Buffer.from('junk'), second]), decompressZstd()) raises ZSTD_error_prefix_unknown ("Unknown frame descriptor"), and Node.js v26.10.0 gives 'a'. That is a zstd decoder difference and this PR does not change it. Related: #33413.

Upstream tests this PR cannot add. The same upstream commit adds testPullStatefulTransformReceiver (test-stream-iter-pull-async.js) and testPullSyncStatefulTransformReceiver (test-stream-iter-pull-sync.js). Bun's copies of both files match v26.3.0. The upstream files at 649a4aba93 also test behaviour that bun does not have yet. I ran each function of those files on its own against bun 1.4.3, with a 3 second limit per function. In test-stream-iter-pull-sync.js 2 of 16 functions fail. In test-stream-iter-pull-async.js 9 of 32 fail, and 2 of those 9 hit the time limit and did not reach an assertion. One function in each file is the receiver test.

Found outside this task. Both upstream receiver tests yield the null flush signal from the transform. Bun throws ERR_INVALID_ARG_TYPE for that. Node.js skips it since nodejs/node@5c0578cbf7 ("stream: skip null output from stateful transforms"). It is tracked in #44132. Between v26.3.0 and v26.10.0, lib/internal/streams/iter/pull.js has 22 commits. This PR ports one. I did not compare the other 21 with bun's pull.ts.

Other runs on the debug build, all pass: test-fs-promises-file-handle-pull.js, test-fs-promises-file-handle-pullsync.js, test-fs-promises-file-handle-writer.js, test-quic-stream-iteration-pull.mjs, test-quic-stream-iteration-pipeto.mjs, test-quic-h3-post-filehandle.mjs, test-quic-stream-body-filehandle.mjs. bun lint and tsc --noEmit -p src/js/tsconfig.json pass.

Runs of node-stream.test.js on the debug build. The machine had a load average of 600 to 800 during all runs. The new case in its final form passed in 4 of 4 runs (1.7 to 2.9 s). A first version of the case also loaded node:zlib/iter in the same process. It timed out at 5000 ms in 2 of 2 runs on this machine. That is why the case leaves the node:zlib/iter path to the vendored file. The full file was not green in either of 2 runs. In the second run the failures were two existing cases, createReadStream > should emit readable on end and process.stdin > should pipe correctly. Run alone, each of them passed once and timed out once. Neither uses node:stream/iter.

Size. src/codegen/bundle-modules.ts --debug=OFF on the base and on this PR: InternalModuleRegistryConstants.bin is 2,630,120 and 2,630,231 bytes. The module internal/streams/iter/pull.js is 19,125 and 19,236 bytes. The size report of the CI build for 245fa9f shows +0.0 KB for every target.

A transform that is not callable at the call. transform can be a getter that returns a function while pull() validates its arguments and another value, for example 42, when the pipeline reads the property to call it. bun 1.4.3 throws TypeError: transform is not a function. (In 'transform(withFlushAsync(source), options)', 'transform' is 42). This PR throws TypeError: transform.@call is not a function. (In 'transform.@call(receiver, withFlushAsync(source), options)', 'transform.@call' is 42). Node.js v26.10.0 throws TypeError: Function.prototype.call was called on 42, which is a number and not a function. The class is the same and no error has a code. The message text changes, and it shows the internal name @call. pullSync() gives the same result. The debug build does not crash.

Not done. I made no release build of this PR. My own runs used the debug build only. No instruction count was taken. No design review and no separate self-review ran. I read the diff against the upstream commit line by line.


no test proof · iteration 0 · platform-specific test(s) that do not run on this machine, deferring to CI, which covers all platforms: test/js/node/stream/node-stream.test.js

…ect as this

pull(), pullSync(), pipeTo() and pipeToSync() read the transform function
from a transform object and called it with no receiver. The three helpers
that start a stateful transform now take the object and call the function
with it as `this`, as Node.js does since nodejs/node@649a4aba93.

test-stream-iter-transform-roundtrip.js is the upstream file at that commit.
@robobun

robobun commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator Author
Updated 8:49 AM PT - Sep 27th, 2026

❌ @robobun, your commit 5bc495f has 1 failures in Build #121259 (All Failures):


🧪   To try this PR locally:

bunx bun-pr 44131

That installs a local version of the PR into your bun-44131 executable, so you can run:

bun-44131 --bun

@robobun

robobun commented Sep 27, 2026 •

Copy link
Copy Markdown
Collaborator Author

Status

Ready for a maintainer. The change is complete and the tests of this PR pass in CI. CI is red for a reason that this PR does not cause.

Repro, with the script in the Notes of the PR body, run with --experimental-stream-iter:

Runtime this === descriptor
Node.js v26.10.0 true
Node.js v26.3.0 false
bun 1.4.3 canary 367d939 false
this PR (debug build) true

CI, build 121259:

  • 180 of 181 jobs passed. The run before it, build 121254, had the same result. A retry of the one failed job is enough for a green run.
  • The job that failed is debian 13 x64-asan - test-bun, shard 0. The test is test/js/bun/spawn/spawn.test.ts, "an idle reader stopped at the highwater mark does not keep the process alive".
  • The test expects an empty stderr. On that lane the child prints WARNING: ptrace appears to be blocked (is seccomp enabled?). LeakSanitizer may hang.
  • The test does not load internal/streams/iter/pull, the one module that this PR changes.
  • The same failure is on other branches. On 2026-09-27 it was on 23 of their last 45 builds (5 red, 18 passed on a retry). On 2026-09-28 it was on 14 of the last 21 builds that ran tests (4 red, 10 passed on a retry). PipeReader: label a chunk Eof only when the reader is at its end #44036 saw it on build 120963, before this PR was open.
  • The tests of this PR ran and passed: node-stream.test.js (x64-asan shard 2, x64 shard 13) and test-stream-iter-transform-roundtrip.js (x64-asan shard 13). I read the job logs for those three shards only.
  • The claude-find-issues check failed. It failed on 24 of its last 25 runs, on other branches too.

Limits of this PR that a reviewer must know:

@coderabbitai

coderabbitai Bot commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

Review in Change Stack →

Navigate logical layers of code changes, visualize relationships, and explore their blast radius.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Repository: oven-sh/bun/.coderabbit.yaml

Review profile: ASSERTIVE

Plan: Essentials

Run ID: 815f686a-9ef8-4e16-ba6e-379bfae16a00

📥 Commits

Reviewing files that changed from the base of the PR and between a4f1429 and 245fa9f.

📒 Files selected for processing (3)
  • src/js/internal/streams/iter/pull.ts
  • test/js/node/stream/node-stream.test.js
  • test/js/node/test/parallel/test-stream-iter-transform-roundtrip.js

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 6 remain after this review.


Walkthrough

Stateful sync and async transforms now run with their transform object as the receiver. Regression tests cover pull, pullSync, pipeTo, pipeToSync, and validated gzip transforms.

Changes

Stream transform receivers

Layer / File(s) Summary
Receiver invocation and regression tests
src/js/internal/streams/iter/pull.ts, test/js/node/stream/node-stream.test.js, test/js/node/test/parallel/test-stream-iter-transform-roundtrip.js
The sync and async stateful transform paths invoke transforms with their transform object as the receiver. Tests check receiver identity and verify the stream operations preserve the input or produce nonempty gzip output.

Suggested reviewers: cirospaciari

Priority: ⬇️ Low

Merge Risk: ⚪ Minimal · up to 5bc49

The receiver change appears ready to merge after normal checks; no actionable merge-blocking issue is identified.

🚥 Pre-merge checks | ✅ 4
✅ Passed checks (4 passed)
Check name Status Explanation
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title clearly and specifically describes the primary change: calling a transform object's transform() method with the object as this.
Description check ✅ Passed The description provides detailed problem, fix, verification results, limitations, test coverage, and known unrelated failures. It does not use the template headings exactly, but it includes the requi…

Comment @coderabbitai help to get the list of available commands.

@claude claude Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nothing blocking. The comments below are optional suggestions. There is no need to push a fix for them before merging.

Beyond the inline finding, I also checked the sibling call sites in pull.ts for the receiver change: all three stateful helpers and all five callers (sync, async, and the kValidatedTransform branch) were updated with the same parameter order, and the fused stateless paths take bare functions so no receiver applies there. The transform.transform re-read at each call site happens in the same loop iteration as the isTransformObject guard, so a non-callable cannot reach $call.

Extended reasoning...

The change threads the transform descriptor object as the this receiver through applyStatefulSyncTransform, applyStatefulAsyncTransform and applyValidatedStatefulAsyncTransform in src/js/internal/streams/iter/pull.ts, using $call per the src/js conventions, and adds a spawned -e test plus a ported upstream test function. It touches no security-sensitive surface. The inline finding is a pre-existing null-flush divergence from Node rather than a regression from this diff; approval is withheld only because verified findings are attached to the review.

Comment thread src/js/internal/streams/iter/pull.ts
@robobun

robobun commented Sep 27, 2026

Copy link
Copy Markdown
Collaborator Author

A correction to one point of the review summary, so that nobody relies on it. A value that is not callable can reach the call. transform can be a getter that returns a function while pull() validates its arguments, and another value when the pipeline reads the property to call it. I ran that case with 42:

Runtime Error
bun 1.4.3 (before this PR) TypeError: transform is not a function. (In 'transform(withFlushAsync(source), options)', 'transform' is 42)
this PR (debug build) TypeError: transform.@call is not a function. (In 'transform.@call(receiver, withFlushAsync(source), options)', 'transform.@call' is 42)
Node.js v26.10.0 TypeError: Function.prototype.call was called on 42, which is a number and not a function

The error is a TypeError with no code in all three, and the debug build does not crash. The message text changes with this PR: it now shows the internal name @call. pullSync() gives the same result. I added this to the Notes of the PR body.

The review thread about the null flush signal is answered there and tracked in #44132.

@robobun

robobun commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

@robobun wake up!!

@Jarred-Sumner

Copy link
Copy Markdown
Collaborator

@robobun wake up!!

@robobun

robobun commented Sep 28, 2026

Copy link
Copy Markdown
Collaborator Author

Awake. The state of this PR:

  • Change: complete. No review thread is open.
  • CI: 180 of 181 jobs passed on build 121259. The red job is debian 13 x64-asan - test-bun, shard 0, on test/js/bun/spawn/spawn.test.ts. The child prints a LeakSanitizer warning (ptrace appears to be blocked), and the test expects an empty stderr.
  • Cause of the red job: not this PR. The same failure is on 14 of the last 21 builds of other branches that ran tests (4 red, 10 passed on a retry). PipeReader: label a chunk Eof only when the reader is at its end #44036 saw it on build 120963, before this PR was open. The test does not load the module that this PR changes.
  • Tests of this PR in CI: node-stream.test.js passed on the x64-asan and x64 lanes. test-stream-iter-transform-roundtrip.js passed on the x64-asan lane. These are the lanes whose logs I read.
  • Branch: main is still at a4f1429, the base of this branch. No update is necessary.

To get a green run, retry the one failed job of build 121259. I used my one CI rerun on this PR already. If you want a full run, tell me and I push an empty commit.

Not in this PR:

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants