Repository navigation
DecompressionStream("zstd"): decode concatenated frames instead of rejecting as trailing junk - #36021
Conversation
…cting as trailing junk
A zstd stream is one or more concatenated frames (RFC 8878 3.1); pzstd
output is multi-frame by construction, as is `cat a.zst b.zst`.
DecompressionStream("zstd") threw ERR_TRAILING_JUNK_AFTER_STREAM_END at
the second frame because the trailing-junk check added for deflate/brotli
treats any input remaining after a frame boundary as junk.
When the stream is zstd-decompress and opted in via rejectGarbageAfterEnd
(only DecompressionStream does), re-enter the write loop with the
remaining input so the native decoder starts the next frame. Actual
trailing garbage then surfaces as ZSTD_error_prefix_unknown from zstd
itself. The plain node:zlib zstd stream keeps stopping at the first
frame to match Node.js.
WalkthroughChangesZstandard decompression now processes concatenated and skippable frames while rejecting trailing garbage. Tests cover frame boundaries, split writes, large concatenations, and expected errors. Zstandard decompression
Suggested reviewers: 🚥 Pre-merge checks | ✅ 4✅ Passed checks (4 passed)
Comment |
|
Updated 12:45 AM PT - Jul 27th, 2026
❌ @robobun, your commit f5a1ac9 has 1 failures in
🧪 To try this PR locally: bunx bun-pr 36021That installs a local version of the PR into your bun-36021 --bun |
![github-actions[bot]](https://avatars.githubusercontent.com/in/15368?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Additional findings (outside current diff — PR may have been updated during review):
-
🟡
test/js/web/streams/compression.test.ts:277— The bare.rejects.toThrow()will pass on any rejection, so a future regression that rejects for the wrong reason (or reverts toERR_TRAILING_JUNK_AFTER_STREAM_END) would still go green. Since the PR description states garbage now surfaces asZSTD_error_prefix_unknownwrapped in aTypeError, consider asserting that — e.g..rejects.toThrow(TypeError)plus a check thate.code(or message) containsprefix_unknown.Extended reasoning...
What the issue is
The new test
rejects trailing garbage after a zstd frameat test/js/web/streams/compression.test.ts:277 ends with:await expect(read).rejects.toThrow();
There is no error class, code, or message argument. REVIEW.md's test rules explicitly call this out under "Every assertion must be able to fail, and assert the strongest invariant":
specific error class/code/message (never bare
toThrow())Why this matters here
This test exists specifically to prove that the new multi-frame handling in
processCallbackdidn't accidentally start accepting trailing garbage. But the assertion only proves that something rejected. Any of the following futures would still pass:- A regression re-introduces the old
ERR_TRAILING_JUNK_AFTER_STREAM_ENDpath (i.e., the_continueOnFrameEndgate stops firing) — the test would happily accept that error even though it means multi-frame streams are broken again for a different-shaped input. - An unrelated stream-setup bug causes
.text()to reject before the zstd decoder ever sees the bytes. - The native decoder crashes with a different error class entirely.
The PR description itself states the intended behavior precisely: "actual garbage surfaces as
ZSTD_error_prefix_unknownfrom zstd itself rather thanERR_TRAILING_JUNK_AFTER_STREAM_END". That's the invariant the test should pin.Step-by-step proof
withJunkis one valid frame followed by0xde 0xad 0xbe 0xef.- With this PR,
processCallbackseesavailInAfter > 0 && _continueOnFrameEnd, re-entershandle.write, and zstd fails on the 4-byte prefix withZSTD_error_prefix_unknown.zlibOnErrordestroys the stream, andDecompressionStreamsurfaces it as aTypeErrorwith the zstd code attached (same wrapping the neighboring brotli test at line ~390 asserts on). await expect(read).rejects.toThrow()matches — but it would equally match if step 2 were replaced by any thrown value.
Suggested fix
Match the pattern already used by the brotli-decoder-error test in this same file — assert the class and the code:
await expect(read).rejects.toThrow(TypeError); await expect(read).rejects.toMatchObject({ code: expect.stringContaining("prefix_unknown") });
(or a try/catch asserting
e instanceof TypeErrorande.code/e.messagementionsprefix_unknown, whichever reads cleaner). Matching a substring rather than the exact code string keeps it robust to how the native layer prefixes zstd error names.Severity
This is a test-quality nit per the repo guidelines — the runtime behavior is correct and the
.rejectsis properly awaited so the assertion can fail; it just isn't asserting the strongest invariant available. - A regression re-introduces the old
|
Addressed in 0c375bf: the trailing-garbage test now asserts |
There was a problem hiding this comment.
Beyond the inline nit, I also checked the re-entry loop for non-termination — ZSTD_decompressStream always consumes input, fills output, or errors when availInAfter > 0, so continueNextFrame cannot spin. Short (<4-byte) trailing garbage was also checked — zstd buffers partial magic and errors on the finish flush rather than silently accepting it.
Extended reasoning...
The change is small and correctly gated (_continueOnFrameEnd only fires when rejectGarbageAfterEnd is set, so createZstdDecompress keeps Node's stop-at-first-frame behavior). The native do_work in NativeZstd.rs calls ZSTD_decompressStream once per write, and on re-entry with leftover input it will either decode the next frame, buffer a partial header (driving availInAfter to 0), or error — no path leaves input unconsumed with output space and no error, so the loop terminates. Leaving human review for the inline nit (trailing-garbage error class changed from TypeError to plain Error) and the layering question of whether multi-frame handling belongs in the native do_work alongside gunzip's multi-member loop.
Peek the 4 bytes following a completed frame: the zstd frame magic 0xFD2FB528 or a skippable-frame magic 0x184D2A5? means another frame follows, so call ZSTD_decompressStream again (mirrors the GUNZIP multi-member loop). A 1-3 byte tail that is a prefix of either magic is fed through so a header split across chunks decodes; anything else is left unconsumed so the JS layer's ERR_TRAILING_JUNK_AFTER_STREAM_END check still fires for DecompressionStream and createZstdDecompress keeps silently ignoring trailing data like Node.js. Drops the JS-side _continueOnFrameEnd gate; the peek makes it unnecessary. Adds coverage for a leading/trailing skippable frame.
There was a problem hiding this comment.
Beyond the inline nit: my earlier TypeError-regression concern is resolved by the native magic-peek redesign — non-magic trailing bytes are left unconsumed, so ERR_TRAILING_JUNK_AFTER_STREAM_END (a TypeError) still fires, and createZstdDecompress still silently discards them since _rejectGarbageAfterEnd is unset. Also checked the while ret == 0 loop for non-termination — each iteration either advances input.pos (libzstd buffers partial headers) or fills output, so it's bounded — and the unsafe slice in next_input_is_frame, where avail > 0 guarantees pos < size within the caller-kept-alive set_buffers region.
Extended reasoning...
The author reworked the fix after my previous review: instead of a JS-side _continueOnFrameEnd flag that fed all leftover bytes back to libzstd (which changed the trailing-garbage error from TypeError/ERR_TRAILING_JUNK_AFTER_STREAM_END to plain Error/ZSTD_error_prefix_unknown), the loop now lives in native do_work and only re-enters when the next bytes are a zstd/skippable magic or magic prefix. That mirrors NativeZlib::do_work_inflate's GUNZIP multi-member loop and preserves both the DecompressionStream trailing-junk TypeError and the Node-compat silent-discard path. The remaining nit (1-3 trailing bytes matching a magic prefix) is narrow and rooted in a pre-existing gap; deferring the approve/don't-approve call to a human given the unsafe pointer arithmetic on untrusted decompression input.
Jarred-Sumner
left a comment
Jarred-Sumner
left a comment
There was a problem hiding this comment.
Can we simplify? Do we need to use unsafe?
…, one comparison path
|
Simplified in 021aa84: the peek now copies up to 4 bytes into a stack The const ZSTD_MAGICNUMBER: [u8; 4] = 0xFD2FB528u32.to_le_bytes();
const ZSTD_MAGIC_SKIPPABLE: [u8; 4] = 0x184D2A50u32.to_le_bytes();
fn next_input_is_frame(&self) -> bool {
let n = (self.input.size - self.input.pos).min(4);
if n == 0 {
return false;
}
let mut head = [0u8; 4];
// SAFETY: input.src[pos..pos+n] lies within the slice installed by set_buffers.
unsafe {
ptr::copy_nonoverlapping(
self.input.src.cast::<u8>().add(self.input.pos),
head.as_mut_ptr(),
n,
);
}
head[..n] == Self::ZSTD_MAGICNUMBER[..n]
|| (head[0] & 0xF0 == Self::ZSTD_MAGIC_SKIPPABLE[0]
&& head[1..n] == Self::ZSTD_MAGIC_SKIPPABLE[1..n])
}libzstd does ship |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/web/streams/compression.test.ts`:
- Around line 266-278: Extend the zstd decompression tests around the existing
leading-skippable-frame case with a stream containing a zstd frame, a skippable
frame, and a second zstd frame. Split the skippable-frame magic across writes
after 1, 2, and 3 magic bytes, then assert the decompressed output includes both
payloads, covering the partial-prefix path in next_input_is_frame().
- Around line 234-246: Update the “decompresses a multi-frame zstd stream split
across writes” test to start consuming ds.readable and await both writer.write
operations and writer.close, ensuring any writable-side failure rejects the test
deterministically while preserving the existing split-boundary scenario.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 5172bef6-f29f-40af-bd4a-5d543346b4a9
📒 Files selected for processing (2)
src/runtime/node/zlib/NativeZstd.rstest/js/web/streams/compression.test.ts
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@test/js/web/streams/compression.test.ts`:
- Around line 241-261: Flatten the offset loop in the zstd multi-frame test into
the test.each matrix so each combination of next-frame shape and offset is an
independent, named test case. Add offsets 1, 2, and 3 as matrix parameters, use
the selected offset when computing split, and remove the sequential for-loop
while preserving the existing setup and expected-output assertions.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: ASSERTIVE
Plan: Pro
Run ID: 11b2b915-3ad3-4e26-9ca0-1554e5266000
📒 Files selected for processing (1)
test/js/web/streams/compression.test.ts
There was a problem hiding this comment.
LGTM — the multi-frame loop mirrors NativeZlib::do_work_inflate's GUNZIP member loop, and all prior feedback has been addressed.
What was reviewed:
next_input_is_framebounds: the 1–4 byte raw-pointer copy stays within[input.src+pos .. input.src+size)as installed byset_buffers; the buffer is rooted byCompressionStream::writefor the duration.- Re-drive when output fills at a frame boundary:
output.pos < output.sizegates the loop, soavailOutAfter == 0still hands control back toprocessCallback/processChunkSyncfor a fresh output buffer. - Skippable-magic match:
head[0] & 0xF0 == 0x50covers0x184D2A50..5F; the leading/trailing skippable-frame test exercises the loop across three consecutiveret == 0returns. zlib.zstdDecompressSyncnow decodes all frames (diverging from Node 26.3.0's first-frame-only behavior toward RFC 8878); pinned by the new test inzlib.test.js.
Extended reasoning...
Overview
Changes NativeZstd::do_work's ZSTD_DECOMPRESS arm to loop while ZSTD_decompressStream returns 0 (frame complete), the output buffer has room, and the unconsumed input begins with a zstd frame magic (0xFD2FB528) or skippable-frame magic (0x184D2A5?) — or a 1–3 byte prefix of either, so a header split across writes still decodes. Non-matching bytes are left unconsumed so processCallback's existing ERR_TRAILING_JUNK_AFTER_STREAM_END check still fires. Adds six DecompressionStream("zstd") tests (single-chunk multi-frame, split-header at offsets 1/2/3 × zstd/skippable next-frame, 64 frames spanning multiple output chunks, leading+trailing skippable frames, and trailing garbage still rejected) plus one zlib.zstdDecompressSync multi-frame test.
Security risks
None identified. Input is untrusted compressed data, but the new code only peeks up to 4 bytes within the caller-supplied input slice whose bounds libzstd just validated (input.pos <= input.size), then hands the same pointers back to ZSTD_decompressStream. No new allocation sizing or length arithmetic on untrusted values. The loop is bounded by forward progress: each iteration either advances input.pos, fills the output buffer, or returns non-zero, so it cannot spin.
Level of scrutiny
Medium. The single unsafe block is a bounded copy_nonoverlapping from a pointer libzstd just read through, into a stack [u8; 4]. The control-flow change directly mirrors the existing GUNZIP multi-member loop at NativeZlib.rs:577-588, so the interaction with processCallback/processChunkSync's re-drive logic is already exercised for gzip. I traced the availOutAfter == 0 re-drive path in both processCallback and processChunkSync to confirm output-buffer-full at a frame boundary correctly resumes on the next call.
Other factors
All three of my prior inline comments and both CodeRabbit findings are resolved: comments were condensed, writer ops are awaited, split-header coverage was extended to skippable frames at offsets 1/2/3, and a zstdDecompressSync multi-frame test now pins the node:zlib behavior change. The remaining known edge case — 1–3 trailing garbage bytes that coincidentally prefix a magic are consumed rather than rejected — was acknowledged as the necessary cost of split-header support and is closed by the pre-existing truncated-frame-on-finish-flush work in #33413. The intentional divergence from Node 26.3.0 (which returns only the first frame) moves zlib.zstdDecompressSync toward RFC 8878 conformance and matches Bun.zstdDecompressSync, the fetch Content-Encoding: zstd decoder, and gunzip's multi-member handling; the old behavior was silent data loss.
|
CI on f5a1ac9: 193/196 jobs passed. The only red lane is |
…jecting as trailing junk (#36021) ## Repro ```js import zlib from "node:zlib"; const f1 = zlib.zstdCompressSync(Buffer.from("first frame\n")); const f2 = zlib.zstdCompressSync(Buffer.from("second frame\n")); const cat = Buffer.concat([f1, f2]); // valid 2-frame stream (cat a.zst b.zst / pzstd) await new Response(new Blob([cat]).stream() .pipeThrough(new DecompressionStream("zstd"))).text(); // TypeError: Trailing junk found after the end of the compressed stream // code: "ERR_TRAILING_JUNK_AFTER_STREAM_END" Bun.zstdDecompressSync(cat); // "first frame\nsecond frame\n" (correct) ``` A zstd stream is one or more concatenated frames (RFC 8878 section 3.1); `zstd -d`, pzstd output (multi-frame by construction), `Bun.zstdDecompressSync`, and the `fetch()` `Content-Encoding: zstd` decoder all decode every frame. Only `DecompressionStream("zstd")` rejects the second frame as trailing junk, so a real pzstd file is unreadable through it while every other zstd decoder in the process accepts it. A leading skippable frame (magic `0x184D2A50..5F`) is rejected the same way. ## Cause The native zstd `do_work()` calls `ZSTD_decompressStream` once per write; libzstd stops at every frame boundary (returns 0 with the next frame unconsumed), so `processCallback` sees `availInAfter > 0` with output room and, because `DecompressionStream` passes `rejectGarbageAfterEnd: true`, raises `ERR_TRAILING_JUNK_AFTER_STREAM_END`. `Gunzip` avoids this because `NativeZlib::do_work_inflate` already loops on multi-member input. ## Fix In `NativeZstd::do_work`, after `ZSTD_decompressStream` returns 0 with input and output room remaining, peek the next four bytes: if they are the frame magic `0xFD2FB528` or a skippable-frame magic `0x184D2A5?` (or a prefix of either when fewer than four remain, so a header split across chunks decodes), call `ZSTD_decompressStream` again. libzstd resets the DCtx to start-of-frame itself, so no explicit reset is needed. Bytes that are not a magic or magic prefix are left unconsumed so the existing `ERR_TRAILING_JUNK_AFTER_STREAM_END` check still fires for genuine garbage, and `createZstdDecompress` still silently ignores trailing non-frame data the way Node.js does. `src/js/node/zlib.ts` is unchanged. ## Verification New tests in `test/js/web/streams/compression.test.ts` cover a 2-frame stream written as one chunk, a 2-frame stream split mid-header across two writes, 64 frames totalling more than one output chunk, a stream with leading and trailing skippable frames, and trailing non-frame garbage still rejected with `ERR_TRAILING_JUNK_AFTER_STREAM_END`. The four multi-frame tests fail on `main` with that error and pass with this change; the Node-compat `createZstdDecompress` trailing-data tests in `test/js/node/zlib/zlib.test.js` still pass. <!-- robobun:evidence:begin --> --- **[review]** gate passed · iteration 2 · 3 files touched <details><summary>fails on main (without fix)</summary> ```console ASAN without fix: 6 failed, 2 skipped $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/node/zlib/zlib.test.js test/js/web/streams/compression.test.ts bun test v1.4.0 (f5a1ac9) test/js/web/streams/compression.test.ts: (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [302.30ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [113.72ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [341.86ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [35.81ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [58.35ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [223.26ms] TypeError: Trailing junk found after the end of the compressed stream code: "ERR_TRAILING_JUNK_AFTER_STREAM_END" at unknown:1:1 at processCallback (node:zlib:431:37) (fail) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream [45.46ms] TypeError: Trailing junk found afte ... (truncated) release without fix: 2 skipped bun test v1.4.0-canary.1 (f62fd6a) test/js/web/streams/compression.test.ts: (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [5.42ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [2.23ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [6.12ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [2.48ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [1.10ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [4.71ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream [0.51ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream split across writes (next = zstd frame) [1.23ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream split across writes (next = skippable frame) [2.40ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses many concatenated zstd frames larger than one output ... (truncated) ``` </details> <details><summary>passes on PR (with fix)</summary> ```console ASAN with fix: 2 skipped $ BUN_DEBUG_QUIET_LOGS=1 bun scripts/build.ts --profile=debug --quiet test "--reporter=junit" "--reporter-outfile=/tmp/mechgate.xml" test/js/node/zlib/zlib.test.js test/js/web/streams/compression.test.ts bun test v1.4.0 (f5a1ac9) test/js/web/streams/compression.test.ts: (pass) CompressionStream and DecompressionStream > brotli > compresses data with brotli [377.58ms] (pass) CompressionStream and DecompressionStream > brotli > decompresses brotli data [115.46ms] (pass) CompressionStream and DecompressionStream > brotli > round-trip compression with brotli [324.69ms] (pass) CompressionStream and DecompressionStream > zstd > compresses data with zstd [44.76ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses zstd data [62.49ms] (pass) CompressionStream and DecompressionStream > zstd > round-trip compression with zstd [186.59ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream [22.91ms] (pass) CompressionStream and DecompressionStream > zstd > decompresses a multi-frame zstd stream split across writes (next = zstd frame) [64.47ms] (pass) CompressionStream and DecompressionStream > zstd > decompress ... (truncated) release with fix: 2 skipped $ bun scripts/build.ts --profile=release [configured] bun-profile → bun (stripped) in 766ms (unchanged) ninja: Entering directory `/workspace/bun/build/release' [1/6] gen generated_host_exports.rs generated_host_exports.rs: 94 exports (host=3, lazy=10, generic=81, rust=0); 240 extern-C blocks audited [1/6] cargo bun_bin → libbun_rust.a (--target x86_64-unknown-linux-gnu) nightly-2026-07-20-x86_64-unknown-linux-gnu unchanged - rustc 1.99.0-nightly (9f36de775 2026-07-19) �[1m�[92m Compiling�[0m bun_core v0.0.0 (/workspace/bun/src/bun_core) �[1m�[92m Compiling�[0m bun_errno v0.0.0 (/workspace/bun/src/errno) �[1m�[92m Compiling�[0m bun_ptr v0.0.0 (/workspace/bun/src/ptr) �[1m�[92m Compiling�[0m bun_boringssl_sys v0.0.0 (/workspace/bun/src/boringssl_sys) �[1m�[92m Compiling�[0m bun_safety v0.0.0 (/workspace/bun/src/safety) �[1m�[92m Compiling�[0m bun_zlib_sys v0.0.0 (/workspace/bun/src/zlib_sys) �[1m�[92m Compiling�[0m bun_cares_sys v0.0.0 (/workspace/bun/src/cares_sys) �[1m�[92m Compiling�[0m bun_zstd v0.0.0 (/workspace/bun/src/zstd) �[1m�[92m Compiling�[0m bun_picohttp v0.0.0 (/workspace/bun/src/picohttp) �[1m�[92m Compiling�[0m bun_output v ... (truncated) ``` </details> <details><summary>diff hotspot</summary> ``` src/runtime/node/zlib/NativeZstd.rs | 56 +++++++++++++++++++++---- test/js/node/zlib/zlib.test.js | 6 +++ test/js/web/streams/compression.test.ts | 73 +++++++++++++++++++++++++++++++++ 3 files changed, 127 insertions(+), 8 deletions(-) ``` </details> **gate history** · 4 passed · 0 rejected · iteration 2 <details><summary>evidence per changed file</summary> ``` file reads edits tests src/runtime/node/zlib/NativeZstd.rs 6 8 0 test/js/node/zlib/zlib.test.js 2 1 0 test/js/web/streams/compression.test.ts 6 8 0 ``` </details> <!-- robobun:evidence:end --> --------- Co-authored-by: autofix-ci[bot] <114827586+autofix-ci[bot]@users.noreply.github.com>
2 participants
Repro
A zstd stream is one or more concatenated frames (RFC 8878 section 3.1);
zstd -d, pzstd output (multi-frame by construction),Bun.zstdDecompressSync, and thefetch()Content-Encoding: zstddecoder all decode every frame. OnlyDecompressionStream("zstd")rejects the second frame as trailing junk, so a real pzstd file is unreadable through it while every other zstd decoder in the process accepts it. A leading skippable frame (magic0x184D2A50..5F) is rejected the same way.Cause
The native zstd
do_work()callsZSTD_decompressStreamonce per write; libzstd stops at every frame boundary (returns 0 with the next frame unconsumed), soprocessCallbackseesavailInAfter > 0with output room and, becauseDecompressionStreampassesrejectGarbageAfterEnd: true, raisesERR_TRAILING_JUNK_AFTER_STREAM_END.Gunzipavoids this becauseNativeZlib::do_work_inflatealready loops on multi-member input.Fix
In
NativeZstd::do_work, afterZSTD_decompressStreamreturns 0 with input and output room remaining, peek the next four bytes: if they are the frame magic0xFD2FB528or a skippable-frame magic0x184D2A5?(or a prefix of either when fewer than four remain, so a header split across chunks decodes), callZSTD_decompressStreamagain. libzstd resets the DCtx to start-of-frame itself, so no explicit reset is needed. Bytes that are not a magic or magic prefix are left unconsumed so the existingERR_TRAILING_JUNK_AFTER_STREAM_ENDcheck still fires for genuine garbage, andcreateZstdDecompressstill silently ignores trailing non-frame data the way Node.js does.src/js/node/zlib.tsis unchanged.Verification
New tests in
test/js/web/streams/compression.test.tscover a 2-frame stream written as one chunk, a 2-frame stream split mid-header across two writes, 64 frames totalling more than one output chunk, a stream with leading and trailing skippable frames, and trailing non-frame garbage still rejected withERR_TRAILING_JUNK_AFTER_STREAM_END. The four multi-frame tests fail onmainwith that error and pass with this change; the Node-compatcreateZstdDecompresstrailing-data tests intest/js/node/zlib/zlib.test.jsstill pass.[review] gate passed · iteration 2 · 3 files touched
fails on main (without fix)
passes on PR (with fix)
diff hotspot
gate history · 4 passed · 0 rejected · iteration 2
evidence per changed file