fix: use Buffer.concat instead of Array.push for IDAT chunks to prevent OOM#83
Open
xbfld wants to merge 1 commit intofoliojs:masterfrom
Open
fix: use Buffer.concat instead of Array.push for IDAT chunks to prevent OOM#83xbfld wants to merge 1 commit intofoliojs:masterfrom
xbfld wants to merge 1 commit intofoliojs:masterfrom
Conversation
…nt OOM The IDAT chunk parser accumulated compressed image data by pushing individual bytes into a JavaScript Array. For large PNG files (100MB+), this created arrays with 100M+ elements. Since V8 stores each Smi element using 8 bytes on 64-bit platforms, a 107MB PNG would consume ~860MB of heap memory just for the JS Array backing store, far exceeding the actual data size. This caused V8 to crash with 'Fatal JavaScript invalid size error' when the array's backing FixedArray exceeded V8's maximum allocation size during capacity growth. Replace the byte-by-byte Array.push approach with Buffer.from/slice to collect IDAT chunk data as Buffer slices, then Buffer.concat to merge them. This reduces memory usage from ~8x the data size to ~1x. Before: 107MB IDAT data -> ~860MB heap (JS Array) -> V8 crash After: 107MB IDAT data -> ~107MB heap (Buffer) -> success
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #84
Problem
The IDAT chunk parser accumulated compressed image data by pushing individual bytes into a JavaScript Array. For large PNG files (100MB+), this created arrays with 100M+ elements. Since V8 stores each Smi element using 8 bytes on 64-bit platforms, a 107MB PNG would consume ~860MB of heap memory, causing V8 to crash with
Fatal JavaScript invalid size error.Solution
Replace the byte-by-byte
Array.pushapproach withBuffer.from/sliceto collect IDAT chunk data as Buffer slices, thenBuffer.concatto merge them.This PR description was written by Claude Code (claude-opus-4-6)