Skip to content

fix: writeFile for buffers >= 2GiB - #261

Closed
dyk1454683243-sudo wants to merge 1 commit into
isaacs:mainfrom
dyk1454683243-sudo:cursor/writefile-2gib-c5b9
Closed

dyk1454683243-sudo wants to merge 1 commit into
isaacs:mainfrom
dyk1454683243-sudo:cursor/writefile-2gib-c5b9

Conversation

@dyk1454683243-sudo

Copy link
Copy Markdown

Problem

gracefulFS.writeFile fails for buffers of 2GiB or larger:

RangeError [ERR_OUT_OF_RANGE]: The value of "length" is out of range. It must be >= 0 && <= 2147483647. Received 2147483648
    at Object.write (node:fs:848:5)
    at writeAll (node:fs:2224:6)

Node's callback writeFile (writeAll) calls the exported fs.write with the full buffer length. fs.write validates length as a signed 32-bit integer (uv_fs_write). fs.promises.writeFile already chunks, which is why native writeFile works on recent Node.

graceful-fs wraps writeFile for EMFILE retry, but writeAll still calls the real fs.write, so the patched path hits this cap.

Fix

Wrap the exported fs.write / fs.writeSync (the functions writeAll actually calls) and split binary writes larger than INT32_MAX (2147483647). String writes and smaller buffers are unchanged.

Tests

test/write-large-buffer.js spies on fs.write to prove chunking without requiring CI to write 2GiB. A writeFile case allocates a 2GiB buffer when possible and skips if allocation fails.

Fixes #256

Node's writeFile writeAll calls fs.write with the full buffer length.
fs.write rejects length > INT32_MAX, so graceful-fs writeFile failed
for buffers >= 2GiB while fs.promises.writeFile already chunks.

Wrap the exported fs.write/writeSync to split large writes, matching
current Node behavior.

Fixes isaacs#256

Co-authored-by: David <dyk1454683243-sudo@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gracefulFS.writeFile fails for buffers with size equal or greater than 2GiB, while fs.writeFile can write arbitrary lengths on latest Node.js

3 participants