fix(buffer): use Latin-1 encoding for atob() per WHATWG spec

[chessbyte]

Issue # (if available)

Fixes #966

Description of changes

The atob() function was incorrectly treating decoded bytes as UTF-8, causing bytes >= 128 to be replaced with U+FFFD (replacement character).

Per the WHATWG spec, atob() should return a "binary string" where each character's code point directly represents a byte value (0-255). This matches Latin-1 (ISO-8859-1) encoding where each byte maps directly to a Unicode code point.

This fix resolves JWT verification issues (issue #966) where signature bytes containing values >= 128 were being corrupted, causing signature length mismatches (248 vs expected 256 bytes) and verification failures.

Added comprehensive tests for atob/btoa including:

• Basic encoding/decoding
• High-byte values (128-255) as Latin-1 characters
• Full 0-255 byte range verification
• Comparison with Buffer.from() for binary data

Sources:

• https://developer.mozilla.org/en-US/docs/Web/API/Window/atob
• https://infra.spec.whatwg.org/#forgiving-base64-decode
• https://web.dev/articles/base64-encoding

Checklist

• Created unit tests in tests/unit and/or in Rust for my feature if needed
• Ran make fix to format JS and apply Clippy auto fixes
• Made sure my code didn't add any additional warnings: make check
• Added relevant type info in types/ directory
• Updated documentation if needed (API.md/README.md/Other)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

[chessbyte]

Tried fixing seemingly unrelated Windows test in #1285

[richarddavison]

Nice! LGTM thank you

[Sytten]

I had to do some tricks in the string decoder. The problem though is that the rust to js only work with UTF-8. They added a raw 2 bytes read but not write.

[chessbyte]

@Sytten can you provide a failing scenario here that I will convert to a test and ensure it passes

[Sytten]

Honestly its a bit too far in my head, I would need to recheck. It was something to do with unsurrogate pair. Might not be relevant here, just know that UTF16 to UTF8 is a PITA.

[chessbyte]

@Sytten I re-read your concerns and added 2nd commit:

btoa() was incorrectly encoding the UTF-8 bytes of the input string
instead of treating each character as a byte value 0-255 (Latin-1).

This caused:

• btoa(String.fromCharCode(255)) to return "w78=" instead of "/w=="
• btoa("€") to return "4oKs" instead of throwing InvalidCharacterError

The fix iterates over character code points, validates each is ≤ 255, and converts directly to bytes. This ensures proper roundtrip with atob().

This addresses concerns raised by @Sytten about UTF-16 to UTF-8 string handling edge cases. Sitter noted that rquickjs's Rust-to-JS string interface primarily works with UTF-8 and mentioned potential issues with "unsurrogate pairs" - referring to lone surrogate code units.

Background on surrogate pairs: JavaScript strings are UTF-16 encoded internally. Characters outside the Basic Multilingual Plane (code points U+10000 and above, like emoji) are represented as surrogate pairs - two 16-bit code units where a high surrogate (0xD800-0xDBFF) is followed by a low surrogate (0xDC00-0xDFFF). A "lone surrogate" occurs when one
appears without its pair, which cannot be validly converted to UTF-8.

For btoa(), surrogate code units (0xD800-0xDFFF) are > 255, so they correctly trigger an error per the WHATWG spec requirement that btoa() only accepts characters with code points 0-255.

Node.js Comparison Tests:

Test	Node.js	LLRT (before)	LLRT (after)
btoa(String.fromCharCode(255))	"/w=="	"w78=" ❌	"/w==" ✅
btoa("€")	throws "Invalid character"	"4oKs" ❌	throws ✅
btoa(surrogate)	throws "Invalid character"	throws (UTF-8 error)	throws ✅

[chessbyte]

@Sytten @richarddavison what is the status of this PR? It has been sitting since my changes on Monday without any activity.

[richarddavison]

Sorry for the delay, xmas times... Minor perf suggestion!

[chessbyte]

The failing tests should be addressed by #1295