Conversation
The parallel reader positions each worker by the stream index, so a block area that begins with the index indicator (0x00) means the index and the block area disagree — corruption. decodeBlock returned the bare errIndexIndicator sentinel unwrapped, which matches neither ErrCorrupt nor anything else a caller could act on; the same class of gap the block-header padding fix closed. The sentinel keeps its meaning in the sequential reader, where reaching the index indicator is how the block loop legitimately ends; only the parallel site, where the index has already promised a block, wraps it as ErrCorrupt. The test plants 0x00 at the first block's offset and asserts both readers reject the input and the parallel error matches ErrCorrupt; it fails on the previous code with the exact message observed. Signed-off-by: apostasie <spam_blackhole@farcloser.world> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The classic-format decoder pulls compressed bytes one at a time through an io.ByteReader. NewReader used the input directly when it already was one, and otherwise wrapped it in breader — a one-byte Read per byte, by design, so the reader would never consume past the stream. On an *os.File that is one system call per compressed byte: 7 MB/s against 84 MB/s for the same file behind a bufio.Reader, an 11.6× penalty for the most natural possible call, NewReader(f). The repo's own gxz CLI quietly avoided it by wrapping its files in bufio first. Wrap a plain reader in bufio.NewReader inside NewReader. An input that is already an io.ByteReader — bufio.Reader, bytes.Reader, strings.Reader — is left alone, so its exact position stays under the caller's control and nothing changes for it. The trade is documented on Reader: the wrapper may read past the stream's end, and a caller who needs the input positioned exactly there passes a ByteReader of its own. TestReaderInputConsumption pins both halves: a ByteReader input is consumed exactly to the end of the stream (this passed on the previous code too — the exact path is untouched), and a plain reader decodes identically with far fewer Reads (63,672 → a handful for a 63,684-byte stream; that subtest fails on the previous code). Benchmarks: plain file 7.2 → 84 MB/s, now indistinguishable from the bufio-wrapped case. Signed-off-by: apostasie <spam_blackhole@farcloser.world> Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.