Add streaming extract parity with p7zip

[mjc]

Summary

• add streaming extract_to_writer APIs and use them for extract_to_memory/extract_all
• compose folder decoders as readers, including incremental BCJ x86 filtering
• add bidirectional p7zip parity coverage for LZMA, LZMA2, and BCJ+LZMA2
• document streaming extraction and p7zip parity guarantees

Verification

• cargo fmt -- --check
• cargo clippy -- -D warnings
• cargo test
• cargo test --test p7zip_extract_parity_test -- --nocapture
• cargo test --test interop_test -- --nocapture
• cargo test --test interop_write_test -- --nocapture
• RUSTDOCFLAGS='-D warnings' cargo doc --no-deps --document-private-items

[Copilot]

Pull request overview

This PR expands r7z’s extraction and interoperability capabilities by introducing streaming extraction APIs, composing folder decoders as readers (including incremental x86 BCJ filtering), and adding AES + BCJ/LZMA(2) parity/interop coverage against p7zip—alongside updated documentation and fixtures.

Changes:

• Add streaming extraction APIs (extract_to_writer*) and refactor extract_to_memory / extract_all to use streaming.
• Implement multi-coder folder decoding via reader composition (BCJ x86 filter, AES-256-SHA-256 decryption support).
• Add extensive p7zip parity/interop tests + new fixtures, plus documentation and benchmark additions.

Reviewed changes

Copilot reviewed 31 out of 33 changed files in this pull request and generated 7 comments.

Show a summary per file

File	Description
tests/support/mod.rs	Adds shared helpers for p7zip/r7z extraction and filesystem tree comparisons.
tests/read_parity_test.rs	Adds tests for path safety and corruption/truncation behavior.
tests/p7zip_extract_parity_test.rs	Adds bidirectional p7zip parity tests for LZMA/LZMA2/BCJ+LZMA2 and streaming extraction error behavior.
tests/lzma_perf_test.rs	Adds an ignored perf comparison harness (Rust LZMA vs liblzma via xz2).
tests/interop_write_test.rs	Extends interop tests (notably BCJ+LZMA2) and formatting/robustness improvements.
tests/interop_test.rs	Expands p7zip read interop coverage (non-solid, Copy, names, BCJ+LZMA2, AES scenarios, unsupported codecs).
tests/fixtures/bcj_lzma2.7z	Adds a p7zip-created BCJ+LZMA2 fixture for regression coverage.
tests/fixtures/aes256.7z	Adds an AES-encrypted fixture for password-related API/testing.
tests/decompression_test.rs	Adds fixture-based BCJ+LZMA2 decompression verification.
tests/archive_test.rs	Minor formatting adjustments around codec-id assertions.
src/stream_info.rs	Uses shared bitmap helper and minor refactors in stream parsing/tests.
src/parsers.rs	Introduces bitmap_is_set helper and updates digest bitmap scanning logic/tests.
src/pack_info.rs	Tightens property tag parsing, adds bounds checks, and updates digest parsing to use bitmap helper.
src/lib.rs	Updates crate docs and exports (AES module, BCJ module, password-aware APIs).
src/headers.rs	Minor formatting change in cached StreamsInfo parsing.
src/files_info.rs	Adds anti-item bitmap parsing + directory/anti helpers; improves parsing robustness for empty blocks.
src/error.rs	Adds password/path/directory-related error variants.
src/codec.rs	Implements reader-composed folder decoding, BCJ reader integration, AES coder handling, and execution order resolution.
src/builder.rs	Adds Codec::Lzma2Bcj write support and updates folder/header encoding for multi-coder folders.
src/bin/parse_flamegraph.rs	Formatting-only refactor in a utility binary.
src/bin/build_n64.rs	Formatting-only refactor in a utility binary.
src/bcj.rs	Adds x86 BCJ filter implementation + streaming reader + tests.
src/archive.rs	Adds password-aware open/parse APIs, streaming extraction to writer, safe-path extraction, and extraction refactors.
src/aes.rs	Adds AES-256-SHA-256 property parsing, key derivation, and CBC decryption + tests.
mempalace.yaml	Adds MemPalace wing/room configuration.
entities.json	Adds MemPalace entities registry content.
docs/mempalace/r7z-manifest.json	Adds MemPalace manifest describing canonical rooms/drawers/tunnels.
docs/mempalace/legacy-r7z-snapshot.json	Adds snapshot documenting legacy MemPalace state assessment.
docs/mempalace-workflow.md	Documents MemPalace workflow, filing rules, and acceptance snapshot.
benches/comparison.rs	Adds optional benchmarks for a large real-world N64 archive if present locally.
README.md	Updates documented feature set and adds examples for streaming + encrypted archives + BCJ+LZMA2 writes.
Cargo.toml	Adds AES/crypto deps and dev-deps for new tests/benchmarks.
CLAUDE.md	Updates project file map to include BCJ and new fixtures.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

src/codec.rs:94

• The doc comment for decompress_folder still says it “just calls decompress once”, but the decompress helper was removed and decompression is now implemented via folder_reader/coder_reader. This is misleading and references a non-existent function; update the comment to describe the current behavior (streaming reader chain) and/or reference the correct helper(s).

/// Decompress all folders in a Folder chain and return the concatenated output.
///
/// For simple single-coder folders this just calls `decompress` once.
/// BCJ+LZMA chaining (bind pairs) is resolved in order.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.