Repository navigation
fix: prepare 0.8.8 blob integrity checks - #81
Merged
Merged
Conversation
Signed-off-by: wchwawa <wch19961116@gmail.com>
Compact records only authenticate a prefix of a reused slot. Read that prefix, clear the rest, and verify independent blobs in parallel for large Linux batches. Keep the per-blob result order and reject damaged pages before callers receive data. Signed-off-by: wchwawa <wch19961116@gmail.com>
The soak and benchmark harnesses use locked path dependencies. Update only their Holt package versions so release validation can run against this candidate without changing third-party packages. Signed-off-by: wchwawa <wch19961116@gmail.com>
wchwawa
marked this pull request as ready for review
October 26, 2026 22:54
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
Prepare Holt 0.8.8 to reject damaged file-backed blob reads on NoKV's current dependency line.
Problem
A changed byte in an active blob could reach a reader without a storage error. This is a private security follow-up; no public issue was opened under
SECURITY.md.Consequences
A caller could consume a wrong value, and a reused slot could expose bytes beyond a compact blob's persisted extent.
Example
A read of a damaged page now returns
NodeCorrupt. A range read beyond the authenticated extent returns zeros.Solution
Bind per-page checksums to each blob slot, sync the checksum record before the manifest mapping, and verify reads before returning them. Version 6 stores migrate on writable open. Compact reads fetch only the authenticated prefix; large Linux batches verify independent blobs in parallel and preserve result order.
Gain
Persistent blob damage fails closed. Sparse blobs avoid full-slot reads, and batch verification no longer runs serially.
Tradeoffs
CRC detects accidental damage, not an attacker who can rewrite storage metadata. Existing damage at migration time cannot be authenticated. Version 6 read-only opens need a prior writable migration. Dense-frame checksums still have material cost, so performance acceptance remains open and this PR stays Draft. NoKV remains locked to published Holt 0.8.7 until 0.8.8 is released and separately integrated.
Validation
cargo test --workspace --locked -- --test-threads=2passed on macOS before a final formatting-only change. Exact committed source passed macOS file-store tests 52/52 and Linux/arm64 Docker io-uring file-store tests 58/58, including a 40-blob damaged-page batch.cargo fmt --all -- --check, macOS workspace Clippy, Linux io-uring Clippy, andgit diff --checkpassed.cargo publish --dry-run --lockedpackaged and verified 128 files.crc32fast 1.5.0: sparse full reads changed by -47.1% on macOS and -13.2% in Linux/arm64 Docker; dense full reads changed by +33.6% and +56.6%, respectively. Dense batch reads changed by +8.5% and +35.8%. These are fresh-open microbenchmarks, not NoKV product latency evidence. Linux used relaxed Docker seccomp for io-uring.release/0.8. The soak and benchmark lockfiles now match Holt 0.8.8, and both locked harness checks pass. A manual Nightly Validation run with 60-second soak/fuzz inputs passed on this exact HEAD (Verus was not enabled); it is not a full nightly or a PR check.This is the 0.8-line backport of #80. No crate was published.